From patchwork Thu May 19 01:29:32 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Haigh X-Patchwork-Id: 9123451 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 3DCA9BF29F for ; Thu, 19 May 2016 01:32:31 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E92462022D for ; Thu, 19 May 2016 01:32:29 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ACD082022A for ; Thu, 19 May 2016 01:32:28 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b3Cmb-0004UU-5y; Thu, 19 May 2016 01:29:49 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b3CmZ-0004UK-AC for xen-devel@lists.xen.org; Thu, 19 May 2016 01:29:47 +0000 Received: from [85.158.143.35] by server-1.bemta-6.messagelabs.com id 9F/87-18833-A071D375; Thu, 19 May 2016 01:29:46 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrLIsWRWlGSWpSXmKPExsVy2uJbjC6TuG2 4wTdziyUfF7M4MHoc3f2bKYAxijUzLym/IoE1Y8fcL0wFzQEVM9evY21gvO/dxcjFISQwiUli 4t1VbF2MnBwsAl+ZJI4eVwSxeQUEJU7OfMICYjML+Eq8bVzHBFGjKtF3dS47iM0GZK9echmsV 0RAWuLa58uMEPXWEh1v94PVCAt4Sjx+cxtsDqeApcSO5lmMEIunMEqcfHCCGWKZhcSVxz/ABk kISEm8/7IfbJAo0IJJd7ayTWDkm4XkpllIboKw5SW2v53DDGFbSXRv3AkVV5SY0v2QHcIOldj +9Q9Ub7bExMUfGDHV+EhsPXcIak6SxIJbj5gWMHKtYlQvTi0qSy3SNdRLKspMzyjJTczM0TU0 MNPLTS0uTkxPzUlMKtZLzs/dxAiMCgYg2MG487nTIUZJDiYlUd6LIrbhQnxJ+SmVGYnFGfFFp TmpxYcYZTg4lCR4d4gC5QSLUtNTK9Iyc4DxCZOW4OBREuG9DpLmLS5IzC3OTIdInWLU5Viy68 FaJiGWvPy8VClx3t0gRQIgRRmleXAjYKniEqOslDAvI9BRQjwFqUW5mSWo8q8YxTkYlYR5X4J M4cnMK4Hb9AroCCagI26J2YAcUZKIkJJqYBQNk7jHsfDy1FONaQZcMkdDczRTt665wCQkx5vR osocoGNnvbsqXOuypWTM5DkdwmY1BovlC36p915bvvhK3uKoeyJGFivfnZ4yueLe8R17XrEHc jMEFrlciSv27s2ZXnWUU4/1j8ViTe33zwqYUk5PEu+93ruz/ImXZlH5wyub5ijO6A1zVGIpzk g01GIuKk4EAGx2h7EQAwAA X-Env-Sender: netwiz@crc.id.au X-Msg-Ref: server-10.tower-21.messagelabs.com!1463621376!14732931!1 X-Originating-IP: [203.56.246.92] X-SpamReason: No, hits=0.0 required=7.0 tests=ML_RADAR_SPEW_LINKS_8, spamassassin: ,async_handler: YXN5bmNfZGVsYXk6IDAgKHRpbWVvdXQp\n X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 40073 invoked from network); 19 May 2016 01:29:37 -0000 Received: from mail.crc.id.au (HELO mail.crc.id.au) (203.56.246.92) by server-10.tower-21.messagelabs.com with SMTP; 19 May 2016 01:29:37 -0000 Received: from lamp.crc.id.au (lamp.crc.id.au [IPv6:2407:e400:b000:200::8e8e]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.crc.id.au (Postfix) with ESMTPSA id E17203A005E; Thu, 19 May 2016 11:29:32 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crc.id.au; s=default; t=1463621373; bh=plKZPBA2BQiKmqc/V7vrey5SvXgUqbYfAvTFFjrwIzg=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=PwthKFYgn1xmv5kJC1cc+8gfsL4KCjUO4ApGN+TWF/MAgu92r3/ZqaAFbCLAYFfyH s2LVNOvXF6GbmEe/BtmaSJjhM8/L2FJcfgLpJPtIwnz1AQtlXg3bNxcEGJm8DSSoHy jPeXG2uybvQzsAruGLOtziT3VU9yxXabUcJJ+ueY= MIME-Version: 1.0 Date: Thu, 19 May 2016 11:29:32 +1000 From: Steven Haigh To: xen-devel@lists.xen.org In-Reply-To: References: <7096b642a7ecfe36f8896c4b2b840457@crc.id.au> Message-ID: X-Sender: netwiz@crc.id.au User-Agent: Roundcube Webmail/1.2-git Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: [Xen-devel] Resend: [PATCH] v3 - Add exclusive locking option to block-iscsi X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 2016-05-09 14:22, Steven Haigh wrote: > On 2016-05-05 15:52, Steven Haigh wrote: >> On 2016-05-05 12:32, Steven Haigh wrote: >>> Overview >>> >>> If you're using iSCSI, you can mount a target by multiple Dom0 >>> machines on the same target. For non-cluster aware filesystems, this >>> can lead to disk corruption and general bad times by all. The iSCSI >>> protocol allows the use of persistent reservations as per the SCSI >>> disk spec. Low level SCSI commands for locking are handled by the >>> sg_persist program (bundled with sg3_utils package in EL). >>> >>> The aim of this patch is to create a 'locktarget=y' option specified >>> within the disk 'target' command for iSCSI to lock the target in >>> exclusive mode on VM start with a key generated from the local >>> systems >>> IP, and release this lock on the shutdown of the DomU. >>> >>> Example Config: >>> disk = >>> ['script=block-iscsi,vdev=xvda,target=iqn=iqn.1986-03.com.sun:02:mytarget,portal=iscsi.example.com,locktarget=y'] >>> >>> In writing this, I have also re-factored parts of the script to put >>> some things in what I believe to be a better place to make expansion >>> easier. This is mainly in removing functions that purely call other >>> functions with no actual code execution. >>> >>> Signed-off-by: Steven Haigh >>> >>> (on a side note, first time I've submitted a patch to the list and >>> I'm >>> currently stuck on a webmail client, so apologies in advance if this >>> all goes wrong ;) >> >> Changes in v2: >> Bugfix: Call find_device to locate the /dev/sdX component of the iSCSI >> target before trying to run unlock_device(). >> >> Apologies for this oversight. >> > > Changes in v3: > * Split the block-iscsi cleanup into a seperate patch > (block-iscsi-locking-v3_01_simplify_block-iscsi.patch). > * Add locking in second patch file > (block-iscsi-locking-v3_02_add_locking.patch) Resend of patches. There was a mention of having to add further documentation to xl-disk-configuration.txt - however there are no mentions of block-iscsi script within the documentation to add. As such, it probably would be out of place to add things here. The locktarget option is presented directly to the block-iscsi script and not evaluated anywhere outside this script. --- block-iscsi.orig 2016-05-09 15:16:35.447480532 +1000 +++ block-iscsi 2016-05-05 15:43:58.222159351 +1000 @@ -37,8 +37,7 @@ { # set multipath default value multipath="n" - for param in $(echo "$1" | tr "," "\n") - do + for param in $(echo "$1" | tr "," "\n"); do case $param in iqn=*) iqn=$(remove_label $param "iqn=") @@ -55,6 +54,15 @@ fatal "Multipath selected, but no multipath tools found" fi ;; + locktarget=*) + locktarget=$(remove_label $param "locktarget=") + if ! command -v sg_persist > /dev/null 2>&1; then + fatal "Locking requested but no sg_persist found" + fi + if ! command -v gethostip > /dev/null 2>&1; then + fatal "Locking requested but no gethostip found for key generation" + fi + ;; esac done if [ -z "$iqn" ] || [ -z "$portal" ]; then @@ -92,6 +100,31 @@ fi } + +lock_device() +{ + ## Lock the iSCSI target as Exclusive Access. + key=$(gethostip -x $(uname -n)) + if ! sg_persist -d ${dev} -o -G -S ${key} > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to register with target" + fi + if ! sg_persist -d ${dev} -o -R -K ${key} -T 6 > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to set persistent reservation" + fi +} + +unlock_device() +{ + ## Unlock the iSCSI target. + key=$(gethostip -x $(uname -n)) + sg_persist -d ${dev} -o -L -K ${key} -T 6 > /dev/null || true + sg_persist -d ${dev} -o -G -K ${key} -S 0 > /dev/null || true +} + command=$1 target=$(xenstore-read $XENBUS_PATH/params || true) if [ -z "$target" ]; then @@ -110,10 +143,17 @@ ## Login to the iSCSI target. do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null - + find_device + if [ "$locktarget" = "y" ]; then + lock_device + fi write_dev $dev ;; remove) + find_device + if [ "$locktarget" = "y" ]; then + unlock_device + fi do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null ;; *)