From patchwork Thu May 5 05:52:30 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Haigh X-Patchwork-Id: 9021131 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C570EBF29F for ; Thu, 5 May 2016 05:55:50 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C5061203EC for ; Thu, 5 May 2016 05:55:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C09FA203E3 for ; Thu, 5 May 2016 05:55:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ayCDQ-0002TA-Ay; Thu, 05 May 2016 05:52:48 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ayCDO-0002T4-NW for xen-devel@lists.xen.org; Thu, 05 May 2016 05:52:47 +0000 Received: from [85.158.139.211] by server-13.bemta-5.messagelabs.com id EC/8A-26485-DAFDA275; Thu, 05 May 2016 05:52:45 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrJIsWRWlGSWpSXmKPExsVy2uJbjO7S+1r hBi+P6Vgs+biYxYHR4+ju30wBjFGsmXlJ+RUJrBk7775iK3hkW/F6JUcD41WrLkYuDiGBSUwS D/deZgNxWAS+MkmceLqEpYuRk4NXQFDi5MwnYDazgK/EjumvmEFsFgFViSUNB8HibED26iUgz ZwcIgLSEtc+X2aEqLeW6Hi7nx3EFhZwlPiw9wFYPaeApcT585eA6jmANltITDpvBbHKQuLbqW tgYyQEpCTef9kPNkYUaPykO1vZJjDyzUJy0SwkF0HY8hLb386Bsq0kujfuZIKwFSWmdD9kh7B tJU7PWwzVGyzxrPso4wJG9lWMGsWpRWWpRbqGRnpJRZnpGSW5iZk5uoYGpnq5qcXFiempOYlJ xXrJ+bmbGIHBzAAEOxj7ZjkfYpTkYFIS5WXcqBUuxJeUn1KZkVicEV9UmpNafIhRhoNDSYL38 D2gnGBRanpqRVpmDjCuYNISHDxKIrx2IGne4oLE3OLMdIjUKUZdjiW7HqxlEmLJy89LlRLnnQ JSJABSlFGaBzcCFuOXGGWlhHkZgY4S4ilILcrNLEGVf8UozsGoJMy7GmQKT2ZeCdymV0BHMAE d8X6uJsgRJYkIKakGxtw2k5PX1+9cc/J2+f1Ghhy+o6qnPI3q1n9IlAj+Mnm7nrBcxZ+3t4/3 Wvtekfc79aDBMMnsh4KPFJ9nXFhP45dZXPevVGuplLi01BVHqBr1L7bV2LdixtejZQparlkzu mZNzGyI2Zwmu1fnM7dhcviOyxPevVp1NPaDf8eMWcf/+XXtaPV8qsRSnJFoqMVcVJwIAOEZST /sAgAA X-Env-Sender: netwiz@crc.id.au X-Msg-Ref: server-11.tower-206.messagelabs.com!1462427556!25792881!1 X-Originating-IP: [203.56.246.92] X-SpamReason: No, hits=0.0 required=7.0 tests=ML_RADAR_SPEW_LINKS_8, spamassassin: ,async_handler: YXN5bmNfZGVsYXk6IDAgKHRpbWVvdXQp\n X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 12626 invoked from network); 5 May 2016 05:52:36 -0000 Received: from mail.crc.id.au (HELO mail.crc.id.au) (203.56.246.92) by server-11.tower-206.messagelabs.com with SMTP; 5 May 2016 05:52:36 -0000 Received: from lamp.crc.id.au (lamp.crc.id.au [IPv6:2407:e400:b000:200::8e8e]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.crc.id.au (Postfix) with ESMTPSA id F03683A008F; Thu, 5 May 2016 15:52:30 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crc.id.au; s=default; t=1462427551; bh=i0S2XgnWq+FORmNOhyLbbaBHiDCacteQlC0VvGb235I=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=LZEJ7Z+JNk44eAiXbZPXbdfn0nYntqXuSzZklUPsJUUNqh8H12hy7gKLtfBvsPSZW wiEieEsuH4T+l6OqUMUn+JaHV8HGMznmPlvVyd1XzC6t1Eqye2XH3bzrRL1aOD/+7n WDHAbVYakYKxT5MDsSUlYVqL/tX0cXGI5uX+Gi1Y= MIME-Version: 1.0 Date: Thu, 05 May 2016 15:52:30 +1000 From: Steven Haigh To: xen-devel@lists.xen.org In-Reply-To: <7096b642a7ecfe36f8896c4b2b840457@crc.id.au> References: <7096b642a7ecfe36f8896c4b2b840457@crc.id.au> Message-ID: X-Sender: netwiz@crc.id.au User-Agent: Roundcube Webmail/1.2-git Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: [Xen-devel] [PATCH] v2 - Add exclusive locking option to block-iscsi X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 2016-05-05 12:32, Steven Haigh wrote: > Overview > > If you're using iSCSI, you can mount a target by multiple Dom0 > machines on the same target. For non-cluster aware filesystems, this > can lead to disk corruption and general bad times by all. The iSCSI > protocol allows the use of persistent reservations as per the SCSI > disk spec. Low level SCSI commands for locking are handled by the > sg_persist program (bundled with sg3_utils package in EL). > > The aim of this patch is to create a 'locktarget=y' option specified > within the disk 'target' command for iSCSI to lock the target in > exclusive mode on VM start with a key generated from the local systems > IP, and release this lock on the shutdown of the DomU. > > Example Config: > disk = > ['script=block-iscsi,vdev=xvda,target=iqn=iqn.1986-03.com.sun:02:mytarget,portal=iscsi.example.com,locktarget=y'] > > In writing this, I have also re-factored parts of the script to put > some things in what I believe to be a better place to make expansion > easier. This is mainly in removing functions that purely call other > functions with no actual code execution. > > Signed-off-by: Steven Haigh > > (on a side note, first time I've submitted a patch to the list and I'm > currently stuck on a webmail client, so apologies in advance if this > all goes wrong ;) Changes in v2: Bugfix: Call find_device to locate the /dev/sdX component of the iSCSI target before trying to run unlock_device(). Apologies for this oversight. --- block-iscsi 2016-02-10 01:44:19.000000000 +1100 +++ block-iscsi-lock 2016-05-05 15:42:09.557191235 +1000 @@ -31,33 +31,37 @@ echo $1 | sed "s/^\("$2"\)//" } -check_tools() -{ - if ! command -v iscsiadm > /dev/null 2>&1; then - fatal "Unable to find iscsiadm tool" - fi - if [ "$multipath" = "y" ] && ! command -v multipath > /dev/null 2>&1; then - fatal "Unable to find multipath" - fi -} - # Sets the following global variables based on the params field passed in as # a parameter: iqn, portal, auth_method, user, multipath, password parse_target() { # set multipath default value multipath="n" - for param in $(echo "$1" | tr "," "\n") - do + for param in $(echo "$1" | tr "," "\n"); do case $param in iqn=*) iqn=$(remove_label $param "iqn=") + if ! command -v iscsiadm > /dev/null 2>&1; then + fatal "Could not find iscsiadm tool." + fi ;; portal=*) portal=$(remove_label $param "portal=") ;; multipath=*) multipath=$(remove_label $param "multipath=") + if ! command -v multipath > /dev/null 2>&1; then + fatal "Multipath selected, but no multipath tools found" + fi + ;; + locktarget=*) + locktarget=$(remove_label $param "locktarget=") + if ! command -v sg_persist > /dev/null 2>&1; then + fatal "Locking requested but no sg_persist found" + fi + if ! command -v gethostip > /dev/null 2>&1; then + fatal "Locking requested but no gethostip found for key generation" + fi ;; esac done @@ -96,38 +100,29 @@ fi } -# Attaches the target $iqn in $portal and sets $dev to point to the -# multipath device -attach() -{ - do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null - find_device -} -# Discovers targets in $portal and checks that $iqn is one of those targets -# Also sets the auth parameters to attach the device -prepare() +lock_device() { - # Check if target is already opened - iscsiadm -m session 2>&1 | grep -q "$iqn" && fatal "Device already opened" - # Discover portal targets - iscsiadm -m discovery -t st -p $portal 2>&1 | grep -q "$iqn" || \ - fatal "No matching target iqn found" -} - -# Attaches the device and writes xenstore backend entries to connect -# the device -add() -{ - attach - write_dev $dev + ## Lock the iSCSI target as Exclusive Access. + key=$(gethostip -x $(uname -n)) + if ! sg_persist -d ${dev} -o -G -S ${key} > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to register with target" + fi + if ! sg_persist -d ${dev} -o -R -K ${key} -T 6 > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to set persistent reservation" + fi } -# Disconnects the device -remove() +unlock_device() { - find_device - do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + ## Unlock the iSCSI target. + key=$(gethostip -x $(uname -n)) + sg_persist -d ${dev} -o -L -K ${key} -T 6 > /dev/null || true + sg_persist -d ${dev} -o -G -K ${key} -S 0 > /dev/null || true } command=$1 @@ -138,15 +133,28 @@ parse_target "$target" -check_tools || exit 1 - case $command in add) - prepare - add + # Check if target is already opened + iscsiadm -m session 2>&1 | grep -q "$iqn" && fatal "Device already opened" + # Discover portal targets + iscsiadm -m discovery -t st -p $portal 2>&1 | grep -q "$iqn" || \ + fatal "No matching target iqn found" + + ## Login to the iSCSI target. + do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null + find_device + if [ "$locktarget" = "y" ]; then + lock_device + fi + write_dev $dev ;; remove) - remove + find_device + if [ "$locktarget" = "y" ]; then + unlock_device + fi + do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null ;; *) exit 1