From patchwork Thu May 5 02:32:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Haigh X-Patchwork-Id: 9019871 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 64590BF29F for ; Thu, 5 May 2016 02:35:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3986C203C0 for ; Thu, 5 May 2016 02:35:05 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 315DB2035D for ; Thu, 5 May 2016 02:35:04 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ay95o-0008Ke-4x; Thu, 05 May 2016 02:32:44 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ay95n-0008KY-3z for xen-devel@lists.xen.org; Thu, 05 May 2016 02:32:43 +0000 Received: from [85.158.137.68] by server-1.bemta-3.messagelabs.com id 18/54-07924-AC0BA275; Thu, 05 May 2016 02:32:42 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrMIsWRWlGSWpSXmKPExsVy2uJbjO7BDVr hBrtO6Fss+biYxYHR4+ju30wBjFGsmXlJ+RUJrBk3ugULDlpXPJm0j6mB8alFFyMnh5DAJCaJ uQfyuhi5OFgE7jFJ3Hz1gQ0kwSsgKHFy5hMWEJtZwFdi1ZoDQHEOoCJViYffMkDCbEDm6iWXw cpFBKQlrn2+zAhRbi3R8XY/O4gtLOAo8W1fHwvESAuJD19fgdVLCEhJvP+yH6xeFGjOpDtb2S Yw8sxCsnkWks0QtrzE9rdzmCFsK4nujTuZIGxFiSndD9khbFuJace2QtUESzx9cpltASP7Kkb 14tSistQiXXO9pKLM9IyS3MTMHF1DA2O93NTi4sT01JzEpGK95PzcTYzAwGQAgh2Mjd+dDjFK cjApifJOStMKF+JLyk+pzEgszogvKs1JLT7EKMPBoSTBO3U9UE6wKDU9tSItMwcYIzBpCQ4eJ RHeDyBp3uKCxNzizHSI1ClGRSlx3r0gCQGQREZpHlwbLC4vMcpKCfMyAh0ixFOQWpSbWYIq/4 pRnINRSZj3FsgUnsy8Erjpr4AWMwEtfj9XE2RxSSJCSqqBcd9zMw+xx//V/k459dii229+Ddv hRaJCUUaq4uuqFhtMVlc4eO7i+73zHJ/PXfn8ivWSywzi96zaPF5/45gWtG5t19ctN6/mf+oU uTanMnZT6ztPoxXiW2eLyr0089V6FxQ+2zE7zmt5iolJ/vJX6XWp14Ru6RVM12vft3/eh3Vr2 JawuCw/3KzEUpyRaKjFXFScCADyGIXBxgIAAA== X-Env-Sender: netwiz@crc.id.au X-Msg-Ref: server-9.tower-31.messagelabs.com!1462415552!6966670!1 X-Originating-IP: [203.56.246.92] X-SpamReason: No, hits=0.0 required=7.0 tests=ML_RADAR_SPEW_LINKS_8, spamassassin: ,async_handler: YXN5bmNfZGVsYXk6IDAgKHRpbWVvdXQp\n X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 42053 invoked from network); 5 May 2016 02:32:33 -0000 Received: from mail.crc.id.au (HELO mail.crc.id.au) (203.56.246.92) by server-9.tower-31.messagelabs.com with SMTP; 5 May 2016 02:32:33 -0000 Received: from lamp.crc.id.au (lamp.crc.id.au [IPv6:2407:e400:b000:200::8e8e]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.crc.id.au (Postfix) with ESMTPSA id 4C1523A008F; Thu, 5 May 2016 12:32:29 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crc.id.au; s=default; t=1462415549; bh=I93/4G/His3huMSHANYt8S4+DZQfLw6vqVfAgUksdXo=; h=Date:From:To:Cc:Subject; b=OHJ3j/+KGII6nYtigEYoJY1tWZIEsgEu7nW1pmag+8gaf4JOLDs7G9DmtbfihDseC D9BapNQeLu+mEMIQVPlG6nfNOmUQQA0veC+5+z12XL1mcNb5/GZXzJWyUVI6EG6GZA V4W/zcTNU+D6HDgJ7K1iHtCxv6swvZWscPUf4AYU= MIME-Version: 1.0 Date: Thu, 05 May 2016 12:32:29 +1000 From: Steven Haigh To: xen-devel@lists.xen.org Message-ID: <7096b642a7ecfe36f8896c4b2b840457@crc.id.au> X-Sender: netwiz@crc.id.au User-Agent: Roundcube Webmail/1.2-git Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: [Xen-devel] [PATCH] v1 - Add exclusive locking option to block-iscsi X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Overview If you're using iSCSI, you can mount a target by multiple Dom0 machines on the same target. For non-cluster aware filesystems, this can lead to disk corruption and general bad times by all. The iSCSI protocol allows the use of persistent reservations as per the SCSI disk spec. Low level SCSI commands for locking are handled by the sg_persist program (bundled with sg3_utils package in EL). The aim of this patch is to create a 'locktarget=y' option specified within the disk 'target' command for iSCSI to lock the target in exclusive mode on VM start with a key generated from the local systems IP, and release this lock on the shutdown of the DomU. Example Config: disk = ['script=block-iscsi,vdev=xvda,target=iqn=iqn.1986-03.com.sun:02:mytarget,portal=iscsi.example.com,locktarget=y'] In writing this, I have also re-factored parts of the script to put some things in what I believe to be a better place to make expansion easier. This is mainly in removing functions that purely call other functions with no actual code execution. Signed-off-by: Steven Haigh (on a side note, first time I've submitted a patch to the list and I'm currently stuck on a webmail client, so apologies in advance if this all goes wrong ;) --- block-iscsi 2016-02-10 01:44:19.000000000 +1100 +++ block-iscsi-lock 2016-05-05 12:30:24.831903983 +1000 @@ -31,33 +31,37 @@ echo $1 | sed "s/^\("$2"\)//" } -check_tools() -{ - if ! command -v iscsiadm > /dev/null 2>&1; then - fatal "Unable to find iscsiadm tool" - fi - if [ "$multipath" = "y" ] && ! command -v multipath > /dev/null 2>&1; then - fatal "Unable to find multipath" - fi -} - # Sets the following global variables based on the params field passed in as # a parameter: iqn, portal, auth_method, user, multipath, password parse_target() { # set multipath default value multipath="n" - for param in $(echo "$1" | tr "," "\n") - do + for param in $(echo "$1" | tr "," "\n"); do case $param in iqn=*) iqn=$(remove_label $param "iqn=") + if ! command -v iscsiadm > /dev/null 2>&1; then + fatal "Could not find iscsiadm tool." + fi ;; portal=*) portal=$(remove_label $param "portal=") ;; multipath=*) multipath=$(remove_label $param "multipath=") + if ! command -v multipath > /dev/null 2>&1; then + fatal "Multipath selected, but no multipath tools found" + fi + ;; + locktarget=*) + locktarget=$(remove_label $param "locktarget=") + if ! command -v sg_persist > /dev/null 2>&1; then + fatal "Locking requested but no sg_persist found" + fi + if ! command -v gethostip > /dev/null 2>&1; then + fatal "Locking requested but no gethostip found for key generation" + fi ;; esac done @@ -96,38 +100,29 @@ fi } -# Attaches the target $iqn in $portal and sets $dev to point to the -# multipath device -attach() -{ - do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null - find_device -} - -# Discovers targets in $portal and checks that $iqn is one of those targets -# Also sets the auth parameters to attach the device -prepare() -{ - # Check if target is already opened - iscsiadm -m session 2>&1 | grep -q "$iqn" && fatal "Device already opened" - # Discover portal targets - iscsiadm -m discovery -t st -p $portal 2>&1 | grep -q "$iqn" || \ - fatal "No matching target iqn found" -} -# Attaches the device and writes xenstore backend entries to connect -# the device -add() +lock_device() { - attach - write_dev $dev + ## Lock the iSCSI target as Exclusive Access. + key=$(gethostip -x $(uname -n)) + if ! sg_persist -d ${dev} -o -G -S ${key} > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to register with target" + fi + if ! sg_persist -d ${dev} -o -R -K ${key} -T 6 > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to set persistent reservation" + fi } -# Disconnects the device -remove() +unlock_device() { - find_device - do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + ## Unlock the iSCSI target. + key=$(gethostip -x $(uname -n)) + sg_persist -d ${dev} -o -L -K ${key} -T 6 > /dev/null || true + sg_persist -d ${dev} -o -G -K ${key} -S 0 > /dev/null || true } command=$1 @@ -138,14 +133,27 @@ parse_target "$target" -check_tools || exit 1 - case $command in add) - prepare - add + # Check if target is already opened + iscsiadm -m session 2>&1 | grep -q "$iqn" && fatal "Device already opened" + # Discover portal targets + iscsiadm -m discovery -t st -p $portal 2>&1 | grep -q "$iqn" || \ + fatal "No matching target iqn found" + + ## Login to the iSCSI target. + do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null + find_device + if [ "$locktarget" = "y" ]; then + lock_device + fi + write_dev $dev ;; remove) + if [ "$locktarget" = "y" ]; then + unlock_device + fi + do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null remove ;; *)