From patchwork Mon May 9 04:22:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Haigh X-Patchwork-Id: 9041081 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 527499F30C for ; Mon, 9 May 2016 04:26:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 332F120138 for ; Mon, 9 May 2016 04:26:03 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 030E62012D for ; Mon, 9 May 2016 04:26:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1azcio-00083m-Ib; Mon, 09 May 2016 04:23:06 +0000 Received: from mail6.bemta6.messagelabs.com ([85.158.143.247]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1azcin-00083g-2M for xen-devel@lists.xen.org; Mon, 09 May 2016 04:23:05 +0000 Received: from [85.158.143.35] by server-2.bemta-6.messagelabs.com id E5/6A-09532-8A010375; Mon, 09 May 2016 04:23:04 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrDIsWRWlGSWpSXmKPExsVy2uJbjO58AYN wg62XTSyWfFzM4sDocXT3b6YAxijWzLyk/IoE1ow5c3eyFizwqdi0q4G9gfG8RxcjF4eQwCQm iTV/tzGCOCwCX5kkLu67ydLFyMnBKyAocXLmEyCbg4NZwFfi0GxzkDCLgKrEpN2r2UBsNiB79 ZLLYLaIgLTEtc+XGUFsZgFriY63+9lBbGEBR4n5F36DjeQUsJQ4u7oBrEZIIE3i/PfZbBCrLC S6p1xgBbElBKQk3n/ZD1YjCrLrzla2CYx8s5BcNAvhollg2+Qltr+dwwxhW0l0b9zJBGErSkz pfsgOYYdKbP/6hwXCzpaYuPgDI6YaH4mt5w5BzUmSWHDrEdMCRq5VjOrFqUVlqUW6xnpJRZnp GSW5iZk5uoYGZnq5qcXFiempOYlJxXrJ+bmbGIExwQAEOxg7/jkdYpTkYFIS5XVl1AsX4kvKT 6nMSCzOiC8qzUktPsQow8GhJMFry28QLiRYlJqeWpGWmQOMTpi0BAePkghvP0iat7ggMbc4Mx 0idYpRl2PJrgdrmYRY8vLzUqXEeXVBigRAijJK8+BGwBLFJUZZKWFeRqCjhHgKUotyM0tQ5V8 xinMwKgnzmoFM4cnMK4Hb9AroCCagI+TY9EGOKElESEk1MC42Y1a1WeMucz3g1N9JS09+qX6/ ag/7/7gojZ4if4N1MVfmz5vT2Wv35uHEorcvHAN3rZm8bXKS9wLD7cJJsaWbXxvnvb5x1bl4v vwa3TVLdY4/Eb4nclN3z8uQx1vMef6XxsS/rZnUL/yBc63glL4/kRf23jx/vLkqZ4KsJ5uB2Z Ht+2Q0V5xWYinOSDTUYi4qTgQAILj0YQ8DAAA= X-Env-Sender: netwiz@crc.id.au X-Msg-Ref: server-7.tower-21.messagelabs.com!1462767774!13273220!1 X-Originating-IP: [203.56.246.92] X-SpamReason: No, hits=0.0 required=7.0 tests=ML_RADAR_SPEW_LINKS_8, spamassassin: ,async_handler: YXN5bmNfZGVsYXk6IDAgKHRpbWVvdXQp\n X-StarScan-Received: X-StarScan-Version: 8.34; banners=-,-,- X-VirusChecked: Checked Received: (qmail 46488 invoked from network); 9 May 2016 04:22:54 -0000 Received: from mail.crc.id.au (HELO mail.crc.id.au) (203.56.246.92) by server-7.tower-21.messagelabs.com with SMTP; 9 May 2016 04:22:54 -0000 Received: from lamp.crc.id.au (lamp.crc.id.au [IPv6:2407:e400:b000:200::8e8e]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.crc.id.au (Postfix) with ESMTPSA id 23D793A008C; Mon, 9 May 2016 14:22:49 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crc.id.au; s=default; t=1462767769; bh=PPTjwat2B8XylJD/CH9qKduJVJK9QaP1FTjPToGkubM=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=JjyygmGBDVsTorE6kZVUVfESHSjZ2frQVMIP43Vu4ohJ8edyy14WMZA2kapIq08e9 6s4uQAzM1lDZBPpSRwThF4N62N21VxGdMJ2RWXWPKdbNo+8SMDNMZVmKohEhXpOCq2 0cdyzi1KG8fkoyLzvP6Jku9heTi11wWpRROUV5Vw= MIME-Version: 1.0 Date: Mon, 09 May 2016 14:22:48 +1000 From: Steven Haigh To: xen-devel@lists.xen.org In-Reply-To: References: <7096b642a7ecfe36f8896c4b2b840457@crc.id.au> Message-ID: X-Sender: netwiz@crc.id.au User-Agent: Roundcube Webmail/1.2-git Cc: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: [Xen-devel] [PATCH] v3 - Add exclusive locking option to block-iscsi X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 2016-05-05 15:52, Steven Haigh wrote: > On 2016-05-05 12:32, Steven Haigh wrote: >> Overview >> >> If you're using iSCSI, you can mount a target by multiple Dom0 >> machines on the same target. For non-cluster aware filesystems, this >> can lead to disk corruption and general bad times by all. The iSCSI >> protocol allows the use of persistent reservations as per the SCSI >> disk spec. Low level SCSI commands for locking are handled by the >> sg_persist program (bundled with sg3_utils package in EL). >> >> The aim of this patch is to create a 'locktarget=y' option specified >> within the disk 'target' command for iSCSI to lock the target in >> exclusive mode on VM start with a key generated from the local systems >> IP, and release this lock on the shutdown of the DomU. >> >> Example Config: >> disk = >> ['script=block-iscsi,vdev=xvda,target=iqn=iqn.1986-03.com.sun:02:mytarget,portal=iscsi.example.com,locktarget=y'] >> >> In writing this, I have also re-factored parts of the script to put >> some things in what I believe to be a better place to make expansion >> easier. This is mainly in removing functions that purely call other >> functions with no actual code execution. >> >> Signed-off-by: Steven Haigh >> >> (on a side note, first time I've submitted a patch to the list and I'm >> currently stuck on a webmail client, so apologies in advance if this >> all goes wrong ;) > > Changes in v2: > Bugfix: Call find_device to locate the /dev/sdX component of the iSCSI > target before trying to run unlock_device(). > > Apologies for this oversight. > Changes in v3: * Split the block-iscsi cleanup into a seperate patch (block-iscsi-locking-v3_01_simplify_block-iscsi.patch). * Add locking in second patch file (block-iscsi-locking-v3_02_add_locking.patch) --- block-iscsi 2016-05-09 15:16:35.447480532 +1000 +++ block-iscsi-lock 2016-05-05 15:43:58.222159351 +1000 @@ -37,8 +37,7 @@ { # set multipath default value multipath="n" - for param in $(echo "$1" | tr "," "\n") - do + for param in $(echo "$1" | tr "," "\n"); do case $param in iqn=*) iqn=$(remove_label $param "iqn=") @@ -55,6 +54,15 @@ fatal "Multipath selected, but no multipath tools found" fi ;; + locktarget=*) + locktarget=$(remove_label $param "locktarget=") + if ! command -v sg_persist > /dev/null 2>&1; then + fatal "Locking requested but no sg_persist found" + fi + if ! command -v gethostip > /dev/null 2>&1; then + fatal "Locking requested but no gethostip found for key generation" + fi + ;; esac done if [ -z "$iqn" ] || [ -z "$portal" ]; then @@ -92,6 +100,31 @@ fi } + +lock_device() +{ + ## Lock the iSCSI target as Exclusive Access. + key=$(gethostip -x $(uname -n)) + if ! sg_persist -d ${dev} -o -G -S ${key} > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to register with target" + fi + if ! sg_persist -d ${dev} -o -R -K ${key} -T 6 > /dev/null; then + unlock_device + iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null + fatal "iSCSI LOCK: Failed to set persistent reservation" + fi +} + +unlock_device() +{ + ## Unlock the iSCSI target. + key=$(gethostip -x $(uname -n)) + sg_persist -d ${dev} -o -L -K ${key} -T 6 > /dev/null || true + sg_persist -d ${dev} -o -G -K ${key} -S 0 > /dev/null || true +} + command=$1 target=$(xenstore-read $XENBUS_PATH/params || true) if [ -z "$target" ]; then @@ -110,10 +143,17 @@ ## Login to the iSCSI target. do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --login > /dev/null - + find_device + if [ "$locktarget" = "y" ]; then + lock_device + fi write_dev $dev ;; remove) + find_device + if [ "$locktarget" = "y" ]; then + unlock_device + fi do_or_die iscsiadm -m node --targetname "$iqn" -p "$portal" --logout > /dev/null ;; *)