From patchwork Thu Nov 12 14:00:11 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 7602461 Return-Path: X-Original-To: patchwork-linux-scsi@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 50B80BF90C for ; Thu, 12 Nov 2015 14:00:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 8423120765 for ; Thu, 12 Nov 2015 14:00:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3FFE12075F for ; Thu, 12 Nov 2015 14:00:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752773AbbKLOAO (ORCPT ); Thu, 12 Nov 2015 09:00:14 -0500 Received: from mx2.suse.de ([195.135.220.15]:37501 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752589AbbKLOAN (ORCPT ); Thu, 12 Nov 2015 09:00:13 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id DCA49AD02; Thu, 12 Nov 2015 13:59:49 +0000 (UTC) Message-ID: <56449B6B.5030903@suse.de> Date: Thu, 12 Nov 2015 15:00:11 +0100 From: Hannes Reinecke User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: Steffen Maier , "Martin K. Petersen" CC: Christoph Hellwig , Ewan Milne , linux-scsi@vger.kernel.org, James Bottomley Subject: Re: [PATCH 0/2] scsi_transport_fc: LUN masking References: <1447238080-109942-1-git-send-email-hare@suse.de> <56449526.5090405@linux.vnet.ibm.com> In-Reply-To: <56449526.5090405@linux.vnet.ibm.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, T_TVD_MIME_EPI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 11/12/2015 02:33 PM, Steffen Maier wrote: > Hi Hannes, > > On 11/11/2015 11:34 AM, Hannes Reinecke wrote: >> having been subjected to the pain of trying to bootstrap a really >> large machine with systemd I decided to implement LUN masking in > > This is very interesting. > > I also thought about initiator-based LUN masking a while ago. > While scsi_transport_fc would be sufficient to allow zfcp to get rid > of its own implementation, I wonder why we wouldn't do it in the > midlayer's scsi_scan.c. This would provide initiator-based LUN > masking for any and all transports and LLDDs? > > (I also thought about initiator-based zoning, and that would indeed > be something for scsi_transport_fc because it is inherently > transport specific.) > Well, it's not _that_ easy. The one problem you're facing here is addressing; at one point you'll have to identify which LUNs you'd like to unmask. While it's true that the LUN itself is stable, the (logical) target number from the SCSI-ID is not (disregarding SCSI parallel, of course). The same goes for the 'bus' and 'host' number. So how would you go about identifying them from the SCSI midlayer? Hence my approach of moving the addressing into the transport layer, where we _can_ identify the targets and hosts uniquely. >> scsi_transport_fc. >> The principle is simple: disallow the automated LUN scanning when >> discovering a rport, and create udev rules which selectively >> enable individual LUNs by echoing the relevant values in the 'scan' >> attribute of the SCSI host. > > I imagined exactly the same existing user space interface. > > Now we "only" need some user-friendly user space tooling, which > provides transport specific user arguments automatically translated > into the midlayer naming scheme, such as converting FC SAN > addressing (HBA device bus-ID,WWPN,FCPLUN) into scsi_device names > (Scsi_Host,0,scsi_id,scsi_lun), > and some persistency mechanism (such as managing udev rules). > Oh, that's easy. Attached is a simple dracut module which will enable LUN-masking during booting. >> With that I'm able to boot an arbitrary large machine without >> running into any udev or systemd imposed timeout. >> To _disable_ LUN masking and restoring the original behaviour >> I've noticed that the 'scan' sysfs attribute is actually synchronous, >> ie the calling process will be blocked until the entire LUN scan >> is completed. >> So I've added another module parameter 'async_user_scan' to >> move the scanning onto the existing scan workqueue, and unblock >> the calling process. > > Just curious: Which means would a user of async_user_scan have to > sync against the completion? > Or is this only meant for user space--I currently cannot > imagine--which does not care about the appearance of what was > (newly) discovered? > The problem I've been running into is that I have a machine where LUN enumeration alone take 1800s (and no, that's not a typo). So with the current implementation of 'scan' the calling process would be stuck for that time, too. If you were to try to rescan all LUNs during startup with say a systemd service it would be killed halfway through as the initialisation would take too long. Plus any sanely defined service requiring any of those disks would only startup once all disks are discovered, so there is no need to wait for the discovery to finish. Or that's the hope, at least :-) Cheers, Hannes From 1784a28f26011ff0dd29b1c40494cadfa71df564 Mon Sep 17 00:00:00 2001 From: Hannes Reinecke Date: Wed, 11 Nov 2015 12:20:33 +0100 Subject: [PATCH] 95lunmask: Add module to handle LUN masking Using the module option 'scsi_transport_fc.disable_target_scan' this implements LUN masking by selectively enable only those devices required for booting. References: bsc#954600 Signed-off-by: Hannes Reinecke --- modules.d/95lunmask/fc_transport_scan_lun.sh | 26 +++++++++++++ modules.d/95lunmask/module-setup.sh | 55 ++++++++++++++++++++++++++++ modules.d/95lunmask/parse-lunmask.sh | 28 ++++++++++++++ 3 files changed, 109 insertions(+) create mode 100755 modules.d/95lunmask/fc_transport_scan_lun.sh create mode 100755 modules.d/95lunmask/module-setup.sh create mode 100755 modules.d/95lunmask/parse-lunmask.sh diff --git a/modules.d/95lunmask/fc_transport_scan_lun.sh b/modules.d/95lunmask/fc_transport_scan_lun.sh new file mode 100755 index 0000000..d9f84a3 --- /dev/null +++ b/modules.d/95lunmask/fc_transport_scan_lun.sh @@ -0,0 +1,26 @@ +#!/bin/bash +# +# fc_transport_lun_scan +# +# Selectively enable individual LUNs behind an FC remote port +# +# ACTION=="add", SUBSYSTEM=="fc_transport", ATTR{port_name}=="wwpn", \ +# PROGRAM="fc_transport_lun_scan lun" +# + +[ -z $DEVPATH ] && exit 1 + +if [ -n "$1" ] ; then + LUN=$1 +else + LUN=- +fi +ID=${DEVPATH##*/rport-} +HOST=${ID%%:*} +CHANNEL=${ID%%-*} +CHANNEL=${CHANNEL#*:} +if [ -f /sys$DEVPATH/scsi_target_id ] ; then + TARGET=$(cat /sys$DEVPATH/scsi_target_id) +fi +[ -z "$TARGET" ] && exit 1 +echo $CHANNEL $TARGET $LUN > /sys/class/scsi_host/host$HOST/scan diff --git a/modules.d/95lunmask/module-setup.sh b/modules.d/95lunmask/module-setup.sh new file mode 100755 index 0000000..d8d559b --- /dev/null +++ b/modules.d/95lunmask/module-setup.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +# called by dracut +cmdline() { + get_lunmask() { + local _dev=$1 + local _devpath=$(cd -P /sys/dev/block/$_dev ; echo $PWD) + local _sdev _lun _rport _classdev _wwpn + + [ "${_devpath#*/sd}" == "$_devpath" ] && return 1 + _sdev="${_devpath%%/block/*}" + _lun="${_sdev##*:}" + _rport="${_devpath##*/rport-}" + [ "$_rport" == "$_devpath" ] && return 1 + _rport="${_rport%%/*}" + _classdev="/sys/class/fc_remote_ports/rport-${_rport}" + [ -d "$_classdev" ] || return 1 + _wwpn=$(cat ${_classdev}/port_name) + echo "rd.lunmask=${_wwpn},${_lun}" + return 0 + } + [[ $hostonly ]] || [[ $mount_needs ]] && { + for_each_host_dev_and_slaves_all get_lunmask + } | sort | uniq +} + +# called by dracut +check() { + [[ $hostonly ]] || [[ $mount_needs ]] && { + [ -f /sys/module/scsi_transport_fc/parameters/disable_target_scan ] \ + && return 0 + return 255 + } + return 0 +} + +# called by dracut +depends() { + return 0 +} + +# called by dracut +install() { + inst_script "$moddir/fc_transport_scan_lun.sh" /usr/lib/udev/fc_transport_scan_lun.sh + inst_hook cmdline 30 "$moddir/parse-lunmask.sh" + if [[ $hostonly_cmdline == "yes" ]] ; then + local _lunmask + + for _lunmask in $(cmdline) ; do + printf "%s\n" "$_lunmask" >> "${initdir}/etc/cmdline.d/95lunmask.conf" + done + fi +} diff --git a/modules.d/95lunmask/parse-lunmask.sh b/modules.d/95lunmask/parse-lunmask.sh new file mode 100755 index 0000000..af990dd --- /dev/null +++ b/modules.d/95lunmask/parse-lunmask.sh @@ -0,0 +1,28 @@ +#!/bin/sh +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*- +# ex: ts=8 sw=4 sts=4 et filetype=sh + +create_udev_rule() { + local wwpn=$1 + local lun=$2 + local _rule=/etc/udev/rules.d/51-fc-lunmask-${wwpn}.rules + local _cu_type _dev_type + + [ -e ${_rule} ] && return 0 + + if [ ! -f "$_rule" ] ; then + cat > $_rule <