From patchwork Thu Jun 30 21:57:38 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lin X-Patchwork-Id: 9208997 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AD51D6075F for ; Thu, 30 Jun 2016 22:02:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9848D2868C for ; Thu, 30 Jun 2016 22:02:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8CCC028691; Thu, 30 Jun 2016 22:02:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2EDEE2868C for ; Thu, 30 Jun 2016 22:02:08 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5ULwHt2031897; Thu, 30 Jun 2016 17:58:17 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5ULwFJ8003674 for ; Thu, 30 Jun 2016 17:58:15 -0400 Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u5ULwF2R025175 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 30 Jun 2016 17:58:15 -0400 Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3DF6781F03; Thu, 30 Jun 2016 21:58:14 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0F20F2027D; Thu, 30 Jun 2016 21:58:13 +0000 (UTC) Received: from [10.8.0.10] (unknown [159.203.220.84]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3B84320279; Thu, 30 Jun 2016 21:58:12 +0000 (UTC) Message-ID: <1467323858.15863.3.camel@ssi> From: Ming Lin To: Mike Snitzer Date: Thu, 30 Jun 2016 14:57:38 -0700 In-Reply-To: References: Mime-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP X-Greylist: Sender IP whitelisted by DNSRBL, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 30 Jun 2016 21:58:14 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 30 Jun 2016 21:58:14 +0000 (UTC) for IP:'198.145.29.136' DOMAIN:'mail.kernel.org' HELO:'mail.kernel.org' FROM:'mlin@kernel.org' RCPT:'' X-RedHat-Spam-Score: -2.787 (BAYES_50, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD) 198.145.29.136 mail.kernel.org 198.145.29.136 mail.kernel.org X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Scanned-By: MIMEDefang 2.75 on 10.5.110.26 X-loop: dm-devel@redhat.com Cc: device-mapper development , linux-nvme@lists.infradead.org Subject: Re: [dm-devel] NVMeoF multi-path setup X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Virus-Scanned: ClamAV using ClamSMTP On Thu, 2016-06-30 at 14:08 -0700, Ming Lin wrote: > Hi Mike, > > I'm trying to test NVMeoF multi-path. > > root@host:~# lsmod |grep dm_multipath > dm_multipath 24576 0 > root@host:~# ps aux |grep multipath > root 13183 0.0 0.1 238452 4972 ? SLl 13:41 0:00 > /sbin/multipathd > > I have nvme0 and nvme1 that are 2 paths to the same NVMe subsystem. > > root@host:/sys/class/nvme# grep . nvme*/address > nvme0/address:traddr=192.168.3.2,trsvcid=1023 > nvme1/address:traddr=192.168.2.2,trsvcid=1023 > > root@host:/sys/class/nvme# grep . nvme*/subsysnqn > nvme0/subsysnqn:nqn.testiqn > nvme1/subsysnqn:nqn.testiqn > > root@host:~# /lib/udev/scsi_id --export --whitelisted -d /dev/nvme1n1 > ID_SCSI=1 > ID_VENDOR=NVMe > ID_VENDOR_ENC=NVMe\x20\x20\x20\x20 > ID_MODEL=Linux > ID_MODEL_ENC=Linux > ID_REVISION=0-rc > ID_TYPE=disk > ID_SERIAL=SNVMe_Linux > ID_SERIAL_SHORT= > ID_SCSI_SERIAL=1122334455667788 > > root@host:~# /lib/udev/scsi_id --export --whitelisted -d /dev/nvme0n1 > ID_SCSI=1 > ID_VENDOR=NVMe > ID_VENDOR_ENC=NVMe\x20\x20\x20\x20 > ID_MODEL=Linux > ID_MODEL_ENC=Linux > ID_REVISION=0-rc > ID_TYPE=disk > ID_SERIAL=SNVMe_Linux > ID_SERIAL_SHORT= > ID_SCSI_SERIAL=1122334455667788 > > But seems multipathd didn't recognize these 2 devices. > > What else I'm missing? There are two problems: 1. there is no "/block/" in the path /sys/devices/virtual/nvme-fabrics/block/nvme0/nvme0n1 2. nvme was blacklisted. I added below quick hack to just make it work. root@host:~# cat /proc/partitions 259 0 937692504 nvme0n1 252 0 937692504 dm-0 259 1 937692504 nvme1n1 --- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel diff --git a/libmultipath/blacklist.c b/libmultipath/blacklist.c index 2400eda..a143383 100644 --- a/libmultipath/blacklist.c +++ b/libmultipath/blacklist.c @@ -190,9 +190,11 @@ setup_default_blist (struct config * conf) if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT)) return 1; +#if 0 str = STRDUP("^nvme.*"); if (!str) return 1; +#endif if (store_ble(conf->blist_devnode, str, ORIGIN_DEFAULT)) return 1; diff --git a/multipathd/main.c b/multipathd/main.c index c0ca571..1364070 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -1012,6 +1012,7 @@ uxsock_trigger (char * str, char ** reply, int * len, void * trigger_data) static int uev_discard(char * devpath) { +#if 0 char *tmp; char a[11], b[11]; @@ -1028,6 +1029,7 @@ uev_discard(char * devpath) condlog(4, "discard event on %s", devpath); return 1; } +#endif return 0; }