From patchwork Thu Feb 27 21:16:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Simmons X-Patchwork-Id: 11410651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 760AC138D for ; Thu, 27 Feb 2020 21:43:29 +0000 (UTC) Received: from pdx1-mailman02.dreamhost.com (pdx1-mailman02.dreamhost.com [64.90.62.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F00224690 for ; Thu, 27 Feb 2020 21:43:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F00224690 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lustre-devel-bounces@lists.lustre.org Received: from pdx1-mailman02.dreamhost.com (localhost [IPv6:::1]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 8B01A349326; Thu, 27 Feb 2020 13:35:00 -0800 (PST) X-Original-To: lustre-devel@lists.lustre.org Delivered-To: lustre-devel-lustre.org@pdx1-mailman02.dreamhost.com Received: from smtp3.ccs.ornl.gov (smtp3.ccs.ornl.gov [160.91.203.39]) by pdx1-mailman02.dreamhost.com (Postfix) with ESMTP id 9BBEE348862 for ; Thu, 27 Feb 2020 13:20:54 -0800 (PST) Received: from star.ccs.ornl.gov (star.ccs.ornl.gov [160.91.202.134]) by smtp3.ccs.ornl.gov (Postfix) with ESMTP id 2D73D918F; Thu, 27 Feb 2020 16:18:19 -0500 (EST) Received: by star.ccs.ornl.gov (Postfix, from userid 2004) id 2C48C46A; Thu, 27 Feb 2020 16:18:19 -0500 (EST) From: James Simmons To: Andreas Dilger , Oleg Drokin , NeilBrown Date: Thu, 27 Feb 2020 16:16:10 -0500 Message-Id: <1582838290-17243-503-git-send-email-jsimmons@infradead.org> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> References: <1582838290-17243-1-git-send-email-jsimmons@infradead.org> Subject: [lustre-devel] [PATCH 502/622] lnet: o2ib: Reintroduce kiblnd_dev_search X-BeenThere: lustre-devel@lists.lustre.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "For discussing Lustre software development." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lustre Development List MIME-Version: 1.0 Errors-To: lustre-devel-bounces@lists.lustre.org Sender: "lustre-devel" From: Chris Horn If we add an interface to multiple nets then we need to re-use the struct ib_dev object for each of the nets. Cray-bug-id: LUS-7935 Fixes: 3aa523159321 ("lnet: consoldate secondary IP address handling") WC-bug-id: https://jira.whamcloud.com/browse/LU-12824 Lustre-commit: e25e45c612a0 ("LU-12824 o2ib: Reintroduce kiblnd_dev_search") Signed-off-by: Chris Horn Reviewed-on: https://review.whamcloud.com/36326 Reviewed-by: James Simmons Reviewed-by: Olaf Weber Reviewed-by: Amir Shehata Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- net/lnet/klnds/o2iblnd/o2iblnd.c | 85 +++++++++++++++++++++++++++++----------- 1 file changed, 63 insertions(+), 22 deletions(-) diff --git a/net/lnet/klnds/o2iblnd/o2iblnd.c b/net/lnet/klnds/o2iblnd/o2iblnd.c index d162b0a7..1cc5358 100644 --- a/net/lnet/klnds/o2iblnd/o2iblnd.c +++ b/net/lnet/klnds/o2iblnd/o2iblnd.c @@ -2821,7 +2821,8 @@ static int kiblnd_start_schedulers(struct kib_sched_info *sched) return rc; } -static int kiblnd_dev_start_threads(struct kib_dev *dev, u32 *cpts, int ncpts) +static int kiblnd_dev_start_threads(struct kib_dev *dev, bool newdev, u32 *cpts, + int ncpts) { int cpt; int rc; @@ -2833,7 +2834,7 @@ static int kiblnd_dev_start_threads(struct kib_dev *dev, u32 *cpts, int ncpts) cpt = !cpts ? i : cpts[i]; sched = kiblnd_data.kib_scheds[cpt]; - if (sched->ibs_nthreads > 0) + if (!newdev && sched->ibs_nthreads > 0) continue; rc = kiblnd_start_schedulers(kiblnd_data.kib_scheds[cpt]); @@ -2846,6 +2847,39 @@ static int kiblnd_dev_start_threads(struct kib_dev *dev, u32 *cpts, int ncpts) return 0; } +static struct kib_dev * +kiblnd_dev_search(char *ifname) +{ + struct kib_dev *alias = NULL; + struct kib_dev *dev; + char *colon; + char *colon2; + + colon = strchr(ifname, ':'); + list_for_each_entry(dev, &kiblnd_data.kib_devs, ibd_list) { + if (strcmp(&dev->ibd_ifname[0], ifname) == 0) + return dev; + + if (alias) + continue; + + colon2 = strchr(dev->ibd_ifname, ':'); + if (colon) + *colon = 0; + if (colon2) + *colon2 = 0; + + if (strcmp(&dev->ibd_ifname[0], ifname) == 0) + alias = dev; + + if (colon) + *colon = ':'; + if (colon2) + *colon2 = ':'; + } + return alias; +} + static int kiblnd_startup(struct lnet_ni *ni) { char *ifname = NULL; @@ -2855,6 +2889,7 @@ static int kiblnd_startup(struct lnet_ni *ni) unsigned long flags; int rc; int i; + bool newdev; LASSERT(ni->ni_net->net_lnd == &the_o2iblnd); @@ -2916,36 +2951,42 @@ static int kiblnd_startup(struct lnet_ni *ni) goto failed; } - ibdev = kzalloc(sizeof(*ibdev), GFP_KERNEL); - if (!ibdev) { - rc = -ENOMEM; - goto failed; - } + ibdev = kiblnd_dev_search(ifname); + newdev = !ibdev; + /* hmm...create kib_dev even for alias */ + if (!ibdev || strcmp(&ibdev->ibd_ifname[0], ifname) != 0) { + ibdev = kzalloc(sizeof(*ibdev), GFP_NOFS); + if (!ibdev) { + rc = -ENOMEM; + goto failed; + } - ibdev->ibd_ifip = ifaces[i].li_ipaddr; - strlcpy(ibdev->ibd_ifname, ifaces[i].li_name, - sizeof(ibdev->ibd_ifname)); - ibdev->ibd_can_failover = !!(ifaces[i].li_flags & IFF_MASTER); + ibdev->ibd_ifip = ifaces[i].li_ipaddr; + strlcpy(ibdev->ibd_ifname, ifaces[i].li_name, + sizeof(ibdev->ibd_ifname)); + ibdev->ibd_can_failover = !!(ifaces[i].li_flags & IFF_MASTER); - INIT_LIST_HEAD(&ibdev->ibd_nets); - INIT_LIST_HEAD(&ibdev->ibd_list); /* not yet in kib_devs */ - INIT_LIST_HEAD(&ibdev->ibd_fail_list); + INIT_LIST_HEAD(&ibdev->ibd_nets); + INIT_LIST_HEAD(&ibdev->ibd_list); /* not yet in kib_devs */ + INIT_LIST_HEAD(&ibdev->ibd_fail_list); - /* initialize the device */ - rc = kiblnd_dev_failover(ibdev, ni->ni_net_ns); - if (rc) { - CERROR("ko2iblnd: Can't initialize device: rc = %d\n", rc); - goto failed; - } + /* initialize the device */ + rc = kiblnd_dev_failover(ibdev, ni->ni_net_ns); + if (rc) { + CERROR("ko2iblnd: Can't initialize device: rc = %d\n", + rc); + goto failed; + } - list_add_tail(&ibdev->ibd_list, &kiblnd_data.kib_devs); + list_add_tail(&ibdev->ibd_list, &kiblnd_data.kib_devs); + } net->ibn_dev = ibdev; ni->ni_nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), ibdev->ibd_ifip); ni->ni_dev_cpt = ifaces[i].li_cpt; - rc = kiblnd_dev_start_threads(ibdev, ni->ni_cpts, ni->ni_ncpts); + rc = kiblnd_dev_start_threads(ibdev, newdev, ni->ni_cpts, ni->ni_ncpts); if (rc) goto failed;