From patchwork Wed Jun 14 14:00:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13280090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24148EB64D9 for ; Wed, 14 Jun 2023 14:00:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245311AbjFNOAo (ORCPT ); Wed, 14 Jun 2023 10:00:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245289AbjFNOAn (ORCPT ); Wed, 14 Jun 2023 10:00:43 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C628511B for ; Wed, 14 Jun 2023 07:00:42 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5C70763C9D for ; Wed, 14 Jun 2023 14:00:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6AC54C433C0; Wed, 14 Jun 2023 14:00:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686751241; bh=yXdH0Bj1EtydNi/S+U3oAgvHc7WfEz5wPLW7IdVlhCo=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=FLtN8gQzb10lushZZgeOyUMSe9WabagiUZOyDRfu/wDxZbwMvGQGsL5fgOiCOCvSX Hf920KnW0LMSlHd4CEA3KoWTMhwxdf9nL2QEp5gXYYqchiWHgzX/w1HJrNMkABGPXk nBofM/3LGmkBGATt/RZhbo14xfVjUQKZOXPh1Hi30hOv44Lhz0LVZTpw6YEjfalIzS R2dfQKhYSd88Q0/ox+ukoCOS8WbGHW+s31cWzlvI+5sOKVUZD17r1JjCs3nwiHcNZD TxxnRaT6gLa2thdWqBHEPO35Fy61GZwqyv33LJZo+GT8Omk/xWSyGJVupilMCoJiuA mWLXdYbJl3WbQ== Subject: [PATCH v3 1/4] RDMA/siw: Fabricate a GID on tun and loopback devices From: Chuck Lever To: jgg@nvidia.com Cc: Tom Talpey , Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com, BMT@zurich.ibm.com Date: Wed, 14 Jun 2023 10:00:40 -0400 Message-ID: <168675124033.2279.4244453854641171409.stgit@manet.1015granger.net> In-Reply-To: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> References: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever LOOPBACK and NONE (tunnel) devices have all-zero MAC addresses. Currently, siw_device_create() falls back to copying the IB device's name in those cases, because an all-zero MAC address breaks the RDMA core address resolution mechanism. However, at the point when siw_device_create() constructs a GID, the ib_device::name field is uninitialized, leaving the MAC address to remain in an all-zero state. Fabricate a random artificial GID for such devices, and ensure that artificial GID is returned for all device query operations. Reported-by: Tom Talpey Link: https://lore.kernel.org/linux-rdma/SA0PR15MB391986C07C4D41E107E79659994FA@SA0PR15MB3919.namprd15.prod.outlook.com/T/#t Fixes: a2d36b02c15d ("RDMA/siw: Enable siw on tunnel devices") Signed-off-by: Chuck Lever Reviewed-by: Bernard Metzler Reviewed-by: Tom Talpey --- drivers/infiniband/sw/siw/siw.h | 1 + drivers/infiniband/sw/siw/siw_main.c | 22 ++++++++-------------- drivers/infiniband/sw/siw/siw_verbs.c | 4 ++-- 3 files changed, 11 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/sw/siw/siw.h b/drivers/infiniband/sw/siw/siw.h index 2f3a9cda3850..8b4a710b82bc 100644 --- a/drivers/infiniband/sw/siw/siw.h +++ b/drivers/infiniband/sw/siw/siw.h @@ -74,6 +74,7 @@ struct siw_device { u32 vendor_part_id; int numa_node; + char raw_gid[ETH_ALEN]; /* physical port state (only one port per device) */ enum ib_port_state state; diff --git a/drivers/infiniband/sw/siw/siw_main.c b/drivers/infiniband/sw/siw/siw_main.c index 65b5cda5457b..f45600d169ae 100644 --- a/drivers/infiniband/sw/siw/siw_main.c +++ b/drivers/infiniband/sw/siw/siw_main.c @@ -75,8 +75,7 @@ static int siw_device_register(struct siw_device *sdev, const char *name) return rv; } - siw_dbg(base_dev, "HWaddr=%pM\n", sdev->netdev->dev_addr); - + siw_dbg(base_dev, "HWaddr=%pM\n", sdev->raw_gid); return 0; } @@ -313,24 +312,19 @@ static struct siw_device *siw_device_create(struct net_device *netdev) return NULL; base_dev = &sdev->base_dev; - sdev->netdev = netdev; - if (netdev->type != ARPHRD_LOOPBACK && netdev->type != ARPHRD_NONE) { - addrconf_addr_eui48((unsigned char *)&base_dev->node_guid, - netdev->dev_addr); + if (netdev->addr_len) { + memcpy(sdev->raw_gid, netdev->dev_addr, + min_t(unsigned int, netdev->addr_len, ETH_ALEN)); } else { /* - * This device does not have a HW address, - * but connection mangagement lib expects gid != 0 + * This device does not have a HW address, but + * connection mangagement requires a unique gid. */ - size_t len = min_t(size_t, strlen(base_dev->name), 6); - char addr[6] = { }; - - memcpy(addr, base_dev->name, len); - addrconf_addr_eui48((unsigned char *)&base_dev->node_guid, - addr); + eth_random_addr(sdev->raw_gid); } + addrconf_addr_eui48((u8 *)&base_dev->node_guid, sdev->raw_gid); base_dev->uverbs_cmd_mask |= BIT_ULL(IB_USER_VERBS_CMD_POST_SEND); diff --git a/drivers/infiniband/sw/siw/siw_verbs.c b/drivers/infiniband/sw/siw/siw_verbs.c index 398ec13db624..32b0befd25e2 100644 --- a/drivers/infiniband/sw/siw/siw_verbs.c +++ b/drivers/infiniband/sw/siw/siw_verbs.c @@ -157,7 +157,7 @@ int siw_query_device(struct ib_device *base_dev, struct ib_device_attr *attr, attr->vendor_part_id = sdev->vendor_part_id; addrconf_addr_eui48((u8 *)&attr->sys_image_guid, - sdev->netdev->dev_addr); + sdev->raw_gid); return 0; } @@ -218,7 +218,7 @@ int siw_query_gid(struct ib_device *base_dev, u32 port, int idx, /* subnet_prefix == interface_id == 0; */ memset(gid, 0, sizeof(*gid)); - memcpy(&gid->raw[0], sdev->netdev->dev_addr, 6); + memcpy(gid->raw, sdev->raw_gid, ETH_ALEN); return 0; } From patchwork Wed Jun 14 14:00:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13280091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DE40EB64D8 for ; Wed, 14 Jun 2023 14:00:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245289AbjFNOAu (ORCPT ); Wed, 14 Jun 2023 10:00:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245264AbjFNOAu (ORCPT ); Wed, 14 Jun 2023 10:00:50 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4483911B for ; Wed, 14 Jun 2023 07:00:49 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CEE3563C9D for ; Wed, 14 Jun 2023 14:00:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DDDB2C433C8; Wed, 14 Jun 2023 14:00:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686751248; bh=lJguwL74pHMBTj150wwXK6RGyrGPW//UR6ztnL09Fas=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=RrKyI3/9BsGwiYmI7sKSDPpwnDQDPMHEbCjVXkjrBaZXvrliJ9KZuvZIxpGuxhciB 5AngzZrHaHE40l0AzV5pTmZvYSQuvG6P7jOuusxLoIGkejfjJaahHe7KwFqFmRxXLU y/+65zitWekZORwKN7WZwoAobTlK710ezzxsJs3V97XRK1vQytzenhUHpSnQMG/5/1 26yWY6bCFWRVzqrVD8k2rivH/t8pB/Z+gYFWJVxizXZ1pOIRGNFcj/VfCek0PbfwZs cmd0rgHmmZQ9nXTkdnh9Jpx0WYZeumH0DwqxIdOog+iXzJ9hrPHJckSY2GG+AbxJN4 EebthM3Q7ol6Q== Subject: [PATCH v3 2/4] RDMA/core: Set gid_attr.ndev for iWARP devices From: Chuck Lever To: jgg@nvidia.com Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com, BMT@zurich.ibm.com Date: Wed, 14 Jun 2023 10:00:47 -0400 Message-ID: <168675124698.2279.15699248221119454150.stgit@manet.1015granger.net> In-Reply-To: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> References: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Have the iwarp side properly set the ndev in the device's sgid_attrs so that address resolution can treat it more like a RoCE device. Suggested-by: Jason Gunthorpe Signed-off-by: Chuck Lever Reviewed-by: Tom Talpey --- drivers/infiniband/core/cache.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 2e91d8879326..717524fe8a39 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -1439,6 +1439,7 @@ static int config_non_roce_gid_cache(struct ib_device *device, { struct ib_gid_attr gid_attr = {}; struct ib_gid_table *table; + struct net_device *ndev; int ret = 0; int i; @@ -1457,10 +1458,21 @@ static int config_non_roce_gid_cache(struct ib_device *device, i); goto err; } + + ndev = NULL; + if (rdma_protocol_iwarp(device, port)) { + ndev = ib_device_get_netdev(device, port); + if (!ndev) + continue; + RCU_INIT_POINTER(gid_attr.ndev, ndev); + } + gid_attr.index = i; tprops->subnet_prefix = be64_to_cpu(gid_attr.gid.global.subnet_prefix); add_modify_gid(table, &gid_attr); + + dev_put(ndev); } err: mutex_unlock(&table->lock); From patchwork Wed Jun 14 14:00:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13280092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3726EEB64D9 for ; Wed, 14 Jun 2023 14:00:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245313AbjFNOA5 (ORCPT ); Wed, 14 Jun 2023 10:00:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245264AbjFNOA4 (ORCPT ); Wed, 14 Jun 2023 10:00:56 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C649D11B for ; Wed, 14 Jun 2023 07:00:55 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 5BE5563F79 for ; Wed, 14 Jun 2023 14:00:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6CC7DC433C0; Wed, 14 Jun 2023 14:00:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686751254; bh=QYfiTehHwNsyhrFEAAsXvpslipmwSWUHlNZkYctCvIc=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=VZznJmM91AaqDHi3YzW0rMCRxMpBO67AyNm8ycdQgizEtdlyOzAqXhmqjjPLSTtFH SoeG+hmMu1H0jnBm2Ku1RWdAgneaMqJY0EQ2cSpAEie8t4navN6UKxHLZ63OPiGANK yPxLgcrerZYlmi7DSTpaWD+uL1Y6NV4KM2ptHJkt43X+yv3iFMmyQrkfXHZKHrWCSV bwL1rxd61lizDXVa8RUIsA0wZUH2eFAROW7CSA/e4+2kPaafu3qDZfiUYb8KFquwnG Zn4WdvAgBnsj6n8il4bi6uWf8eco4UFAaPGmiK9fqoBTUfSnvud5ULbmsSQQqSkooV BlpuNozF1zbvg== Subject: [PATCH v3 3/4] RDMA/cma: Deduplicate error flow in cma_validate_port() From: Chuck Lever To: jgg@nvidia.com Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com, BMT@zurich.ibm.com Date: Wed, 14 Jun 2023 10:00:53 -0400 Message-ID: <168675125344.2279.13016950228751332493.stgit@manet.1015granger.net> In-Reply-To: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> References: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever Clean up to prepare for the addition of new logic. Signed-off-by: Chuck Lever --- drivers/infiniband/core/cma.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 93a1c48d0c32..a1756ed1faa1 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -686,30 +686,31 @@ cma_validate_port(struct ib_device *device, u32 port, struct rdma_id_private *id_priv) { struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr; + const struct ib_gid_attr *sgid_attr = ERR_PTR(-ENODEV); int bound_if_index = dev_addr->bound_dev_if; - const struct ib_gid_attr *sgid_attr; int dev_type = dev_addr->dev_type; struct net_device *ndev = NULL; if (!rdma_dev_access_netns(device, id_priv->id.route.addr.dev_addr.net)) - return ERR_PTR(-ENODEV); + goto out; if ((dev_type == ARPHRD_INFINIBAND) && !rdma_protocol_ib(device, port)) - return ERR_PTR(-ENODEV); + goto out; if ((dev_type != ARPHRD_INFINIBAND) && rdma_protocol_ib(device, port)) - return ERR_PTR(-ENODEV); + goto out; if (dev_type == ARPHRD_ETHER && rdma_protocol_roce(device, port)) { ndev = dev_get_by_index(dev_addr->net, bound_if_index); if (!ndev) - return ERR_PTR(-ENODEV); + goto out; } else { gid_type = IB_GID_TYPE_IB; } sgid_attr = rdma_find_gid_by_port(device, gid, gid_type, port, ndev); dev_put(ndev); +out: return sgid_attr; } From patchwork Wed Jun 14 14:00:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13280101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADB00EB64D8 for ; Wed, 14 Jun 2023 14:01:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236308AbjFNOBL (ORCPT ); Wed, 14 Jun 2023 10:01:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245324AbjFNOBD (ORCPT ); Wed, 14 Jun 2023 10:01:03 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AAB51BC6 for ; Wed, 14 Jun 2023 07:01:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D3E5F63F79 for ; Wed, 14 Jun 2023 14:01:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DEF8CC433C8; Wed, 14 Jun 2023 14:01:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686751261; bh=BQsvPIcbmiqCskEL61I0tER2ROKUt45q7hKBes3HzJo=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ej3NjMo0VHMaeDmfPXyWkl32vCr6gN1QI8wZrgEu5BaFPtzQoBmMhNTa5N1De9fYU crzrbtPbuXKLos+P5IsS6A4dZ+jQ+tjM+/tOuag74WHdR+OoafBbwjPCszwQwTLqC3 hq/PKlKxP9I58pcTjzPVMdZ3Lsjp5O81hSJARDIokgLDXU+zSxfeV7fL/RB7/jCnZn YIHGzvWvUWMtP/2uSG7wj5lfNjJnVKHB7qLFWs0Y/7Z+2eU9vQQ+I6kGlvA9JcgxpT o5DIqBg+9mF6iTZwGqI56q2I+P89hnnnloEnp02evSEtJjVGPsrDf7mno2uIVW1uAP OW0VSYz8zJGrw== Subject: [PATCH v3 4/4] RDMA/cma: Avoid GID lookups on iWARP devices From: Chuck Lever To: jgg@nvidia.com Cc: Chuck Lever , linux-rdma@vger.kernel.org, tom@talpey.com, BMT@zurich.ibm.com Date: Wed, 14 Jun 2023 10:00:59 -0400 Message-ID: <168675125998.2279.7297073638926155456.stgit@manet.1015granger.net> In-Reply-To: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> References: <168675101993.2279.4985978457935843722.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Chuck Lever We would like to enable the use of siw on top of a VPN that is constructed and managed via a tun device. That hasn't worked up until now because ARPHRD_NONE devices (such as tun devices) have no GID for the RDMA/core to look up. But it turns out that the egress device has already been picked for us -- no GID is necessary. addr_handler() just has to do the right thing with it. Suggested-by: Jason Gunthorpe Signed-off-by: Chuck Lever --- drivers/infiniband/core/cma.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index a1756ed1faa1..50b8da2c4720 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -700,6 +700,19 @@ cma_validate_port(struct ib_device *device, u32 port, if ((dev_type != ARPHRD_INFINIBAND) && rdma_protocol_ib(device, port)) goto out; + if (rdma_protocol_iwarp(device, port)) { + sgid_attr = rdma_get_gid_attr(device, port, 0); + if (IS_ERR(sgid_attr)) + goto out; + + /* XXX: I don't think this is RCU-safe, but does it need to be? */ + ndev = rcu_dereference(sgid_attr->ndev); + if (!net_eq(dev_net(ndev), dev_addr->net) || + ndev->ifindex != bound_if_index) + sgid_attr = ERR_PTR(-ENODEV); + goto out; + } + if (dev_type == ARPHRD_ETHER && rdma_protocol_roce(device, port)) { ndev = dev_get_by_index(dev_addr->net, bound_if_index); if (!ndev)