From patchwork Fri Jun 23 09:57:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28BBCEB64D7 for ; Fri, 23 Jun 2023 09:58:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230338AbjFWJ6c (ORCPT ); Fri, 23 Jun 2023 05:58:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230120AbjFWJ6b (ORCPT ); Fri, 23 Jun 2023 05:58:31 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26ECAC6 for ; Fri, 23 Jun 2023 02:58:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514310; x=1719050310; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=srdNvnLtJM4cXhiaa6KO7TQ2+8tSqelv/CbI9RrmN3k=; b=lT8dX8anU+kst2O4Nwe5E+F1FQR9xGgyQQKYnjeBLH3p0eHegoL8pjf4 9GwYu0Ki9Y6n71JHQXc9lb8hfUPub7LNcssOOxya+KOemxmT3z/hIdE+0 OU0JssJVMASgkPwscmm3ESdjNmD79Ph3CX1532QbFSKOjKl5bwrTyyEd6 cNn9rQbJ7//VV3jELQ1T945gYJ36Hp2P5YmSHGAorCzkzZ/cjVCD8XcTG CFRL+axBpJqAOcMQJ9RLz97splXBHRZiHpbiFbJt8Ye734eJ+tHH5awRT 2o/GHfHgco9ezjGiHI9uCqpFCIkW+P+euX7koaxBjTLjZm7nl4kn/k1nO g==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411353" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411353" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715262988" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715262988" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:27 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 1/8] RDMA/rxe: Creating listening sock in newlink function Date: Fri, 23 Jun 2023 17:57:42 +0800 Message-Id: <20230623095749.485873-2-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Originally when the module rdma_rxe is loaded, the sock listening on udp port 4791 is created. Currently moving the creating listening port to newlink function. So when running "rdma link add" command, the sock listening on udp port 4791 is created. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 7a7e713de52d..89b24bc34299 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -194,6 +194,10 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev) goto err; } + err = rxe_net_init(); + if (err) + return err; + err = rxe_net_add(ibdev_name, ndev); if (err) { rxe_err("failed to add %s\n", ndev->name); @@ -210,12 +214,6 @@ static struct rdma_link_ops rxe_link_ops = { static int __init rxe_module_init(void) { - int err; - - err = rxe_net_init(); - if (err) - return err; - rdma_link_register(&rxe_link_ops); pr_info("loaded\n"); return 0; From patchwork Fri Jun 23 09:57:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290222 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 823F8EB64DD for ; Fri, 23 Jun 2023 09:58:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230120AbjFWJ6f (ORCPT ); Fri, 23 Jun 2023 05:58:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231332AbjFWJ6e (ORCPT ); Fri, 23 Jun 2023 05:58:34 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35BF910D2 for ; Fri, 23 Jun 2023 02:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514313; x=1719050313; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iENf/iiU5uaw3/3Q2YFd6N3f+fEw/dYrZ6aZUcw4lWg=; b=LkG9b3pJ9SV7jYt2wB+unUZoMbhC7K/dCXE3gaKSLEyIk3+1GRLYMu30 e9injftIRO9ge1Hf5YzpmzBvlw+Sp4Ya4VICV88UAC57BsMooVHL+vvlo O57+x7LCNX5TygrpP3hAn5fRnkmZAbFCfB/Cw6KR13e/QVxiH8UPLiNYf /6VFvPhw9UarAxwKcVfT1pbBYnHKF6KD3AWJVEW8q+Tsephyh2rK06tjV f+wTV+3vb/XrxH1ePPzfdpBLP9jWypQYxGcSjFyndvBGeNUBLKIH2fQYG 9P4Eo3lDenMdWJvptRF0Xz1Su0xMJcYG6pV5n1xyD9DF70OgZ37unvTsD A==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411362" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411362" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:32 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715262995" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715262995" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:30 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 2/8] RDMA/rxe: Support more rdma links in init_net Date: Fri, 23 Jun 2023 17:57:43 +0800 Message-Id: <20230623095749.485873-3-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun In init_net, when several rdma links are created with the command "rdma link add", newlink will check whether the udp port 4791 is listening or not. If not, creating a sock listening on udp port 4791. If yes, increasing the reference count of the sock. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 12 ++++++- drivers/infiniband/sw/rxe/rxe_net.c | 55 +++++++++++++++++++++-------- drivers/infiniband/sw/rxe/rxe_net.h | 1 + 3 files changed, 52 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 89b24bc34299..c15d3c5d7a6f 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -8,6 +8,7 @@ #include #include "rxe.h" #include "rxe_loc.h" +#include "rxe_net.h" MODULE_AUTHOR("Bob Pearson, Frank Zago, John Groves, Kamal Heib"); MODULE_DESCRIPTION("Soft RDMA transport"); @@ -207,14 +208,23 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev) return err; } -static struct rdma_link_ops rxe_link_ops = { +struct rdma_link_ops rxe_link_ops = { .type = "rxe", .newlink = rxe_newlink, }; static int __init rxe_module_init(void) { + int err; + rdma_link_register(&rxe_link_ops); + + err = rxe_register_notifier(); + if (err) { + pr_err("Failed to register netdev notifier\n"); + return -1; + } + pr_info("loaded\n"); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index a38fab19bed1..141020bfcf3b 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -627,13 +627,23 @@ static struct notifier_block rxe_net_notifier = { static int rxe_net_ipv4_init(void) { - recv_sockets.sk4 = rxe_setup_udp_tunnel(&init_net, - htons(ROCE_V2_UDP_DPORT), false); - if (IS_ERR(recv_sockets.sk4)) { - recv_sockets.sk4 = NULL; + struct sock *sk; + struct socket *sock; + + rcu_read_lock(); + sk = udp4_lib_lookup(&init_net, 0, 0, htonl(INADDR_ANY), + htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (sk) + return 0; + + sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), false); + if (IS_ERR(sock)) { pr_err("Failed to create IPv4 UDP tunnel\n"); + recv_sockets.sk4 = NULL; return -1; } + recv_sockets.sk4 = sock; return 0; } @@ -641,24 +651,46 @@ static int rxe_net_ipv4_init(void) static int rxe_net_ipv6_init(void) { #if IS_ENABLED(CONFIG_IPV6) + struct sock *sk; + struct socket *sock; + + rcu_read_lock(); + sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, + htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (sk) + return 0; - recv_sockets.sk6 = rxe_setup_udp_tunnel(&init_net, - htons(ROCE_V2_UDP_DPORT), true); - if (PTR_ERR(recv_sockets.sk6) == -EAFNOSUPPORT) { + sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), true); + if (PTR_ERR(sock) == -EAFNOSUPPORT) { recv_sockets.sk6 = NULL; pr_warn("IPv6 is not supported, can not create a UDPv6 socket\n"); return 0; } - if (IS_ERR(recv_sockets.sk6)) { + if (IS_ERR(sock)) { recv_sockets.sk6 = NULL; pr_err("Failed to create IPv6 UDP tunnel\n"); return -1; } + recv_sockets.sk6 = sock; #endif return 0; } +int rxe_register_notifier(void) +{ + int err; + + err = register_netdevice_notifier(&rxe_net_notifier); + if (err) { + pr_err("Failed to register netdev notifier\n"); + return -1; + } + + return 0; +} + void rxe_net_exit(void) { rxe_release_udp_tunnel(recv_sockets.sk6); @@ -670,19 +702,12 @@ int rxe_net_init(void) { int err; - recv_sockets.sk6 = NULL; - err = rxe_net_ipv4_init(); if (err) return err; err = rxe_net_ipv6_init(); if (err) goto err_out; - err = register_netdevice_notifier(&rxe_net_notifier); - if (err) { - pr_err("Failed to register netdev notifier\n"); - goto err_out; - } return 0; err_out: rxe_net_exit(); diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index 45d80d00f86b..a222c3eeae12 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -18,6 +18,7 @@ struct rxe_recv_sockets { int rxe_net_add(const char *ibdev_name, struct net_device *ndev); +int rxe_register_notifier(void); int rxe_net_init(void); void rxe_net_exit(void); From patchwork Fri Jun 23 09:57:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F301EB64D7 for ; Fri, 23 Jun 2023 09:58:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231799AbjFWJ6j (ORCPT ); Fri, 23 Jun 2023 05:58:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231754AbjFWJ6h (ORCPT ); Fri, 23 Jun 2023 05:58:37 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F49C10C2 for ; Fri, 23 Jun 2023 02:58:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514316; x=1719050316; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=riLZi2effh8u4Uro4E8PQpRTP+7UfnPIysrb7hhnVKA=; b=JRosjZSC9V/1RpdpyRc/HmApIUDoxOb5ilq3uy+0ZOPzZKRSNU0LuHKZ hESmzvhHUEfXWKEcPF1w3w7oGq+pQpXEE9C8EC2jNNPQcF6iQa82S0w0J vBqEOl55jzEdvGL5EzsnG55yrxmtLShNr6xxdz8PoIyweXmW4eFHv4UR+ H0ZdFO1aDc/4dwtF/xy+AVn1lu1YfU+XeKqeiz5i8KqiO7r5BCNLrNEmH QxugLjK14eKjKNhrzqFaPMYJabbqeYQ49w/2N/0hNDRxXGM3Sn0u+zpq/ 1a6y4yMM4RXQOf2HxbpgCXWSrhRcauCtHIEVhBHTLRUgAiVA1rnmaZOop g==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411382" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411382" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263002" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263002" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:33 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 3/8] RDMA/nldev: Add dellink function pointer Date: Fri, 23 Jun 2023 17:57:44 +0800 Message-Id: <20230623095749.485873-4-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun The newlink function pointer is added. And the sock listening on port 4791 is added in the newlink function. So the dellink function is needed to remove the sock. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/core/nldev.c | 6 ++++++ include/rdma/rdma_netlink.h | 2 ++ 2 files changed, 8 insertions(+) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index d5d3e4f0de77..97a62685ed5b 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1758,6 +1758,12 @@ static int nldev_dellink(struct sk_buff *skb, struct nlmsghdr *nlh, return -EINVAL; } + if (device->link_ops) { + err = device->link_ops->dellink(device); + if (err) + return err; + } + ib_unregister_device_and_put(device); return 0; } diff --git a/include/rdma/rdma_netlink.h b/include/rdma/rdma_netlink.h index c2a79aeee113..bf9df004061f 100644 --- a/include/rdma/rdma_netlink.h +++ b/include/rdma/rdma_netlink.h @@ -5,6 +5,7 @@ #include #include +#include enum { RDMA_NLDEV_ATTR_EMPTY_STRING = 1, @@ -114,6 +115,7 @@ struct rdma_link_ops { struct list_head list; const char *type; int (*newlink)(const char *ibdev_name, struct net_device *ndev); + int (*dellink)(struct ib_device *dev); }; void rdma_link_register(struct rdma_link_ops *ops); From patchwork Fri Jun 23 09:57:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64E29C001B0 for ; Fri, 23 Jun 2023 09:58:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230149AbjFWJ6p (ORCPT ); Fri, 23 Jun 2023 05:58:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231790AbjFWJ6k (ORCPT ); Fri, 23 Jun 2023 05:58:40 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AE5DE75 for ; Fri, 23 Jun 2023 02:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514319; x=1719050319; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vU0GioiXJOlgyFJaHHjgyPJBN1lMUb+xTOHwuCATeh0=; b=d6gnIHeI1k4Y1enIecGXZfvBENk+cAALGCU/uh77H2Npg1B+62X1ZWUq LUzeMZv/UaPAcBXHDIldAfXrHUTHMj08FqVpltj4E1SA/UIIqb+G/xUeU xI7IcMhwRRxlKbfvMzs3r5FslePko3S3o7TP0bT0ylib9v1ntTIvbz9b2 OcZH4TMs8rdsJz8odvpz194zv9TIY0qMmGMNn3mnRiHhxfsC3ds9eu5cz wF5KL9OwH29B2HMyX+i6Bs+ypUOQ9aggkrbu6lNOkTzSMmYU94v00nB8o DiO1nRldWUByHuCGX6al87O7WKrf4xlOE4TcrKqDl0D2hyuDf/LQrmjxQ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411399" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411399" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263010" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263010" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:36 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 4/8] RDMA/rxe: Implement dellink in rxe Date: Fri, 23 Jun 2023 17:57:45 +0800 Message-Id: <20230623095749.485873-5-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun When running "rdma link del" command, dellink function will be called. If the sock refcnt is greater than the refcnt needed for udp tunnel, the sock refcnt will be decreased by 1. If equal, the last rdma link is removed. The udp tunnel will be destroyed. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 12 +++++++++++- drivers/infiniband/sw/rxe/rxe_net.c | 17 +++++++++++++++-- drivers/infiniband/sw/rxe/rxe_net.h | 1 + 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index c15d3c5d7a6f..ac7e7b0a9dc9 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -168,10 +168,12 @@ void rxe_set_mtu(struct rxe_dev *rxe, unsigned int ndev_mtu) /* called by ifc layer to create new rxe device. * The caller should allocate memory for rxe by calling ib_alloc_device. */ +static struct rdma_link_ops rxe_link_ops; int rxe_add(struct rxe_dev *rxe, unsigned int mtu, const char *ibdev_name) { rxe_init(rxe); rxe_set_mtu(rxe, mtu); + rxe->ib_dev.link_ops = &rxe_link_ops; return rxe_register_device(rxe, ibdev_name); } @@ -208,9 +210,17 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev) return err; } -struct rdma_link_ops rxe_link_ops = { +static int rxe_dellink(struct ib_device *dev) +{ + rxe_net_del(dev); + + return 0; +} + +static struct rdma_link_ops rxe_link_ops = { .type = "rxe", .newlink = rxe_newlink, + .dellink = rxe_dellink, }; static int __init rxe_module_init(void) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 141020bfcf3b..3c58764339a0 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -534,6 +534,21 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev) return 0; } +#define SK_REF_FOR_TUNNEL 2 +void rxe_net_del(struct ib_device *dev) +{ + if (refcount_read(&recv_sockets.sk6->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(recv_sockets.sk6->sk); + else + rxe_release_udp_tunnel(recv_sockets.sk6); + + if (refcount_read(&recv_sockets.sk4->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(recv_sockets.sk4->sk); + else + rxe_release_udp_tunnel(recv_sockets.sk4); +} +#undef SK_REF_FOR_TUNNEL + static void rxe_port_event(struct rxe_dev *rxe, enum ib_event_type event) { @@ -693,8 +708,6 @@ int rxe_register_notifier(void) void rxe_net_exit(void) { - rxe_release_udp_tunnel(recv_sockets.sk6); - rxe_release_udp_tunnel(recv_sockets.sk4); unregister_netdevice_notifier(&rxe_net_notifier); } diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index a222c3eeae12..f48f22f3353b 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -17,6 +17,7 @@ struct rxe_recv_sockets { }; int rxe_net_add(const char *ibdev_name, struct net_device *ndev); +void rxe_net_del(struct ib_device *dev); int rxe_register_notifier(void); int rxe_net_init(void); From patchwork Fri Jun 23 09:57:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2458FEB64DD for ; Fri, 23 Jun 2023 09:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231790AbjFWJ6s (ORCPT ); Fri, 23 Jun 2023 05:58:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231823AbjFWJ6o (ORCPT ); Fri, 23 Jun 2023 05:58:44 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8383C2113 for ; Fri, 23 Jun 2023 02:58:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514322; x=1719050322; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EIBbTu67woXrlhL80cHYmrDeIyAOYI/qf1QyhUe03To=; b=EZbrLi8l3FNBfAeEybh0J4IlyOVFrclmCcywT7qqLky94SPjSdNonB7o ce46Uou5+yMPN+z+r12HAOfMHWISUK1CuEL2IjJi8ZQKV4bE4FLWKpD0Z 1fsiggrMr+rBPiMpS1epTXqEkdM2AKc39jVIMReYYpKTbxLT2mgCXXvXm LhF/DPTTu1E7kjibdEc8wReAUYr0lhxY8HPoyANrvFlDyEv+qcJR4jUE5 6PuJCy7Ohb7JuONVijpdSvUOBWezT7bWJLfENxma1S9A99LQHPICwjuId l6nXG8HuOnBXWltZxQDZwINQnbSFYaTR+LqZVuaFNRry+v+IFPrqbqP5v Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411410" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411410" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263022" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263022" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:39 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 5/8] RDMA/rxe: Replace global variable with sock lookup functions Date: Fri, 23 Jun 2023 17:57:46 +0800 Message-Id: <20230623095749.485873-6-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Originally a global variable is to keep the sock of udp listening on port 4791. In fact, sock lookup functions can be used to get the sock. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 1 + drivers/infiniband/sw/rxe/rxe_net.c | 58 ++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_net.h | 5 --- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 + 4 files changed, 45 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index ac7e7b0a9dc9..c9b3125b26d0 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -74,6 +74,7 @@ static void rxe_init_device_param(struct rxe_dev *rxe) rxe->ndev->dev_addr); rxe->max_ucontext = RXE_MAX_UCONTEXT; + rxe->l_sk6 = NULL; } /* initialize port attributes */ diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 3c58764339a0..85ca842f3bad 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -18,8 +18,6 @@ #include "rxe_net.h" #include "rxe_loc.h" -static struct rxe_recv_sockets recv_sockets; - static struct dst_entry *rxe_find_route4(struct rxe_qp *qp, struct net_device *ndev, struct in_addr *saddr, @@ -51,6 +49,23 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, { struct dst_entry *ndst; struct flowi6 fl6 = { { 0 } }; + struct rxe_dev *rxe; + + rxe = rxe_get_dev_from_net(ndev); + if (!rxe->l_sk6) { + struct sock *sk; + + rcu_read_lock(); + sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) { + rxe_dbg_qp(qp, "file: %s +%d, error\n", __FILE__, __LINE__); + return (struct dst_entry *)sk; + } + __sock_put(sk); + rxe->l_sk6 = sk->sk_socket; + } + memset(&fl6, 0, sizeof(fl6)); fl6.flowi6_oif = ndev->ifindex; @@ -58,8 +73,8 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, memcpy(&fl6.daddr, daddr, sizeof(*daddr)); fl6.flowi6_proto = IPPROTO_UDP; - ndst = ipv6_stub->ipv6_dst_lookup_flow(sock_net(recv_sockets.sk6->sk), - recv_sockets.sk6->sk, &fl6, + ndst = ipv6_stub->ipv6_dst_lookup_flow(dev_net(ndev), + rxe->l_sk6->sk, &fl6, NULL); if (IS_ERR(ndst)) { rxe_dbg_qp(qp, "no route to %pI6\n", daddr); @@ -537,15 +552,33 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev) #define SK_REF_FOR_TUNNEL 2 void rxe_net_del(struct ib_device *dev) { - if (refcount_read(&recv_sockets.sk6->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) - __sock_put(recv_sockets.sk6->sk); + struct sock *sk; + + rcu_read_lock(); + sk = udp4_lib_lookup(&init_net, 0, 0, htonl(INADDR_ANY), htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) + return; + + __sock_put(sk); + + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(sk); else - rxe_release_udp_tunnel(recv_sockets.sk6); + rxe_release_udp_tunnel(sk->sk_socket); + + rcu_read_lock(); + sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) + return; + + __sock_put(sk); - if (refcount_read(&recv_sockets.sk4->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) - __sock_put(recv_sockets.sk4->sk); + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(sk); else - rxe_release_udp_tunnel(recv_sockets.sk4); + rxe_release_udp_tunnel(sk->sk_socket); } #undef SK_REF_FOR_TUNNEL @@ -655,10 +688,8 @@ static int rxe_net_ipv4_init(void) sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), false); if (IS_ERR(sock)) { pr_err("Failed to create IPv4 UDP tunnel\n"); - recv_sockets.sk4 = NULL; return -1; } - recv_sockets.sk4 = sock; return 0; } @@ -678,17 +709,14 @@ static int rxe_net_ipv6_init(void) sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), true); if (PTR_ERR(sock) == -EAFNOSUPPORT) { - recv_sockets.sk6 = NULL; pr_warn("IPv6 is not supported, can not create a UDPv6 socket\n"); return 0; } if (IS_ERR(sock)) { - recv_sockets.sk6 = NULL; pr_err("Failed to create IPv6 UDP tunnel\n"); return -1; } - recv_sockets.sk6 = sock; #endif return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index f48f22f3353b..027b20e1bab6 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -11,11 +11,6 @@ #include #include -struct rxe_recv_sockets { - struct socket *sk4; - struct socket *sk6; -}; - int rxe_net_add(const char *ibdev_name, struct net_device *ndev); void rxe_net_del(struct ib_device *dev); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 26a20f088692..0aa3817770a5 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -382,6 +382,7 @@ struct rxe_dev { struct rxe_port port; struct crypto_shash *tfm; + struct socket *l_sk6; }; static inline void rxe_counter_inc(struct rxe_dev *rxe, enum rxe_counters index) From patchwork Fri Jun 23 09:57:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E161EB64D7 for ; Fri, 23 Jun 2023 09:58:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231294AbjFWJ6v (ORCPT ); Fri, 23 Jun 2023 05:58:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231784AbjFWJ6r (ORCPT ); Fri, 23 Jun 2023 05:58:47 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 664AA1A3 for ; Fri, 23 Jun 2023 02:58:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514325; x=1719050325; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mNvcMF5hZgoubXeH1rnMEegqssZgCZ4e+zYFWMPaV3U=; b=fJWlgnAIhM5JqtPu/D++S4PcxKaA/tFMvDz1VzzOn2iPDDX5X/BZ6OAi i9p9oCwP6MgGwoqXrQw83ByhhCSkX6JHGXC/QBKx1o/Dau3+XP+fSxSfL wPGmvL7axcwFzyk+Ubpw0CSaM0OQBkcEVzrNIEfS/mHbO96LUsuhEW4a+ q/WQ0uwUACZDtYKFTbSkg8eO4YiCddgeI7tLzjSbvh3oOlSlWByLPM2o+ HZR2aql2FAFVr2FjOvJeZ/vHDi0MWhgYukc51jeWIKRxCU0CE+V6bGpOU gRgw0vJt6f6triqofb4K3Hn6D/iilNCMJlNU/VfJdXDKlS2+VtusQfUBd w==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411427" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411427" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263030" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263030" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:42 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 6/8] RDMA/rxe: add the support of net namespace Date: Fri, 23 Jun 2023 17:57:47 +0800 Message-Id: <20230623095749.485873-7-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Originally init_net is used to indicate the current net namespace. Currently more net namespaces are supported. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 2 +- drivers/infiniband/sw/rxe/rxe_net.c | 33 +++++++++++++++++------------ drivers/infiniband/sw/rxe/rxe_net.h | 2 +- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index c9b3125b26d0..ef632be05e38 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -198,7 +198,7 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev) goto err; } - err = rxe_net_init(); + err = rxe_net_init(ndev); if (err) return err; diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 85ca842f3bad..607382a41e82 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -32,7 +32,7 @@ static struct dst_entry *rxe_find_route4(struct rxe_qp *qp, memcpy(&fl.daddr, daddr, sizeof(*daddr)); fl.flowi4_proto = IPPROTO_UDP; - rt = ip_route_output_key(&init_net, &fl); + rt = ip_route_output_key(dev_net(ndev), &fl); if (IS_ERR(rt)) { rxe_dbg_qp(qp, "no route to %pI4\n", &daddr->s_addr); return NULL; @@ -56,7 +56,8 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, struct sock *sk; rcu_read_lock(); - sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + sk = udp6_lib_lookup(dev_net(ndev), NULL, 0, &in6addr_any, + htons(ROCE_V2_UDP_DPORT), 0); rcu_read_unlock(); if (!sk) { rxe_dbg_qp(qp, "file: %s +%d, error\n", __FILE__, __LINE__); @@ -553,9 +554,13 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev) void rxe_net_del(struct ib_device *dev) { struct sock *sk; + struct rxe_dev *rxe; + + rxe = container_of(dev, struct rxe_dev, ib_dev); rcu_read_lock(); - sk = udp4_lib_lookup(&init_net, 0, 0, htonl(INADDR_ANY), htons(ROCE_V2_UDP_DPORT), 0); + sk = udp4_lib_lookup(dev_net(rxe->ndev), 0, 0, htonl(INADDR_ANY), + htons(ROCE_V2_UDP_DPORT), 0); rcu_read_unlock(); if (!sk) return; @@ -568,7 +573,8 @@ void rxe_net_del(struct ib_device *dev) rxe_release_udp_tunnel(sk->sk_socket); rcu_read_lock(); - sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + sk = udp6_lib_lookup(dev_net(rxe->ndev), NULL, 0, &in6addr_any, + htons(ROCE_V2_UDP_DPORT), 0); rcu_read_unlock(); if (!sk) return; @@ -640,6 +646,7 @@ static int rxe_notify(struct notifier_block *not_blk, switch (event) { case NETDEV_UNREGISTER: ib_unregister_device_queued(&rxe->ib_dev); + rxe_net_del(&rxe->ib_dev); break; case NETDEV_UP: rxe_port_up(rxe); @@ -673,19 +680,19 @@ static struct notifier_block rxe_net_notifier = { .notifier_call = rxe_notify, }; -static int rxe_net_ipv4_init(void) +static int rxe_net_ipv4_init(struct net_device *ndev) { struct sock *sk; struct socket *sock; rcu_read_lock(); - sk = udp4_lib_lookup(&init_net, 0, 0, htonl(INADDR_ANY), + sk = udp4_lib_lookup(dev_net(ndev), 0, 0, htonl(INADDR_ANY), htons(ROCE_V2_UDP_DPORT), 0); rcu_read_unlock(); if (sk) return 0; - sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), false); + sock = rxe_setup_udp_tunnel(dev_net(ndev), htons(ROCE_V2_UDP_DPORT), false); if (IS_ERR(sock)) { pr_err("Failed to create IPv4 UDP tunnel\n"); return -1; @@ -694,20 +701,20 @@ static int rxe_net_ipv4_init(void) return 0; } -static int rxe_net_ipv6_init(void) +static int rxe_net_ipv6_init(struct net_device *ndev) { #if IS_ENABLED(CONFIG_IPV6) struct sock *sk; struct socket *sock; rcu_read_lock(); - sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, + sk = udp6_lib_lookup(dev_net(ndev), NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); rcu_read_unlock(); if (sk) return 0; - sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), true); + sock = rxe_setup_udp_tunnel(dev_net(ndev), htons(ROCE_V2_UDP_DPORT), true); if (PTR_ERR(sock) == -EAFNOSUPPORT) { pr_warn("IPv6 is not supported, can not create a UDPv6 socket\n"); return 0; @@ -739,14 +746,14 @@ void rxe_net_exit(void) unregister_netdevice_notifier(&rxe_net_notifier); } -int rxe_net_init(void) +int rxe_net_init(struct net_device *ndev) { int err; - err = rxe_net_ipv4_init(); + err = rxe_net_ipv4_init(ndev); if (err) return err; - err = rxe_net_ipv6_init(); + err = rxe_net_ipv6_init(ndev); if (err) goto err_out; return 0; diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index 027b20e1bab6..56249677d692 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -15,7 +15,7 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev); void rxe_net_del(struct ib_device *dev); int rxe_register_notifier(void); -int rxe_net_init(void); +int rxe_net_init(struct net_device *ndev); void rxe_net_exit(void); #endif /* RXE_NET_H */ From patchwork Fri Jun 23 09:57:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1F9CEB64DD for ; Fri, 23 Jun 2023 09:59:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231795AbjFWJ7C (ORCPT ); Fri, 23 Jun 2023 05:59:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231451AbjFWJ7A (ORCPT ); Fri, 23 Jun 2023 05:59:00 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30D0D2113 for ; Fri, 23 Jun 2023 02:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514337; x=1719050337; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0Q/N86RVgXiby66kCCPZhxmNxHFMLEwe897xlWtwZnE=; b=NA1K6NZMjXv3muKAL0CZN71JpoAp1Y0Y9oFugRTAeU4QQLWgFt6EOSct bUUoLcnTgmBEn2D+pKTfa/lTULOFtXJJN9vuwOnwO5NIG+dVQp1J9jM68 Tm/39Ga41ctv1/FtPLU/tYkq2GC7XKeUpiKwc2t4aGuCr+lG1K+kxqLyd 87a+aJncs1i8KnWqZiW4oXQqRjopcUYs0Z+BvMcKJR9/xzqdo57O/VqoT KbNu9B7sIj7g50AOVFYi0cMZAbuH9BfMhZ2R/aVko9Qd/butlTyUr+UXu w06a5YlcFanDaRr62RWAE5LqryNTiga/ZlEqsRXG7SmsCjVNx9PC0r9RR Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411437" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411437" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263045" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263045" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:45 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 7/8] RDMA/rxe: Add the support of net namespace notifier Date: Fri, 23 Jun 2023 17:57:48 +0800 Message-Id: <20230623095749.485873-8-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun The functions register_pernet_subsys/unregister_pernet_subsys register a notifier of net namespace. When a new net namespace is created, the init function of rxe will be called to initialize sk4 and sk6 socks. When a net namespace is destroyed, the exit function will be called to handle sk4 and sk6 socks. The functions rxe_ns_pernet_sk4 and rxe_ns_pernet_sk6 are used to get sk4 and sk6 socks. The functions rxe_ns_pernet_set_sk4 and rxe_ns_pernet_set_sk6 are used to set sk4 and sk6 socks. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/Makefile | 3 +- drivers/infiniband/sw/rxe/rxe.c | 9 ++ drivers/infiniband/sw/rxe/rxe_net.c | 56 ++++++------ drivers/infiniband/sw/rxe/rxe_ns.c | 134 ++++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_ns.h | 17 ++++ 5 files changed, 193 insertions(+), 26 deletions(-) create mode 100644 drivers/infiniband/sw/rxe/rxe_ns.c create mode 100644 drivers/infiniband/sw/rxe/rxe_ns.h diff --git a/drivers/infiniband/sw/rxe/Makefile b/drivers/infiniband/sw/rxe/Makefile index 5395a581f4bb..8380f97674cb 100644 --- a/drivers/infiniband/sw/rxe/Makefile +++ b/drivers/infiniband/sw/rxe/Makefile @@ -22,4 +22,5 @@ rdma_rxe-y := \ rxe_mcast.o \ rxe_task.o \ rxe_net.o \ - rxe_hw_counters.o + rxe_hw_counters.o \ + rxe_ns.o diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index ef632be05e38..96841c56ff3a 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -9,6 +9,7 @@ #include "rxe.h" #include "rxe_loc.h" #include "rxe_net.h" +#include "rxe_ns.h" MODULE_AUTHOR("Bob Pearson, Frank Zago, John Groves, Kamal Heib"); MODULE_DESCRIPTION("Soft RDMA transport"); @@ -236,6 +237,12 @@ static int __init rxe_module_init(void) return -1; } + err = rxe_namespace_init(); + if (err) { + pr_err("Failed to register net namespace notifier\n"); + return -1; + } + pr_info("loaded\n"); return 0; } @@ -246,6 +253,8 @@ static void __exit rxe_module_exit(void) ib_unregister_driver(RDMA_DRIVER_RXE); rxe_net_exit(); + rxe_namespace_exit(); + pr_info("unloaded\n"); } diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 607382a41e82..57b8c5593e3c 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -17,6 +17,7 @@ #include "rxe.h" #include "rxe_net.h" #include "rxe_loc.h" +#include "rxe_ns.h" static struct dst_entry *rxe_find_route4(struct rxe_qp *qp, struct net_device *ndev, @@ -558,33 +559,30 @@ void rxe_net_del(struct ib_device *dev) rxe = container_of(dev, struct rxe_dev, ib_dev); - rcu_read_lock(); - sk = udp4_lib_lookup(dev_net(rxe->ndev), 0, 0, htonl(INADDR_ANY), - htons(ROCE_V2_UDP_DPORT), 0); - rcu_read_unlock(); + sk = rxe_ns_pernet_sk4(dev_net(rxe->ndev)); if (!sk) return; - __sock_put(sk); - if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) { __sock_put(sk); - else + } else { rxe_release_udp_tunnel(sk->sk_socket); + sk = NULL; + rxe_ns_pernet_set_sk4(dev_net(rxe->ndev), sk); + } - rcu_read_lock(); - sk = udp6_lib_lookup(dev_net(rxe->ndev), NULL, 0, &in6addr_any, - htons(ROCE_V2_UDP_DPORT), 0); - rcu_read_unlock(); + sk = rxe_ns_pernet_sk6(dev_net(rxe->ndev)); if (!sk) return; - __sock_put(sk); - - if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) { __sock_put(sk); - else + } else { rxe_release_udp_tunnel(sk->sk_socket); + sk = NULL; + rxe_ns_pernet_set_sk6(dev_net(rxe->ndev), sk); + } } #undef SK_REF_FOR_TUNNEL @@ -685,18 +683,18 @@ static int rxe_net_ipv4_init(struct net_device *ndev) struct sock *sk; struct socket *sock; - rcu_read_lock(); - sk = udp4_lib_lookup(dev_net(ndev), 0, 0, htonl(INADDR_ANY), - htons(ROCE_V2_UDP_DPORT), 0); - rcu_read_unlock(); - if (sk) + sk = rxe_ns_pernet_sk4(dev_net(ndev)); + if (sk) { + sock_hold(sk); return 0; + } sock = rxe_setup_udp_tunnel(dev_net(ndev), htons(ROCE_V2_UDP_DPORT), false); if (IS_ERR(sock)) { pr_err("Failed to create IPv4 UDP tunnel\n"); return -1; } + rxe_ns_pernet_set_sk4(dev_net(ndev), sock->sk); return 0; } @@ -707,12 +705,11 @@ static int rxe_net_ipv6_init(struct net_device *ndev) struct sock *sk; struct socket *sock; - rcu_read_lock(); - sk = udp6_lib_lookup(dev_net(ndev), NULL, 0, &in6addr_any, - htons(ROCE_V2_UDP_DPORT), 0); - rcu_read_unlock(); - if (sk) + sk = rxe_ns_pernet_sk6(dev_net(ndev)); + if (sk) { + sock_hold(sk); return 0; + } sock = rxe_setup_udp_tunnel(dev_net(ndev), htons(ROCE_V2_UDP_DPORT), true); if (PTR_ERR(sock) == -EAFNOSUPPORT) { @@ -724,6 +721,9 @@ static int rxe_net_ipv6_init(struct net_device *ndev) pr_err("Failed to create IPv6 UDP tunnel\n"); return -1; } + + rxe_ns_pernet_set_sk6(dev_net(ndev), sock->sk); + #endif return 0; } @@ -753,11 +753,17 @@ int rxe_net_init(struct net_device *ndev) err = rxe_net_ipv4_init(ndev); if (err) return err; + err = rxe_net_ipv6_init(ndev); if (err) goto err_out; + return 0; + err_out: + /* If ipv6 error, release ipv4 resource */ + udp_tunnel_sock_release(rxe_ns_pernet_sk4(dev_net(ndev))->sk_socket); + rxe_ns_pernet_set_sk4(dev_net(ndev), NULL); rxe_net_exit(); return err; } diff --git a/drivers/infiniband/sw/rxe/rxe_ns.c b/drivers/infiniband/sw/rxe/rxe_ns.c new file mode 100644 index 000000000000..29d08899dcda --- /dev/null +++ b/drivers/infiniband/sw/rxe/rxe_ns.c @@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * Copyright (c) 2016 Mellanox Technologies Ltd. All rights reserved. + * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "rxe_ns.h" + +/* + * Per network namespace data + */ +struct rxe_ns_sock { + struct sock __rcu *rxe_sk4; + struct sock __rcu *rxe_sk6; +}; + +/* + * Index to store custom data for each network namespace. + */ +static unsigned int rxe_pernet_id; + +/* + * Called for every existing and added network namespaces + */ +static int __net_init rxe_ns_init(struct net *net) +{ + /* + * create (if not present) and access data item in network namespace + * (net) using the id (net_id) + */ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + + rcu_assign_pointer(ns_sk->rxe_sk4, NULL); /* initialize sock 4 socket */ + rcu_assign_pointer(ns_sk->rxe_sk6, NULL); /* initialize sock 6 socket */ + synchronize_rcu(); + + return 0; +} + +static void __net_exit rxe_ns_exit(struct net *net) +{ + /* + * called when the network namespace is removed + */ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + struct sock *rxe_sk4 = NULL; + struct sock *rxe_sk6 = NULL; + + rcu_read_lock(); + rxe_sk4 = rcu_dereference(ns_sk->rxe_sk4); + rxe_sk6 = rcu_dereference(ns_sk->rxe_sk6); + rcu_read_unlock(); + + /* close socket */ + if (rxe_sk4 && rxe_sk4->sk_socket) { + udp_tunnel_sock_release(rxe_sk4->sk_socket); + rcu_assign_pointer(ns_sk->rxe_sk4, NULL); + synchronize_rcu(); + } + + if (rxe_sk6 && rxe_sk6->sk_socket) { + udp_tunnel_sock_release(rxe_sk6->sk_socket); + rcu_assign_pointer(ns_sk->rxe_sk6, NULL); + synchronize_rcu(); + } +} + +/* + * callback to make the module network namespace aware + */ +static struct pernet_operations rxe_net_ops __net_initdata = { + .init = rxe_ns_init, + .exit = rxe_ns_exit, + .id = &rxe_pernet_id, + .size = sizeof(struct rxe_ns_sock), +}; + +struct sock *rxe_ns_pernet_sk4(struct net *net) +{ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + struct sock *sk; + + rcu_read_lock(); + sk = rcu_dereference(ns_sk->rxe_sk4); + rcu_read_unlock(); + + return sk; +} + +void rxe_ns_pernet_set_sk4(struct net *net, struct sock *sk) +{ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + + rcu_assign_pointer(ns_sk->rxe_sk4, sk); + synchronize_rcu(); +} + +struct sock *rxe_ns_pernet_sk6(struct net *net) +{ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + struct sock *sk; + + rcu_read_lock(); + sk = rcu_dereference(ns_sk->rxe_sk6); + rcu_read_unlock(); + + return sk; +} + +void rxe_ns_pernet_set_sk6(struct net *net, struct sock *sk) +{ + struct rxe_ns_sock *ns_sk = net_generic(net, rxe_pernet_id); + + rcu_assign_pointer(ns_sk->rxe_sk6, sk); + synchronize_rcu(); +} + +int __init rxe_namespace_init(void) +{ + return register_pernet_subsys(&rxe_net_ops); +} + +void __exit rxe_namespace_exit(void) +{ + unregister_pernet_subsys(&rxe_net_ops); +} diff --git a/drivers/infiniband/sw/rxe/rxe_ns.h b/drivers/infiniband/sw/rxe/rxe_ns.h new file mode 100644 index 000000000000..da5bfcea1274 --- /dev/null +++ b/drivers/infiniband/sw/rxe/rxe_ns.h @@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* + * Copyright (c) 2016 Mellanox Technologies Ltd. All rights reserved. + * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved. + */ + +#ifndef RXE_NS_H +#define RXE_NS_H + +struct sock *rxe_ns_pernet_sk4(struct net *net); +struct sock *rxe_ns_pernet_sk6(struct net *net); +void rxe_ns_pernet_set_sk4(struct net *net, struct sock *sk); +void rxe_ns_pernet_set_sk6(struct net *net, struct sock *sk); +int __init rxe_namespace_init(void); +void __exit rxe_namespace_exit(void); + +#endif /* RXE_NS_H */ From patchwork Fri Jun 23 09:57:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35AAAC001B0 for ; Fri, 23 Jun 2023 09:59:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231803AbjFWJ7D (ORCPT ); Fri, 23 Jun 2023 05:59:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231754AbjFWJ7A (ORCPT ); Fri, 23 Jun 2023 05:59:00 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4DDBE75 for ; Fri, 23 Jun 2023 02:58:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514337; x=1719050337; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k6rNKO7P0g0cwzqXZF+L2T39+CBNTEATEb9gtwNbeuM=; b=ea8pqfxdbJX+sA3cuFw/UeYCzPNK0w1UzFrv7GUB9lKnCOZ9a5AUJnI1 Kr/Yjbdy8s2WoqZJeJKhinOng2v8a0KeJFFGcELvNU0sRb7Ni+jdaitly mBT0IFptIWGfZNpmUqLc0x0Dimqq3UbjXLf2mkLBu2FdIPbFnYCKIT/iD uMJkBsknyq4fwj2U3DGPoaRoIQl7vtAvjTnKG/4XEiEzcp5C2npoy9oyG 5jksCClDfEIogrjjpFch2pCt4I6JG3v0iYkbD+Bzm0HHIaURAXr2vyroL xIwkwuG4HtjOxVGXtLHzgw0SCekoHihLsMZ69Su1vt+qRLIMUeo3moWOf g==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411446" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411446" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263062" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263062" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:48 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 8/8] RDMA/rxe: Replace l_sk6 with sk6 in net namespace Date: Fri, 23 Jun 2023 17:57:49 +0800 Message-Id: <20230623095749.485873-9-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun The net namespace variable sk6 can be used. As such, l_sk6 can be replaced with it. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 1 - drivers/infiniband/sw/rxe/rxe_net.c | 20 +------------------- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 3 files changed, 1 insertion(+), 21 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 96841c56ff3a..b1dfba2fdf15 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -75,7 +75,6 @@ static void rxe_init_device_param(struct rxe_dev *rxe) rxe->ndev->dev_addr); rxe->max_ucontext = RXE_MAX_UCONTEXT; - rxe->l_sk6 = NULL; } /* initialize port attributes */ diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 57b8c5593e3c..1f4b8ccfdbb5 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -50,24 +50,6 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, { struct dst_entry *ndst; struct flowi6 fl6 = { { 0 } }; - struct rxe_dev *rxe; - - rxe = rxe_get_dev_from_net(ndev); - if (!rxe->l_sk6) { - struct sock *sk; - - rcu_read_lock(); - sk = udp6_lib_lookup(dev_net(ndev), NULL, 0, &in6addr_any, - htons(ROCE_V2_UDP_DPORT), 0); - rcu_read_unlock(); - if (!sk) { - rxe_dbg_qp(qp, "file: %s +%d, error\n", __FILE__, __LINE__); - return (struct dst_entry *)sk; - } - __sock_put(sk); - rxe->l_sk6 = sk->sk_socket; - } - memset(&fl6, 0, sizeof(fl6)); fl6.flowi6_oif = ndev->ifindex; @@ -76,7 +58,7 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, fl6.flowi6_proto = IPPROTO_UDP; ndst = ipv6_stub->ipv6_dst_lookup_flow(dev_net(ndev), - rxe->l_sk6->sk, &fl6, + rxe_ns_pernet_sk6(dev_net(ndev)), &fl6, NULL); if (IS_ERR(ndst)) { rxe_dbg_qp(qp, "no route to %pI6\n", daddr); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 0aa3817770a5..26a20f088692 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -382,7 +382,6 @@ struct rxe_dev { struct rxe_port port; struct crypto_shash *tfm; - struct socket *l_sk6; }; static inline void rxe_counter_inc(struct rxe_dev *rxe, enum rxe_counters index)