From patchwork Fri Jun 23 09:57:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2458FEB64DD for ; Fri, 23 Jun 2023 09:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231790AbjFWJ6s (ORCPT ); Fri, 23 Jun 2023 05:58:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231823AbjFWJ6o (ORCPT ); Fri, 23 Jun 2023 05:58:44 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8383C2113 for ; Fri, 23 Jun 2023 02:58:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514322; x=1719050322; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EIBbTu67woXrlhL80cHYmrDeIyAOYI/qf1QyhUe03To=; b=EZbrLi8l3FNBfAeEybh0J4IlyOVFrclmCcywT7qqLky94SPjSdNonB7o ce46Uou5+yMPN+z+r12HAOfMHWISUK1CuEL2IjJi8ZQKV4bE4FLWKpD0Z 1fsiggrMr+rBPiMpS1epTXqEkdM2AKc39jVIMReYYpKTbxLT2mgCXXvXm LhF/DPTTu1E7kjibdEc8wReAUYr0lhxY8HPoyANrvFlDyEv+qcJR4jUE5 6PuJCy7Ohb7JuONVijpdSvUOBWezT7bWJLfENxma1S9A99LQHPICwjuId l6nXG8HuOnBXWltZxQDZwINQnbSFYaTR+LqZVuaFNRry+v+IFPrqbqP5v Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411410" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411410" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263022" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263022" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:39 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 5/8] RDMA/rxe: Replace global variable with sock lookup functions Date: Fri, 23 Jun 2023 17:57:46 +0800 Message-Id: <20230623095749.485873-6-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun Originally a global variable is to keep the sock of udp listening on port 4791. In fact, sock lookup functions can be used to get the sock. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 1 + drivers/infiniband/sw/rxe/rxe_net.c | 58 ++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_net.h | 5 --- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 + 4 files changed, 45 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index ac7e7b0a9dc9..c9b3125b26d0 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -74,6 +74,7 @@ static void rxe_init_device_param(struct rxe_dev *rxe) rxe->ndev->dev_addr); rxe->max_ucontext = RXE_MAX_UCONTEXT; + rxe->l_sk6 = NULL; } /* initialize port attributes */ diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 3c58764339a0..85ca842f3bad 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -18,8 +18,6 @@ #include "rxe_net.h" #include "rxe_loc.h" -static struct rxe_recv_sockets recv_sockets; - static struct dst_entry *rxe_find_route4(struct rxe_qp *qp, struct net_device *ndev, struct in_addr *saddr, @@ -51,6 +49,23 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, { struct dst_entry *ndst; struct flowi6 fl6 = { { 0 } }; + struct rxe_dev *rxe; + + rxe = rxe_get_dev_from_net(ndev); + if (!rxe->l_sk6) { + struct sock *sk; + + rcu_read_lock(); + sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) { + rxe_dbg_qp(qp, "file: %s +%d, error\n", __FILE__, __LINE__); + return (struct dst_entry *)sk; + } + __sock_put(sk); + rxe->l_sk6 = sk->sk_socket; + } + memset(&fl6, 0, sizeof(fl6)); fl6.flowi6_oif = ndev->ifindex; @@ -58,8 +73,8 @@ static struct dst_entry *rxe_find_route6(struct rxe_qp *qp, memcpy(&fl6.daddr, daddr, sizeof(*daddr)); fl6.flowi6_proto = IPPROTO_UDP; - ndst = ipv6_stub->ipv6_dst_lookup_flow(sock_net(recv_sockets.sk6->sk), - recv_sockets.sk6->sk, &fl6, + ndst = ipv6_stub->ipv6_dst_lookup_flow(dev_net(ndev), + rxe->l_sk6->sk, &fl6, NULL); if (IS_ERR(ndst)) { rxe_dbg_qp(qp, "no route to %pI6\n", daddr); @@ -537,15 +552,33 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev) #define SK_REF_FOR_TUNNEL 2 void rxe_net_del(struct ib_device *dev) { - if (refcount_read(&recv_sockets.sk6->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) - __sock_put(recv_sockets.sk6->sk); + struct sock *sk; + + rcu_read_lock(); + sk = udp4_lib_lookup(&init_net, 0, 0, htonl(INADDR_ANY), htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) + return; + + __sock_put(sk); + + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(sk); else - rxe_release_udp_tunnel(recv_sockets.sk6); + rxe_release_udp_tunnel(sk->sk_socket); + + rcu_read_lock(); + sk = udp6_lib_lookup(&init_net, NULL, 0, &in6addr_any, htons(ROCE_V2_UDP_DPORT), 0); + rcu_read_unlock(); + if (!sk) + return; + + __sock_put(sk); - if (refcount_read(&recv_sockets.sk4->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) - __sock_put(recv_sockets.sk4->sk); + if (refcount_read(&sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(sk); else - rxe_release_udp_tunnel(recv_sockets.sk4); + rxe_release_udp_tunnel(sk->sk_socket); } #undef SK_REF_FOR_TUNNEL @@ -655,10 +688,8 @@ static int rxe_net_ipv4_init(void) sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), false); if (IS_ERR(sock)) { pr_err("Failed to create IPv4 UDP tunnel\n"); - recv_sockets.sk4 = NULL; return -1; } - recv_sockets.sk4 = sock; return 0; } @@ -678,17 +709,14 @@ static int rxe_net_ipv6_init(void) sock = rxe_setup_udp_tunnel(&init_net, htons(ROCE_V2_UDP_DPORT), true); if (PTR_ERR(sock) == -EAFNOSUPPORT) { - recv_sockets.sk6 = NULL; pr_warn("IPv6 is not supported, can not create a UDPv6 socket\n"); return 0; } if (IS_ERR(sock)) { - recv_sockets.sk6 = NULL; pr_err("Failed to create IPv6 UDP tunnel\n"); return -1; } - recv_sockets.sk6 = sock; #endif return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index f48f22f3353b..027b20e1bab6 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -11,11 +11,6 @@ #include #include -struct rxe_recv_sockets { - struct socket *sk4; - struct socket *sk6; -}; - int rxe_net_add(const char *ibdev_name, struct net_device *ndev); void rxe_net_del(struct ib_device *dev); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 26a20f088692..0aa3817770a5 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -382,6 +382,7 @@ struct rxe_dev { struct rxe_port port; struct crypto_shash *tfm; + struct socket *l_sk6; }; static inline void rxe_counter_inc(struct rxe_dev *rxe, enum rxe_counters index)