From patchwork Fri Jun 23 09:57:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhu Yanjun X-Patchwork-Id: 13290224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64E29C001B0 for ; Fri, 23 Jun 2023 09:58:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230149AbjFWJ6p (ORCPT ); Fri, 23 Jun 2023 05:58:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231790AbjFWJ6k (ORCPT ); Fri, 23 Jun 2023 05:58:40 -0400 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AE5DE75 for ; Fri, 23 Jun 2023 02:58:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687514319; x=1719050319; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vU0GioiXJOlgyFJaHHjgyPJBN1lMUb+xTOHwuCATeh0=; b=d6gnIHeI1k4Y1enIecGXZfvBENk+cAALGCU/uh77H2Npg1B+62X1ZWUq LUzeMZv/UaPAcBXHDIldAfXrHUTHMj08FqVpltj4E1SA/UIIqb+G/xUeU xI7IcMhwRRxlKbfvMzs3r5FslePko3S3o7TP0bT0ylib9v1ntTIvbz9b2 OcZH4TMs8rdsJz8odvpz194zv9TIY0qMmGMNn3mnRiHhxfsC3ds9eu5cz wF5KL9OwH29B2HMyX+i6Bs+ypUOQ9aggkrbu6lNOkTzSMmYU94v00nB8o DiO1nRldWUByHuCGX6al87O7WKrf4xlOE4TcrKqDl0D2hyuDf/LQrmjxQ g==; X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="424411399" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="424411399" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2023 02:58:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10749"; a="715263010" X-IronPort-AV: E=Sophos;i="6.01,151,1684825200"; d="scan'208";a="715263010" Received: from unknown (HELO intel-71.bj.intel.com) ([10.238.154.71]) by orsmga002.jf.intel.com with ESMTP; 23 Jun 2023 02:58:36 -0700 From: Zhu Yanjun To: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org, linux-rdma@vger.kernel.org, parav@nvidia.com, lehrer@gmail.com, rpearsonhpe@gmail.com Cc: Zhu Yanjun , Rain River Subject: [PATCH v6 4/8] RDMA/rxe: Implement dellink in rxe Date: Fri, 23 Jun 2023 17:57:45 +0800 Message-Id: <20230623095749.485873-5-yanjun.zhu@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230623095749.485873-1-yanjun.zhu@intel.com> References: <20230623095749.485873-1-yanjun.zhu@intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Zhu Yanjun When running "rdma link del" command, dellink function will be called. If the sock refcnt is greater than the refcnt needed for udp tunnel, the sock refcnt will be decreased by 1. If equal, the last rdma link is removed. The udp tunnel will be destroyed. Tested-by: Rain River Signed-off-by: Zhu Yanjun --- drivers/infiniband/sw/rxe/rxe.c | 12 +++++++++++- drivers/infiniband/sw/rxe/rxe_net.c | 17 +++++++++++++++-- drivers/infiniband/sw/rxe/rxe_net.h | 1 + 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index c15d3c5d7a6f..ac7e7b0a9dc9 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -168,10 +168,12 @@ void rxe_set_mtu(struct rxe_dev *rxe, unsigned int ndev_mtu) /* called by ifc layer to create new rxe device. * The caller should allocate memory for rxe by calling ib_alloc_device. */ +static struct rdma_link_ops rxe_link_ops; int rxe_add(struct rxe_dev *rxe, unsigned int mtu, const char *ibdev_name) { rxe_init(rxe); rxe_set_mtu(rxe, mtu); + rxe->ib_dev.link_ops = &rxe_link_ops; return rxe_register_device(rxe, ibdev_name); } @@ -208,9 +210,17 @@ static int rxe_newlink(const char *ibdev_name, struct net_device *ndev) return err; } -struct rdma_link_ops rxe_link_ops = { +static int rxe_dellink(struct ib_device *dev) +{ + rxe_net_del(dev); + + return 0; +} + +static struct rdma_link_ops rxe_link_ops = { .type = "rxe", .newlink = rxe_newlink, + .dellink = rxe_dellink, }; static int __init rxe_module_init(void) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 141020bfcf3b..3c58764339a0 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -534,6 +534,21 @@ int rxe_net_add(const char *ibdev_name, struct net_device *ndev) return 0; } +#define SK_REF_FOR_TUNNEL 2 +void rxe_net_del(struct ib_device *dev) +{ + if (refcount_read(&recv_sockets.sk6->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(recv_sockets.sk6->sk); + else + rxe_release_udp_tunnel(recv_sockets.sk6); + + if (refcount_read(&recv_sockets.sk4->sk->sk_refcnt) > SK_REF_FOR_TUNNEL) + __sock_put(recv_sockets.sk4->sk); + else + rxe_release_udp_tunnel(recv_sockets.sk4); +} +#undef SK_REF_FOR_TUNNEL + static void rxe_port_event(struct rxe_dev *rxe, enum ib_event_type event) { @@ -693,8 +708,6 @@ int rxe_register_notifier(void) void rxe_net_exit(void) { - rxe_release_udp_tunnel(recv_sockets.sk6); - rxe_release_udp_tunnel(recv_sockets.sk4); unregister_netdevice_notifier(&rxe_net_notifier); } diff --git a/drivers/infiniband/sw/rxe/rxe_net.h b/drivers/infiniband/sw/rxe/rxe_net.h index a222c3eeae12..f48f22f3353b 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.h +++ b/drivers/infiniband/sw/rxe/rxe_net.h @@ -17,6 +17,7 @@ struct rxe_recv_sockets { }; int rxe_net_add(const char *ibdev_name, struct net_device *ndev); +void rxe_net_del(struct ib_device *dev); int rxe_register_notifier(void); int rxe_net_init(void);