From patchwork Tue Feb 8 21:16:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739447 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 335F3C433FE for ; Tue, 8 Feb 2022 22:24:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387539AbiBHWYm (ORCPT ); Tue, 8 Feb 2022 17:24:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37648 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386860AbiBHVRR (ORCPT ); Tue, 8 Feb 2022 16:17:17 -0500 Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8E53C0612BA for ; Tue, 8 Feb 2022 13:17:15 -0800 (PST) Received: by mail-oi1-x230.google.com with SMTP id y23so398393oia.13 for ; Tue, 08 Feb 2022 13:17:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VebRPvsPxA5BA5urDmgWLzjvmvNxF11B7f0mE7t7dCk=; b=mTSEB8/d0Rk5GJshCH3CBxUyfUoRunLto9Awx5Yv2gSA996BJ8Ljo4ct1eNoJ2jSY8 HA2CqsN2VWGygLIfU+G2+kS/zwYeHCYZJCoCH3roX6ct7zXWPKnkaJoRY6ZrqmK6BMT2 042gHa/3rAETsUN4o8eUKmt0mZMVGWfvvvEITnMMpM1PIMnGWNW1BMSdFvA0EmR01oPA ZvgVn3wZWiPwh5RNioO15U/kXm0x3FZLFv59UFcP2hRYoN1KWXz0nv+WN+OY9JqGzP8Q k80dC8/8axDfEhKc73+9tuFLP0+lJcaslj3tUCOOPT1SQMlSmHKR1Fb0FeVLfvek0u5E HkpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VebRPvsPxA5BA5urDmgWLzjvmvNxF11B7f0mE7t7dCk=; b=6WvNUFRKfdut0CzUCk4fhKdwtElRmo/7GOe2kTBi6gFygV96yusugHLNSfkv6/w7tP QiKVm00Witk4Gn5Jysc3k4QcRqcgpejRZXoJfIEAxfqPsFBnJ1oGj88J51BOMaSpyckC pstcJqBO8wGoSKLzZV0N1VxZz0K1pSjwVOktmPoLdvcHU2G+eMrmZISpsH9aAR/FxFh/ kdGaLc6hpYEA6d4TTVXNKTd4zaQpHUW6WNzV1A0TSPkF03fhdf6ECoRdcGw2EtFRdgf4 eIeNUNoVSAGmK0AeiDADGmADDJ7wK352ZMHKgKBCXRC080D2Qgwsb48TmER6wNRbFVYf G38g== X-Gm-Message-State: AOAM5339SQmpvxLJ5mb93K6HIyGDK9r0glUxJibqyN+lxIQlyoU8br1k PPZkfqEo6qP4ISWeRGHSoWk= X-Google-Smtp-Source: ABdhPJxAKCDQWuZpABpSG0Uu0IZxzMg29gQ8NzF7/4Bt8T6D4/H3ndCeU4KkVdudU9HC/HX8isXeIA== X-Received: by 2002:a05:6808:9b3:: with SMTP id e19mr1536102oig.90.1644355034959; Tue, 08 Feb 2022 13:17:14 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:14 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 01/11] RDMA/rxe: Move mcg_lock to rxe Date: Tue, 8 Feb 2022 15:16:35 -0600 Message-Id: <20220208211644.123457-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace mcg->mcg_lock and mc_grp_pool->pool_lock by rxe->mcg_lock. This is the first step of several intended to decouple the mc_grp and mc_elem objects from the rxe pool code. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 2 ++ drivers/infiniband/sw/rxe/rxe_mcast.c | 19 +++++++++---------- drivers/infiniband/sw/rxe/rxe_recv.c | 4 ++-- drivers/infiniband/sw/rxe/rxe_verbs.h | 3 ++- 4 files changed, 15 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index fab291245366..e74c4216b314 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -211,6 +211,8 @@ static int rxe_init(struct rxe_dev *rxe) spin_lock_init(&rxe->pending_lock); INIT_LIST_HEAD(&rxe->pending_mmaps); + spin_lock_init(&rxe->mcg_lock); + mutex_init(&rxe->usdev_lock); return 0; diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 9336295c4ee2..4828274efbd4 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -25,7 +25,7 @@ static int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid) return dev_mc_del(rxe->ndev, ll_addr); } -/* caller should hold mc_grp_pool->pool_lock */ +/* caller should hold rxe->mcg_lock */ static struct rxe_mcg *create_grp(struct rxe_dev *rxe, struct rxe_pool *pool, union ib_gid *mgid) @@ -38,7 +38,6 @@ static struct rxe_mcg *create_grp(struct rxe_dev *rxe, return ERR_PTR(-ENOMEM); INIT_LIST_HEAD(&grp->qp_list); - spin_lock_init(&grp->mcg_lock); grp->rxe = rxe; rxe_add_key_locked(grp, mgid); @@ -62,7 +61,7 @@ static int rxe_mcast_get_grp(struct rxe_dev *rxe, union ib_gid *mgid, if (rxe->attr.max_mcast_qp_attach == 0) return -EINVAL; - write_lock_bh(&pool->pool_lock); + spin_lock_bh(&rxe->mcg_lock); grp = rxe_pool_get_key_locked(pool, mgid); if (grp) @@ -70,13 +69,13 @@ static int rxe_mcast_get_grp(struct rxe_dev *rxe, union ib_gid *mgid, grp = create_grp(rxe, pool, mgid); if (IS_ERR(grp)) { - write_unlock_bh(&pool->pool_lock); + spin_unlock_bh(&rxe->mcg_lock); err = PTR_ERR(grp); return err; } done: - write_unlock_bh(&pool->pool_lock); + spin_unlock_bh(&rxe->mcg_lock); *grp_p = grp; return 0; } @@ -88,7 +87,7 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, struct rxe_mca *elem; /* check to see of the qp is already a member of the group */ - spin_lock_bh(&grp->mcg_lock); + spin_lock_bh(&rxe->mcg_lock); list_for_each_entry(elem, &grp->qp_list, qp_list) { if (elem->qp == qp) { err = 0; @@ -118,7 +117,7 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, err = 0; out: - spin_unlock_bh(&grp->mcg_lock); + spin_unlock_bh(&rxe->mcg_lock); return err; } @@ -132,7 +131,7 @@ static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, if (!grp) goto err1; - spin_lock_bh(&grp->mcg_lock); + spin_lock_bh(&rxe->mcg_lock); list_for_each_entry_safe(elem, tmp, &grp->qp_list, qp_list) { if (elem->qp == qp) { @@ -140,7 +139,7 @@ static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, grp->num_qp--; atomic_dec(&qp->mcg_num); - spin_unlock_bh(&grp->mcg_lock); + spin_unlock_bh(&rxe->mcg_lock); rxe_drop_ref(elem); rxe_drop_ref(grp); /* ref held by QP */ rxe_drop_ref(grp); /* ref from get_key */ @@ -148,7 +147,7 @@ static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, } } - spin_unlock_bh(&grp->mcg_lock); + spin_unlock_bh(&rxe->mcg_lock); rxe_drop_ref(grp); /* ref from get_key */ err1: return -EINVAL; diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index 7ff6b53555f4..a084b5d69937 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -250,7 +250,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) if (!mcg) goto drop; /* mcast group not registered */ - spin_lock_bh(&mcg->mcg_lock); + spin_lock_bh(&rxe->mcg_lock); /* this is unreliable datagram service so we let * failures to deliver a multicast packet to a @@ -298,7 +298,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) } } - spin_unlock_bh(&mcg->mcg_lock); + spin_unlock_bh(&rxe->mcg_lock); rxe_drop_ref(mcg); /* drop ref from rxe_pool_get_key. */ diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 55f8ed2bc621..9940c69cbb63 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -353,7 +353,6 @@ struct rxe_mw { struct rxe_mcg { struct rxe_pool_elem elem; - spinlock_t mcg_lock; /* guard group */ struct rxe_dev *rxe; struct list_head qp_list; union ib_gid mgid; @@ -399,6 +398,8 @@ struct rxe_dev { struct rxe_pool mc_grp_pool; struct rxe_pool mc_elem_pool; + spinlock_t mcg_lock; + spinlock_t pending_lock; /* guard pending_mmaps */ struct list_head pending_mmaps; From patchwork Tue Feb 8 21:16:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739455 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8867EC433EF for ; Tue, 8 Feb 2022 22:25:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387563AbiBHWZE (ORCPT ); Tue, 8 Feb 2022 17:25:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386862AbiBHVRS (ORCPT ); Tue, 8 Feb 2022 16:17:18 -0500 Received: from mail-oo1-xc33.google.com (mail-oo1-xc33.google.com [IPv6:2607:f8b0:4864:20::c33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA078C0612BE for ; Tue, 8 Feb 2022 13:17:16 -0800 (PST) Received: by mail-oo1-xc33.google.com with SMTP id t75-20020a4a3e4e000000b002e9c0821d78so182357oot.4 for ; Tue, 08 Feb 2022 13:17:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=05c42XlWl134ALUFlEKXcrK4+zBYz/latDBJjpL3jn8=; b=IB8WHbIgG3P2/ouWRnaqemPXfmkq30IF37lb9bEwLx4OyqgTvjTB77E/DcYRa+za0w uxkHxgLy6tXX4n494TRp6HHD47MW+VgizWfLKli5DEaM2eKIv+rhi/lBhZh+c/ahLccB R0WQS3x5IlztDEsIxQyIhNzVFFw/hBt8knbM/YA/1cullbL+GmcGc+fwgowKfff3iO3N BTKSOJzQRfOn7SwnZpqVdgI345X2XyqwQEOLyoxTU1IL4bUazGZu3rpJ+rvheQe5harU cjxEqpGeo3WWxHuCgVfi9Z66TCEiLmB2uxd3sVhkVcmCzA7qCNPIEh4fRdliEDL6FQt6 sRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=05c42XlWl134ALUFlEKXcrK4+zBYz/latDBJjpL3jn8=; b=npX3Tpr8OXInOJHGHfR0uP13BAW/foj1g+ddhQ9cCpDnhlBZ2bFrW3tYXFN8d5cByw PMxTtCW6d1B4wbv3DOkvNnT6PzPcnygaMoE9zL7HswCXJ2cMkdbquAqq2b/FN6x8V0Nc nEA088SrrjTLIJH2Y9HYioigglK8jMK//LX6lmhvzwlUQ871HRBAC827YS+AtFNPqora gsI4Ca+gu3ivudCRgsbLgaeql0SPrsEBGRvrYRFwCvrDLcXtQe8CYiGlepyhemOhYG4k LLOvVaNmRM858qrvLpQ+Jy8zZ2WBCq1qS2xZl/fsZ/QeGoP05lsOAUlorltH9NvjC+MM emUQ== X-Gm-Message-State: AOAM532kZ7eu1sd5znuGMKbQgWbDntqlgKkAkUHMCEu6u/IWBV56QwvE 00MLOb84DNZFkzzuKwq6mFw= X-Google-Smtp-Source: ABdhPJxmgzInN8g5kbYQLHQRbmvxWxsc3ldby6MBrtC75vZ/tKAc3CynHVDzADjgIpqR+l/957gEIQ== X-Received: by 2002:a05:6871:4c1:: with SMTP id n1mr951106oai.255.1644355035893; Tue, 08 Feb 2022 13:17:15 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:15 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 02/11] RDMA/rxe: Use kzmalloc/kfree for mca Date: Tue, 8 Feb 2022 15:16:36 -0600 Message-Id: <20220208211644.123457-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Remove rxe_mca (was rxe_mc_elem) from rxe pools and use kzmalloc and kfree to allocate and free in rxe_mcast.c. Call kzalloc outside of spinlocks to avoid having to use GFP_ATOMIC. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 8 -- drivers/infiniband/sw/rxe/rxe_mcast.c | 191 ++++++++++++++++---------- drivers/infiniband/sw/rxe/rxe_pool.c | 5 - drivers/infiniband/sw/rxe/rxe_pool.h | 3 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 2 - 5 files changed, 120 insertions(+), 89 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index e74c4216b314..7386a51b953d 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -29,7 +29,6 @@ void rxe_dealloc(struct ib_device *ib_dev) rxe_pool_cleanup(&rxe->mr_pool); rxe_pool_cleanup(&rxe->mw_pool); rxe_pool_cleanup(&rxe->mc_grp_pool); - rxe_pool_cleanup(&rxe->mc_elem_pool); if (rxe->tfm) crypto_free_shash(rxe->tfm); @@ -163,15 +162,8 @@ static int rxe_init_pools(struct rxe_dev *rxe) if (err) goto err9; - err = rxe_pool_init(rxe, &rxe->mc_elem_pool, RXE_TYPE_MC_ELEM, - rxe->attr.max_total_mcast_qp_attach); - if (err) - goto err10; - return 0; -err10: - rxe_pool_cleanup(&rxe->mc_grp_pool); err9: rxe_pool_cleanup(&rxe->mw_pool); err8: diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 4828274efbd4..3c06b0590c82 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -26,94 +26,102 @@ static int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid) } /* caller should hold rxe->mcg_lock */ -static struct rxe_mcg *create_grp(struct rxe_dev *rxe, - struct rxe_pool *pool, - union ib_gid *mgid) +static struct rxe_mcg *__rxe_create_grp(struct rxe_dev *rxe, + struct rxe_pool *pool, + union ib_gid *mgid) { - int err; struct rxe_mcg *grp; + int err; - grp = rxe_alloc_locked(&rxe->mc_grp_pool); + grp = rxe_alloc_locked(pool); if (!grp) return ERR_PTR(-ENOMEM); - INIT_LIST_HEAD(&grp->qp_list); - grp->rxe = rxe; - rxe_add_key_locked(grp, mgid); - err = rxe_mcast_add(rxe, mgid); if (unlikely(err)) { - rxe_drop_key_locked(grp); rxe_drop_ref(grp); return ERR_PTR(err); } + INIT_LIST_HEAD(&grp->qp_list); + grp->rxe = rxe; + + /* rxe_alloc_locked takes a ref on grp but that will be + * dropped when grp goes out of scope. We need to take a ref + * on the pointer that will be saved in the red-black tree + * by rxe_add_key and used to lookup grp from mgid later. + * Adding key makes object visible to outside so this should + * be done last after the object is ready. + */ + rxe_add_ref(grp); + rxe_add_key_locked(grp, mgid); + return grp; } -static int rxe_mcast_get_grp(struct rxe_dev *rxe, union ib_gid *mgid, - struct rxe_mcg **grp_p) +static struct rxe_mcg *rxe_mcast_get_grp(struct rxe_dev *rxe, + union ib_gid *mgid) { - int err; struct rxe_mcg *grp; struct rxe_pool *pool = &rxe->mc_grp_pool; if (rxe->attr.max_mcast_qp_attach == 0) - return -EINVAL; + return ERR_PTR(-EINVAL); spin_lock_bh(&rxe->mcg_lock); - grp = rxe_pool_get_key_locked(pool, mgid); - if (grp) - goto done; - - grp = create_grp(rxe, pool, mgid); - if (IS_ERR(grp)) { - spin_unlock_bh(&rxe->mcg_lock); - err = PTR_ERR(grp); - return err; - } - -done: + if (!grp) + grp = __rxe_create_grp(rxe, pool, mgid); spin_unlock_bh(&rxe->mcg_lock); - *grp_p = grp; - return 0; + + return grp; } static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, - struct rxe_mcg *grp) + struct rxe_mcg *grp) { + struct rxe_mca *mca, *tmp; int err; - struct rxe_mca *elem; - /* check to see of the qp is already a member of the group */ + /* check to see if the qp is already a member of the group */ + spin_lock_bh(&rxe->mcg_lock); + list_for_each_entry(mca, &grp->qp_list, qp_list) { + if (mca->qp == qp) { + spin_unlock_bh(&rxe->mcg_lock); + return 0; + } + } + spin_unlock_bh(&rxe->mcg_lock); + + /* speculative alloc new mca without using GFP_ATOMIC */ + mca = kzalloc(sizeof(*mca), GFP_KERNEL); + if (!mca) + return -ENOMEM; + spin_lock_bh(&rxe->mcg_lock); - list_for_each_entry(elem, &grp->qp_list, qp_list) { - if (elem->qp == qp) { + /* re-check to see if someone else just attached qp */ + list_for_each_entry(tmp, &grp->qp_list, qp_list) { + if (tmp->qp == qp) { + kfree(mca); err = 0; goto out; } } + /* check limits after checking if already attached */ if (grp->num_qp >= rxe->attr.max_mcast_qp_attach) { + kfree(mca); err = -ENOMEM; goto out; } - elem = rxe_alloc_locked(&rxe->mc_elem_pool); - if (!elem) { - err = -ENOMEM; - goto out; - } - - /* each qp holds a ref on the grp */ - rxe_add_ref(grp); + /* protect pointer to qp in mca */ + rxe_add_ref(qp); + mca->qp = qp; - grp->num_qp++; - elem->qp = qp; atomic_inc(&qp->mcg_num); - - list_add(&elem->qp_list, &grp->qp_list); + grp->num_qp++; + list_add(&mca->qp_list, &grp->qp_list); err = 0; out: @@ -121,45 +129,78 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, return err; } +/* caller should be holding rxe->mcg_lock */ +static void __rxe_destroy_grp(struct rxe_mcg *grp) +{ + /* first remove grp from red-black tree then drop ref */ + rxe_drop_key_locked(grp); + rxe_drop_ref(grp); + + rxe_mcast_delete(grp->rxe, &grp->mgid); +} + +static void rxe_destroy_grp(struct rxe_mcg *grp) +{ + struct rxe_dev *rxe = grp->rxe; + + spin_lock_bh(&rxe->mcg_lock); + __rxe_destroy_grp(grp); + spin_unlock_bh(&rxe->mcg_lock); +} + +void rxe_mc_cleanup(struct rxe_pool_elem *elem) +{ + /* nothing left to do for now */ +} + static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, union ib_gid *mgid) { struct rxe_mcg *grp; - struct rxe_mca *elem, *tmp; - - grp = rxe_pool_get_key(&rxe->mc_grp_pool, mgid); - if (!grp) - goto err1; + struct rxe_mca *mca, *tmp; + int err; spin_lock_bh(&rxe->mcg_lock); + grp = rxe_pool_get_key_locked(&rxe->mc_grp_pool, mgid); + if (!grp) { + /* we didn't find the mcast group for mgid */ + err = -EINVAL; + goto out_unlock; + } + + list_for_each_entry_safe(mca, tmp, &grp->qp_list, qp_list) { + if (mca->qp == qp) { + list_del(&mca->qp_list); - list_for_each_entry_safe(elem, tmp, &grp->qp_list, qp_list) { - if (elem->qp == qp) { - list_del(&elem->qp_list); + /* if the number of qp's attached to the + * mcast group falls to zero go ahead and + * tear it down. This will not free the + * object since we are still holding a ref + * from the get key above. + */ grp->num_qp--; + if (grp->num_qp <= 0) + __rxe_destroy_grp(grp); + atomic_dec(&qp->mcg_num); - spin_unlock_bh(&rxe->mcg_lock); - rxe_drop_ref(elem); - rxe_drop_ref(grp); /* ref held by QP */ - rxe_drop_ref(grp); /* ref from get_key */ - return 0; + /* drop the ref from get key. This will free the + * object if num_qp is zero. + */ + rxe_drop_ref(grp); + kfree(mca); + err = 0; + goto out_unlock; } } - spin_unlock_bh(&rxe->mcg_lock); - rxe_drop_ref(grp); /* ref from get_key */ -err1: - return -EINVAL; -} - -void rxe_mc_cleanup(struct rxe_pool_elem *elem) -{ - struct rxe_mcg *grp = container_of(elem, typeof(*grp), elem); - struct rxe_dev *rxe = grp->rxe; + /* we didn't find the qp on the list */ + rxe_drop_ref(grp); + err = -EINVAL; - rxe_drop_key(grp); - rxe_mcast_delete(rxe, &grp->mgid); +out_unlock: + spin_unlock_bh(&rxe->mcg_lock); + return err; } int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) @@ -170,12 +211,16 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) struct rxe_mcg *grp; /* takes a ref on grp if successful */ - err = rxe_mcast_get_grp(rxe, mgid, &grp); - if (err) - return err; + grp = rxe_mcast_get_grp(rxe, mgid); + if (IS_ERR(grp)) + return PTR_ERR(grp); err = rxe_mcast_add_grp_elem(rxe, qp, grp); + /* if we failed to attach the first qp to grp tear it down */ + if (grp->num_qp == 0) + rxe_destroy_grp(grp); + rxe_drop_ref(grp); return err; } diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 63c594173565..a6756aa93e2b 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -90,11 +90,6 @@ static const struct rxe_type_info { .key_offset = offsetof(struct rxe_mcg, mgid), .key_size = sizeof(union ib_gid), }, - [RXE_TYPE_MC_ELEM] = { - .name = "rxe-mc_elem", - .size = sizeof(struct rxe_mca), - .elem_offset = offsetof(struct rxe_mca, elem), - }, }; static int rxe_pool_init_index(struct rxe_pool *pool, u32 max, u32 min) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 214279310f4d..9201bb6b8b07 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -23,7 +23,6 @@ enum rxe_elem_type { RXE_TYPE_MR, RXE_TYPE_MW, RXE_TYPE_MC_GRP, - RXE_TYPE_MC_ELEM, RXE_NUM_TYPES, /* keep me last */ }; @@ -156,4 +155,6 @@ void rxe_elem_release(struct kref *kref); /* drop a reference on an object */ #define rxe_drop_ref(obj) kref_put(&(obj)->elem.ref_cnt, rxe_elem_release) +#define rxe_read_ref(obj) kref_read(&(obj)->elem.ref_cnt) + #endif /* RXE_POOL_H */ diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 9940c69cbb63..1b0f40881895 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -362,7 +362,6 @@ struct rxe_mcg { }; struct rxe_mca { - struct rxe_pool_elem elem; struct list_head qp_list; struct rxe_qp *qp; }; @@ -396,7 +395,6 @@ struct rxe_dev { struct rxe_pool mr_pool; struct rxe_pool mw_pool; struct rxe_pool mc_grp_pool; - struct rxe_pool mc_elem_pool; spinlock_t mcg_lock; From patchwork Tue Feb 8 21:16:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739443 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AC35C4332F for ; Tue, 8 Feb 2022 22:24:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386620AbiBHWYa (ORCPT ); Tue, 8 Feb 2022 17:24:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386866AbiBHVRT (ORCPT ); Tue, 8 Feb 2022 16:17:19 -0500 Received: from mail-oo1-xc36.google.com (mail-oo1-xc36.google.com [IPv6:2607:f8b0:4864:20::c36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B812CC0612B8 for ; Tue, 8 Feb 2022 13:17:17 -0800 (PST) Received: by mail-oo1-xc36.google.com with SMTP id o192-20020a4a2cc9000000b00300af40d795so113180ooo.13 for ; Tue, 08 Feb 2022 13:17:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fUNciH/hkWjnuuT5P1MnljhT7pAEeg4AIrHOcJWnmPs=; b=io9ytJmCkvpMyiUMweIHCy52gnJncw98AoZhhKYH1Zykn74XQBwhY59T8LrK9+iT4M itR6U0CB3btSrHqliu7bow2d8oE+YiQGEcT0VkKHES4Q7/jDhoO1laRlQ1JTr+J73NqG Mwz1oqicjPXXhy93zSfqhOvYZmXUl5oQrtze1/bZQneWA1uGqol5hvacdUsNLCb8qNWB M/ZAZrsiFjqnT8vJ/5nggZ7ZQVQLYPcrca10unQLWkYcAjshoTrk2Fb2CrajxBPM5sot xB4vXBuJksDMURDhlOHRRYxSLzU8FETxI7MCfKLEF2JS/uV5udeGMoTNvO+Dp9UfhquT d63w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fUNciH/hkWjnuuT5P1MnljhT7pAEeg4AIrHOcJWnmPs=; b=5tgwSqj2uPVRhIIkca7PT+8Qu2KJ3JgtuKVHSz0Ek2jXiYeDuISl7HqNQovzdKiZLu jUoYSHfV1gI7KKt0pbr/vdXBVhCGyOk3iIX3M7TWJpcKp267r1EgLY9GjIn2A2hY9c8X UeI68Q2iCKwvEAWyBoQq3JCzc/54ySCdkxSowjkppc1NQIT2/hE7LWOveVpp19CpUodJ sDS2Bh/rs1ByclJ6ZgoSXijvbfNVI3V+Rj4nOAJH5rRtBdHZf0I/j2vR5nEJPbCveVMC 5VSOUzw0IMQNukvG8b8l7FAqfDcaCLnM7LCrjS3ve+PS00qmoQOE4QG4SaQ21XFd4fWv HLsQ== X-Gm-Message-State: AOAM530wC6Q2BATdezn/zkC07hprbxQLIOUgwyyEAwnx0WiBEK5Wk9CU cT/sNGil914tTM2/dbs/qUM= X-Google-Smtp-Source: ABdhPJwHHFwc7OLTQH1CBkOFrPUNjVCjncdrAwug3lXX8oH9Qf5tOrL05BD6fCIop2qw05rvpxnr1A== X-Received: by 2002:a05:6870:8222:: with SMTP id n34mr1019239oae.67.1644355036957; Tue, 08 Feb 2022 13:17:16 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:16 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 03/11] RDMA/rxe: Replace grp by mcg, mce by mca Date: Tue, 8 Feb 2022 15:16:37 -0600 Message-Id: <20220208211644.123457-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace 'grp' by 'mcg', 'mce' by 'mca'. Shorten subroutine names in rxe_mcast.c. These name uses are more in line with other object names used. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 110 +++++++++++++------------- drivers/infiniband/sw/rxe/rxe_recv.c | 8 +- 2 files changed, 59 insertions(+), 59 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 3c06b0590c82..96dc11a892a4 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -26,66 +26,66 @@ static int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid) } /* caller should hold rxe->mcg_lock */ -static struct rxe_mcg *__rxe_create_grp(struct rxe_dev *rxe, +static struct rxe_mcg *__rxe_create_mcg(struct rxe_dev *rxe, struct rxe_pool *pool, union ib_gid *mgid) { - struct rxe_mcg *grp; + struct rxe_mcg *mcg; int err; - grp = rxe_alloc_locked(pool); - if (!grp) + mcg = rxe_alloc_locked(pool); + if (!mcg) return ERR_PTR(-ENOMEM); err = rxe_mcast_add(rxe, mgid); if (unlikely(err)) { - rxe_drop_ref(grp); + rxe_drop_ref(mcg); return ERR_PTR(err); } - INIT_LIST_HEAD(&grp->qp_list); - grp->rxe = rxe; + INIT_LIST_HEAD(&mcg->qp_list); + mcg->rxe = rxe; - /* rxe_alloc_locked takes a ref on grp but that will be - * dropped when grp goes out of scope. We need to take a ref + /* rxe_alloc_locked takes a ref on mcg but that will be + * dropped when mcg goes out of scope. We need to take a ref * on the pointer that will be saved in the red-black tree - * by rxe_add_key and used to lookup grp from mgid later. + * by rxe_add_key and used to lookup mcg from mgid later. * Adding key makes object visible to outside so this should * be done last after the object is ready. */ - rxe_add_ref(grp); - rxe_add_key_locked(grp, mgid); + rxe_add_ref(mcg); + rxe_add_key_locked(mcg, mgid); - return grp; + return mcg; } -static struct rxe_mcg *rxe_mcast_get_grp(struct rxe_dev *rxe, +static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) { - struct rxe_mcg *grp; + struct rxe_mcg *mcg; struct rxe_pool *pool = &rxe->mc_grp_pool; if (rxe->attr.max_mcast_qp_attach == 0) return ERR_PTR(-EINVAL); spin_lock_bh(&rxe->mcg_lock); - grp = rxe_pool_get_key_locked(pool, mgid); - if (!grp) - grp = __rxe_create_grp(rxe, pool, mgid); + mcg = rxe_pool_get_key_locked(pool, mgid); + if (!mcg) + mcg = __rxe_create_mcg(rxe, pool, mgid); spin_unlock_bh(&rxe->mcg_lock); - return grp; + return mcg; } -static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, - struct rxe_mcg *grp) +static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, + struct rxe_mcg *mcg) { struct rxe_mca *mca, *tmp; int err; /* check to see if the qp is already a member of the group */ spin_lock_bh(&rxe->mcg_lock); - list_for_each_entry(mca, &grp->qp_list, qp_list) { + list_for_each_entry(mca, &mcg->qp_list, qp_list) { if (mca->qp == qp) { spin_unlock_bh(&rxe->mcg_lock); return 0; @@ -100,7 +100,7 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, spin_lock_bh(&rxe->mcg_lock); /* re-check to see if someone else just attached qp */ - list_for_each_entry(tmp, &grp->qp_list, qp_list) { + list_for_each_entry(tmp, &mcg->qp_list, qp_list) { if (tmp->qp == qp) { kfree(mca); err = 0; @@ -109,7 +109,7 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, } /* check limits after checking if already attached */ - if (grp->num_qp >= rxe->attr.max_mcast_qp_attach) { + if (mcg->num_qp >= rxe->attr.max_mcast_qp_attach) { kfree(mca); err = -ENOMEM; goto out; @@ -120,8 +120,8 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, mca->qp = qp; atomic_inc(&qp->mcg_num); - grp->num_qp++; - list_add(&mca->qp_list, &grp->qp_list); + mcg->num_qp++; + list_add(&mca->qp_list, &mcg->qp_list); err = 0; out: @@ -130,21 +130,21 @@ static int rxe_mcast_add_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, } /* caller should be holding rxe->mcg_lock */ -static void __rxe_destroy_grp(struct rxe_mcg *grp) +static void __rxe_destroy_mcg(struct rxe_mcg *mcg) { - /* first remove grp from red-black tree then drop ref */ - rxe_drop_key_locked(grp); - rxe_drop_ref(grp); + /* first remove mcg from red-black tree then drop ref */ + rxe_drop_key_locked(mcg); + rxe_drop_ref(mcg); - rxe_mcast_delete(grp->rxe, &grp->mgid); + rxe_mcast_delete(mcg->rxe, &mcg->mgid); } -static void rxe_destroy_grp(struct rxe_mcg *grp) +static void rxe_destroy_mcg(struct rxe_mcg *mcg) { - struct rxe_dev *rxe = grp->rxe; + struct rxe_dev *rxe = mcg->rxe; spin_lock_bh(&rxe->mcg_lock); - __rxe_destroy_grp(grp); + __rxe_destroy_mcg(mcg); spin_unlock_bh(&rxe->mcg_lock); } @@ -153,22 +153,22 @@ void rxe_mc_cleanup(struct rxe_pool_elem *elem) /* nothing left to do for now */ } -static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, +static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, union ib_gid *mgid) { - struct rxe_mcg *grp; + struct rxe_mcg *mcg; struct rxe_mca *mca, *tmp; int err; spin_lock_bh(&rxe->mcg_lock); - grp = rxe_pool_get_key_locked(&rxe->mc_grp_pool, mgid); - if (!grp) { + mcg = rxe_pool_get_key_locked(&rxe->mc_grp_pool, mgid); + if (!mcg) { /* we didn't find the mcast group for mgid */ err = -EINVAL; goto out_unlock; } - list_for_each_entry_safe(mca, tmp, &grp->qp_list, qp_list) { + list_for_each_entry_safe(mca, tmp, &mcg->qp_list, qp_list) { if (mca->qp == qp) { list_del(&mca->qp_list); @@ -178,16 +178,16 @@ static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, * object since we are still holding a ref * from the get key above. */ - grp->num_qp--; - if (grp->num_qp <= 0) - __rxe_destroy_grp(grp); + mcg->num_qp--; + if (mcg->num_qp <= 0) + __rxe_destroy_mcg(mcg); atomic_dec(&qp->mcg_num); /* drop the ref from get key. This will free the * object if num_qp is zero. */ - rxe_drop_ref(grp); + rxe_drop_ref(mcg); kfree(mca); err = 0; goto out_unlock; @@ -195,7 +195,7 @@ static int rxe_mcast_drop_grp_elem(struct rxe_dev *rxe, struct rxe_qp *qp, } /* we didn't find the qp on the list */ - rxe_drop_ref(grp); + rxe_drop_ref(mcg); err = -EINVAL; out_unlock: @@ -208,20 +208,20 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) int err; struct rxe_dev *rxe = to_rdev(ibqp->device); struct rxe_qp *qp = to_rqp(ibqp); - struct rxe_mcg *grp; + struct rxe_mcg *mcg; - /* takes a ref on grp if successful */ - grp = rxe_mcast_get_grp(rxe, mgid); - if (IS_ERR(grp)) - return PTR_ERR(grp); + /* takes a ref on mcg if successful */ + mcg = rxe_get_mcg(rxe, mgid); + if (IS_ERR(mcg)) + return PTR_ERR(mcg); - err = rxe_mcast_add_grp_elem(rxe, qp, grp); + err = rxe_attach_mcg(rxe, qp, mcg); - /* if we failed to attach the first qp to grp tear it down */ - if (grp->num_qp == 0) - rxe_destroy_grp(grp); + /* if we failed to attach the first qp to mcg tear it down */ + if (mcg->num_qp == 0) + rxe_destroy_mcg(mcg); - rxe_drop_ref(grp); + rxe_drop_ref(mcg); return err; } @@ -230,5 +230,5 @@ int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) struct rxe_dev *rxe = to_rdev(ibqp->device); struct rxe_qp *qp = to_rqp(ibqp); - return rxe_mcast_drop_grp_elem(rxe, qp, mgid); + return rxe_detach_mcg(rxe, qp, mgid); } diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index a084b5d69937..d91c6660e83c 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -234,7 +234,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) { struct rxe_pkt_info *pkt = SKB_TO_PKT(skb); struct rxe_mcg *mcg; - struct rxe_mca *mce; + struct rxe_mca *mca; struct rxe_qp *qp; union ib_gid dgid; int err; @@ -257,8 +257,8 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) * single QP happen and just move on and try * the rest of them on the list */ - list_for_each_entry(mce, &mcg->qp_list, qp_list) { - qp = mce->qp; + list_for_each_entry(mca, &mcg->qp_list, qp_list) { + qp = mca->qp; /* validate qp for incoming packet */ err = check_type_state(rxe, pkt, qp); @@ -273,7 +273,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) * skb and pass to the QP. Pass the original skb to * the last QP in the list. */ - if (mce->qp_list.next != &mcg->qp_list) { + if (mca->qp_list.next != &mcg->qp_list) { struct sk_buff *cskb; struct rxe_pkt_info *cpkt; From patchwork Tue Feb 8 21:16:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739450 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4A5C4332F for ; Tue, 8 Feb 2022 22:24:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345032AbiBHWYz (ORCPT ); Tue, 8 Feb 2022 17:24:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386864AbiBHVRT (ORCPT ); Tue, 8 Feb 2022 16:17:19 -0500 Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A1700C0612BA for ; Tue, 8 Feb 2022 13:17:18 -0800 (PST) Received: by mail-oi1-x22b.google.com with SMTP id x193so516670oix.0 for ; Tue, 08 Feb 2022 13:17:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uUCkWwoN4ANeBFvyn/bUTxsS52qGVJg89kXWa2UrRYQ=; b=l0hsJA5ERe4McX1FG6aRRLlUyqCrIHKKR7ozlQY2DaM6ib7MlvXHpGNm0p0yiRgcEX f7oDEh2yVe1YbHupCfpIGP7/KHJG57tHS0DYZwB4fLSpuSN1OSZkRZUAfy9vS52WMBCq o3VHmo8Lh3ZVhX88CCmrV1UEP3XFPXxfn7/nF5ffRPlubMxMakRinRnkIhmahQH+WXGD RNfahDqTohcJGxq0m3R+aRz4js/tK8IPiCTGS60FjerFiz5R6AscfbM8D0OJF2GS+e3C B6HNSkEv2Fw31VoGa3+VeCz98PVRacSw5CovNxSTV3sHzT6R+Cj//44wBq8LDojB/ePq iotw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uUCkWwoN4ANeBFvyn/bUTxsS52qGVJg89kXWa2UrRYQ=; b=w17M+QWFAU3csZcZg17OVsb1ZWNxWOemPBj83p6rLBG9VYIaQmUkjHbeYNNTuoHNe7 P0Cuf7Pz6Z4jlxZ53Fn5NcoXTTK2fM/bBptU41q214EaENvfFm7FdwG9MnTuiFx9lVF6 Ox5vGCUIMSwFGJ1E/nNh8k6ARO5OZ2YXYYt+UVFrDfHlV5gjmNI4euJ5ZX6QRtIZb6UI Yy6Z854rUElHljRhMaBmyDZ//jdlTsEWIjDe7+TNE1srGSlamHsbkHtnciAlc7BFC5/4 cGibywiPYVT9KISim67vU6z8aSbgxKfu9TxTKEMA4BLsAKuqyaMUSJjQppJP/lleAaTd oeNw== X-Gm-Message-State: AOAM532i8PDp6hr5kiBraPxbomm+anOrc0OV6b1ixV10LG1HRuuYgsAb ue3uZ2dvOfJLoq2TtE/7yUK+daURkDI= X-Google-Smtp-Source: ABdhPJyL6wSTBLyCY+HkIIxZ3AlzX8ZP1EVqsSZx86Fn2G3+NnYXrY/xPQMR3JJ5b6lxJ+Mp7nPLbA== X-Received: by 2002:a05:6808:1181:: with SMTP id j1mr1377100oil.182.1644355037795; Tue, 08 Feb 2022 13:17:17 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:17 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 04/11] RDMA/rxe: Replace int num_qp by atomic_t qp_num Date: Tue, 8 Feb 2022 15:16:38 -0600 Message-Id: <20220208211644.123457-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace int num_qp in struct rxe_mcg by atomic_t qp_num. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 9 ++++----- drivers/infiniband/sw/rxe/rxe_verbs.h | 2 +- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 96dc11a892a4..2c6cb2eb5ac1 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -109,7 +109,8 @@ static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, } /* check limits after checking if already attached */ - if (mcg->num_qp >= rxe->attr.max_mcast_qp_attach) { + if (atomic_inc_return(&mcg->qp_num) > rxe->attr.max_mcast_qp_attach) { + atomic_dec(&mcg->qp_num); kfree(mca); err = -ENOMEM; goto out; @@ -120,7 +121,6 @@ static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, mca->qp = qp; atomic_inc(&qp->mcg_num); - mcg->num_qp++; list_add(&mca->qp_list, &mcg->qp_list); err = 0; @@ -178,8 +178,7 @@ static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, * object since we are still holding a ref * from the get key above. */ - mcg->num_qp--; - if (mcg->num_qp <= 0) + if (atomic_dec_return(&mcg->qp_num) <= 0) __rxe_destroy_mcg(mcg); atomic_dec(&qp->mcg_num); @@ -218,7 +217,7 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) err = rxe_attach_mcg(rxe, qp, mcg); /* if we failed to attach the first qp to mcg tear it down */ - if (mcg->num_qp == 0) + if (atomic_read(&mcg->qp_num) == 0) rxe_destroy_mcg(mcg); rxe_drop_ref(mcg); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 1b0f40881895..3790163bb265 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -356,7 +356,7 @@ struct rxe_mcg { struct rxe_dev *rxe; struct list_head qp_list; union ib_gid mgid; - int num_qp; + atomic_t qp_num; u32 qkey; u16 pkey; }; From patchwork Tue Feb 8 21:16:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739452 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C403C433FE for ; Tue, 8 Feb 2022 22:25:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1353601AbiBHWZA (ORCPT ); Tue, 8 Feb 2022 17:25:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386871AbiBHVRU (ORCPT ); Tue, 8 Feb 2022 16:17:20 -0500 Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42990C0612BC for ; Tue, 8 Feb 2022 13:17:19 -0800 (PST) Received: by mail-oo1-xc2c.google.com with SMTP id o192-20020a4a2cc9000000b00300af40d795so113248ooo.13 for ; Tue, 08 Feb 2022 13:17:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7YtBCKzCWxeoCaS5cF/oPWG1hIXwVDdb+8ZUmhXUBmw=; b=fucyRGOcJv+lS8ga10DBO8uv6WFIPGlamXEvi1eUDsUxs81LjkCFTuiW36gjt+bNbO HKJr1oxKR4LO3q4LOfzicDcDC+D9AEf9wQo4s0axhXH6M50m+QJdJF1e/3tCP4xzJ4kT NSrZyW0xKLETdAvStzwIWe8YSa8Rp137UJakr/NLuYoT06TW6ufqdfQQuPuE+TLt2WLg IccMzafoV/64CGShMw1itMLdzbniH6c18ChuHQole45mxWlV0SbFVoa8fneSTysXrJlY wlENdMdvPDpvYdaipT4M2Mf1TD72msUoU5J/hS2X06ItOYAyH1TdNwF0cmS9P+342/5K GyRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7YtBCKzCWxeoCaS5cF/oPWG1hIXwVDdb+8ZUmhXUBmw=; b=52eggqhKQan0xyau85JDniMO9LjWAescK083HL54HRbrMYIAGxyh8HVCOo1IHFFpeN 6xu00IVDIPJeWALHW+9Nqt/i4ROqeNyvzolyi/CClMVXCDivqYX4cw6jXFQlxNPUN/Xe AJxQnkQ/7bziFrnl7+ehoFsERDzG/P04llrOGl0T+CXdjjOeQfc4zUh4vUoIJkj2MmRb 7sU2Tg+rgJYiy0ITHRl+QrRwSTXys/dyrMAzesxLI8Z6C1/fyVACJ6zq3l8ASUOihCQV YHrbL23UKCcmhW2sBSckYhuXLK/5o0PsGVJw8P4APKyEXBEHRbyjIrLn2r7kR5n8fyK6 3ROA== X-Gm-Message-State: AOAM533aDxjsM6TzqrisstFDlqikAFhYdigPN6QmCuqbGjc1DAz6avAd rRV2STKYqCRHR1Fc6Q7rH4s= X-Google-Smtp-Source: ABdhPJzfifgtR9SdJ+8gl4r4NDf1AHPW8RHw/HQSE+kjeZhxbrLhKCJtznGQqVRx0oLxBGI8p6jdeA== X-Received: by 2002:a05:6870:c90d:: with SMTP id hj13mr1031000oab.332.1644355038593; Tue, 08 Feb 2022 13:17:18 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:18 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 05/11] RDMA/rxe: Replace pool key by rxe->mcg_tree Date: Tue, 8 Feb 2022 15:16:39 -0600 Message-Id: <20220208211644.123457-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Continuing to decouple mcg from rxe pools. Create red-black tree code in rxe_mcast.c to hold mcg index. Replace pool key calls by calls to local red-black routines. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 2 + drivers/infiniband/sw/rxe/rxe_loc.h | 3 +- drivers/infiniband/sw/rxe/rxe_mcast.c | 258 ++++++++++++++++++++------ drivers/infiniband/sw/rxe/rxe_recv.c | 4 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 6 +- 5 files changed, 210 insertions(+), 63 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 7386a51b953d..dc36148272dd 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -203,7 +203,9 @@ static int rxe_init(struct rxe_dev *rxe) spin_lock_init(&rxe->pending_lock); INIT_LIST_HEAD(&rxe->pending_mmaps); + /* init multicast support */ spin_lock_init(&rxe->mcg_lock); + rxe->mcg_tree = RB_ROOT; mutex_init(&rxe->usdev_lock); diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index af40e3c212fb..d41831878fa6 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -40,9 +40,10 @@ void rxe_cq_disable(struct rxe_cq *cq); void rxe_cq_cleanup(struct rxe_pool_elem *arg); /* rxe_mcast.c */ -void rxe_mc_cleanup(struct rxe_pool_elem *arg); +struct rxe_mcg *rxe_lookup_mcg(struct rxe_dev *rxe, union ib_gid *mgid); int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); +void rxe_mc_cleanup(struct rxe_pool_elem *elem); /* rxe_mmap.c */ struct rxe_mmap_info { diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 2c6cb2eb5ac1..78d696cd40d5 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -25,56 +25,225 @@ static int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid) return dev_mc_del(rxe->ndev, ll_addr); } -/* caller should hold rxe->mcg_lock */ -static struct rxe_mcg *__rxe_create_mcg(struct rxe_dev *rxe, - struct rxe_pool *pool, +/** + * __rxe_insert_mcg - insert an mcg into red-black tree (rxe->mcg_tree) + * @mcg: mcg object with an embedded red-black tree node + * + * Context: caller must hold a reference to mcg and rxe->mcg_lock and + * is responsible to avoid adding the same mcg twice to the tree. + */ +static void __rxe_insert_mcg(struct rxe_mcg *mcg) +{ + struct rb_root *tree = &mcg->rxe->mcg_tree; + struct rb_node **link = &tree->rb_node; + struct rb_node *node = NULL; + struct rxe_mcg *tmp; + int cmp; + + while (*link) { + node = *link; + tmp = rb_entry(node, struct rxe_mcg, node); + + cmp = memcmp(&tmp->mgid, &mcg->mgid, sizeof(mcg->mgid)); + if (cmp > 0) + link = &(*link)->rb_left; + else + link = &(*link)->rb_right; + } + + rb_link_node(&mcg->node, node, link); + rb_insert_color(&mcg->node, tree); +} + +/** + * __rxe_remove_mcg - remove an mcg from red-black tree holding lock + * @mcg: mcast group object with an embedded red-black tree node + * + * Context: caller must hold a reference to mcg and rxe->mcg_lock + */ +static void __rxe_remove_mcg(struct rxe_mcg *mcg) +{ + rb_erase(&mcg->node, &mcg->rxe->mcg_tree); +} + +/** + * __rxe_lookup_mcg - lookup mcg in rxe->mcg_tree while holding lock + * @rxe: rxe device object + * @mgid: multicast IP address + * + * Context: caller must hold rxe->mcg_lock + * Returns: mcg on success and takes a ref to mcg else NULL + */ +static struct rxe_mcg *__rxe_lookup_mcg(struct rxe_dev *rxe, union ib_gid *mgid) { + struct rb_root *tree = &rxe->mcg_tree; struct rxe_mcg *mcg; - int err; + struct rb_node *node; + int cmp; - mcg = rxe_alloc_locked(pool); - if (!mcg) - return ERR_PTR(-ENOMEM); + node = tree->rb_node; - err = rxe_mcast_add(rxe, mgid); - if (unlikely(err)) { - rxe_drop_ref(mcg); - return ERR_PTR(err); + while (node) { + mcg = rb_entry(node, struct rxe_mcg, node); + + cmp = memcmp(&mcg->mgid, mgid, sizeof(*mgid)); + + if (cmp > 0) + node = node->rb_left; + else if (cmp < 0) + node = node->rb_right; + else + break; } + if (node) { + rxe_add_ref(mcg); + return mcg; + } + + return NULL; +} + +/** + * rxe_lookup_mcg - lookup up mcg in red-back tree + * @rxe: rxe device object + * @mgid: multicast IP address + * + * Returns: mcg if found else NULL + */ +struct rxe_mcg *rxe_lookup_mcg(struct rxe_dev *rxe, union ib_gid *mgid) +{ + struct rxe_mcg *mcg; + + spin_lock_bh(&rxe->mcg_lock); + mcg = __rxe_lookup_mcg(rxe, mgid); + spin_unlock_bh(&rxe->mcg_lock); + + return mcg; +} + +/** + * __rxe_init_mcg - initialize a new mcg + * @rxe: rxe device + * @mgid: multicast address as a gid + * @mcg: new mcg object + * + * Context: caller should hold rxe->mcg lock + * Returns: 0 on success else an error + */ +static int __rxe_init_mcg(struct rxe_dev *rxe, union ib_gid *mgid, + struct rxe_mcg *mcg) +{ + int err; + + err = rxe_mcast_add(rxe, mgid); + if (unlikely(err)) + return err; + + memcpy(&mcg->mgid, mgid, sizeof(mcg->mgid)); INIT_LIST_HEAD(&mcg->qp_list); mcg->rxe = rxe; - /* rxe_alloc_locked takes a ref on mcg but that will be + /* caller holds a ref on mcg but that will be * dropped when mcg goes out of scope. We need to take a ref * on the pointer that will be saved in the red-black tree - * by rxe_add_key and used to lookup mcg from mgid later. - * Adding key makes object visible to outside so this should + * by __rxe_insert_mcg and used to lookup mcg from mgid later. + * Inserting mcg makes it visible to outside so this should * be done last after the object is ready. */ rxe_add_ref(mcg); - rxe_add_key_locked(mcg, mgid); + __rxe_insert_mcg(mcg); - return mcg; + return 0; } -static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, - union ib_gid *mgid) +/** + * rxe_get_mcg - lookup or allocate a mcg + * @rxe: rxe device object + * @mgid: multicast IP address as a gid + * + * Returns: mcg on success else ERR_PTR(error) + */ +static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) { - struct rxe_mcg *mcg; struct rxe_pool *pool = &rxe->mc_grp_pool; + struct rxe_mcg *mcg, *tmp; + int err; - if (rxe->attr.max_mcast_qp_attach == 0) + if (rxe->attr.max_mcast_grp == 0) return ERR_PTR(-EINVAL); - spin_lock_bh(&rxe->mcg_lock); - mcg = rxe_pool_get_key_locked(pool, mgid); + /* check to see if mcg already exists */ + mcg = rxe_lookup_mcg(rxe, mgid); + if (mcg) + return mcg; + + /* speculative alloc of new mcg */ + mcg = rxe_alloc(pool); if (!mcg) - mcg = __rxe_create_mcg(rxe, pool, mgid); - spin_unlock_bh(&rxe->mcg_lock); + return ERR_PTR(-ENOMEM); + spin_lock_bh(&rxe->mcg_lock); + /* re-check to see if someone else just added it */ + tmp = __rxe_lookup_mcg(rxe, mgid); + if (tmp) { + rxe_drop_ref(mcg); + mcg = tmp; + goto out; + } + + if (atomic_inc_return(&rxe->mcg_num) > rxe->attr.max_mcast_grp) { + err = -ENOMEM; + goto err_dec; + } + + err = __rxe_init_mcg(rxe, mgid, mcg); + if (err) + goto err_dec; +out: + spin_unlock_bh(&rxe->mcg_lock); return mcg; + +err_dec: + atomic_dec(&rxe->mcg_num); + spin_unlock_bh(&rxe->mcg_lock); + rxe_drop_ref(mcg); + return ERR_PTR(err); +} + +/** + * __rxe_destroy_mcg - destroy mcg object holding rxe->mcg_lock + * @mcg: the mcg object + * + * Context: caller is holding rxe->mcg_lock + * no qp's are attached to mcg + */ +static void __rxe_destroy_mcg(struct rxe_mcg *mcg) +{ + /* remove mcg from red-black tree then drop ref */ + __rxe_remove_mcg(mcg); + rxe_drop_ref(mcg); + + rxe_mcast_delete(mcg->rxe, &mcg->mgid); +} + +/** + * rxe_destroy_mcg - destroy mcg object + * @mcg: the mcg object + * + * Context: no qp's are attached to mcg + */ +static void rxe_destroy_mcg(struct rxe_mcg *mcg) +{ + spin_lock_bh(&mcg->rxe->mcg_lock); + __rxe_destroy_mcg(mcg); + spin_unlock_bh(&mcg->rxe->mcg_lock); +} + +void rxe_mc_cleanup(struct rxe_pool_elem *elem) +{ + /* nothing left to do for now */ } static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, @@ -129,30 +298,6 @@ static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, return err; } -/* caller should be holding rxe->mcg_lock */ -static void __rxe_destroy_mcg(struct rxe_mcg *mcg) -{ - /* first remove mcg from red-black tree then drop ref */ - rxe_drop_key_locked(mcg); - rxe_drop_ref(mcg); - - rxe_mcast_delete(mcg->rxe, &mcg->mgid); -} - -static void rxe_destroy_mcg(struct rxe_mcg *mcg) -{ - struct rxe_dev *rxe = mcg->rxe; - - spin_lock_bh(&rxe->mcg_lock); - __rxe_destroy_mcg(mcg); - spin_unlock_bh(&rxe->mcg_lock); -} - -void rxe_mc_cleanup(struct rxe_pool_elem *elem) -{ - /* nothing left to do for now */ -} - static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, union ib_gid *mgid) { @@ -160,17 +305,16 @@ static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, struct rxe_mca *mca, *tmp; int err; - spin_lock_bh(&rxe->mcg_lock); - mcg = rxe_pool_get_key_locked(&rxe->mc_grp_pool, mgid); - if (!mcg) { - /* we didn't find the mcast group for mgid */ - err = -EINVAL; - goto out_unlock; - } + mcg = rxe_lookup_mcg(rxe, mgid); + if (!mcg) + return -EINVAL; + spin_lock_bh(&rxe->mcg_lock); list_for_each_entry_safe(mca, tmp, &mcg->qp_list, qp_list) { if (mca->qp == qp) { list_del(&mca->qp_list); + atomic_dec(&qp->mcg_num); + rxe_drop_ref(qp); /* if the number of qp's attached to the * mcast group falls to zero go ahead and @@ -181,10 +325,8 @@ static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, if (atomic_dec_return(&mcg->qp_num) <= 0) __rxe_destroy_mcg(mcg); - atomic_dec(&qp->mcg_num); - /* drop the ref from get key. This will free the - * object if num_qp is zero. + * object if qp_num is zero. */ rxe_drop_ref(mcg); kfree(mca); diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index d91c6660e83c..fb265902f7e3 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -246,7 +246,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) memcpy(&dgid, &ipv6_hdr(skb)->daddr, sizeof(dgid)); /* lookup mcast group corresponding to mgid, takes a ref */ - mcg = rxe_pool_get_key(&rxe->mc_grp_pool, &dgid); + mcg = rxe_lookup_mcg(rxe, &dgid); if (!mcg) goto drop; /* mcast group not registered */ @@ -300,7 +300,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) spin_unlock_bh(&rxe->mcg_lock); - rxe_drop_ref(mcg); /* drop ref from rxe_pool_get_key. */ + rxe_drop_ref(mcg); if (likely(!skb)) return; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 3790163bb265..caa5b1b05019 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -353,6 +353,7 @@ struct rxe_mw { struct rxe_mcg { struct rxe_pool_elem elem; + struct rb_node node; struct rxe_dev *rxe; struct list_head qp_list; union ib_gid mgid; @@ -396,7 +397,10 @@ struct rxe_dev { struct rxe_pool mw_pool; struct rxe_pool mc_grp_pool; + /* multicast support */ spinlock_t mcg_lock; + struct rb_root mcg_tree; + atomic_t mcg_num; spinlock_t pending_lock; /* guard pending_mmaps */ struct list_head pending_mmaps; @@ -477,6 +481,4 @@ static inline struct rxe_pd *rxe_mw_pd(struct rxe_mw *mw) int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name); -void rxe_mc_cleanup(struct rxe_pool_elem *elem); - #endif /* RXE_VERBS_H */ From patchwork Tue Feb 8 21:16:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739446 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B0EEC433F5 for ; Tue, 8 Feb 2022 22:24:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387531AbiBHWYl (ORCPT ); Tue, 8 Feb 2022 17:24:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386877AbiBHVRU (ORCPT ); Tue, 8 Feb 2022 16:17:20 -0500 Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10C44C0612BA for ; Tue, 8 Feb 2022 13:17:20 -0800 (PST) Received: by mail-oo1-xc2d.google.com with SMTP id t75-20020a4a3e4e000000b002e9c0821d78so182530oot.4 for ; Tue, 08 Feb 2022 13:17:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5NSZgM/eetxo3VfRJ6XMU05MDJxyGmAMp/JPP6GxM7s=; b=gB9xCVx5goJwy3Czrk7Y78TIJSRUa3wf4SvRtNdRrlxJOyf4T6fR8A5ZTH6BOj1dNW tDHcq18D3mqxhfB6sKtaJLRYehNjk/+E+0z+lKRKmHFVBBJxV/vYscT7RnOgdcW4bsfU Ofa7kSzcwbvoy1ySznTtyeTp3QIAinHt4du3NDJX1gasAC2SVkL71x0IdOk+kW+vafSk dlmaSbxU8mFPexAf9gPjRavnIjOK/5qUzJQkey6vS+2niWyhX3r8nzbGnxdd9jPExVYa tDGXi1wC1h1EWFbvILkq9Uzxa4zkhw1kwTtuCj2anm/LRK/OIgAY+9xKCaxSX8bo22ob A9UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5NSZgM/eetxo3VfRJ6XMU05MDJxyGmAMp/JPP6GxM7s=; b=H/JZoDmeIw1UsLH9vxfd3eUvl1B2YbaG5wG/FUJ4F4Hd6tjRhiqS+KjhwuHcmGc4wz qtn0j0V2wsKQX4b6uzPnIXQSwY/4ePdVC3Jbs0qkZvO1KfpywAA0uxw2AhEJd2x6Sk5Q yoTQxEPX7wqt20MammNyW7WvfkSOAsm3iMnld4kK9pMqa7NZZONLGEWPvTWQsxAAOVsQ MdqpnU0uIL9h8QQa+5YsAu009s2l5NEl0Lg881LcbfgQg+LBSjUU3k8xpdzCzaU3bFSh 08Ei6ZKmw8zSa9YxsCQj86/W0EA6JFuXhcBVdNDMYHPD3KRGLsjWWNRJmIvT+qimMdVv Z2rA== X-Gm-Message-State: AOAM530Bom0GVvtE0PcJlsDpg3NRN6zZzqXOxjfn9xbmLUNe2K12LwOA uEKkUUqAakZ/D/p0WYT2cHB0qBKllBw= X-Google-Smtp-Source: ABdhPJwoFybKweBozUvjAUiGzMhFmtWWMIMOmneHUIv4F8htYEYKwzeBY+I+9aSOWHgAcGoc20614w== X-Received: by 2002:a05:6870:1251:: with SMTP id 17mr965219oao.254.1644355039422; Tue, 08 Feb 2022 13:17:19 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:19 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 06/11] RDMA/rxe: Remove key'ed object support Date: Tue, 8 Feb 2022 15:16:40 -0600 Message-Id: <20220208211644.123457-7-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Now that rxe_mcast.c has it's own red-black tree support there is no longer any requirement for key'ed objects in rxe pools. This patch removes the key APIs and related code. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 123 --------------------------- drivers/infiniband/sw/rxe/rxe_pool.h | 38 --------- 2 files changed, 161 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index a6756aa93e2b..49a25f8ceae1 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -16,8 +16,6 @@ static const struct rxe_type_info { enum rxe_pool_flags flags; u32 min_index; u32 max_index; - size_t key_offset; - size_t key_size; } rxe_type_info[RXE_NUM_TYPES] = { [RXE_TYPE_UC] = { .name = "rxe-uc", @@ -147,12 +145,6 @@ int rxe_pool_init( goto out; } - if (pool->flags & RXE_POOL_KEY) { - pool->key.tree = RB_ROOT; - pool->key.key_offset = info->key_offset; - pool->key.key_size = info->key_size; - } - out: return err; } @@ -209,77 +201,6 @@ static int rxe_insert_index(struct rxe_pool *pool, struct rxe_pool_elem *new) return 0; } -static int rxe_insert_key(struct rxe_pool *pool, struct rxe_pool_elem *new) -{ - struct rb_node **link = &pool->key.tree.rb_node; - struct rb_node *parent = NULL; - struct rxe_pool_elem *elem; - int cmp; - - while (*link) { - parent = *link; - elem = rb_entry(parent, struct rxe_pool_elem, key_node); - - cmp = memcmp((u8 *)elem + pool->key.key_offset, - (u8 *)new + pool->key.key_offset, - pool->key.key_size); - - if (cmp == 0) { - pr_warn("key already exists!\n"); - return -EINVAL; - } - - if (cmp > 0) - link = &(*link)->rb_left; - else - link = &(*link)->rb_right; - } - - rb_link_node(&new->key_node, parent, link); - rb_insert_color(&new->key_node, &pool->key.tree); - - return 0; -} - -int __rxe_add_key_locked(struct rxe_pool_elem *elem, void *key) -{ - struct rxe_pool *pool = elem->pool; - int err; - - memcpy((u8 *)elem + pool->key.key_offset, key, pool->key.key_size); - err = rxe_insert_key(pool, elem); - - return err; -} - -int __rxe_add_key(struct rxe_pool_elem *elem, void *key) -{ - struct rxe_pool *pool = elem->pool; - int err; - - write_lock_bh(&pool->pool_lock); - err = __rxe_add_key_locked(elem, key); - write_unlock_bh(&pool->pool_lock); - - return err; -} - -void __rxe_drop_key_locked(struct rxe_pool_elem *elem) -{ - struct rxe_pool *pool = elem->pool; - - rb_erase(&elem->key_node, &pool->key.tree); -} - -void __rxe_drop_key(struct rxe_pool_elem *elem) -{ - struct rxe_pool *pool = elem->pool; - - write_lock_bh(&pool->pool_lock); - __rxe_drop_key_locked(elem); - write_unlock_bh(&pool->pool_lock); -} - int __rxe_add_index_locked(struct rxe_pool_elem *elem) { struct rxe_pool *pool = elem->pool; @@ -443,47 +364,3 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) return obj; } - -void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) -{ - struct rb_node *node; - struct rxe_pool_elem *elem; - void *obj; - int cmp; - - node = pool->key.tree.rb_node; - - while (node) { - elem = rb_entry(node, struct rxe_pool_elem, key_node); - - cmp = memcmp((u8 *)elem + pool->key.key_offset, - key, pool->key.key_size); - - if (cmp > 0) - node = node->rb_left; - else if (cmp < 0) - node = node->rb_right; - else - break; - } - - if (node) { - kref_get(&elem->ref_cnt); - obj = elem->obj; - } else { - obj = NULL; - } - - return obj; -} - -void *rxe_pool_get_key(struct rxe_pool *pool, void *key) -{ - void *obj; - - read_lock_bh(&pool->pool_lock); - obj = rxe_pool_get_key_locked(pool, key); - read_unlock_bh(&pool->pool_lock); - - return obj; -} diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 9201bb6b8b07..2db9d9872cd1 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -9,7 +9,6 @@ enum rxe_pool_flags { RXE_POOL_INDEX = BIT(1), - RXE_POOL_KEY = BIT(2), RXE_POOL_NO_ALLOC = BIT(4), }; @@ -32,9 +31,6 @@ struct rxe_pool_elem { struct kref ref_cnt; struct list_head list; - /* only used if keyed */ - struct rb_node key_node; - /* only used if indexed */ struct rb_node index_node; u32 index; @@ -61,13 +57,6 @@ struct rxe_pool { u32 max_index; u32 min_index; } index; - - /* only used if keyed */ - struct { - struct rb_root tree; - size_t key_offset; - size_t key_size; - } key; }; /* initialize a pool of objects with given limit on @@ -112,26 +101,6 @@ void __rxe_drop_index(struct rxe_pool_elem *elem); #define rxe_drop_index(obj) __rxe_drop_index(&(obj)->elem) -/* assign a key to a keyed object and insert object into - * pool's rb tree holding and not holding pool_lock - */ -int __rxe_add_key_locked(struct rxe_pool_elem *elem, void *key); - -#define rxe_add_key_locked(obj, key) __rxe_add_key_locked(&(obj)->elem, key) - -int __rxe_add_key(struct rxe_pool_elem *elem, void *key); - -#define rxe_add_key(obj, key) __rxe_add_key(&(obj)->elem, key) - -/* remove elem from rb tree holding and not holding the pool_lock */ -void __rxe_drop_key_locked(struct rxe_pool_elem *elem); - -#define rxe_drop_key_locked(obj) __rxe_drop_key_locked(&(obj)->elem) - -void __rxe_drop_key(struct rxe_pool_elem *elem); - -#define rxe_drop_key(obj) __rxe_drop_key(&(obj)->elem) - /* lookup an indexed object from index holding and not holding the pool_lock. * takes a reference on object */ @@ -139,13 +108,6 @@ void *rxe_pool_get_index_locked(struct rxe_pool *pool, u32 index); void *rxe_pool_get_index(struct rxe_pool *pool, u32 index); -/* lookup keyed object from key holding and not holding the pool_lock. - * takes a reference on the objecti - */ -void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key); - -void *rxe_pool_get_key(struct rxe_pool *pool, void *key); - /* cleanup an object when all references are dropped */ void rxe_elem_release(struct kref *kref); From patchwork Tue Feb 8 21:16:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739445 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEA2CC4332F for ; Tue, 8 Feb 2022 22:24:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386942AbiBHWYk (ORCPT ); Tue, 8 Feb 2022 17:24:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386878AbiBHVRW (ORCPT ); Tue, 8 Feb 2022 16:17:22 -0500 Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19225C0612BE for ; Tue, 8 Feb 2022 13:17:21 -0800 (PST) Received: by mail-oo1-xc29.google.com with SMTP id f11-20020a4abb0b000000b002e9abf6bcbcso242159oop.0 for ; Tue, 08 Feb 2022 13:17:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XY3DD4ol7toyhGXGGzuYQIC2OzhUvk7iT9xEabpmvAQ=; b=YQjpn/AqWm+OS6T55zkWPUemCV81lvRUofbpg/Er6LbmhXWrLEUAZQvYkKv4yPJYcR 0YoqOsehVeUXrj9QT8j3qyFeOoiCl8BUPU+NuZZJVLYlu0x0HXTzLh+GunisYXDFzV9+ KjBthkhPrGZFVwIIJwyvxEs+xOsa9oHyr6zs4zoJhV2n28eqp/4cVuQ21axRUX9d5owl Wj4X6RwOoY1AdsDVAIf7Yh8/pek6rvVIpC0xtkhyqVjh4hJND24sbaFo4odqY53qEw+n MALNv4hb9gExGmddkX0KG7hnXGqcf/YyiehQDkNdBW/HNaDPLPSHgaXntqM6zW/SwgYd Pqnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XY3DD4ol7toyhGXGGzuYQIC2OzhUvk7iT9xEabpmvAQ=; b=cAnklYGPc4IIDEpGTMawHuWN8IFknUQ2VSjdDWQL410MdL0biGPYE2adZdKNI6g5x5 MgbCDgIxs/zMGh/ht2alc5Xts1stymbz1ZK3zt0TJ9Pixia+6nuZAtpuAAGLStZpPTTj rFfDW5IDsI32E6NvC0VXcW6DGIieLwxYWsQk9kQvjdRD8Y9yuMqefdte3iA8cjrUB+b1 taQdChjNrMoIoBx31nUE7oG7xHIUuBmlwozIzVMR7kFRvtKPKYPpjiTEhnY1WWTgDGLA HOwC139oem3IRgbb4W34Rq8oINHMvG31+tDv69liT5ttYOTTGr0q9C6ktp06IXLHHwGV CkOQ== X-Gm-Message-State: AOAM533zszAWRrMyf/YMuU6CPEq1yPIKJMJ2qmGZZVzL/vfu531neVXf hsw+Z8hGk6XTsIPTrTgQoT0rl9PjhLQ= X-Google-Smtp-Source: ABdhPJyeZWprEeyhm4/q7U9SgwmMygJoxypJE+vae9i4DbIeQ7ikEtFamJZ2DffvMag2j/G4aciYFQ== X-Received: by 2002:a05:6870:91c6:: with SMTP id c6mr1035039oaf.147.1644355040411; Tue, 08 Feb 2022 13:17:20 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:19 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 07/11] RDMA/rxe: Remove mcg from rxe pools Date: Tue, 8 Feb 2022 15:16:41 -0600 Message-Id: <20220208211644.123457-8-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Finish removing mcg from rxe pools. Replace rxe pools ref counting by kref's. Replace rxe_alloc by kzalloc. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 8 ------ drivers/infiniband/sw/rxe/rxe_loc.h | 2 +- drivers/infiniband/sw/rxe/rxe_mcast.c | 39 ++++++++++++++++----------- drivers/infiniband/sw/rxe/rxe_pool.c | 9 ------- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - drivers/infiniband/sw/rxe/rxe_recv.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 2 +- 7 files changed, 27 insertions(+), 36 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index dc36148272dd..3520eb2db685 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -28,7 +28,6 @@ void rxe_dealloc(struct ib_device *ib_dev) rxe_pool_cleanup(&rxe->cq_pool); rxe_pool_cleanup(&rxe->mr_pool); rxe_pool_cleanup(&rxe->mw_pool); - rxe_pool_cleanup(&rxe->mc_grp_pool); if (rxe->tfm) crypto_free_shash(rxe->tfm); @@ -157,15 +156,8 @@ static int rxe_init_pools(struct rxe_dev *rxe) if (err) goto err8; - err = rxe_pool_init(rxe, &rxe->mc_grp_pool, RXE_TYPE_MC_GRP, - rxe->attr.max_mcast_grp); - if (err) - goto err9; - return 0; -err9: - rxe_pool_cleanup(&rxe->mw_pool); err8: rxe_pool_cleanup(&rxe->mr_pool); err7: diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index d41831878fa6..409efeecd581 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -43,7 +43,7 @@ void rxe_cq_cleanup(struct rxe_pool_elem *arg); struct rxe_mcg *rxe_lookup_mcg(struct rxe_dev *rxe, union ib_gid *mgid); int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); -void rxe_mc_cleanup(struct rxe_pool_elem *elem); +void rxe_cleanup_mcg(struct kref *kref); /* rxe_mmap.c */ struct rxe_mmap_info { diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 78d696cd40d5..07c218788c59 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -98,7 +98,7 @@ static struct rxe_mcg *__rxe_lookup_mcg(struct rxe_dev *rxe, } if (node) { - rxe_add_ref(mcg); + kref_get(&mcg->ref_cnt); return mcg; } @@ -141,6 +141,7 @@ static int __rxe_init_mcg(struct rxe_dev *rxe, union ib_gid *mgid, if (unlikely(err)) return err; + kref_init(&mcg->ref_cnt); memcpy(&mcg->mgid, mgid, sizeof(mcg->mgid)); INIT_LIST_HEAD(&mcg->qp_list); mcg->rxe = rxe; @@ -152,7 +153,7 @@ static int __rxe_init_mcg(struct rxe_dev *rxe, union ib_gid *mgid, * Inserting mcg makes it visible to outside so this should * be done last after the object is ready. */ - rxe_add_ref(mcg); + kref_get(&mcg->ref_cnt); __rxe_insert_mcg(mcg); return 0; @@ -167,7 +168,6 @@ static int __rxe_init_mcg(struct rxe_dev *rxe, union ib_gid *mgid, */ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) { - struct rxe_pool *pool = &rxe->mc_grp_pool; struct rxe_mcg *mcg, *tmp; int err; @@ -180,7 +180,7 @@ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) return mcg; /* speculative alloc of new mcg */ - mcg = rxe_alloc(pool); + mcg = kzalloc(sizeof(*mcg), GFP_KERNEL); if (!mcg) return ERR_PTR(-ENOMEM); @@ -188,7 +188,7 @@ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) /* re-check to see if someone else just added it */ tmp = __rxe_lookup_mcg(rxe, mgid); if (tmp) { - rxe_drop_ref(mcg); + kfree(mcg); mcg = tmp; goto out; } @@ -208,10 +208,21 @@ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) err_dec: atomic_dec(&rxe->mcg_num); spin_unlock_bh(&rxe->mcg_lock); - rxe_drop_ref(mcg); + kfree(mcg); return ERR_PTR(err); } +/** + * rxe_cleanup_mcg - cleanup mcg for kref_put + * @kref: + */ +void rxe_cleanup_mcg(struct kref *kref) +{ + struct rxe_mcg *mcg = container_of(kref, typeof(*mcg), ref_cnt); + + kfree(mcg); +} + /** * __rxe_destroy_mcg - destroy mcg object holding rxe->mcg_lock * @mcg: the mcg object @@ -221,11 +232,14 @@ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) */ static void __rxe_destroy_mcg(struct rxe_mcg *mcg) { + struct rxe_dev *rxe = mcg->rxe; + /* remove mcg from red-black tree then drop ref */ __rxe_remove_mcg(mcg); - rxe_drop_ref(mcg); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); rxe_mcast_delete(mcg->rxe, &mcg->mgid); + atomic_dec(&rxe->mcg_num); } /** @@ -241,11 +255,6 @@ static void rxe_destroy_mcg(struct rxe_mcg *mcg) spin_unlock_bh(&mcg->rxe->mcg_lock); } -void rxe_mc_cleanup(struct rxe_pool_elem *elem) -{ - /* nothing left to do for now */ -} - static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, struct rxe_mcg *mcg) { @@ -328,7 +337,7 @@ static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, /* drop the ref from get key. This will free the * object if qp_num is zero. */ - rxe_drop_ref(mcg); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); kfree(mca); err = 0; goto out_unlock; @@ -336,7 +345,7 @@ static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, } /* we didn't find the qp on the list */ - rxe_drop_ref(mcg); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); err = -EINVAL; out_unlock: @@ -362,7 +371,7 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) if (atomic_read(&mcg->qp_num) == 0) rxe_destroy_mcg(mcg); - rxe_drop_ref(mcg); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); return err; } diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 49a25f8ceae1..b6fe7c93aaab 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -79,15 +79,6 @@ static const struct rxe_type_info { .min_index = RXE_MIN_MW_INDEX, .max_index = RXE_MAX_MW_INDEX, }, - [RXE_TYPE_MC_GRP] = { - .name = "rxe-mc_grp", - .size = sizeof(struct rxe_mcg), - .elem_offset = offsetof(struct rxe_mcg, elem), - .cleanup = rxe_mc_cleanup, - .flags = RXE_POOL_KEY, - .key_offset = offsetof(struct rxe_mcg, mgid), - .key_size = sizeof(union ib_gid), - }, }; static int rxe_pool_init_index(struct rxe_pool *pool, u32 max, u32 min) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 2db9d9872cd1..8fc95c6b7b9b 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -21,7 +21,6 @@ enum rxe_elem_type { RXE_TYPE_CQ, RXE_TYPE_MR, RXE_TYPE_MW, - RXE_TYPE_MC_GRP, RXE_NUM_TYPES, /* keep me last */ }; diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index fb265902f7e3..53924453abef 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -300,7 +300,7 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) spin_unlock_bh(&rxe->mcg_lock); - rxe_drop_ref(mcg); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); if (likely(!skb)) return; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index caa5b1b05019..20fe3ee6589d 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -352,8 +352,8 @@ struct rxe_mw { }; struct rxe_mcg { - struct rxe_pool_elem elem; struct rb_node node; + struct kref ref_cnt; struct rxe_dev *rxe; struct list_head qp_list; union ib_gid mgid; From patchwork Tue Feb 8 21:16:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 812DCC433EF for ; Tue, 8 Feb 2022 22:25:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386864AbiBHWY5 (ORCPT ); Tue, 8 Feb 2022 17:24:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37722 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386881AbiBHVRX (ORCPT ); Tue, 8 Feb 2022 16:17:23 -0500 Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD196C0612BA for ; Tue, 8 Feb 2022 13:17:22 -0800 (PST) Received: by mail-oo1-xc34.google.com with SMTP id k13-20020a4a948d000000b003172f2f6bdfso198883ooi.1 for ; Tue, 08 Feb 2022 13:17:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ulTF2hxvuEkRlHh2KLWnbeUZj1+dl09v2C4+yKaBJY8=; b=MRrj1LnivVzJNwxWl4gmI7zu79uK7gNzcfTtMwf6Dk9jCoiDeld9N7IhgPUbPjSVTF 9KjFe+ddsNtyZfP5NwvIUQqEtSsFLOKkMwNCxaTIIAyxhv2uTNQ6WXWGwo3j9CIfLxbs SbqZX2sUsIP/RIYYZw73AGj44MP19t6u63CvTuZV8SbINOj6sfW2y8aw9dUb1GYek7Zp RwWCywTN/AU8D6szzMnOzlC/yM9wZ+iUHTCgWOlsh5ngDrZDAtkO3KAmgrEL2s1/OR5R qXLEa1Y67KCppal6IavB0wJSC8rEWbhVvE/ZEcoHPqFzytDJbA44Fve9aDFRNDjWq7lH pmdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ulTF2hxvuEkRlHh2KLWnbeUZj1+dl09v2C4+yKaBJY8=; b=FIMomfzM+cRjiKWa6/DJ1ORwWwRcgEE6GLobHQ76MxQ0WOQXmLN0ek26CXVNX9FQru 0N/1/o/cloK5q5hgeyt9fN3UY6dL/Qjv8L49C1FNbOmHyiLrcpjDzQJ2p7jMoruNor89 wY8+62AsejtBsSVOfYxnlAJd96nn556WBVUaB8YCH2lJU7M7g+ih+s+imypPb0hgHjrz gYVJhu8tb+QDW7by5/8L5/mQuTvdrAx1+AbBrcPCCHlBx9j3u28AfZZPQghLeBX1l5+u WfSEqtfZQ0PA9w3B8o4L0ZG44r1TGPS1sCsYwm6FAuX6Gcku16vb2Qd83itklaOucWdS a0Gw== X-Gm-Message-State: AOAM530guGmdbwtxXkPvsAXWlWhNG5KPibrGe97mAbt+KHab5DpwZJbi WyU3x2rJrRZFr7x0qb32HiXQEiM786I= X-Google-Smtp-Source: ABdhPJy61c5SbsTYM+homsrsxlI7Gf1U9RQsU9l9WbU9mLUK2DQ/m4oYR6CLRHFXGVEU2zUei1QwWQ== X-Received: by 2002:a05:6870:3811:: with SMTP id y17mr1039662oal.318.1644355042062; Tue, 08 Feb 2022 13:17:22 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:21 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 08/11] RDMA/rxe: Add code to cleanup mcast memory Date: Tue, 8 Feb 2022 15:16:42 -0600 Message-Id: <20220208211644.123457-9-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Well behaved applications will free all memory allocated by multicast but programs which do not clean up properly can leave behind allocated memory when the rxe driver is unloaded. This patch walks the red-black tree holding multicast group elements and then walks the list of attached qp's freeing the mca's and finally the mcg's. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 2 ++ drivers/infiniband/sw/rxe/rxe_loc.h | 1 + drivers/infiniband/sw/rxe/rxe_mcast.c | 31 +++++++++++++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 3520eb2db685..603b0156f889 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -29,6 +29,8 @@ void rxe_dealloc(struct ib_device *ib_dev) rxe_pool_cleanup(&rxe->mr_pool); rxe_pool_cleanup(&rxe->mw_pool); + rxe_cleanup_mcast(rxe); + if (rxe->tfm) crypto_free_shash(rxe->tfm); } diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index 409efeecd581..0bc1b7e2877c 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -44,6 +44,7 @@ struct rxe_mcg *rxe_lookup_mcg(struct rxe_dev *rxe, union ib_gid *mgid); int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid); void rxe_cleanup_mcg(struct kref *kref); +void rxe_cleanup_mcast(struct rxe_dev *rxe); /* rxe_mmap.c */ struct rxe_mmap_info { diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 07c218788c59..846147878607 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -382,3 +382,34 @@ int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) return rxe_detach_mcg(rxe, qp, mgid); } + +/** + * rxe_cleanup_mcast - cleanup all resources held by mcast + * @rxe: rxe object + * + * Called when rxe device is unloaded. Walk red-black tree to + * find all mcg's and then walk mcg->qp_list to find all mca's and + * free them. These should have been freed already if apps are + * well behaved. + */ +void rxe_cleanup_mcast(struct rxe_dev *rxe) +{ + struct rb_root *root = &rxe->mcg_tree; + struct rb_node *node, *next; + struct rxe_mcg *mcg; + struct rxe_mca *mca, *tmp; + + for (node = rb_first(root); node; node = next) { + next = rb_next(node); + mcg = rb_entry(node, typeof(*mcg), node); + + spin_lock_bh(&rxe->mcg_lock); + list_for_each_entry_safe(mca, tmp, &mcg->qp_list, qp_list) + kfree(mca); + + __rxe_remove_mcg(mcg); + spin_unlock_bh(&rxe->mcg_lock); + + kfree(mcg); + } +} From patchwork Tue Feb 8 21:16:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72CB3C433F5 for ; Tue, 8 Feb 2022 22:24:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386838AbiBHWYi (ORCPT ); Tue, 8 Feb 2022 17:24:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386885AbiBHVRZ (ORCPT ); Tue, 8 Feb 2022 16:17:25 -0500 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10B0DC0612B8 for ; Tue, 8 Feb 2022 13:17:24 -0800 (PST) Received: by mail-oi1-x234.google.com with SMTP id i5so478450oih.1 for ; Tue, 08 Feb 2022 13:17:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iqRAEa887JpXbjOAQTwV/4n6IwZcJpoqvAlHonjohs0=; b=ZpnnO7x/QQWXRS9w7CyT6sMcbSHO/U/OWnXoENCTSq4HUHncFJG29troKP0X77B3Og MzBTMxZst/DtUts/z9FEyhLDi1h6RRyc5vapC11AjfmOPYsSfs1Zl5u4lkRMFcEnc61W tinwqaVQsMi/2EPVKOTanl5ynWp+D9FFopoa3hJCpRhG5ITs76acxPnIX7CQMuNvhj46 MRQOsflBnHzZZ+Ju3c8taWg4xcYi3srUdxEI0H2Dh53Jr5iPVfOUf0zUsjI6ba6Ky6jY kwWQeMJkXgdgbv7nrmZLNDtwC2Aidfi4TT3iNtLc42QERc4wGA/2LUIlQfCFH/W6Iaem oRYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iqRAEa887JpXbjOAQTwV/4n6IwZcJpoqvAlHonjohs0=; b=vMa1hJB6GCkWEDMjDUrAuyGJEXEnZAvOx52YoQ/1V0Fy+lcohDYxrQbMvqg+4W2kzw 7DctYLx5LSNtn+KhCmO/umvRXLMwO7c0T50vDwL/EAlKMDtFf8zOEmCTb4FK3eriJ3sb yjJAXVXjQQ10vcoqy4bZ7LWTXQTPpeTxIdbE1ZDIUag5LsGq3Bcd+nvapQwzofPjMGyF QAneedVyhCkAr7tNJK7RxTmLA4/6c0w2QLHEnd+8vtVkFPN41es8oNF+dHDkIEyLpikO wxJSQkoCcZa6SN8FihVQ2kAxiRDnoysAGvLSe2SdGuWHSQUYIBY1UH5F1jRVvN3hz4Q3 WeeA== X-Gm-Message-State: AOAM5315gdSxLqkXCfDgNi132UO5HVD9tYCXLY0g9g5BztkbEDE4rJAJ xyR9Got2Z2/XoB1gpG+WPzc= X-Google-Smtp-Source: ABdhPJw7VUdzHblzaXmv2ONH7UyldfM8ItX7njYoP0YrKwGu98+QNxjT5tbGw6aIn+kOWHMKRy/YWw== X-Received: by 2002:a05:6808:150d:: with SMTP id u13mr1429205oiw.55.1644355043026; Tue, 08 Feb 2022 13:17:23 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:22 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 09/11] RDMA/rxe: Finish cleanup of rxe_mcast.c Date: Tue, 8 Feb 2022 15:16:43 -0600 Message-Id: <20220208211644.123457-10-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Cleanup rxe_mcast.c code. Minor changes and complete comments. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 189 +++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 + 2 files changed, 144 insertions(+), 46 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 846147878607..b66839276aa6 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -1,12 +1,33 @@ // SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB /* + * Copyright (c) 2022 Hewlett Packard Enterprise, Inc. All rights reserved. * Copyright (c) 2016 Mellanox Technologies Ltd. All rights reserved. * Copyright (c) 2015 System Fabric Works, Inc. All rights reserved. */ +/* + * rxe_mcast.c implements driver support for multicast transport. + * It is based on two data structures struct rxe_mcg ('mcg') and + * struct rxe_mca ('mca'). An mcg is allocated each time a qp is + * attached to a new mgid for the first time. These are indexed by + * a red-black tree using the mgid. This data structure is searched + * for the mcg when a multicast packet is received and when another + * qp is attached to the same mgid. It is cleaned up when the last qp + * is detached from the mcg. Each time a qp is attached to an mcg an + * mca is created. It holds a pointer to the qp and is added to a list + * of qp's that are attached to the mcg. The qp_list is used to replicate + * mcast packets in the rxe receive path. + */ + #include "rxe.h" -#include "rxe_loc.h" +/** + * rxe_mcast_add - add multicast address to rxe device + * @rxe: rxe device object + * @mgid: multicast address as a gid + * + * Returns 0 on success else an error + */ static int rxe_mcast_add(struct rxe_dev *rxe, union ib_gid *mgid) { unsigned char ll_addr[ETH_ALEN]; @@ -16,6 +37,13 @@ static int rxe_mcast_add(struct rxe_dev *rxe, union ib_gid *mgid) return dev_mc_add(rxe->ndev, ll_addr); } +/** + * rxe_mcast_delete - delete multicast address from rxe device + * @rxe: rxe device object + * @mgid: multicast address as a gid + * + * Returns 0 on success else an error + */ static int rxe_mcast_delete(struct rxe_dev *rxe, union ib_gid *mgid) { unsigned char ll_addr[ETH_ALEN]; @@ -214,7 +242,7 @@ static struct rxe_mcg *rxe_get_mcg(struct rxe_dev *rxe, union ib_gid *mgid) /** * rxe_cleanup_mcg - cleanup mcg for kref_put - * @kref: + * @kref: struct kref embedded in mcg */ void rxe_cleanup_mcg(struct kref *kref) { @@ -255,9 +283,57 @@ static void rxe_destroy_mcg(struct rxe_mcg *mcg) spin_unlock_bh(&mcg->rxe->mcg_lock); } -static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, - struct rxe_mcg *mcg) +/** + * __rxe_init_mca - initialize a new mca holding lock + * @qp: qp object + * @mcg: mcg object + * @mca: empty space for new mca + * + * Context: caller must hold references on qp and mcg, rxe->mcg_lock + * and pass memory for new mca + * + * Returns: 0 on success else an error + */ +static int __rxe_init_mca(struct rxe_qp *qp, struct rxe_mcg *mcg, + struct rxe_mca *mca) +{ + struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + int n; + + n = atomic_inc_return(&rxe->mcg_attach); + if (n > rxe->attr.max_total_mcast_qp_attach) { + atomic_dec(&rxe->mcg_attach); + return -ENOMEM; + } + + n = atomic_inc_return(&mcg->qp_num); + if (n > rxe->attr.max_mcast_qp_attach) { + atomic_dec(&mcg->qp_num); + atomic_dec(&rxe->mcg_attach); + return -ENOMEM; + } + + atomic_inc(&qp->mcg_num); + + rxe_add_ref(qp); + mca->qp = qp; + + list_add_tail(&mca->qp_list, &mcg->qp_list); + + return 0; +} + +/** + * rxe_attach_mcg - attach qp to mcg if not already attached + * @qp: qp object + * @mcg: mcg object + * + * Context: caller must hold reference on qp and mcg. + * Returns: 0 on success else an error + */ +static int rxe_attach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) { + struct rxe_dev *rxe = mcg->rxe; struct rxe_mca *mca, *tmp; int err; @@ -286,73 +362,77 @@ static int rxe_attach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, } } - /* check limits after checking if already attached */ - if (atomic_inc_return(&mcg->qp_num) > rxe->attr.max_mcast_qp_attach) { - atomic_dec(&mcg->qp_num); + err = __rxe_init_mca(qp, mcg, mca); + if (err) kfree(mca); - err = -ENOMEM; - goto out; - } - - /* protect pointer to qp in mca */ - rxe_add_ref(qp); - mca->qp = qp; - - atomic_inc(&qp->mcg_num); - list_add(&mca->qp_list, &mcg->qp_list); - - err = 0; out: spin_unlock_bh(&rxe->mcg_lock); return err; } -static int rxe_detach_mcg(struct rxe_dev *rxe, struct rxe_qp *qp, - union ib_gid *mgid) +/** + * __rxe_cleanup_mca - cleanup mca object holding lock + * @mca: mca object + * @mcg: mcg object + * + * Context: caller must hold a reference to mcg and rxe->mcg_lock + */ +static void __rxe_cleanup_mca(struct rxe_mca *mca, struct rxe_mcg *mcg) { - struct rxe_mcg *mcg; - struct rxe_mca *mca, *tmp; - int err; + list_del(&mca->qp_list); - mcg = rxe_lookup_mcg(rxe, mgid); - if (!mcg) - return -EINVAL; + atomic_dec(&mcg->qp_num); + atomic_dec(&mcg->rxe->mcg_attach); + atomic_dec(&mca->qp->mcg_num); + rxe_drop_ref(mca->qp); + + kfree(mca); +} + +/** + * rxe_detach_mcg - detach qp from mcg + * @mcg: mcg object + * @qp: qp object + * + * Returns: 0 on success else an error if qp is not attached. + */ +static int rxe_detach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) +{ + struct rxe_dev *rxe = mcg->rxe; + struct rxe_mca *mca, *tmp; spin_lock_bh(&rxe->mcg_lock); list_for_each_entry_safe(mca, tmp, &mcg->qp_list, qp_list) { if (mca->qp == qp) { - list_del(&mca->qp_list); - atomic_dec(&qp->mcg_num); - rxe_drop_ref(qp); + __rxe_cleanup_mca(mca, mcg); /* if the number of qp's attached to the * mcast group falls to zero go ahead and * tear it down. This will not free the * object since we are still holding a ref - * from the get key above. + * from the caller */ - if (atomic_dec_return(&mcg->qp_num) <= 0) + if (atomic_read(&mcg->qp_num) <= 0) __rxe_destroy_mcg(mcg); - /* drop the ref from get key. This will free the - * object if qp_num is zero. - */ - kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); - kfree(mca); - err = 0; - goto out_unlock; + spin_unlock_bh(&rxe->mcg_lock); + return 0; } } /* we didn't find the qp on the list */ - kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); - err = -EINVAL; - -out_unlock: spin_unlock_bh(&rxe->mcg_lock); - return err; + return -EINVAL; } +/** + * rxe_attach_mcast - attach qp to multicast group (see IBA-11.3.1) + * @ibqp: (IB) qp object + * @mgid: multicast IP address + * @mlid: multicast LID, ignored for RoCEv2 (see IBA-A17.5.6) + * + * Returns: 0 on success else an errno + */ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) { int err; @@ -365,7 +445,7 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) if (IS_ERR(mcg)) return PTR_ERR(mcg); - err = rxe_attach_mcg(rxe, qp, mcg); + err = rxe_attach_mcg(mcg, qp); /* if we failed to attach the first qp to mcg tear it down */ if (atomic_read(&mcg->qp_num) == 0) @@ -375,12 +455,29 @@ int rxe_attach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) return err; } +/** + * rxe_detach_mcast - detach qp from multicast group (see IBA-11.3.2) + * @ibqp: address of (IB) qp object + * @mgid: multicast IP address + * @mlid: multicast LID, ignored for RoCEv2 (see IBA-A17.5.6) + * + * Returns: 0 on success else an errno + */ int rxe_detach_mcast(struct ib_qp *ibqp, union ib_gid *mgid, u16 mlid) { struct rxe_dev *rxe = to_rdev(ibqp->device); struct rxe_qp *qp = to_rqp(ibqp); + struct rxe_mcg *mcg; + int err; + + mcg = rxe_lookup_mcg(rxe, mgid); + if (!mcg) + return -EINVAL; - return rxe_detach_mcg(rxe, qp, mgid); + err = rxe_detach_mcg(mcg, qp); + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); + + return err; } /** diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 20fe3ee6589d..6b15251ff67a 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -401,6 +401,7 @@ struct rxe_dev { spinlock_t mcg_lock; struct rb_root mcg_tree; atomic_t mcg_num; + atomic_t mcg_attach; spinlock_t pending_lock; /* guard pending_mmaps */ struct list_head pending_mmaps; From patchwork Tue Feb 8 21:16:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 696D2C433F5 for ; Tue, 8 Feb 2022 22:25:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386349AbiBHWZF (ORCPT ); Tue, 8 Feb 2022 17:25:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386886AbiBHVRZ (ORCPT ); Tue, 8 Feb 2022 16:17:25 -0500 Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 683F7C0612BA for ; Tue, 8 Feb 2022 13:17:24 -0800 (PST) Received: by mail-oo1-xc29.google.com with SMTP id p190-20020a4a2fc7000000b0031820de484aso145329oop.9 for ; Tue, 08 Feb 2022 13:17:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nHH2kN1qaHun2fFYMfE2C2Cg4rCPKgUgoK974Y7RcTA=; b=ilIuoh611GNfpBpnQeZoqJWrPVcnoGpW2gsQXAdeK1RcEGLp8QKiCMg73lkMLSLj6e TD7EdF7EK5rY/B3efwNPY/1HHcAP9dxA//uLQLgYn9fYA+47VqvUarc7LpiGOSGc8BDJ JLmreTJSUu9UZ7l2o8MLrSZVIGyqnFnHNvIAe5uaDICbu6ikSU5edsXG9f58pULFE+iW V2gYUuhg30HNgd05/rAP/gKrvDSB6nyIYzQN+hylgebgo8iJiQ0zFleAmmBK06o96DNZ Yj4QD247dH+LUpTivc++88TNhI9w2Aa352ZvgCKtKOo79hZyOxqEAzTpXt62q/YqLCho pYQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nHH2kN1qaHun2fFYMfE2C2Cg4rCPKgUgoK974Y7RcTA=; b=5Bo+L3MeOcQ93yP3cIVccbFBqUXlSvKEONp4yFU6SG8ws+WGjES+UZ+vDtmXtm/fQ2 lqr9vHu2my9EFBdBD4w5dDms+j205I/iyFk1gN0mT0U55Oj4stQ46zyqQsiN6mgznavK 4E2UiKNVEqUW/J+JfBWo4QoLDXA/RwxwMEf1A+vnVeIrw2H4tqYIyCbpL/F4ZJlw6W0m RutWysnmxjTt5OunzYyyxKhvUaYyrqed8X5aG+RtAynEUl2bIt/055zkvCeImVtGpDd4 7r+4dEW/ULoLEt2Gp0gy74oW/LFyD6/GysIwfy/mFDcCuEvLqmZxmxXS9smXYXzZbGom e7Og== X-Gm-Message-State: AOAM530FFVVfyIXi/r/LA+xCtFfK981LgUdEnHPEL4wXBmcW+y0oMcwx KdnJK9PKJkhi7r4XdJCZevFvP6BrXQg= X-Google-Smtp-Source: ABdhPJxENCU/6oNJQtq0/Pj7n1bi16vQEIz8j2lLV0Qyn1XRioWUZeY58eb88Yo2sgxlBUqL6QR9PQ== X-Received: by 2002:a05:6870:91c5:: with SMTP id c5mr980251oaf.203.1644355043786; Tue, 08 Feb 2022 13:17:23 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:23 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 10/11] RDMA/rxe: For mcast copy qp list to temp array Date: Tue, 8 Feb 2022 15:16:44 -0600 Message-Id: <20220208211644.123457-11-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Currently rxe_rcv_mcast_pkt performs most of its work under the rxe->mcg_lock and calls into rxe_rcv which queues the packets to the responder and completer tasklets holding the lock which is a very bad idea. This patch walks the qp_list in mcg and copies the qp addresses to a temporary array under the lock but does the rest of the work without holding the lock. The critical section is now very small. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_recv.c | 103 +++++++++++++++++---------- 1 file changed, 64 insertions(+), 39 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index 53924453abef..9b21cbb22602 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -232,11 +232,15 @@ static inline void rxe_rcv_pkt(struct rxe_pkt_info *pkt, struct sk_buff *skb) static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) { + struct sk_buff *skb_copy; struct rxe_pkt_info *pkt = SKB_TO_PKT(skb); + struct rxe_pkt_info *pkt_copy; struct rxe_mcg *mcg; struct rxe_mca *mca; struct rxe_qp *qp; + struct rxe_qp **qp_array; union ib_gid dgid; + int n, nmax; int err; if (skb->protocol == htons(ETH_P_IP)) @@ -248,68 +252,89 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) /* lookup mcast group corresponding to mgid, takes a ref */ mcg = rxe_lookup_mcg(rxe, &dgid); if (!mcg) - goto drop; /* mcast group not registered */ + goto err_drop; /* mcast group not registered */ + + /* this is the current number of qp's attached to mcg plus a + * little room in case new qp's are attached between here + * and when we finish walking the qp list. If someone can + * attach more than 4 new qp's we will miss forwarding + * packets to those qp's. This is actually OK since UD is + * a unreliable service. + */ + nmax = atomic_read(&mcg->qp_num) + 4; + qp_array = kmalloc_array(nmax, sizeof(qp), GFP_KERNEL); + n = 0; spin_lock_bh(&rxe->mcg_lock); - - /* this is unreliable datagram service so we let - * failures to deliver a multicast packet to a - * single QP happen and just move on and try - * the rest of them on the list - */ list_for_each_entry(mca, &mcg->qp_list, qp_list) { - qp = mca->qp; + /* protect the qp pointers in the list */ + rxe_add_ref(mca->qp); + qp_array[n++] = mca->qp; + if (n == nmax) + break; + } + spin_unlock_bh(&rxe->mcg_lock); + nmax = n; + kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); - /* validate qp for incoming packet */ + for (n = 0; n < nmax; n++) { + qp = qp_array[n]; + + /* since this is an unreliable transport if + * one of the qp's fails to pass these checks + * just don't forward a packet and continue + * on to the other qp's. If there aren't any + * drop the skb + */ err = check_type_state(rxe, pkt, qp); - if (err) + if (err) { + rxe_drop_ref(qp); + if (n == nmax - 1) + goto err_free; continue; + } err = check_keys(rxe, pkt, bth_qpn(pkt), qp); - if (err) + if (err) { + rxe_drop_ref(qp); + if (n == nmax - 1) + goto err_free; continue; + } - /* for all but the last QP create a new clone of the - * skb and pass to the QP. Pass the original skb to - * the last QP in the list. + /* for all but the last qp create a new copy(clone) + * of the skb and pass to the qp. Pass the original + * skb to the last qp in the list unless it failed + * checks above */ - if (mca->qp_list.next != &mcg->qp_list) { - struct sk_buff *cskb; - struct rxe_pkt_info *cpkt; - - cskb = skb_clone(skb, GFP_ATOMIC); - if (unlikely(!cskb)) + if (n < nmax - 1) { + skb_copy = skb_clone(skb, GFP_KERNEL); + if (unlikely(!skb_copy)) { + rxe_drop_ref(qp); continue; + } if (WARN_ON(!ib_device_try_get(&rxe->ib_dev))) { - kfree_skb(cskb); - break; + kfree_skb(skb_copy); + rxe_drop_ref(qp); + continue; } - cpkt = SKB_TO_PKT(cskb); - cpkt->qp = qp; - rxe_add_ref(qp); - rxe_rcv_pkt(cpkt, cskb); + pkt_copy = SKB_TO_PKT(skb_copy); + pkt_copy->qp = qp; + rxe_rcv_pkt(pkt_copy, skb_copy); } else { pkt->qp = qp; - rxe_add_ref(qp); rxe_rcv_pkt(pkt, skb); - skb = NULL; /* mark consumed */ } } - spin_unlock_bh(&rxe->mcg_lock); - - kref_put(&mcg->ref_cnt, rxe_cleanup_mcg); - - if (likely(!skb)) - return; - - /* This only occurs if one of the checks fails on the last - * QP in the list above - */ + kfree(qp_array); + return; -drop: +err_free: + kfree(qp_array); +err_drop: kfree_skb(skb); ib_device_put(&rxe->ib_dev); } From patchwork Tue Feb 8 21:16:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12739449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53BF9C433EF for ; Tue, 8 Feb 2022 22:24:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387548AbiBHWYy (ORCPT ); Tue, 8 Feb 2022 17:24:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386888AbiBHVR0 (ORCPT ); Tue, 8 Feb 2022 16:17:26 -0500 Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B226C0612BE for ; Tue, 8 Feb 2022 13:17:25 -0800 (PST) Received: by mail-oo1-xc34.google.com with SMTP id r15-20020a4ae5cf000000b002edba1d3349so189465oov.3 for ; Tue, 08 Feb 2022 13:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=04OBAbMZdx8m+LUHoCK8AJ7Vdo9yDfZsBItW/wXTXIA=; b=Y00SPhT9fdH5mPtMqsTpKlg8nluFKCJ3eosVeKArhbKBSpIgnh4AI8HSj48I1rKFr/ MdCEG/E0FHlE5XLXGh1qeagtaKtFK5kybfnk2hlqZtyesnd0rtUohnRKZBfRizio0U8A BfIZXxGy/W/hKLnVGv5eCN993t89mD8gmWwWhdqu14H9bSKvbzDBMSqv8EbNUtlJi00g jNi/QHJWNfUaMCfj2/V0hxhSx1NL2c5qwLah9MVoKRlwqy9a/JuzOK7XmPRLoJwYP5fH q9DCHKDmcxar7SFM6UQwOnOC1gjyANn5G+nQW/ebxOXFz+nzvECG9npfJUXOxlHTGuMI 4PGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=04OBAbMZdx8m+LUHoCK8AJ7Vdo9yDfZsBItW/wXTXIA=; b=zeuikrqjQKqkLm4zdIvtZyLfF7lhTrsG3ZJSER8NekjQTjFonYRWDdijt84k4MAMRX tXq7jp3NEexryDYlcLzY9WxTNBnhsxQhA+EEobsXbwyJD8GVXcjT2VL5t2n6NX/OZxHD kfZskUjE3mrAKFSxXZs7uzI1uGeDPpuJbaBimGccZslcNUtengbX4D+BVBX6fXKrBjK+ amb+p1zfbyC23trPOD1G32tngJS99d1fn2ALCvzFg55Ld/T4hjdR9zVAi3uP3HNu3dRy gE8t0YiNqP5m7j2jMWvLfPnlgP+PSkTkXFHEaWg+7NxuliuN/iJgacXPO/f7W/PvdiXO QU+Q== X-Gm-Message-State: AOAM532ekqF6W6fcBv3L7k7yd5VpgklCmIOdcZTXh+W9u/J1njDHVxM3 QLhTymqc332/oGGwg/nNEMs= X-Google-Smtp-Source: ABdhPJyaIih8im86GsDGoIT6a//gvhOZxPLCxt9ZFbuWbiQANZT4gwDqvaqh95eXOs505YZ2gAtlNw== X-Received: by 2002:a05:6870:111a:: with SMTP id 26mr705334oaf.142.1644355044747; Tue, 08 Feb 2022 13:17:24 -0800 (PST) Received: from ubuntu-21.tx.rr.com (2603-8081-140c-1a00-2501-ba3f-d39d-75da.res6.spectrum.com. [2603:8081:140c:1a00:2501:ba3f:d39d:75da]) by smtp.googlemail.com with ESMTPSA id bh7sm2145462oib.6.2022.02.08.13.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Feb 2022 13:17:24 -0800 (PST) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v11 11/11] RDMA/rxe: Convert mca read locking to RCU Date: Tue, 8 Feb 2022 15:16:45 -0600 Message-Id: <20220208211644.123457-12-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220208211644.123457-1-rpearsonhpe@gmail.com> References: <20220208211644.123457-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Replace spinlock with rcu read locks for read side operations on mca in rxe_recv.c and rxe_mcast.c. Use rcu list extensions on write side operations and use spinlock to separate write threads. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 73 +++++++++++++++------------ drivers/infiniband/sw/rxe/rxe_recv.c | 6 +-- 2 files changed, 45 insertions(+), 34 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index b66839276aa6..19cfa06c62ec 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -17,6 +17,12 @@ * mca is created. It holds a pointer to the qp and is added to a list * of qp's that are attached to the mcg. The qp_list is used to replicate * mcast packets in the rxe receive path. + * + * The highest performance operations are mca list traversal when + * processing incoming multicast packets which need to be fanned out + * to the attached qp's. This list is protected by RCU locking for read + * operations and a spinlock in the rxe_dev struct for write operations. + * The red-black tree is protected by the same spinlock. */ #include "rxe.h" @@ -284,7 +290,7 @@ static void rxe_destroy_mcg(struct rxe_mcg *mcg) } /** - * __rxe_init_mca - initialize a new mca holding lock + * __rxe_init_mca_rcu - initialize a new mca holding lock * @qp: qp object * @mcg: mcg object * @mca: empty space for new mca @@ -294,8 +300,8 @@ static void rxe_destroy_mcg(struct rxe_mcg *mcg) * * Returns: 0 on success else an error */ -static int __rxe_init_mca(struct rxe_qp *qp, struct rxe_mcg *mcg, - struct rxe_mca *mca) +static int __rxe_init_mca_rcu(struct rxe_qp *qp, struct rxe_mcg *mcg, + struct rxe_mca *mca) { struct rxe_dev *rxe = to_rdev(qp->ibqp.device); int n; @@ -318,7 +324,7 @@ static int __rxe_init_mca(struct rxe_qp *qp, struct rxe_mcg *mcg, rxe_add_ref(qp); mca->qp = qp; - list_add_tail(&mca->qp_list, &mcg->qp_list); + list_add_tail_rcu(&mca->qp_list, &mcg->qp_list); return 0; } @@ -338,14 +344,14 @@ static int rxe_attach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) int err; /* check to see if the qp is already a member of the group */ - spin_lock_bh(&rxe->mcg_lock); - list_for_each_entry(mca, &mcg->qp_list, qp_list) { + rcu_read_lock(); + list_for_each_entry_rcu(mca, &mcg->qp_list, qp_list) { if (mca->qp == qp) { - spin_unlock_bh(&rxe->mcg_lock); + rcu_read_unlock(); return 0; } } - spin_unlock_bh(&rxe->mcg_lock); + rcu_read_unlock(); /* speculative alloc new mca without using GFP_ATOMIC */ mca = kzalloc(sizeof(*mca), GFP_KERNEL); @@ -362,7 +368,7 @@ static int rxe_attach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) } } - err = __rxe_init_mca(qp, mcg, mca); + err = __rxe_init_mca_rcu(qp, mcg, mca); if (err) kfree(mca); out: @@ -371,22 +377,22 @@ static int rxe_attach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) } /** - * __rxe_cleanup_mca - cleanup mca object holding lock + * __rxe_cleanup_mca_rcu - cleanup mca object holding lock * @mca: mca object * @mcg: mcg object * * Context: caller must hold a reference to mcg and rxe->mcg_lock */ -static void __rxe_cleanup_mca(struct rxe_mca *mca, struct rxe_mcg *mcg) +static void __rxe_cleanup_mca_rcu(struct rxe_mca *mca, struct rxe_mcg *mcg) { - list_del(&mca->qp_list); + list_del_rcu(&mca->qp_list); atomic_dec(&mcg->qp_num); atomic_dec(&mcg->rxe->mcg_attach); atomic_dec(&mca->qp->mcg_num); rxe_drop_ref(mca->qp); - kfree(mca); + kfree_rcu(mca); } /** @@ -399,30 +405,35 @@ static void __rxe_cleanup_mca(struct rxe_mca *mca, struct rxe_mcg *mcg) static int rxe_detach_mcg(struct rxe_mcg *mcg, struct rxe_qp *qp) { struct rxe_dev *rxe = mcg->rxe; - struct rxe_mca *mca, *tmp; + struct rxe_mca *mca; + int ret; spin_lock_bh(&rxe->mcg_lock); - list_for_each_entry_safe(mca, tmp, &mcg->qp_list, qp_list) { - if (mca->qp == qp) { - __rxe_cleanup_mca(mca, mcg); - - /* if the number of qp's attached to the - * mcast group falls to zero go ahead and - * tear it down. This will not free the - * object since we are still holding a ref - * from the caller - */ - if (atomic_read(&mcg->qp_num) <= 0) - __rxe_destroy_mcg(mcg); - - spin_unlock_bh(&rxe->mcg_lock); - return 0; - } + list_for_each_entry_rcu(mca, &mcg->qp_list, qp_list) { + if (mca->qp == qp) + goto found; } /* we didn't find the qp on the list */ + ret = -EINVAL; + goto done; + +found: + __rxe_cleanup_mca_rcu(mca, mcg); + + /* if the number of qp's attached to the + * mcast group falls to zero go ahead and + * tear it down. This will not free the + * object since we are still holding a ref + * from the caller + */ + if (atomic_read(&mcg->qp_num) <= 0) + __rxe_destroy_mcg(mcg); + + ret = 0; +done: spin_unlock_bh(&rxe->mcg_lock); - return -EINVAL; + return ret; } /** diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index 9b21cbb22602..c2cab85c6576 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -265,15 +265,15 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb) qp_array = kmalloc_array(nmax, sizeof(qp), GFP_KERNEL); n = 0; - spin_lock_bh(&rxe->mcg_lock); - list_for_each_entry(mca, &mcg->qp_list, qp_list) { + rcu_read_lock(); + list_for_each_entry_rcu(mca, &mcg->qp_list, qp_list) { /* protect the qp pointers in the list */ rxe_add_ref(mca->qp); qp_array[n++] = mca->qp; if (n == nmax) break; } - spin_unlock_bh(&rxe->mcg_lock); + rcu_read_unlock(); nmax = n; kref_put(&mcg->ref_cnt, rxe_cleanup_mcg);