From patchwork Fri Jul 21 20:50:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322522 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B946C001DE for ; Fri, 21 Jul 2023 20:51:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230029AbjGUUvJ (ORCPT ); Fri, 21 Jul 2023 16:51:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjGUUvI (ORCPT ); Fri, 21 Jul 2023 16:51:08 -0400 Received: from mail-oo1-xc2e.google.com (mail-oo1-xc2e.google.com [IPv6:2607:f8b0:4864:20::c2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50C9E30EA for ; Fri, 21 Jul 2023 13:51:07 -0700 (PDT) Received: by mail-oo1-xc2e.google.com with SMTP id 006d021491bc7-55b8f1c930eso1498669eaf.3 for ; Fri, 21 Jul 2023 13:51:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972666; x=1690577466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/JUn5TGcUWqYXziIB1X+oJSW2dprGCzLxbWBVFHQeq4=; b=jVXmpr0DpdHp4gKarA5ejGnFwrs1LYpP3kRy3Raf23WHaZJF+Ej6pGRlSfM53MhI5/ lG3QicstHRJNOz2WWkQOplMb8RJKGs/ZPdJfl6iFKGvrB8G5Cc3Uxe0zrEi7wLN3xsjQ YEI0nDWCPlYwsWlc2Ci0OWz5XlMN2r/dSlRHvOYNFqVcRJt0nCNzr0kKk6gjU8Mht2iq yUPQtiq6046QSB5wLfdhV4Wxmj69/XpfqfFE+lc1JJGYEvkn8haW/+BHDkPT3G5NEDYU 9mnYETagPWVhi3tMNM9TfH7BziDrctqGxZpFO/DRIfKRDMqNWzjfnbMB75f1AUseWxIO wNAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972666; x=1690577466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/JUn5TGcUWqYXziIB1X+oJSW2dprGCzLxbWBVFHQeq4=; b=dhoahdiC3JBVhVLHJ2I+T7ea4qQRqk5NNyGLv5TMURzWjN5++4dZwCQUw14Bop079b eoQhH77qatt0pLWGt8t7mnVQV0AHm5JrVG7+4+p9g5bj1d+5Z0wkl6NbnyS5eCYq9oCF hpjwVgzx7Nmi8q3+OH6yCcq90c4ERK+JMyGa7DjUap9rEexshmFVQZs8hGSYcuXbvU5Z CX0FRvTzCVebyFPkr9J7JDsXNn2GS4GuTC/HEsreNdZUrfSy0tvN/bBErSSmkeu6xPSu u/kurJjvHQlyCf+9DNJJrJjJXhjC66DYFvKJcKkX61JD6LkwDwsaSvxSKv0FGEK/hs7N C/4Q== X-Gm-Message-State: ABy/qLaEgUa7ppEgpIryIDlb6Sy4G0rtef69c8jlBi1yTVdpAfYNU12p 09WN6qwL9wBrmLEKCmJmO1E= X-Google-Smtp-Source: APBJJlF3E5XsHan7gpa8hQEAocf9wNFNCXNp3qgHkWGgUZXQSFfo4MHnQVuxIQ3pVenoumDNbrIxvg== X-Received: by 2002:a05:6808:2005:b0:3a4:11a1:4cd8 with SMTP id q5-20020a056808200500b003a411a14cd8mr3998076oiw.4.1689972666598; Fri, 21 Jul 2023 13:51:06 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:06 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 1/9] RDMA/rxe: Fix handling sleepable in rxe_pool.c Date: Fri, 21 Jul 2023 15:50:14 -0500 Message-Id: <20230721205021.5394-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Since the AH creation and destroy verbs APIs can be called in interrupt context it is necessary to not sleep in these cases. This was partially dealt with in previous fixes but some cases remain where the code could still potentially sleep. This patch fixes this by extending the __rxe_finalize() call to have a sleepable parameter and using this in the AH verbs calls. It also fixes one call to rxe_cleanup which did not use the sleepable API. Fixes: 215d0a755e1b ("RDMA/rxe: Stop lookup of partially built objects") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 5 +++-- drivers/infiniband/sw/rxe/rxe_pool.h | 5 +++-- drivers/infiniband/sw/rxe/rxe_verbs.c | 11 ++++++----- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 6215c6de3a84..de0043b6d3f3 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -247,10 +247,11 @@ int __rxe_put(struct rxe_pool_elem *elem) return kref_put(&elem->ref_cnt, rxe_elem_release); } -void __rxe_finalize(struct rxe_pool_elem *elem) +void __rxe_finalize(struct rxe_pool_elem *elem, bool sleepable) { void *xa_ret; + gfp_t gfp_flags = sleepable ? GFP_KERNEL : GFP_ATOMIC; - xa_ret = xa_store(&elem->pool->xa, elem->index, elem, GFP_KERNEL); + xa_ret = xa_store(&elem->pool->xa, elem->index, elem, gfp_flags); WARN_ON(xa_err(xa_ret)); } diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index ef2f6f88e254..d764c51ed6f7 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -77,7 +77,8 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable); #define rxe_read(obj) kref_read(&(obj)->elem.ref_cnt) -void __rxe_finalize(struct rxe_pool_elem *elem); -#define rxe_finalize(obj) __rxe_finalize(&(obj)->elem) +void __rxe_finalize(struct rxe_pool_elem *elem, bool sleepable); +#define rxe_finalize(obj) __rxe_finalize(&(obj)->elem, true) +#define rxe_finalize_ah(obj, sleepable) __rxe_finalize(&(obj)->elem, sleepable) #endif /* RXE_POOL_H */ diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index b899924b81eb..5e93dbac17b3 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -265,6 +265,7 @@ static int rxe_create_ah(struct ib_ah *ibah, struct rxe_dev *rxe = to_rdev(ibah->device); struct rxe_ah *ah = to_rah(ibah); struct rxe_create_ah_resp __user *uresp = NULL; + bool sleepable = init_attr->flags & RDMA_CREATE_AH_SLEEPABLE; int err, cleanup_err; if (udata) { @@ -276,8 +277,7 @@ static int rxe_create_ah(struct ib_ah *ibah, ah->is_user = false; } - err = rxe_add_to_pool_ah(&rxe->ah_pool, ah, - init_attr->flags & RDMA_CREATE_AH_SLEEPABLE); + err = rxe_add_to_pool_ah(&rxe->ah_pool, ah, sleepable); if (err) { rxe_dbg_dev(rxe, "unable to create ah"); goto err_out; @@ -307,12 +307,12 @@ static int rxe_create_ah(struct ib_ah *ibah, } rxe_init_av(init_attr->ah_attr, &ah->av); - rxe_finalize(ah); + rxe_finalize_ah(ah, sleepable); return 0; err_cleanup: - cleanup_err = rxe_cleanup(ah); + cleanup_err = rxe_cleanup_ah(ah, sleepable); if (cleanup_err) rxe_err_ah(ah, "cleanup failed, err = %d", cleanup_err); err_out: @@ -354,9 +354,10 @@ static int rxe_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *attr) static int rxe_destroy_ah(struct ib_ah *ibah, u32 flags) { struct rxe_ah *ah = to_rah(ibah); + bool sleepable = flags & RDMA_DESTROY_AH_SLEEPABLE; int err; - err = rxe_cleanup_ah(ah, flags & RDMA_DESTROY_AH_SLEEPABLE); + err = rxe_cleanup_ah(ah, sleepable); if (err) rxe_err_ah(ah, "cleanup failed, err = %d", err); From patchwork Fri Jul 21 20:50:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322523 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CDA5C001DC for ; Fri, 21 Jul 2023 20:51:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229529AbjGUUvK (ORCPT ); Fri, 21 Jul 2023 16:51:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjGUUvJ (ORCPT ); Fri, 21 Jul 2023 16:51:09 -0400 Received: from mail-oo1-xc2a.google.com (mail-oo1-xc2a.google.com [IPv6:2607:f8b0:4864:20::c2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EE2730E2 for ; Fri, 21 Jul 2023 13:51:08 -0700 (PDT) Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-5636426c1b3so1502260eaf.1 for ; Fri, 21 Jul 2023 13:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972667; x=1690577467; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2d0ewW30mk8n21Bpio/h6Obd9GorFFkBE725pzkhJz4=; b=dgAnlFAnmhKFDDirHHb5N5O8gKE6uqKNnKH4NVkt8AF7ldqUC83lnW79OlOkbIr/M4 TwiEIZv3FbDuKJ+kVAKLSp6wVkD1+a73UwUmmgQCrhlB1+AwRz6moCoudMlydwH0+20R XlvyRPeis4qsye4Bm5rRbmyuG5t0L2jH2/7dyy5QLgcilcacIf02P0iBGEUjYDVhaFx3 eyw+nE5b48mkN874KxaqMut9gilf5050yP50xUh3dGjXYfUhNiKqUzfR1tpK/zRSMCwC K3ID4fThACVkVdJWt9Q5l1wMFvzH0w1BDBsFbEBiGSNpCthoEBgD8Pn45J7Iynix/2SR 4KmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972667; x=1690577467; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2d0ewW30mk8n21Bpio/h6Obd9GorFFkBE725pzkhJz4=; b=Lcs4Uj932rbBryZAnVn8JLCakwVfcz3TM/c94XWD7Gy8aI5atP9lM7RlFMhpEWwiL4 zR1b5kUqVQW+TOJILASIHinCqHvFtxcH70NZCT3KjmkI5ky+CfYebc2jmyS/TcfVv3oj 2oXKfNikGpPaDZAMVYHiNXgA/TEJ0fqTv1FoqPyY+q9FSCnSpbbglGjK/Ji1v1KEVnjl t5XEPPnuWeHEyOuVd5IyndwxZtYnUa55GOkZg8ns/XW+8ZVvVjzEhIbciyq++wrRxjZR af3/rucngcCdpt37CZ0anNjt85MC1Aby5vVsOw465jovTomsvnGqU0XXJ6DbbZ0XSoZF iBig== X-Gm-Message-State: ABy/qLYn4rcfWNHNjl0hZ6aufZyKlj04U9M9FPnC6b8F6UwQNtu9pqjw UQnBPozjP7ZWaVnB/7svr3mpwfwNQXs= X-Google-Smtp-Source: APBJJlEdFS/vyyjyLGLN1SdYiLzYowT9yfcbb21NAe6hQYiwS4o+mcjeGNESVsnYew8tTU1QWyoGZg== X-Received: by 2002:a54:4883:0:b0:3a3:47c5:1de3 with SMTP id r3-20020a544883000000b003a347c51de3mr3110708oic.49.1689972667510; Fri, 21 Jul 2023 13:51:07 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:07 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 2/9] RDMA/rxe: Fix xarray locking in rxe_pool.c Date: Fri, 21 Jul 2023 15:50:15 -0500 Message-Id: <20230721205021.5394-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The kernel rdma verbs API calls are not documented to require that the caller be in process context and it is well known that the address handle calls sometimes are in interrupt context while the other ones are in process context. If we replace the xarray spin_lock in the rxe_pool APIs by irqsave spin_locks we make most of the verbs API calls safe in any context. This patch replaces the xa locks with xa_lock_irqsave. Fixes: 3225717f6dfa ("RDMA/rxe: Replace red-black trees by xarrays") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index de0043b6d3f3..b88492f5f300 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -119,8 +119,10 @@ void rxe_pool_cleanup(struct rxe_pool *pool) int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem, bool sleepable) { - int err; + struct xarray *xa = &pool->xa; + unsigned long flags; gfp_t gfp_flags; + int err; if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto err_cnt; @@ -138,8 +140,10 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem, if (sleepable) might_sleep(); - err = xa_alloc_cyclic(&pool->xa, &elem->index, NULL, pool->limit, + xa_lock_irqsave(xa, flags); + err = __xa_alloc_cyclic(xa, &elem->index, NULL, pool->limit, &pool->next, gfp_flags); + xa_unlock_irqrestore(xa, flags); if (err < 0) goto err_cnt; @@ -179,6 +183,7 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) struct rxe_pool *pool = elem->pool; struct xarray *xa = &pool->xa; static int timeout = RXE_POOL_TIMEOUT; + unsigned long flags; int ret, err = 0; void *xa_ret; @@ -188,7 +193,9 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) /* erase xarray entry to prevent looking up * the pool elem from its index */ - xa_ret = xa_erase(xa, elem->index); + xa_lock_irqsave(xa, flags); + xa_ret = __xa_erase(xa, elem->index); + xa_unlock_irqrestore(xa, flags); WARN_ON(xa_err(xa_ret)); /* if this is the last call to rxe_put complete the @@ -249,9 +256,13 @@ int __rxe_put(struct rxe_pool_elem *elem) void __rxe_finalize(struct rxe_pool_elem *elem, bool sleepable) { - void *xa_ret; gfp_t gfp_flags = sleepable ? GFP_KERNEL : GFP_ATOMIC; + struct xarray *xa = &elem->pool->xa; + unsigned long flags; + void *xa_ret; - xa_ret = xa_store(&elem->pool->xa, elem->index, elem, gfp_flags); + xa_lock_irqsave(xa, flags); + xa_ret = __xa_store(xa, elem->index, elem, gfp_flags); + xa_unlock_irqrestore(xa, flags); WARN_ON(xa_err(xa_ret)); } From patchwork Fri Jul 21 20:50:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322524 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 110E3EB64DD for ; Fri, 21 Jul 2023 20:51:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230264AbjGUUvL (ORCPT ); Fri, 21 Jul 2023 16:51:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230127AbjGUUvK (ORCPT ); Fri, 21 Jul 2023 16:51:10 -0400 Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5583C30EA for ; Fri, 21 Jul 2023 13:51:09 -0700 (PDT) Received: by mail-oi1-x22b.google.com with SMTP id 5614622812f47-3a3b7f992e7so1617741b6e.2 for ; Fri, 21 Jul 2023 13:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972668; x=1690577468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=piWlo3/tFoxJPMOCyPDxN1hN4WpGqotBjw4+5hWeBIw=; b=N3ouiJxZNu+IYfsBXrbv0SITQKyVAEaJIUJsryFcFVTJ74dY8MOeJ6VoPYPROdY5Zx nMb5ThWtd8HCUGQqy0k1o7BAwujDEmpjiDdTOFeetp06VFIEIeEtDKtrw7QBWwvbcsRT Cboaf2kB3NCWHdtl/oMP58rj5ONGUNEHAAY7k/ZxB/yrXJss+dFUTy1GHoybK/sl8CdR alZbyJyTx4QfC+4BjFGs8VHGEv7dKAWZpFrZmgMthtWOsfOKXo96jPonZSLJH7+m8+7U aelLApWDkMTrC20M+YbeSfUKlbgZxU6Ax8kg0KZsU5Va3w/zQQr+VZB0R/42oMOq7/fW mhbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972668; x=1690577468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=piWlo3/tFoxJPMOCyPDxN1hN4WpGqotBjw4+5hWeBIw=; b=VanE+ETTrtr2lXlcTYWYRq7aSpoymnLp6qmY/xFcq70dkxApIlPiALp8FEmxoHa/OZ vNlJXyyi1OIMsXgqeSXBd+hY3t66BAZ5AMySqHihVvITSgILeyUJ/VFg5ZqCX5S6rF7g 0ALQUvJF1GN0h1tqMN7QqeFcv5NkZCiZGwG+EfqaIpYRicTqo7KzTDIgRD78sAPvjwB3 zftnB41zkSldckxJ//ni1hujv4r28tnTNEvwhUgnH7/7C9JjC8aWE46t8hpoD4eAfJH9 Yskart6J52O2TDdNI44Z4lQL1RGngl6ULES13055ehHJC4qS8WC17ZuTVLf6ij2jGejj bQgg== X-Gm-Message-State: ABy/qLYhsKrXriWGaBjDTuDWJ/dzKpWwD6duD99WFPaG5z019y+MAnAB lpG5cmuzkt5+X7noOmaL7sI= X-Google-Smtp-Source: APBJJlGAsh/ktYStvvs8T3mEVTy2DB3PS4cqEFNhGyX/xdNdr7iigryVn4p0qMJb7Yi1FG6otRPQZA== X-Received: by 2002:a54:4487:0:b0:3a4:8a13:9891 with SMTP id v7-20020a544487000000b003a48a139891mr3122256oiv.16.1689972668506; Fri, 21 Jul 2023 13:51:08 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:08 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 3/9] RDMA/rxe: Fix freeing busy objects Date: Fri, 21 Jul 2023 15:50:16 -0500 Message-Id: <20230721205021.5394-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Currently the rxe driver calls wait_for_completion_timeout() in rxe_complete() to wait for the references to the object to be freed before final cleanup and returning to the rdma-core destroy verb. If this does not happen within the timeout interval it prints a WARN_ON and returns to the 'destroy' verb caller without any indication that there was an error. This is incorrect. A heavily loaded system can take an arbitrarily long time to complete the work needed before freeing all the references with no guarantees of performance within a specific time. Another frequent cause of these timeouts is due to ref counting bugs introduced by changes in the driver so it is helpful to report the timeouts if they occur. This patch changes the type of the rxe_cleanup() subroutine to void and fixes calls to reflect this API change in rxe_verbs.c. This is better aligned with the code in rdma-core which sometimes fails to check the return value of destroy verb calls assuming they will always succeed. Specifically this is the case for kernel qp's. Not able to return an error, this patch puts the completion timeout or busy wait code into a subroutine called wait_until_done(). And places the call in a loop with a 10 second timeout that issues a WARN_ON each pass through the loop. This is slow enough to not overload the logs. Fixes: 215d0a755e1b ("RDMA/rxe: Stop lookup of partially built objects") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 39 ++++++------- drivers/infiniband/sw/rxe/rxe_pool.h | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 79 +++++++-------------------- 3 files changed, 38 insertions(+), 82 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index b88492f5f300..9877a798258a 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -6,7 +6,7 @@ #include "rxe.h" -#define RXE_POOL_TIMEOUT (200) +#define RXE_POOL_TIMEOUT (10000) /* 10 seconds */ #define RXE_POOL_ALIGN (16) static const struct rxe_type_info { @@ -175,16 +175,17 @@ static void rxe_elem_release(struct kref *kref) { struct rxe_pool_elem *elem = container_of(kref, typeof(*elem), ref_cnt); - complete(&elem->complete); + complete_all(&elem->complete); } -int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) +void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) { struct rxe_pool *pool = elem->pool; struct xarray *xa = &pool->xa; - static int timeout = RXE_POOL_TIMEOUT; + int timeout = RXE_POOL_TIMEOUT; + unsigned long until; unsigned long flags; - int ret, err = 0; + int ret; void *xa_ret; if (sleepable) @@ -209,39 +210,31 @@ int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) * return to rdma-core */ if (sleepable) { - if (!completion_done(&elem->complete) && timeout) { + while (!completion_done(&elem->complete) && timeout) { ret = wait_for_completion_timeout(&elem->complete, timeout); - - /* Shouldn't happen. There are still references to - * the object but, rather than deadlock, free the - * object or pass back to rdma-core. - */ - if (WARN_ON(!ret)) - err = -EINVAL; + WARN_ON(!ret); } } else { - unsigned long until = jiffies + timeout; - /* AH objects are unique in that the destroy_ah verb * can be called in atomic context. This delay * replaces the wait_for_completion call above * when the destroy_ah call is not sleepable */ - while (!completion_done(&elem->complete) && - time_before(jiffies, until)) - mdelay(1); - - if (WARN_ON(!completion_done(&elem->complete))) - err = -EINVAL; + while (!completion_done(&elem->complete) && timeout) { + until = jiffies + timeout; + while (!completion_done(&elem->complete) && + time_before(jiffies, until)) { + mdelay(10); + } + WARN_ON(!completion_done(&elem->complete)); + } } if (pool->cleanup) pool->cleanup(elem); atomic_dec(&pool->num_elem); - - return err; } int __rxe_get(struct rxe_pool_elem *elem) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index d764c51ed6f7..efef4b05d1ed 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -71,7 +71,7 @@ int __rxe_get(struct rxe_pool_elem *elem); int __rxe_put(struct rxe_pool_elem *elem); #define rxe_put(obj) __rxe_put(&(obj)->elem) -int __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable); +void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable); #define rxe_cleanup(obj) __rxe_cleanup(&(obj)->elem, true) #define rxe_cleanup_ah(obj, sleepable) __rxe_cleanup(&(obj)->elem, sleepable) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 5e93dbac17b3..67995c259916 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -218,11 +218,8 @@ static int rxe_alloc_ucontext(struct ib_ucontext *ibuc, struct ib_udata *udata) static void rxe_dealloc_ucontext(struct ib_ucontext *ibuc) { struct rxe_ucontext *uc = to_ruc(ibuc); - int err; - err = rxe_cleanup(uc); - if (err) - rxe_err_uc(uc, "cleanup failed, err = %d", err); + rxe_cleanup(uc); } /* pd */ @@ -248,11 +245,8 @@ static int rxe_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata) static int rxe_dealloc_pd(struct ib_pd *ibpd, struct ib_udata *udata) { struct rxe_pd *pd = to_rpd(ibpd); - int err; - err = rxe_cleanup(pd); - if (err) - rxe_err_pd(pd, "cleanup failed, err = %d", err); + rxe_cleanup(pd); return 0; } @@ -266,7 +260,7 @@ static int rxe_create_ah(struct ib_ah *ibah, struct rxe_ah *ah = to_rah(ibah); struct rxe_create_ah_resp __user *uresp = NULL; bool sleepable = init_attr->flags & RDMA_CREATE_AH_SLEEPABLE; - int err, cleanup_err; + int err; if (udata) { /* test if new user provider */ @@ -312,9 +306,7 @@ static int rxe_create_ah(struct ib_ah *ibah, return 0; err_cleanup: - cleanup_err = rxe_cleanup_ah(ah, sleepable); - if (cleanup_err) - rxe_err_ah(ah, "cleanup failed, err = %d", cleanup_err); + rxe_cleanup_ah(ah, sleepable); err_out: rxe_err_ah(ah, "returned err = %d", err); return err; @@ -355,11 +347,8 @@ static int rxe_destroy_ah(struct ib_ah *ibah, u32 flags) { struct rxe_ah *ah = to_rah(ibah); bool sleepable = flags & RDMA_DESTROY_AH_SLEEPABLE; - int err; - err = rxe_cleanup_ah(ah, sleepable); - if (err) - rxe_err_ah(ah, "cleanup failed, err = %d", err); + rxe_cleanup_ah(ah, sleepable); return 0; } @@ -372,7 +361,7 @@ static int rxe_create_srq(struct ib_srq *ibsrq, struct ib_srq_init_attr *init, struct rxe_pd *pd = to_rpd(ibsrq->pd); struct rxe_srq *srq = to_rsrq(ibsrq); struct rxe_create_srq_resp __user *uresp = NULL; - int err, cleanup_err; + int err; if (udata) { if (udata->outlen < sizeof(*uresp)) { @@ -414,9 +403,7 @@ static int rxe_create_srq(struct ib_srq *ibsrq, struct ib_srq_init_attr *init, return 0; err_cleanup: - cleanup_err = rxe_cleanup(srq); - if (cleanup_err) - rxe_err_srq(srq, "cleanup failed, err = %d", cleanup_err); + rxe_cleanup(srq); err_out: rxe_err_dev(rxe, "returned err = %d", err); return err; @@ -515,11 +502,8 @@ static int rxe_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr, static int rxe_destroy_srq(struct ib_srq *ibsrq, struct ib_udata *udata) { struct rxe_srq *srq = to_rsrq(ibsrq); - int err; - err = rxe_cleanup(srq); - if (err) - rxe_err_srq(srq, "cleanup failed, err = %d", err); + rxe_cleanup(srq); return 0; } @@ -532,7 +516,7 @@ static int rxe_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init, struct rxe_pd *pd = to_rpd(ibqp->pd); struct rxe_qp *qp = to_rqp(ibqp); struct rxe_create_qp_resp __user *uresp = NULL; - int err, cleanup_err; + int err; if (udata) { if (udata->inlen) { @@ -581,9 +565,7 @@ static int rxe_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init, return 0; err_cleanup: - cleanup_err = rxe_cleanup(qp); - if (cleanup_err) - rxe_err_qp(qp, "cleanup failed, err = %d", cleanup_err); + rxe_cleanup(qp); err_out: rxe_err_dev(rxe, "returned err = %d", err); return err; @@ -649,9 +631,7 @@ static int rxe_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) goto err_out; } - err = rxe_cleanup(qp); - if (err) - rxe_err_qp(qp, "cleanup failed, err = %d", err); + rxe_cleanup(qp); return 0; @@ -1062,7 +1042,7 @@ static int rxe_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, struct rxe_dev *rxe = to_rdev(dev); struct rxe_cq *cq = to_rcq(ibcq); struct rxe_create_cq_resp __user *uresp = NULL; - int err, cleanup_err; + int err; if (udata) { if (udata->outlen < sizeof(*uresp)) { @@ -1101,9 +1081,7 @@ static int rxe_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, return 0; err_cleanup: - cleanup_err = rxe_cleanup(cq); - if (cleanup_err) - rxe_err_cq(cq, "cleanup failed, err = %d", cleanup_err); + rxe_cleanup(cq); err_out: rxe_err_dev(rxe, "returned err = %d", err); return err; @@ -1208,9 +1186,7 @@ static int rxe_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata) goto err_out; } - err = rxe_cleanup(cq); - if (err) - rxe_err_cq(cq, "cleanup failed, err = %d", err); + rxe_cleanup(cq); return 0; @@ -1258,7 +1234,7 @@ static struct ib_mr *rxe_reg_user_mr(struct ib_pd *ibpd, u64 start, struct rxe_dev *rxe = to_rdev(ibpd->device); struct rxe_pd *pd = to_rpd(ibpd); struct rxe_mr *mr; - int err, cleanup_err; + int err; if (access & ~RXE_ACCESS_SUPPORTED_MR) { rxe_err_pd(pd, "access = %#x not supported (%#x)", access, @@ -1290,9 +1266,7 @@ static struct ib_mr *rxe_reg_user_mr(struct ib_pd *ibpd, u64 start, return &mr->ibmr; err_cleanup: - cleanup_err = rxe_cleanup(mr); - if (cleanup_err) - rxe_err_mr(mr, "cleanup failed, err = %d", cleanup_err); + rxe_cleanup(mr); err_free: kfree(mr); rxe_err_pd(pd, "returned err = %d", err); @@ -1339,7 +1313,7 @@ static struct ib_mr *rxe_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type, struct rxe_dev *rxe = to_rdev(ibpd->device); struct rxe_pd *pd = to_rpd(ibpd); struct rxe_mr *mr; - int err, cleanup_err; + int err; if (mr_type != IB_MR_TYPE_MEM_REG) { err = -EINVAL; @@ -1370,9 +1344,7 @@ static struct ib_mr *rxe_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type, return &mr->ibmr; err_cleanup: - cleanup_err = rxe_cleanup(mr); - if (cleanup_err) - rxe_err_mr(mr, "cleanup failed, err = %d", err); + rxe_cleanup(mr); err_free: kfree(mr); err_out: @@ -1383,25 +1355,16 @@ static struct ib_mr *rxe_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type, static int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata) { struct rxe_mr *mr = to_rmr(ibmr); - int err, cleanup_err; /* See IBA 10.6.7.2.6 */ if (atomic_read(&mr->num_mw) > 0) { - err = -EINVAL; - rxe_dbg_mr(mr, "mr has mw's bound"); - goto err_out; + rxe_err_mr(mr, "mr has mw's bound"); + return -EINVAL; } - cleanup_err = rxe_cleanup(mr); - if (cleanup_err) - rxe_err_mr(mr, "cleanup failed, err = %d", cleanup_err); - + rxe_cleanup(mr); kfree_rcu(mr, elem.rcu); return 0; - -err_out: - rxe_err_mr(mr, "returned err = %d", err); - return err; } static ssize_t parent_show(struct device *device, From patchwork Fri Jul 21 20:50:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322525 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D63CC001DE for ; Fri, 21 Jul 2023 20:51:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230283AbjGUUvM (ORCPT ); Fri, 21 Jul 2023 16:51:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230268AbjGUUvL (ORCPT ); Fri, 21 Jul 2023 16:51:11 -0400 Received: from mail-oo1-xc33.google.com (mail-oo1-xc33.google.com [IPv6:2607:f8b0:4864:20::c33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4311B30E2 for ; Fri, 21 Jul 2023 13:51:10 -0700 (PDT) Received: by mail-oo1-xc33.google.com with SMTP id 006d021491bc7-563439ea4a2so1503066eaf.0 for ; Fri, 21 Jul 2023 13:51:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972669; x=1690577469; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ab5HfWQqqxPkl4GvltNoLygUb3ZXkv4d8gCaU1bGBsQ=; b=YZPXi5egj7X0sgjP4sHOeKcFh5bM9spTAGMDfhJjL5rtmoKsFgouPLBqJAtp1357L+ PvnOgh2HvVVa83YPoNEZbGieoUGUEss5stWnFZ9Z5tiWaI1MlMILEZ0yrjSHh6T8d0Lk /a30ZMqb3mcjZ8aMsZX/5lTZ9otlfbS71BJdLxPJNO3+Evqr8QFcyr5eT7EDoRb9Ny2J MTMH++p3gE7qNNRQEtCF/Fx0KbmnOP3yC2HDFXRGoNAVDUKYMPBGM5kX/YNNhm1bP3EM JYXjTdf9EW3uCf/YKCFzxWHj1323b9BxP4ddUlLoF5nZS/s0SA/FM2KEYKddR5PxhWtN SIlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972669; x=1690577469; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ab5HfWQqqxPkl4GvltNoLygUb3ZXkv4d8gCaU1bGBsQ=; b=Gq7kvN4b6U5OZ1ERVX1JA5UpjoZwi199wbCd6G+ZH3Va85BUcM04N11O9tGu9txNya RNH13JAABDcSMfz1jOBCZ/qsYF5zVtyFRruKvarx6l/V1wsuw/uSvSKIROZVfjD2UoA6 FpEuVg91lh02yTRB6m29h0gNBTsXFLyJoHUQEh+gUtzKX7sVCcjRZi6W5vAsjwH4Jbhl o0y98BcLwORLbVa67LeXzOOFNUSexL2fLBGJExviAhxyZ2sMsJ+U1hx8bNSCWhwRz0tT fBd2Dtq/73GGDb4WXiBYxg4YqbGyWbI9HbpcCmu8YdUXu2Ri/6+2xjSNGp5EmVluvMQp ewgg== X-Gm-Message-State: ABy/qLZnryBIAwiaW3mIqFmBEGpi1WuEwDPJPcFz1UpzOgekUFfIiVSL 62CnrYleAZ7j5dvSILYXGLY= X-Google-Smtp-Source: APBJJlFIMFxMKhEDOPYe56XzjfBvmq+Zcp4tmsgvXAMqg2DULJOZHfcHblQbbcpuHio9SesCb9K8ug== X-Received: by 2002:aca:2212:0:b0:3a3:7e62:fca2 with SMTP id b18-20020aca2212000000b003a37e62fca2mr3494771oic.0.1689972669420; Fri, 21 Jul 2023 13:51:09 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:09 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 4/9] RDMA/rxe: Fix delayed send packet handling Date: Fri, 21 Jul 2023 15:50:17 -0500 Message-Id: <20230721205021.5394-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org In cable pull testing some NICs can hold a send packet long enough to allow ulp protocol stacks to destroy the qp and the cleanup routines to timeout waiting for all qp references to be released. When the NIC driver finally frees the SKB the qp pointer is no longer valid and causes a seg fault in rxe_skb_tx_dtor(). This patch passes the qp index instead of the qp to the skb destructor callback function. The call back is required to lookup the qp from the index and if it has been destroyed the lookup will return NULL and the qp will not be referenced avoiding the seg fault. Fixes: 8700e3e7c485 ("Soft RoCE driver") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_net.c | 87 ++++++++++++++++++++++------- drivers/infiniband/sw/rxe/rxe_qp.c | 1 - 2 files changed, 67 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index cd59666158b1..10e4a752ff7c 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -131,19 +131,28 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev, return dst; } +static struct rxe_dev *get_rxe_from_skb(struct sk_buff *skb) +{ + struct rxe_dev *rxe; + struct net_device *ndev = skb->dev; + + rxe = rxe_get_dev_from_net(ndev); + if (!rxe && is_vlan_dev(ndev)) + rxe = rxe_get_dev_from_net(vlan_dev_real_dev(ndev)); + + return rxe; +} + static int rxe_udp_encap_recv(struct sock *sk, struct sk_buff *skb) { struct udphdr *udph; struct rxe_dev *rxe; - struct net_device *ndev = skb->dev; struct rxe_pkt_info *pkt = SKB_TO_PKT(skb); /* takes a reference on rxe->ib_dev * drop when skb is freed */ - rxe = rxe_get_dev_from_net(ndev); - if (!rxe && is_vlan_dev(ndev)) - rxe = rxe_get_dev_from_net(vlan_dev_real_dev(ndev)); + rxe = get_rxe_from_skb(skb); if (!rxe) goto drop; @@ -345,46 +354,84 @@ int rxe_prepare(struct rxe_av *av, struct rxe_pkt_info *pkt, static void rxe_skb_tx_dtor(struct sk_buff *skb) { - struct sock *sk = skb->sk; - struct rxe_qp *qp = sk->sk_user_data; - int skb_out = atomic_dec_return(&qp->skb_out); + struct rxe_dev *rxe; + unsigned int index; + struct rxe_qp *qp; + int skb_out; + + /* takes a ref on ib device if success */ + rxe = get_rxe_from_skb(skb); + if (!rxe) + goto out; + + /* recover source qp index from sk->sk_user_data + * free the reference taken in rxe_send + */ + index = (int)(uintptr_t)skb->sk->sk_user_data; + sock_put(skb->sk); + /* lookup qp from index, takes a ref on success */ + qp = rxe_pool_get_index(&rxe->qp_pool, index); + if (!qp) + goto out_put_ibdev; + + skb_out = atomic_dec_return(&qp->skb_out); if (unlikely(qp->need_req_skb && skb_out < RXE_INFLIGHT_SKBS_PER_QP_LOW)) rxe_sched_task(&qp->req.task); rxe_put(qp); +out_put_ibdev: + ib_device_put(&rxe->ib_dev); +out: + return; } static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt) { + struct rxe_qp *qp = pkt->qp; + struct sock *sk; int err; - skb->destructor = rxe_skb_tx_dtor; - skb->sk = pkt->qp->sk->sk; + /* qp can be destroyed while this packet is waiting on + * the tx queue. So need to protect sk. + */ + sk = qp->sk->sk; + sock_hold(sk); + skb->sk = sk; - rxe_get(pkt->qp); atomic_inc(&pkt->qp->skb_out); + sk->sk_user_data = (void *)(long)qp->elem.index; + skb->destructor = rxe_skb_tx_dtor; + if (skb->protocol == htons(ETH_P_IP)) { - err = ip_local_out(dev_net(skb_dst(skb)->dev), skb->sk, skb); + err = ip_local_out(dev_net(skb_dst(skb)->dev), sk, skb); } else if (skb->protocol == htons(ETH_P_IPV6)) { - err = ip6_local_out(dev_net(skb_dst(skb)->dev), skb->sk, skb); + err = ip6_local_out(dev_net(skb_dst(skb)->dev), sk, skb); } else { - rxe_dbg_qp(pkt->qp, "Unknown layer 3 protocol: %d\n", - skb->protocol); - atomic_dec(&pkt->qp->skb_out); - rxe_put(pkt->qp); - kfree_skb(skb); - return -EINVAL; + rxe_dbg_qp(qp, "Unknown layer 3 protocol: %d", + skb->protocol); + err = -EINVAL; + goto err_not_sent; } + /* IP consumed the packet, the destructor will handle cleanup */ if (unlikely(net_xmit_eval(err))) { - rxe_dbg_qp(pkt->qp, "error sending packet: %d\n", err); - return -EAGAIN; + rxe_dbg_qp(qp, "Error sending packet: %d", err); + err = -EAGAIN; + goto err_out; } return 0; + +err_not_sent: + skb->destructor = NULL; + atomic_dec(&pkt->qp->skb_out); + kfree_skb(skb); + sock_put(sk); +err_out: + return err; } /* fix up a send packet to match the packets diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index a569b111a9d2..dcbf71031453 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -194,7 +194,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, err = sock_create_kern(&init_net, AF_INET, SOCK_DGRAM, 0, &qp->sk); if (err < 0) return err; - qp->sk->sk->sk_user_data = qp; /* pick a source UDP port number for this QP based on * the source QPN. this spreads traffic for different QPs From patchwork Fri Jul 21 20:50:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322526 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CB01C001DC for ; Fri, 21 Jul 2023 20:51:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230127AbjGUUvN (ORCPT ); Fri, 21 Jul 2023 16:51:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229699AbjGUUvM (ORCPT ); Fri, 21 Jul 2023 16:51:12 -0400 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0E1A030ED for ; Fri, 21 Jul 2023 13:51:11 -0700 (PDT) Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3a3fbfb616dso1540874b6e.3 for ; Fri, 21 Jul 2023 13:51:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972670; x=1690577470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A7gNvQz3sS+3ucrtBSdSo4m5tG8c2ZAqY/c8seuYI3I=; b=d9OSBN5vetV6sxipO6w5WcSpVWaX9qzw/HadXZAqmWx5Xm18XyOUsmfULcAhUvvptZ L6Grw4jWUTfpQamQOWHbxHcTrauDDgebINt3ZIyHazElcpfqjMtN5L63uLQW7ETMHmsx G6mjBpYJlPhUYUF2v+0GQMgwgNtxKGHa07Vkumfua+NYiMu1zWDa1RZDOUjBKW5rqeEp MexisZTGnvYaaa1wMSH1YBUbP+ApAHLKL3wW2/zRZtkYfaIoB4s5vs3ZyjzOjPJ8ZOLb 6UpkQFUpS8uMeI49ue51MR+GF7meR8KqZzvJFX0ULv/ODi2k7qU0wKazQgqohnJ0msIN EMyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972670; x=1690577470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A7gNvQz3sS+3ucrtBSdSo4m5tG8c2ZAqY/c8seuYI3I=; b=eti+Ks0F8o3q+BSQmnb0l3mSypJPo7PhquBhcw4HWweQvDDOCDGLGvgnd8/SB8V1JG UzxA7Jq7gMvpvs6d2yTUWLMSrDXC37nOUeKy0GqRInVlGuYb1tEbGyZMP8/XhvUOHaqR PpJkAYX3rLuQSJdXVFqUrZsbe+Rokd/lf+C/AfjBdQZRdtSFaVUimyjY1WkIrC5+22wM M5TsvLl4sRhGQixb4zW/FqwEWzfY16To1vTYxV34eJrRywqE8G0EKtq/MLyRhS9RKHZR vG0LFKJlsDFxganBssJ8XdmXaZF3FQ9xH+a2+UbNPzYieH97toMJP+XyGAHpatE2guO3 7Myw== X-Gm-Message-State: ABy/qLZHJzkpMWoCzBTSuDj+XNvm500x6YyogK5+tx+xfqPFojDKKhPN xirkdyjhllNjRrYXltPNpJ0= X-Google-Smtp-Source: APBJJlHQ3XzkvISKXVO66yMSwkVOfCvsG7B+1Z1aoI6MFvNlERRRNU7gsOgDa+eLThFHGQ0mjibfSg== X-Received: by 2002:a05:6808:18a:b0:3a4:1c0b:85d0 with SMTP id w10-20020a056808018a00b003a41c0b85d0mr3085656oic.55.1689972670372; Fri, 21 Jul 2023 13:51:10 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:09 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 5/9] RDMA/rxe: Optimize rxe_init_packet in rxe_net.c Date: Fri, 21 Jul 2023 15:50:18 -0500 Message-Id: <20230721205021.5394-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This patch replaces code that looks up the ndev from the ib_device and port number of a rxe device with the saved value of the ndev used when the rxe device was created. Since the rxe driver does not support multi port operation or path migration, the ndev is constant for the life of the rxe device. The ndev lookup code in rxe_init_packet has a significant performance impact under heavy load and can consume a noticeable fraction of the overall cpu time. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_net.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 10e4a752ff7c..c1b2eaf82334 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -508,14 +508,9 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, { unsigned int hdr_len; struct sk_buff *skb = NULL; - struct net_device *ndev; - const struct ib_gid_attr *attr; + struct net_device *ndev = rxe->ndev; const int port_num = 1; - attr = rdma_get_gid_attr(&rxe->ib_dev, port_num, av->grh.sgid_index); - if (IS_ERR(attr)) - return NULL; - if (av->network_type == RXE_NETWORK_TYPE_IPV4) hdr_len = ETH_HLEN + sizeof(struct udphdr) + sizeof(struct iphdr); @@ -523,25 +518,14 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, hdr_len = ETH_HLEN + sizeof(struct udphdr) + sizeof(struct ipv6hdr); - rcu_read_lock(); - ndev = rdma_read_gid_attr_ndev_rcu(attr); - if (IS_ERR(ndev)) { - rcu_read_unlock(); - goto out; - } - skb = alloc_skb(paylen + hdr_len + LL_RESERVED_SPACE(ndev), - GFP_ATOMIC); - - if (unlikely(!skb)) { - rcu_read_unlock(); + skb = alloc_skb(paylen + hdr_len + LL_RESERVED_SPACE(ndev), GFP_ATOMIC); + if (unlikely(!skb)) goto out; - } skb_reserve(skb, hdr_len + LL_RESERVED_SPACE(ndev)); /* FIXME: hold reference to this netdev until life of this skb. */ - skb->dev = ndev; - rcu_read_unlock(); + skb->dev = ndev; if (av->network_type == RXE_NETWORK_TYPE_IPV4) skb->protocol = htons(ETH_P_IP); @@ -554,7 +538,6 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, pkt->mask |= RXE_GRH_MASK; out: - rdma_put_gid_attr(attr); return skb; } From patchwork Fri Jul 21 20:50:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322527 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBFC9C00528 for ; Fri, 21 Jul 2023 20:51:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230291AbjGUUvO (ORCPT ); Fri, 21 Jul 2023 16:51:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230286AbjGUUvN (ORCPT ); Fri, 21 Jul 2023 16:51:13 -0400 Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E74D30E2 for ; Fri, 21 Jul 2023 13:51:12 -0700 (PDT) Received: by mail-oi1-x22e.google.com with SMTP id 5614622812f47-3a3ad1f39ebso2213344b6e.1 for ; Fri, 21 Jul 2023 13:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972671; x=1690577471; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v6c0b63QWNURv57zuWgFkjRO3WOLWYqhIFu4BCOLKlg=; b=kmnAbSd+vtg0GhXoZcDkvrgx9Vquuva7SErkSDQEzNR22J1OXwub0vNBjVYK41r//T g7kbwjbsiP/yqQubQuvAvZFAFv23LXbCk1DZIMJT8KVFMcmKxaLSCojUF7lyItpQCRjZ AYORi/WQrH/JkWytX8RsyuNEClsLfc9VgbxnPXR0g6tN5SwPyOe1s8eHm6T56zjRVCwz 7AUWUn62ZwfzrV1PTAFmvcuC8Y/82B2M1/hLVHHy3e5xKBuYBJAsvwgEIc3idbG/Yljc tvc2Z9dtnjpRAasmjobVYC8cquqYUBHpkyUYTAulEps21KgTeDEFg9aaBes9uIQwxb1J ckmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972671; x=1690577471; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v6c0b63QWNURv57zuWgFkjRO3WOLWYqhIFu4BCOLKlg=; b=aQUl+06BTtyz+GeOtYD4kHaLKe7LcvfLkM+xFKgx6wKkOcLUnccuyozILM7KMWfAQ7 uX5jPBfbSs9NtHs+LnFfvdN+Lg58ZkytB1OAyNfoyX8hmq/A1cU8NNMhZ7yWBEbHCIvl rMLCbSAp5Fxxc3kwSc3/M4LzLe1SVVp3AxFcQSxsDGIP/k81uu8Ykm4pLBFUfDv70sL5 r2NWNpOXT8eeVtlePqRzeLRDoD8fQfgtvvCyNasbtbu8xzMQOtfi22tKy56o7LvDrjl5 FyOzPVZj41qtMLbLVspYi7FoG796AF47IMZv3bUSGteKR6IIUHZk6W2jEHGV5tvxQm2p sMVw== X-Gm-Message-State: ABy/qLZdod7q7rloseRmrSD6wtVUSQGZUdCQpXpN1YTYWJBKimZIbVri FbZLy+XwTc46kECf1QsO0jk= X-Google-Smtp-Source: APBJJlFM8hJ2k1sj3iyym6LGYueDgUoQJkbGX22oTLLdRpfi/SOd4E4aZ8k9UZUjoRepDhK9ogbwyQ== X-Received: by 2002:a05:6808:2a5b:b0:3a1:e792:86f3 with SMTP id fa27-20020a0568082a5b00b003a1e79286f3mr2110611oib.2.1689972671400; Fri, 21 Jul 2023 13:51:11 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:11 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 6/9] RDMA/rxe: Delete unused field elem->list Date: Fri, 21 Jul 2023 15:50:19 -0500 Message-Id: <20230721205021.5394-7-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org struct rxe_pool_elem field list is not used. This patch removes it. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index efef4b05d1ed..daef1ed72722 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -23,7 +23,6 @@ struct rxe_pool_elem { struct rxe_pool *pool; void *obj; struct kref ref_cnt; - struct list_head list; struct completion complete; struct rcu_head rcu; u32 index; From patchwork Fri Jul 21 20:50:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322529 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C44DEB64DD for ; Fri, 21 Jul 2023 20:51:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229699AbjGUUvR (ORCPT ); Fri, 21 Jul 2023 16:51:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229643AbjGUUvQ (ORCPT ); Fri, 21 Jul 2023 16:51:16 -0400 Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CC26359C for ; Fri, 21 Jul 2023 13:51:13 -0700 (PDT) Received: by mail-oi1-x230.google.com with SMTP id 5614622812f47-3a4062577c0so2216571b6e.0 for ; Fri, 21 Jul 2023 13:51:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972672; x=1690577472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BPT+HSty6dekpK34u9m2eE59lcPueQMgpbY9nuwDHAg=; b=AOlJlS5oKvPecN8vV3Gwx1KcZyEw6ITeQp1xXOHbnJRrMt7/NLmL0a6g80lbdgKOin k8XRuPB2CDGmJMkplsFOKKr3TQSiq1XKwOjZ4YgBzA9Q6rMgWHSQoFj2flOJChA6+aQH jecuVbz7hbVE/fXXnjYp1SX+LHniNKEyA41ET2yc5PJYOhWYCKYDMHXS24/7tQsHK31j BP4BcZ+zUJpnn2EpvrFQoJt7eho5Job27wHQGE6XNpSyJpBbCgykQq2pInM6YjKnydUP nvCyFVy7BZopGs3ycGTMqL3o5EiUrbdo2nPlkpey8NyRLiN5OpNdsWuRgKwjs4dQdDC3 xJvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972672; x=1690577472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BPT+HSty6dekpK34u9m2eE59lcPueQMgpbY9nuwDHAg=; b=QmytG/uArTaQfkFSttrDWwfJcOmIjNazS987/mTdZPivjabZbXr3zC7BG7hk/Aqor1 CE2AOZ3GF2y3YrL/lTvgemnk7gr3CwWldI2CkXF9V0tIUKNetMoI7o1HOyY6AQu9BZ3E epOjtwMq6wIyETZOJ3/tIqfFVHEiBwBPBj5qI1dSijxLcHYC+aW7DJRmljcPVFr031B2 IpMaEppihnYCqCjufzLsrbJeaWznVZnYQ1yUsFnFFv8uSUt6LlYzo0d9qYlzIefPA3XQ 7Ax9SVsuiamZKHat8vePZQ7XS51cHGNd+2Yq7oYoMvQQjiNI3XGlU7ZBtIKtz9L7bR+j pnXg== X-Gm-Message-State: ABy/qLaPYTxbqsu6G3wEy7LAyxKsz/tKW/R+cecRtXYb4UgIiEXz3mrL zBlB23TapYYcsXXlFNgX7CQ= X-Google-Smtp-Source: APBJJlF5jb/1d3s87DTirjoagRUJkAvucNF3bZ8TKQmGQCtOBt7CXjXH3a2A1lp6OFnQO37/bJz1Pg== X-Received: by 2002:a05:6808:2025:b0:3a3:9b4a:3959 with SMTP id q37-20020a056808202500b003a39b4a3959mr3448339oiw.17.1689972672377; Fri, 21 Jul 2023 13:51:12 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:11 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 7/9] RDMA/rxe: Add elem->valid field Date: Fri, 21 Jul 2023 15:50:20 -0500 Message-Id: <20230721205021.5394-8-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Currently the rxe driver stores NULL in the pool xarray to indicate that the pool element should not be looked up from its index. This prevents looping over pool elements during driver cleanup. This patch adds a new valid field to struct rxe_pool_elem as an alternative way to accomplish the same thing. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 32 ++++++++++++---------------- drivers/infiniband/sw/rxe/rxe_pool.h | 1 + 2 files changed, 15 insertions(+), 18 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 9877a798258a..cb812bd969c6 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -141,7 +141,7 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem, if (sleepable) might_sleep(); xa_lock_irqsave(xa, flags); - err = __xa_alloc_cyclic(xa, &elem->index, NULL, pool->limit, + err = __xa_alloc_cyclic(xa, &elem->index, elem, pool->limit, &pool->next, gfp_flags); xa_unlock_irqrestore(xa, flags); if (err < 0) @@ -158,11 +158,14 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) { struct rxe_pool_elem *elem; struct xarray *xa = &pool->xa; + int valid; void *obj; rcu_read_lock(); elem = xa_load(xa, index); - if (elem && kref_get_unless_zero(&elem->ref_cnt)) + /* get current value of elem->valid */ + valid = elem ? smp_load_acquire(&elem->valid) : 0; + if (valid && kref_get_unless_zero(&elem->ref_cnt)) obj = elem->obj; else obj = NULL; @@ -191,13 +194,8 @@ void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) if (sleepable) might_sleep(); - /* erase xarray entry to prevent looking up - * the pool elem from its index - */ - xa_lock_irqsave(xa, flags); - xa_ret = __xa_erase(xa, elem->index); - xa_unlock_irqrestore(xa, flags); - WARN_ON(xa_err(xa_ret)); + /* prevent looking up element from index */ + smp_store_release(&elem->valid, 0); /* if this is the last call to rxe_put complete the * object. It is safe to touch obj->elem after this since @@ -231,6 +229,11 @@ void __rxe_cleanup(struct rxe_pool_elem *elem, bool sleepable) } } + xa_lock_irqsave(xa, flags); + xa_ret = __xa_erase(xa, elem->index); + xa_unlock_irqrestore(xa, flags); + WARN_ON(xa_err(xa_ret)); + if (pool->cleanup) pool->cleanup(elem); @@ -249,13 +252,6 @@ int __rxe_put(struct rxe_pool_elem *elem) void __rxe_finalize(struct rxe_pool_elem *elem, bool sleepable) { - gfp_t gfp_flags = sleepable ? GFP_KERNEL : GFP_ATOMIC; - struct xarray *xa = &elem->pool->xa; - unsigned long flags; - void *xa_ret; - - xa_lock_irqsave(xa, flags); - xa_ret = __xa_store(xa, elem->index, elem, gfp_flags); - xa_unlock_irqrestore(xa, flags); - WARN_ON(xa_err(xa_ret)); + /* allow looking up element from index */ + smp_store_release(&elem->valid, 1); } diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index daef1ed72722..5cca33c5cfb5 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -26,6 +26,7 @@ struct rxe_pool_elem { struct completion complete; struct rcu_head rcu; u32 index; + int valid; }; struct rxe_pool { From patchwork Fri Jul 21 20:50:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322528 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69A62C001DE for ; Fri, 21 Jul 2023 20:51:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230286AbjGUUvQ (ORCPT ); Fri, 21 Jul 2023 16:51:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230268AbjGUUvP (ORCPT ); Fri, 21 Jul 2023 16:51:15 -0400 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E35D2D7B for ; Fri, 21 Jul 2023 13:51:14 -0700 (PDT) Received: by mail-oi1-x234.google.com with SMTP id 5614622812f47-3a412653352so1597673b6e.0 for ; Fri, 21 Jul 2023 13:51:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972673; x=1690577473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=50adx+O5i2zetcnORsbaLKi+BjncFeH1wEkN4/15/TI=; b=WdHLi7ryjjxUAanqqFoDmBn5fI9JswywJm79+cnEGlQaeiwtVcNHg9b6G0mIyCClwy 5ylZ4s+JbCceIdIytK/sohRRDzOEMD1ZdILNgNLPx19UqoGNX3kyVXTF8T4zF+1KYOvr diudTbpR20teRTJ1IBTFprOGLpzc+n9j9t+hrlxmM7aW+ePgODg7pICR5BHJpaLjr4G/ YtOxOe3Ws5cMD295PCAy4+V0WaHF8Ld2S2WzcxeaP0OtAZ5uvfO2vhHdLUsrhZZ/VOw2 gXtTV55UYc5yI9AymHSqteFdGax+2VucyhyLvEST1NLfjGGsLbcePJrr9U1fLaYbW62Z xOFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972673; x=1690577473; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=50adx+O5i2zetcnORsbaLKi+BjncFeH1wEkN4/15/TI=; b=QZL/yJODiWZJBN8JmYhY5W8nw1vJR0p5LJ+mt3Q1nRRfuqv1t3nDnFwSl773ZHNJHc ePc78yqm0rDtu/Z1XqJcanooOgn8ci5YQEGnglFtgDAP9QqjYoui6bIGXEw1rxL2v65h AKD8WMg0Nm8RKGd1oQZrYw6ipwJTuU6FG4JB84GxKraBcDM5EUi41YH3/0+vO5C72TwL +NJbuL+xPLIZpSDX6EDAA5eM22If0fE2lAaC8CX7nTFC0qAX6w2Q8eA/5Vd6XbyFPSuM guowssEtV2pNWeUAwFMFgWgObG1vFkUzhvJ18nTMi6d0EgcXnDTrqagNPcaSvi8AI/S2 vaPQ== X-Gm-Message-State: ABy/qLZAZXuFSrHJNHYRE1vOQ4FBftBvPeTyGkVujtuF202YneDXu5Gv KieJxxMfQgA/FTmgUysnR3I= X-Google-Smtp-Source: APBJJlHSCUSGXMIzJiHreJ9UU28DMpIV/ZspKt8eMM0uJ8FkJec07nguWKQZmw1acFDokgqe1I04hQ== X-Received: by 2002:a05:6808:2a6f:b0:3a4:45b7:ae06 with SMTP id fu15-20020a0568082a6f00b003a445b7ae06mr2852249oib.56.1689972673389; Fri, 21 Jul 2023 13:51:13 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:12 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 8/9] RDMA/rxe: Report leaked objects Date: Fri, 21 Jul 2023 15:50:21 -0500 Message-Id: <20230721205021.5394-9-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This patch gives a more detailed list of objects that are not freed in case of error before the module exits. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index cb812bd969c6..3249c2741491 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -113,7 +113,17 @@ void rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, void rxe_pool_cleanup(struct rxe_pool *pool) { - WARN_ON(!xa_empty(&pool->xa)); + unsigned long index; + struct rxe_pool_elem *elem; + + xa_lock(&pool->xa); + xa_for_each(&pool->xa, index, elem) { + rxe_err_dev(pool->rxe, "%s#%d: Leaked", pool->name, + elem->index); + } + xa_unlock(&pool->xa); + + xa_destroy(&pool->xa); } int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_elem *elem, From patchwork Fri Jul 21 20:50:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13322530 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64C55C001DC for ; Fri, 21 Jul 2023 20:51:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229560AbjGUUvS (ORCPT ); Fri, 21 Jul 2023 16:51:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230289AbjGUUvR (ORCPT ); Fri, 21 Jul 2023 16:51:17 -0400 Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 276A435AC for ; Fri, 21 Jul 2023 13:51:15 -0700 (PDT) Received: by mail-ot1-x331.google.com with SMTP id 46e09a7af769-6b9af1826b6so1958036a34.1 for ; Fri, 21 Jul 2023 13:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689972674; x=1690577474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cGzb7HB3gzwMWGLewlnlIN14EkqKNso8cg2HkdT4ARY=; b=Tn38PM8ExM/ggHuf5stlX9txx/Wwjf34aeZws8nSnOIFIAi/HK4sNsGZWR+75Oce8n 9GYDE//o24Fi6OAo2x8344b2kiuGASp4AI71fH6t/JQNem8RqtLDewmzsVyClV2eryzs upfum2JtIzBJH6uTuuYcP/56QvgqdiM4hlfpz+vchMmRn3oP9UepnzkRsik9gBMChjTi tbafLDGhEDtatbd5Uwi5DXAZT58cgxrf9hVeg7HObylXFcId4HB0C4VyMFW2Cs0KAduB OmqTvvjxVhH4GMrllj6yyJHD+Xf8/jXcbqPpdwNjNHHvh5AtBxMPp5k2AolfJOk+OgJa koQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972674; x=1690577474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cGzb7HB3gzwMWGLewlnlIN14EkqKNso8cg2HkdT4ARY=; b=dR06U9f0S56nJZokKPfbfsXhMbZ+LHgAo5/fJGJhoNUQRJXF2+e4jV3vzbblftg4R7 ARsBl6XA7pW26qSV0RdvezN/HSTC6bHoyFgXczmmwytrmgipmD1XLswLh1AEel9uT23H RjFuvTBoGiWHdcxgXVtxpgCUjKEGZA4OZkBYZ0wsN7OJGfgLYvLJKWUaAp9bx1g7tIGh e+9nyQcUco7vQlE6BPauAeeDXtfc5QZ0H2L1uXVperHm7906H4leAgJL8VrqLLf7C7tE pRAp/ptvsV+ZM3dyvhif/BUfW+SU8FXwhJFiVnyIVaAgw7+On4P2fkyQL2GKhfLd5wW/ I7Cg== X-Gm-Message-State: ABy/qLbKb4AKzdQRyW//NxNksgXt1X6ejsFS6O6bYx8b9Ons8JMdheE5 meRZiP+JLRPhAE30moVmhqc= X-Google-Smtp-Source: APBJJlEg+OgNgCwAuvvLlRL8lzAgXxHh+ZdnMsJxqMiYufsijdziZykJIB4oH5bHYtYdH6sIWd244A== X-Received: by 2002:a05:6808:10c2:b0:398:5d57:3d08 with SMTP id s2-20020a05680810c200b003985d573d08mr4503358ois.37.1689972674264; Fri, 21 Jul 2023 13:51:14 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-3742-d596-b265-a511.res6.spectrum.com. [2603:8081:140c:1a00:3742:d596:b265:a511]) by smtp.gmail.com with ESMTPSA id o188-20020acaf0c5000000b003a375c11aa5sm1909551oih.30.2023.07.21.13.51.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:51:13 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 9/9] RDMA/rxe: Protect pending send packets Date: Fri, 21 Jul 2023 15:50:22 -0500 Message-Id: <20230721205021.5394-10-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230721205021.5394-1-rpearsonhpe@gmail.com> References: <20230721205021.5394-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Network interruptions may cause long delays in the processing of send packets during which time the rxe driver may be unloaded. This will cause seg faults when the packet is ultimately freed as it calls the destructor function in the rxe driver. This has been observed in cable pull fail over fail back testing. This patch takes a reference on the driver to protect the packet from this happening. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe.c | 26 ++++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe.h | 3 +++ drivers/infiniband/sw/rxe/rxe_net.c | 7 +++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe.c b/drivers/infiniband/sw/rxe/rxe.c index 54c723a6edda..6b55c595f8f8 100644 --- a/drivers/infiniband/sw/rxe/rxe.c +++ b/drivers/infiniband/sw/rxe/rxe.c @@ -208,10 +208,33 @@ static struct rdma_link_ops rxe_link_ops = { .newlink = rxe_newlink, }; +static struct rxe_module { + struct kref ref_cnt; + struct completion complete; +} rxe_module; + +static void rxe_module_release(struct kref *kref) +{ + complete(&rxe_module.complete); +} + +int rxe_module_get(void) +{ + return kref_get_unless_zero(&rxe_module.ref_cnt); +} + +int rxe_module_put(void) +{ + return kref_put(&rxe_module.ref_cnt, rxe_module_release); +} + static int __init rxe_module_init(void) { int err; + kref_init(&rxe_module.ref_cnt); + init_completion(&rxe_module.complete); + err = rxe_alloc_wq(); if (err) return err; @@ -229,6 +252,9 @@ static int __init rxe_module_init(void) static void __exit rxe_module_exit(void) { + rxe_module_put(); + wait_for_completion(&rxe_module.complete); + rdma_link_unregister(&rxe_link_ops); ib_unregister_driver(RDMA_DRIVER_RXE); rxe_net_exit(); diff --git a/drivers/infiniband/sw/rxe/rxe.h b/drivers/infiniband/sw/rxe/rxe.h index d33dd6cf83d3..077e3ad8f39a 100644 --- a/drivers/infiniband/sw/rxe/rxe.h +++ b/drivers/infiniband/sw/rxe/rxe.h @@ -158,4 +158,7 @@ void rxe_port_up(struct rxe_dev *rxe); void rxe_port_down(struct rxe_dev *rxe); void rxe_set_port_state(struct rxe_dev *rxe); +int rxe_module_get(void); +int rxe_module_put(void); + #endif /* RXE_H */ diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index c1b2eaf82334..0e447420a441 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -384,6 +384,7 @@ static void rxe_skb_tx_dtor(struct sk_buff *skb) out_put_ibdev: ib_device_put(&rxe->ib_dev); out: + rxe_module_put(); return; } @@ -400,6 +401,12 @@ static int rxe_send(struct sk_buff *skb, struct rxe_pkt_info *pkt) sock_hold(sk); skb->sk = sk; + /* the module may be potentially removed while this packet + * is waiting on the tx queue causing seg faults. So need + * to protect the module. + */ + rxe_module_get(); + atomic_inc(&pkt->qp->skb_out); sk->sk_user_data = (void *)(long)qp->elem.index;