From patchwork Mon Jan 25 21:16:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044395 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7785FC433E0 for ; Mon, 25 Jan 2021 21:28:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31EEB208C7 for ; Mon, 25 Jan 2021 21:28:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732472AbhAYVTR (ORCPT ); Mon, 25 Jan 2021 16:19:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732510AbhAYVR4 (ORCPT ); Mon, 25 Jan 2021 16:17:56 -0500 Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E80A4C0613ED for ; Mon, 25 Jan 2021 13:16:45 -0800 (PST) Received: by mail-oi1-x22e.google.com with SMTP id i25so4898076oie.10 for ; Mon, 25 Jan 2021 13:16:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1fNm7hAq2LAMRsJr67mUat+oeMfmUk+H6CjsCCKEX0M=; b=Bi/VUbbHtXrQjZcxa1pcz+vp/LHWHYCMuE93sV0lHcmo1oAnYJwNyH4mrnCIbmhrzP M28hMiyG6n+UFW272cqwlTypTwYQLE9ThLfC5c6vv/5tDUsIwE+5T0w6DJhrH5yx97RV Kh+QGXfgKHcbRZBlwpwBE7YM/1YDAsDs7TorxQyfu00EpF90ryVvcP93Re/u5LU8iSJh Wj0vu47+33T6UPtlAfdeNymt4YkPvLsqpeF1osYFP5Hgx8hx9UL51YzyG6thAHnwga36 L3xW15BZWUKxNYmY2biXaNBXVmo+I9qjdXtYWvEsAOKVDRZNrg/whk/rdyI1sPJ7RgwH kWvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1fNm7hAq2LAMRsJr67mUat+oeMfmUk+H6CjsCCKEX0M=; b=Jcf1BCE6EKAYJHd6/gixnwRnUDv2hMwdRBSp6qqXe/RGIvlgeUPEpf+Q1zMSnQjYs2 5hhWWPArwDAAFTBc7FWiUzz3eMBbhCYtsuosSISKfP4RAsCWFdpJ2Zy20ISCq3Wxlq15 0htQ+xxumegbgFZTNVNrRPKEo8L03UJoGpcnx7el6qlhXqadBue17CQNPsmUN60RLMtV fy5vilFXzsfl8iLm+j6fdP8Wr0g7sWxxj+4W5voqCQsLW27hH6meJFZmUX8Q7fm9AK5F 368dn7YAgCwiaZmiKgOlaNHg1P9XIEQaR74jAKb/RlqUdBmbAFzPr4mzf4X0tSTSBz0J sqdQ== X-Gm-Message-State: AOAM5315muDLqMcBIc4JC+Pwje/PIBj3DRvAjQ58ZVjr2ml4wx7By+1H ArVyE95dDwb8YFeembVUBZA= X-Google-Smtp-Source: ABdhPJzAZ4Gk9aveDWT57ndsYNtBjFI02sE/DwTDmCiTImN//JzmqQsSdtiK6y/tLeKIaWk9llpO2Q== X-Received: by 2002:aca:f00a:: with SMTP id o10mr1334677oih.128.1611609405425; Mon, 25 Jan 2021 13:16:45 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:45 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson , syzbot+ec2fd72374785d0e558e@syzkaller.appspotmail.com Subject: [PATCH for-next v3 1/6] RDMA/rxe: Fix bug in rxe_alloc Date: Mon, 25 Jan 2021 15:16:36 -0600 Message-Id: <20210125211641.2694-2-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v1] A recent patch which added an 'unlocked' version of rxe_alloc introduced a bug causing kzalloc(..., GFP_KERNEL) to be called while holding a spin lock. This patch corrects that error. rxe_alloc_nl() should always be called while holding the pool->pool_lock so the 2nd argument to kzalloc there should be GFP_ATOMIC. rxe_alloc() prior to the change only locked the code around checking that pool->state is RXE_POOL_STATE_VALID to avoid races between working threads and a thread shutting down the rxe driver. This patch reverts rxe_alloc() to this behavior so the lock is not held when kzalloc() is called. Reported-by: syzbot+ec2fd72374785d0e558e@syzkaller.appspotmail.com Fixes: 3853c35e243d ("RDMA/rxe: Add unlocked versions of pool APIs") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 41 ++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index d26730eec720..cfcd55175572 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -343,8 +343,6 @@ void *rxe_alloc_nl(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); - if (pool->state != RXE_POOL_STATE_VALID) return NULL; @@ -356,8 +354,7 @@ void *rxe_alloc_nl(struct rxe_pool *pool) if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; - obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? - GFP_ATOMIC : GFP_KERNEL); + obj = kzalloc(info->size, GFP_ATOMIC); if (!obj) goto out_cnt; @@ -378,14 +375,46 @@ void *rxe_alloc_nl(struct rxe_pool *pool) void *rxe_alloc(struct rxe_pool *pool) { - u8 *obj; unsigned long flags; + struct rxe_type_info *info = &rxe_type_info[pool->type]; + struct rxe_pool_entry *elem; + u8 *obj; + + might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); read_lock_irqsave(&pool->pool_lock, flags); - obj = rxe_alloc_nl(pool); + if (pool->state != RXE_POOL_STATE_VALID) { + read_unlock_irqrestore(&pool->pool_lock, flags); + return NULL; + } + + kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); + if (!ib_device_try_get(&pool->rxe->ib_dev)) + goto out_put_pool; + + if (atomic_inc_return(&pool->num_elem) > pool->max_elem) + goto out_cnt; + + obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? + GFP_ATOMIC : GFP_KERNEL); + if (!obj) + goto out_cnt; + + elem = (struct rxe_pool_entry *)(obj + info->elem_offset); + + elem->pool = pool; + kref_init(&elem->ref_cnt); + return obj; + +out_cnt: + atomic_dec(&pool->num_elem); + ib_device_put(&pool->rxe->ib_dev); +out_put_pool: + rxe_pool_put(pool); + return NULL; } int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) From patchwork Mon Jan 25 21:16:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044371 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54515C433E0 for ; Mon, 25 Jan 2021 21:19:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 150A720679 for ; Mon, 25 Jan 2021 21:19:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731495AbhAYVSr (ORCPT ); Mon, 25 Jan 2021 16:18:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732506AbhAYVR4 (ORCPT ); Mon, 25 Jan 2021 16:17:56 -0500 Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A854DC061786 for ; Mon, 25 Jan 2021 13:16:46 -0800 (PST) Received: by mail-oi1-x236.google.com with SMTP id h6so14956383oie.5 for ; Mon, 25 Jan 2021 13:16:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z019pMrIdk7T2xOU6DabaUb3qTUzcQVh5XZ76sNd5OE=; b=gMooDl+gAfH+vFofvoxtgZIGrddAZCnWtkHwbFYDx6x2Ph9CZ4vyqL952l6Wk1sYen QjhCmxoToKDPNo8J5MFC4XJGrHXpcapjUFysN7bPPTnx+EAYRnCV/E2+SSz+sR4lQn1h 1gNF+YdAztezEH79KWWu8DhOPiYaz+L7n40GQ1J0BYO2sUmSpyk/r1NxzS4Vs4xCeQcF cCISAMwToS+GEi8W/jPR3mnip4kiAiuk8gOb31laTeZ28sTj9DhL5Sp+CK0On/HkPA5e Cd/h5035efaRJXU+pgzaDVts3KxsoI4KRu0F+kxz+9CILlq5fKdtloFtpxfeBg9iodHw kG3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z019pMrIdk7T2xOU6DabaUb3qTUzcQVh5XZ76sNd5OE=; b=skxoSm5RFSgkowOvtiLFN2pqTbzx3+Ua7+SNM2jnnvEg5PMeLHSeOP0hayiNZbf7Im XNF1L/Kcp9cOxp18W9GwBWD7hG8SJWdNf4YGZ9N0wyRroM/ttjFK0YscObuABiRGvFB9 x0lolvUxveFlKzjDGo0BwzfiQUU51tOm/tm1Gql0/wba7cRns7E5RcRFrodQaehrvh2T tTTQcI+4CpeaOAhT61mfrqCN3gIZDQ3nGrkG+ZIr1ovo6wjVs5MQ+JwaNa9+e/1R08Vj xlq2nYe+FVxFOic6KpDXhxfTCRGAW3nuHmsXnb3aPyD3Lmx/YdJczAAUVyBallZBZoz8 WchQ== X-Gm-Message-State: AOAM533exN+LojzNu9gbhIto6CR+tmADCN/LALGTzFGuGzxs2RJj4CVy qx6b7pFpUDiStUuq6kiFto4= X-Google-Smtp-Source: ABdhPJycnUu4Rs5QxAvU6AtljOFZ7sHOcrOzAc0TSUESL+rHSQVtFe1ltNMtJdmyyCNax6lJjlERqg== X-Received: by 2002:aca:c085:: with SMTP id q127mr1331043oif.70.1611609406121; Mon, 25 Jan 2021 13:16:46 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:45 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson , Hillf Danton Subject: [PATCH for-next v3 2/6] RDMA/rxe: Fix misleading comments and names Date: Mon, 25 Jan 2021 15:16:37 -0600 Message-Id: <20210125211641.2694-3-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v3] Was rxe_xxx__() changed to rxe_xxx_locked() Suggested-by: jgg@nvidia.com The names and comments of the 'unlocked' pool APIs are very misleading and not what was intended. This patch replaces 'rxe_xxx_nl' with 'rxe_xxx_locked' with comments indicating that the caller is expected to hold the rxe pool lock. Reported-by: Hillf Danton Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 8 ++-- drivers/infiniband/sw/rxe/rxe_pool.c | 22 +++++------ drivers/infiniband/sw/rxe/rxe_pool.h | 55 +++++++++++++-------------- 3 files changed, 42 insertions(+), 43 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 5be47ce7d319..0ea9a5aa4ec0 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -15,18 +15,18 @@ static struct rxe_mc_grp *create_grp(struct rxe_dev *rxe, int err; struct rxe_mc_grp *grp; - grp = rxe_alloc_nl(&rxe->mc_grp_pool); + grp = rxe_alloc_locked(&rxe->mc_grp_pool); if (!grp) return ERR_PTR(-ENOMEM); INIT_LIST_HEAD(&grp->qp_list); spin_lock_init(&grp->mcg_lock); grp->rxe = rxe; - rxe_add_key_nl(grp, mgid); + rxe_add_key_locked(grp, mgid); err = rxe_mcast_add(rxe, mgid); if (unlikely(err)) { - rxe_drop_key_nl(grp); + rxe_drop_key_locked(grp); rxe_drop_ref(grp); return ERR_PTR(err); } @@ -47,7 +47,7 @@ int rxe_mcast_get_grp(struct rxe_dev *rxe, union ib_gid *mgid, write_lock_irqsave(&pool->pool_lock, flags); - grp = rxe_pool_get_key_nl(pool, mgid); + grp = rxe_pool_get_key_locked(pool, mgid); if (grp) goto done; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index cfcd55175572..5ca54e09cd0e 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -266,7 +266,7 @@ static void insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) return; } -void __rxe_add_key_nl(struct rxe_pool_entry *elem, void *key) +void __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key) { struct rxe_pool *pool = elem->pool; @@ -280,11 +280,11 @@ void __rxe_add_key(struct rxe_pool_entry *elem, void *key) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_key_nl(elem, key); + __rxe_add_key_locked(elem, key); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_drop_key_nl(struct rxe_pool_entry *elem) +void __rxe_drop_key_locked(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -297,11 +297,11 @@ void __rxe_drop_key(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_drop_key_nl(elem); + __rxe_drop_key_locked(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_add_index_nl(struct rxe_pool_entry *elem) +void __rxe_add_index_locked(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -315,11 +315,11 @@ void __rxe_add_index(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_index_nl(elem); + __rxe_add_index_locked(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_drop_index_nl(struct rxe_pool_entry *elem) +void __rxe_drop_index_locked(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -333,11 +333,11 @@ void __rxe_drop_index(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_drop_index_nl(elem); + __rxe_drop_index_locked(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void *rxe_alloc_nl(struct rxe_pool *pool) +void *rxe_alloc_locked(struct rxe_pool *pool) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rxe_pool_entry *elem; @@ -507,7 +507,7 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) return obj; } -void *rxe_pool_get_key_nl(struct rxe_pool *pool, void *key) +void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; @@ -551,7 +551,7 @@ void *rxe_pool_get_key(struct rxe_pool *pool, void *key) unsigned long flags; read_lock_irqsave(&pool->pool_lock, flags); - obj = rxe_pool_get_key_nl(pool, key); + obj = rxe_pool_get_key_locked(pool, key); read_unlock_irqrestore(&pool->pool_lock, flags); return obj; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 373e08554c1c..a75ac2d2847a 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -106,11 +106,10 @@ int rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, /* free resources from object pool */ void rxe_pool_cleanup(struct rxe_pool *pool); -/* allocate an object from pool */ -void *rxe_alloc(struct rxe_pool *pool); +/* allocate an object from pool holding and not holding the pool lock */ +void *rxe_alloc_locked(struct rxe_pool *pool); -/* allocate an object from pool - no lock */ -void *rxe_alloc_nl(struct rxe_pool *pool); +void *rxe_alloc(struct rxe_pool *pool); /* connect already allocated object to pool */ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem); @@ -118,60 +117,60 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem); #define rxe_add_to_pool(pool, obj) __rxe_add_to_pool(pool, &(obj)->pelem) /* assign an index to an indexed object and insert object into - * pool's rb tree with and without holding the pool_lock + * pool's rb tree holding and not holding the pool_lock */ -void __rxe_add_index(struct rxe_pool_entry *elem); +void __rxe_add_index_locked(struct rxe_pool_entry *elem); -#define rxe_add_index(obj) __rxe_add_index(&(obj)->pelem) +#define rxe_add_index_locked(obj) __rxe_add_index_locked(&(obj)->pelem) -void __rxe_add_index_nl(struct rxe_pool_entry *elem); +void __rxe_add_index(struct rxe_pool_entry *elem); -#define rxe_add_index_nl(obj) __rxe_add_index_nl(&(obj)->pelem) +#define rxe_add_index(obj) __rxe_add_index(&(obj)->pelem) /* drop an index and remove object from rb tree - * with and without holding the pool_lock + * holding and not holding the pool_lock */ -void __rxe_drop_index(struct rxe_pool_entry *elem); +void __rxe_drop_index_locked(struct rxe_pool_entry *elem); -#define rxe_drop_index(obj) __rxe_drop_index(&(obj)->pelem) +#define rxe_drop_index_locked(obj) __rxe_drop_index_locked(&(obj)->pelem) -void __rxe_drop_index_nl(struct rxe_pool_entry *elem); +void __rxe_drop_index(struct rxe_pool_entry *elem); -#define rxe_drop_index_nl(obj) __rxe_drop_index_nl(&(obj)->pelem) +#define rxe_drop_index(obj) __rxe_drop_index(&(obj)->pelem) /* assign a key to a keyed object and insert object into - * pool's rb tree with and without holding pool_lock + * pool's rb tree holding and not holding pool_lock */ +void __rxe_add_key_locked(struct rxe_pool_entry *elem, void *key); + +#define rxe_add_key_locked(obj, key) __rxe_add_key_locked(&(obj)->pelem, key) + void __rxe_add_key(struct rxe_pool_entry *elem, void *key); #define rxe_add_key(obj, key) __rxe_add_key(&(obj)->pelem, key) -void __rxe_add_key_nl(struct rxe_pool_entry *elem, void *key); +/* remove elem from rb tree holding and not holding the pool_lock */ +void __rxe_drop_key_locked(struct rxe_pool_entry *elem); -#define rxe_add_key_nl(obj, key) __rxe_add_key_nl(&(obj)->pelem, key) +#define rxe_drop_key_locked(obj) __rxe_drop_key_locked(&(obj)->pelem) -/* remove elem from rb tree with and without holding pool_lock */ void __rxe_drop_key(struct rxe_pool_entry *elem); #define rxe_drop_key(obj) __rxe_drop_key(&(obj)->pelem) -void __rxe_drop_key_nl(struct rxe_pool_entry *elem); - -#define rxe_drop_key_nl(obj) __rxe_drop_key_nl(&(obj)->pelem) - -/* lookup an indexed object from index with and without holding pool_lock. +/* lookup an indexed object from index holding and not holding the pool_lock. * takes a reference on object */ -void *rxe_pool_get_index(struct rxe_pool *pool, u32 index); +void *rxe_pool_get_index_locked(struct rxe_pool *pool, u32 index); -void *rxe_pool_get_index_nl(struct rxe_pool *pool, u32 index); +void *rxe_pool_get_index(struct rxe_pool *pool, u32 index); -/* lookup keyed object from key with and without holding pool_lock. +/* lookup keyed object from key holding and not holding the pool_lock. * takes a reference on the objecti */ -void *rxe_pool_get_key(struct rxe_pool *pool, void *key); +void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key); -void *rxe_pool_get_key_nl(struct rxe_pool *pool, void *key); +void *rxe_pool_get_key(struct rxe_pool *pool, void *key); /* cleanup an object when all references are dropped */ void rxe_elem_release(struct kref *kref); From patchwork Mon Jan 25 21:16:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044375 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B42B0C433E0 for ; Mon, 25 Jan 2021 21:19:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7F1D920679 for ; Mon, 25 Jan 2021 21:19:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732552AbhAYVTL (ORCPT ); Mon, 25 Jan 2021 16:19:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732573AbhAYVSX (ORCPT ); Mon, 25 Jan 2021 16:18:23 -0500 Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69BEBC061788 for ; Mon, 25 Jan 2021 13:16:47 -0800 (PST) Received: by mail-ot1-x32a.google.com with SMTP id a109so14236718otc.1 for ; Mon, 25 Jan 2021 13:16:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ak10mtEKMMemekcMv4+yfdQKMMStyMtW7BlCZsMxFLo=; b=aaFyttkx0Ad8A/9CM6wJRlU8aAS8qrqeHuvqu126lmog5IOSSWlbr+mTUruLV1gFqf CvKljhc77KwPnJ/kNslLlLQRUGEJUjcvNA3jCoW5HzWhfoxF2JPtam6MI7aLUUOcDE3n IP1pOFtT+G6IWjs6XmhcKW5ieuZz0LoVn5430r6TFVoIcOdE3AE39+VT52EeThfy8vAN 2KNOtdweMoZRGgnuS/65d73f861r//VufNPgrhSVp5PC5piVF6BI5j3dXZng3YHVIJHD z5iSIF1sIAxNCWTvm/qnPoYRIEB4J9X8TkU/m20v0Xv/PiQ7PG3lAy8pu9Ds/wH706eA I1ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ak10mtEKMMemekcMv4+yfdQKMMStyMtW7BlCZsMxFLo=; b=Qu6S0igVXhCN+MpArEbT2mrEnYVaCVAjur3qoh6fO1gUW1LHS/qxVR2zkfMuSBbawa KjhBD0jMZFfB2AGkRxkU7kNwL7PlQZH83hExrgJ3vKhXXmFat2BLIcNj/DiOvsrFwe3u WQDNvSLwMRBvyhYPnIVtkHWvg25vhk+ICcftXWMtGq8gouEC5XHER9UBjyr7IAXKdoiv zD2R3iuKgSZAo/rElOvKBb9TTfMjz/ZfrXvM8R+IhR3rfppr4bV6jNIVwuzK0qxJKZOW RHJDJHyjZJaj5I6KgZCpIZu+Tb+s0hgbButzEGP/El2uvDyOb4YsisfQ1ZD0dqLeTbyr e5RA== X-Gm-Message-State: AOAM530GYEDPV9WDtMn8Q3POHGRryj60gerAarm4TPIiVWEYMPUaY8md 51gPUOc0hqgH0gyXuGzAI0Q= X-Google-Smtp-Source: ABdhPJwZvx+Of7khi6wAVRhPeC2DutpwhZ/HCh4uZoqOGlDOrEspXPYsulDaRkeYJ0d+27ywmYpEEA== X-Received: by 2002:a9d:84d:: with SMTP id 71mr1770740oty.338.1611609406899; Mon, 25 Jan 2021 13:16:46 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:46 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v3 3/6] RDMA/rxe: Remove RXE_POOL_ATOMIC Date: Mon, 25 Jan 2021 15:16:38 -0600 Message-Id: <20210125211641.2694-4-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v2] rxe_alloc() used the RXE_POOL_ATOMIC flag in rxe_type_info to select GFP_ATOMIC in calls to kzalloc(). This was intended to handle cases where an object could be created in interrupt context. This no longer occurs since allocating those objects has moved into the core so this flag is not necessary. An incorrect use of this flag was still present for rxe_mc_elem objects and is removed. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 6 +----- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 5ca54e09cd0e..0ca46bd8be51 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -84,7 +84,6 @@ struct rxe_type_info rxe_type_info[RXE_NUM_TYPES] = { .name = "rxe-mc_elem", .size = sizeof(struct rxe_mc_elem), .elem_offset = offsetof(struct rxe_mc_elem, pelem), - .flags = RXE_POOL_ATOMIC, }, }; @@ -380,8 +379,6 @@ void *rxe_alloc(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); - read_lock_irqsave(&pool->pool_lock, flags); if (pool->state != RXE_POOL_STATE_VALID) { read_unlock_irqrestore(&pool->pool_lock, flags); @@ -397,8 +394,7 @@ void *rxe_alloc(struct rxe_pool *pool) if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; - obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? - GFP_ATOMIC : GFP_KERNEL); + obj = kzalloc(info->size, GFP_KERNEL); if (!obj) goto out_cnt; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index a75ac2d2847a..22714dafe00d 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -11,7 +11,6 @@ #define RXE_POOL_CACHE_FLAGS (0) enum rxe_pool_flags { - RXE_POOL_ATOMIC = BIT(0), RXE_POOL_INDEX = BIT(1), RXE_POOL_KEY = BIT(2), RXE_POOL_NO_ALLOC = BIT(4), From patchwork Mon Jan 25 21:16:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044373 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52146C433E0 for ; Mon, 25 Jan 2021 21:19:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1CE5B20679 for ; Mon, 25 Jan 2021 21:19:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732470AbhAYVSw (ORCPT ); Mon, 25 Jan 2021 16:18:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732571AbhAYVSX (ORCPT ); Mon, 25 Jan 2021 16:18:23 -0500 Received: from mail-ot1-x329.google.com (mail-ot1-x329.google.com [IPv6:2607:f8b0:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DDDAC06178A for ; Mon, 25 Jan 2021 13:16:48 -0800 (PST) Received: by mail-ot1-x329.google.com with SMTP id s2so12017956otp.5 for ; Mon, 25 Jan 2021 13:16:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=71W3O1dOs9YUN5fkCQYTn3c6Ta+pMpza3pa9aXNa424=; b=FBfF1eQC1CkEEXaRtP+LAMJkgxKyzW4oqtVpNBXyQoC6l9h6WENAlUXj5POyjPrfzq Iyzi+tQ13w+UQn0dprFymynnMpt9IlYQU+MpIOtZZH8iQUK8SitaQgGMtqOqktaZv+iV /tvHcYaUakQiKr2Fq6hiFBHCaONZegPTWVAWPUYQabDe2inZhLYdrhSqpwpIxjH51MF9 bpXY72rZkN4x5W8+Ggqs42tSFBselME4/GqNGGj2g1GKUfa1S0cttVXNgapkN7KwlEfF LjzfyPQofUk9Ccq7uUqiTKX//5N9qshk2Ewxm00LRCTNl3JJXAUN/8+rykca7Cg15Esq b8hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=71W3O1dOs9YUN5fkCQYTn3c6Ta+pMpza3pa9aXNa424=; b=fMgGwVfoPAcPJOunOT3cIJ4p8Iz6Qw2g5TRRPdrRqutAS+dve7/eTCTDgxp/DoiSCv 5bjuW5wtUc84GiIzkmtDd+URWH96yUqjwSsD7q+Y3pYJzAIzvSvpgausw4G0QDKsJcf/ FpO9RVyP7LsttWxmO30VIVV6t8vUO0V7p9dFz1N9t2+2nclVMWnKkxwdFW2DnPLkvAb0 Vnp9z+R90MEj7hKuRSFrDrxGLk7nro8KU8nJhfGSbwWKhmADpfORbv1hw9Uyclozn/eX COOVnFbSeTekgeJ8gmRlNxSscExm6p0GBwrpfWyX/ULAypMtKI/33wEkZO7PAD5xi69t 8dLw== X-Gm-Message-State: AOAM532SyIF6D3emSyryBHo2Y+klh3C6NpxwX3C8KAITp9BJCuUrvGaN JkQN13AdgYBX805r7WB+n1Y= X-Google-Smtp-Source: ABdhPJz6jVnwe1Kkm4Zwr1eUTH9Y23o9uOSoWZUi/55CbIfOQ6rKSmb+QE9P0kw/6+cbgnTRsDEF1w== X-Received: by 2002:a9d:3b43:: with SMTP id z61mr1788903otb.217.1611609407579; Mon, 25 Jan 2021 13:16:47 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:47 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v3 4/6] RDMA/rxe: Remove references to ib_device and pool Date: Mon, 25 Jan 2021 15:16:39 -0600 Message-Id: <20210125211641.2694-5-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v2] rxe_pool.c takes references to the pool and ib_device structs for each object allocated and also keeps an atomic num_elem count in each pool. This is more work than is needed. Pool allocation is only called from verbs APIs which already have references to ib_device and pools are only diasbled when the driver is removed so no protection of the pool addresses are needed. The elem count is used to warn if elements are still present in a pool when it is cleaned up which is useful. This patch eliminates the references to the ib_device and pool structs. [v1] The previous version only removed the ib_device reference. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 42 ++-------------------------- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 2 files changed, 2 insertions(+), 41 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 0ca46bd8be51..5f85a90e5a5a 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -142,8 +142,6 @@ int rxe_pool_init( atomic_set(&pool->num_elem, 0); - kref_init(&pool->ref_cnt); - rwlock_init(&pool->pool_lock); if (rxe_type_info[type].flags & RXE_POOL_INDEX) { @@ -165,19 +163,6 @@ int rxe_pool_init( return err; } -static void rxe_pool_release(struct kref *kref) -{ - struct rxe_pool *pool = container_of(kref, struct rxe_pool, ref_cnt); - - pool->state = RXE_POOL_STATE_INVALID; - kfree(pool->index.table); -} - -static void rxe_pool_put(struct rxe_pool *pool) -{ - kref_put(&pool->ref_cnt, rxe_pool_release); -} - void rxe_pool_cleanup(struct rxe_pool *pool) { unsigned long flags; @@ -189,7 +174,8 @@ void rxe_pool_cleanup(struct rxe_pool *pool) pool_name(pool)); write_unlock_irqrestore(&pool->pool_lock, flags); - rxe_pool_put(pool); + pool->state = RXE_POOL_STATE_INVALID; + kfree(pool->index.table); } static u32 alloc_index(struct rxe_pool *pool) @@ -345,11 +331,6 @@ void *rxe_alloc_locked(struct rxe_pool *pool) if (pool->state != RXE_POOL_STATE_VALID) return NULL; - kref_get(&pool->ref_cnt); - - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -366,9 +347,6 @@ void *rxe_alloc_locked(struct rxe_pool *pool) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return NULL; } @@ -385,12 +363,8 @@ void *rxe_alloc(struct rxe_pool *pool) return NULL; } - kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -407,9 +381,6 @@ void *rxe_alloc(struct rxe_pool *pool) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return NULL; } @@ -422,12 +393,8 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) read_unlock_irqrestore(&pool->pool_lock, flags); return -EINVAL; } - kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -438,9 +405,6 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return -EINVAL; } @@ -461,8 +425,6 @@ void rxe_elem_release(struct kref *kref) } atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); - rxe_pool_put(pool); } void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 22714dafe00d..8f8de746ca17 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -68,7 +68,6 @@ struct rxe_pool { struct rxe_dev *rxe; rwlock_t pool_lock; /* protects pool add/del/search */ size_t elem_size; - struct kref ref_cnt; void (*cleanup)(struct rxe_pool_entry *obj); enum rxe_pool_state state; enum rxe_pool_flags flags; From patchwork Mon Jan 25 21:16:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044923 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA97C433E0 for ; Mon, 25 Jan 2021 23:47:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F88722AAC for ; Mon, 25 Jan 2021 23:47:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732536AbhAYVTE (ORCPT ); Mon, 25 Jan 2021 16:19:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732575AbhAYVSX (ORCPT ); Mon, 25 Jan 2021 16:18:23 -0500 Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD8B0C06178B for ; Mon, 25 Jan 2021 13:16:48 -0800 (PST) Received: by mail-oi1-x22a.google.com with SMTP id n7so4408246oic.11 for ; Mon, 25 Jan 2021 13:16:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tsTol3rhzUxL2jFVkJX/iOKqAbW+O0NXRl4F0Z6OsCw=; b=rQ4NEQuk+PgwD8WI/Vy/TP2ucL9CR4Z2T3ohkZ30Pz2zw1H16h99uhtN105emhE3al xhuEjabdIPvS8UR4D1+hSbJzwzYrrdfQZmMBwZV3VHm+ZzPMPfbOTdq8OfrsDyuYLZ+J 0BLKnRoWRnD9GYaleLzl59bO2lxR5vfW+GRP0M61OTSrd9n0SSQMs7PGrjP6n2nLPxKs R/V/5DP/LdfDuASZs0ZHSfax3tGcqCz9UVq9KVDc6fPTnOTjKbh0NZAbW5O1K7JKxYJE LqoBunVEPBFdIaW+WZSZc0j6h97LU6W3OsNdspXIy0yTUFuou1z4XY9Sm9QVRVFyaNNO xf+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tsTol3rhzUxL2jFVkJX/iOKqAbW+O0NXRl4F0Z6OsCw=; b=MUYfW+x2dqSWeG/jK4LvTD/OZIYwx4CQ+iQ9bPNtKv/cLWfji4jY1znkS53p6JmIBM q5IkT0YkNDZrBpEnGs8eCYtx6YGZ8lTQorpd9p/YrY+uKZQ4ARhloJEKT2fbVQ0BndLc LSWApuYPOpophe/zKDUaqpMkPYGQbyQhsv9LdmP8spvWeEHZKkBturYO3gru9jjG+OIJ zHisi9TGDhBYMmY618VUMYnAyeYNvoOR20VwfO4gWcwFDbAOKquVQ5AAO88438dVQIgx VHInDVIGBXmjYyj+p/wlugtXipmKU1QzieCTClXxj3aA7bFP+HoG63D3jgk65ft0JQ+k ro5g== X-Gm-Message-State: AOAM530MOnf4dqHQfOMlIQEkFZjjI/A4hQ+1K2JphvdtBY3XZccfaMFa 7DDm3yebXAj4jBqwMM67ycU= X-Google-Smtp-Source: ABdhPJxliPQZKlf+Ud3tXg/1cSBA/NET5jw7rBWQr1RCRbndvYudbzol+0TBiPTcRPhEn2DqepaPTg== X-Received: by 2002:aca:a844:: with SMTP id r65mr1282430oie.35.1611609408275; Mon, 25 Jan 2021 13:16:48 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:47 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson , zyjzyj2000@gmail.c Subject: [PATCH for-next v3 5/6] RDMA/rxe: Remove unneeded pool->state Date: Mon, 25 Jan 2021 15:16:40 -0600 Message-Id: <20210125211641.2694-6-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v3] Fixed spelling error. rxe_pool.c uses the field pool->state to mark a pool as invalid when it is shut down and checks it in several pool APIs to verify that the pool has not been shut down. This is unneeded because the pools are not marked invalid unless the entire driver is being removed at which point no functional APIs should or could be executing. This patch removes this field and associated code. Suggested-by: zyjzyj2000@gmail.c Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 38 +--------------------------- drivers/infiniband/sw/rxe/rxe_pool.h | 6 ----- 2 files changed, 1 insertion(+), 43 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 5f85a90e5a5a..5aa835028460 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -157,24 +157,16 @@ int rxe_pool_init( pool->key.key_size = rxe_type_info[type].key_size; } - pool->state = RXE_POOL_STATE_VALID; - out: return err; } void rxe_pool_cleanup(struct rxe_pool *pool) { - unsigned long flags; - - write_lock_irqsave(&pool->pool_lock, flags); - pool->state = RXE_POOL_STATE_INVALID; if (atomic_read(&pool->num_elem) > 0) pr_warn("%s pool destroyed with unfree'd elem\n", pool_name(pool)); - write_unlock_irqrestore(&pool->pool_lock, flags); - pool->state = RXE_POOL_STATE_INVALID; kfree(pool->index.table); } @@ -328,9 +320,6 @@ void *rxe_alloc_locked(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - if (pool->state != RXE_POOL_STATE_VALID) - return NULL; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -352,19 +341,10 @@ void *rxe_alloc_locked(struct rxe_pool *pool) void *rxe_alloc(struct rxe_pool *pool) { - unsigned long flags; struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rxe_pool_entry *elem; u8 *obj; - read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) { - read_unlock_irqrestore(&pool->pool_lock, flags); - return NULL; - } - - read_unlock_irqrestore(&pool->pool_lock, flags); - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -386,15 +366,6 @@ void *rxe_alloc(struct rxe_pool *pool) int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) { - unsigned long flags; - - read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) { - read_unlock_irqrestore(&pool->pool_lock, flags); - return -EINVAL; - } - read_unlock_irqrestore(&pool->pool_lock, flags); - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -437,9 +408,6 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) - goto out; - node = pool->index.tree.rb_node; while (node) { @@ -460,8 +428,8 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) obj = NULL; } -out: read_unlock_irqrestore(&pool->pool_lock, flags); + return obj; } @@ -473,9 +441,6 @@ void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) u8 *obj = NULL; int cmp; - if (pool->state != RXE_POOL_STATE_VALID) - goto out; - node = pool->key.tree.rb_node; while (node) { @@ -499,7 +464,6 @@ void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) obj = NULL; } -out: return obj; } diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 8f8de746ca17..61210b300a78 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -46,11 +46,6 @@ struct rxe_type_info { extern struct rxe_type_info rxe_type_info[]; -enum rxe_pool_state { - RXE_POOL_STATE_INVALID, - RXE_POOL_STATE_VALID, -}; - struct rxe_pool_entry { struct rxe_pool *pool; struct kref ref_cnt; @@ -69,7 +64,6 @@ struct rxe_pool { rwlock_t pool_lock; /* protects pool add/del/search */ size_t elem_size; void (*cleanup)(struct rxe_pool_entry *obj); - enum rxe_pool_state state; enum rxe_pool_flags flags; enum rxe_elem_type type; From patchwork Mon Jan 25 21:16:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12044377 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3A4CC433DB for ; Mon, 25 Jan 2021 21:19:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9EAC120679 for ; Mon, 25 Jan 2021 21:19:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726716AbhAYVTP (ORCPT ); Mon, 25 Jan 2021 16:19:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732568AbhAYVSX (ORCPT ); Mon, 25 Jan 2021 16:18:23 -0500 Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 88143C06178C for ; Mon, 25 Jan 2021 13:16:49 -0800 (PST) Received: by mail-oi1-x233.google.com with SMTP id m13so8358532oig.8 for ; Mon, 25 Jan 2021 13:16:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9E77g67+kgLL6DKT76WJ3KWOIjkCnglA8Markr+v0xM=; b=BDF99oER9+ZDKZBKJTJWKWwC9oF8n6iAgJ8CI8uRN4vAT+PUpbawcg6pfgKzT0KfhY vkZ2ac5tELvjTL4LhJH1iPnCDx9toG4MJZVgQ1Gt83DQSpENBy7tVp6aGvvtXPiS/1uS 64t37lEDjYEzgqQdpocjpEN1SYGpEj6zxiMQaHl0tzPfwEGZAxiIRSK/ltc+0bXzmiUs JJi2TCq3PMJvCCUvm47LRj6dtDjU+I9PY21fOHmwpiVHl3rusSnBhXzLJ+f8Ptwoyh48 QoYfbR+fFJx8zLO0WJ/z/CCzMi7/FE56lJZrCG6ioowt+jw4y20YKm5yRm+rP9IieaLB Kd/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9E77g67+kgLL6DKT76WJ3KWOIjkCnglA8Markr+v0xM=; b=gk+1DNgD/lijpMN8Bfv0WJjO8Ip+oh3VYqlI5bBEHwD+xVreZsaw1yFMDi/NIbHAUQ cOkwC1jsBDyPCoyY8ryuO/ePw0HuUmbGPAOPyHGW+nfnyCP1mC8wIhW5DJ+bHMo1fVMA qMPweJ5CQU9gBuZYkm5i4Efw04FIFh1pNxXVCxbgw4UtWGejoSP5k5DzoacOEvHFrxC0 8X5ja8twjC41sIaKLfIrw/idnw3T+SSd8hsEgSAobrlBNpQMh9Qnq3ZVh58ATstjXBZW i2ZTsfIAZZ5oYynusc222wPCN7n2OUjKv2oFiAiNCRhaQ388xAhFxk/LMI8YcI03conn D/kg== X-Gm-Message-State: AOAM530t0usq3lFW1TNJBeUvky18+R9W2km3scL9NCB3Ah8vwDPZsn9B Z1OelAeTsLREO+xkStAshFrVbSsgUYM= X-Google-Smtp-Source: ABdhPJw407eB2lShKQHeG2AXnTEdKf2uXBBliDV/rNGcQ+oElzG6KLzBUK78CdTeQ8i8Z+8cPEuVxg== X-Received: by 2002:aca:f009:: with SMTP id o9mr1312380oih.133.1611609409070; Mon, 25 Jan 2021 13:16:49 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-8baa-bb83-cd49-4ca4.res6.spectrum.com. [2603:8081:140c:1a00:8baa:bb83:cd49:4ca4]) by smtp.gmail.com with ESMTPSA id w11sm3722564otg.58.2021.01.25.13.16.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 13:16:48 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v3 6/6] RDMA/rxe: Replace missing rxe_pool_get_index_locked Date: Mon, 25 Jan 2021 15:16:41 -0600 Message-Id: <20210125211641.2694-7-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210125211641.2694-1-rpearson@hpe.com> References: <20210125211641.2694-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org [v2] One of the pool APIs for when caller is holding lock was not defined but is declared in rxe_pool.h. This patch adds the definition. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 5aa835028460..307d8986e7c9 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -398,15 +398,12 @@ void rxe_elem_release(struct kref *kref) atomic_dec(&pool->num_elem); } -void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) +void *rxe_pool_get_index_locked(struct rxe_pool *pool, u32 index) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; struct rxe_pool_entry *elem; - u8 *obj = NULL; - unsigned long flags; - - read_lock_irqsave(&pool->pool_lock, flags); + u8 *obj; node = pool->index.tree.rb_node; @@ -428,6 +425,16 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) obj = NULL; } + return obj; +} + +void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) +{ + u8 *obj; + unsigned long flags; + + read_lock_irqsave(&pool->pool_lock, flags); + obj = rxe_pool_get_index_locked(pool, index); read_unlock_irqrestore(&pool->pool_lock, flags); return obj; @@ -438,7 +445,7 @@ void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; struct rxe_pool_entry *elem; - u8 *obj = NULL; + u8 *obj; int cmp; node = pool->key.tree.rb_node; @@ -469,7 +476,7 @@ void *rxe_pool_get_key_locked(struct rxe_pool *pool, void *key) void *rxe_pool_get_key(struct rxe_pool *pool, void *key) { - u8 *obj = NULL; + u8 *obj; unsigned long flags; read_lock_irqsave(&pool->pool_lock, flags);