From patchwork Fri Jan 22 19:29:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F066C433E9 for ; Fri, 22 Jan 2021 19:40:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1986C23B08 for ; Fri, 22 Jan 2021 19:40:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729635AbhAVTiU (ORCPT ); Fri, 22 Jan 2021 14:38:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730669AbhAVTap (ORCPT ); Fri, 22 Jan 2021 14:30:45 -0500 Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6732EC061786 for ; Fri, 22 Jan 2021 11:30:05 -0800 (PST) Received: by mail-ot1-x332.google.com with SMTP id s2so3996987otp.5 for ; Fri, 22 Jan 2021 11:30:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9nJ7JTUyYxu5fQxugnuhPqq/LbS6Yhpu3mLRRsHkwpo=; b=d7vIkAp6p4fwmJ54DXqWblOtqPdNUaIUIaRgyen6sBVnBZEWDQYti4oKRr0cIHl1GB m3JszNwqgRLwVL1UooU5S31zCQ/ol5phkIqQE7JPZXbZhjMD0te7uXJ4rZXIqeDg30h0 oogobo1a9gTjfoFaSQ19n/asQcUqHlVEhkb/hY74kXwdw6ilmAhVPz6TCCxZPZyJ0O3F 4nabuqJUqY2ZFNoDZEcd97FLdygfAEKe88dfCXrKZ7WFX32fdVapeA8GLi4dgzNg/xve uQSrfyGXYnYZOCJsZlh1NmTzKYKdftrUgg7mDjaY/R8eWkM465tfSbTjPHVL75tDDpgb 55TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9nJ7JTUyYxu5fQxugnuhPqq/LbS6Yhpu3mLRRsHkwpo=; b=tUE+D+DfU9KLwQEEySWSF/K0qBjzYFy2+WCQ1n0EI+9FADSDmeO6+cYCkPDvV7/lyx FtNelRvp9ciJoJTLClEK58UT/+ZqUhvY2N0zn9QUZnOvPcGQguWCB+VqdewnvHffJG+d FM3D7EaL1z968xbD67d6qugHxdPz0GLkzRtbs2Dh9OPvWr+yQhCfqv8MGY/9ZfPnzo5B hE8nnab+PfK/bQ6qHiD6QNVsDslM4YjPqrk5bu+6TBo2XxWfnqpq9P9RQHJqI5jFiJdR EXzAKmLn65csjTprbI6rJ/dLyHphohWFnok14Zm0sO4FB3BGqURnfFLYS0t6Age9kiyY yh+A== X-Gm-Message-State: AOAM533OLGY56uSwYKrDP2xj4SvMPlXuyUgGWoFKYWcgicAW+dTxTbBS BUrQO5RZQ1GhzLgu0UnceLc= X-Google-Smtp-Source: ABdhPJxaWM2Kqw2AyzlTtYfG9przv5SeWiDQSA4A1yDh5FHQINj2jNZqfn09cooFAb3QsjfJHyHlpQ== X-Received: by 2002:a9d:4d05:: with SMTP id n5mr4532458otf.99.1611343803338; Fri, 22 Jan 2021 11:30:03 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:02 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson , syzbot+ec2fd72374785d0e558e@syzkaller.appspotmail.com Subject: [PATCH for-next v2 1/6] RDMA/rxe: Fix bug in rxe_alloc Date: Fri, 22 Jan 2021 13:29:38 -0600 Message-Id: <20210122192943.5538-2-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org A recent patch which added an 'unlocked' version of rxe_alloc introduced a bug causing kzalloc(..., GFP_KERNEL) to be called while holding a spin lock. This patch corrects that error. rxe_alloc_nl() should always be called while holding the pool->pool_lock so the 2nd argument to kzalloc there should be GFP_ATOMIC. rxe_alloc() prior to the change only locked the code around checking that pool->state is RXE_POOL_STATE_VALID to avoid races between working threads and a thread shutting down the rxe driver. This patch reverts rxe_alloc() to this behavior so the lock is not held when kzalloc() is called. Reported-by: syzbot+ec2fd72374785d0e558e@syzkaller.appspotmail.com Fixes: 3853c35e243d ("RDMA/rxe: Add unlocked versions of pool APIs") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 41 ++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index d26730eec720..cfcd55175572 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -343,8 +343,6 @@ void *rxe_alloc_nl(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); - if (pool->state != RXE_POOL_STATE_VALID) return NULL; @@ -356,8 +354,7 @@ void *rxe_alloc_nl(struct rxe_pool *pool) if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; - obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? - GFP_ATOMIC : GFP_KERNEL); + obj = kzalloc(info->size, GFP_ATOMIC); if (!obj) goto out_cnt; @@ -378,14 +375,46 @@ void *rxe_alloc_nl(struct rxe_pool *pool) void *rxe_alloc(struct rxe_pool *pool) { - u8 *obj; unsigned long flags; + struct rxe_type_info *info = &rxe_type_info[pool->type]; + struct rxe_pool_entry *elem; + u8 *obj; + + might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); read_lock_irqsave(&pool->pool_lock, flags); - obj = rxe_alloc_nl(pool); + if (pool->state != RXE_POOL_STATE_VALID) { + read_unlock_irqrestore(&pool->pool_lock, flags); + return NULL; + } + + kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); + if (!ib_device_try_get(&pool->rxe->ib_dev)) + goto out_put_pool; + + if (atomic_inc_return(&pool->num_elem) > pool->max_elem) + goto out_cnt; + + obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? + GFP_ATOMIC : GFP_KERNEL); + if (!obj) + goto out_cnt; + + elem = (struct rxe_pool_entry *)(obj + info->elem_offset); + + elem->pool = pool; + kref_init(&elem->ref_cnt); + return obj; + +out_cnt: + atomic_dec(&pool->num_elem); + ib_device_put(&pool->rxe->ib_dev); +out_put_pool: + rxe_pool_put(pool); + return NULL; } int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) From patchwork Fri Jan 22 19:29:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA303C43381 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B00723B55 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730329AbhAVTjE (ORCPT ); Fri, 22 Jan 2021 14:39:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730673AbhAVTar (ORCPT ); Fri, 22 Jan 2021 14:30:47 -0500 Received: from mail-ot1-x334.google.com (mail-ot1-x334.google.com [IPv6:2607:f8b0:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33DEBC06178B for ; Fri, 22 Jan 2021 11:30:07 -0800 (PST) Received: by mail-ot1-x334.google.com with SMTP id s2so3997027otp.5 for ; Fri, 22 Jan 2021 11:30:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KcQeKXoCKnWw4LoGTmnpc6rvmrTrMihBUZAP9LEh1Fk=; b=K5afm634qhOx596XDyMhXpmrgRiVp5RY8KbjVoDuj++Qz82SyVmP2yqnhorSTKSeRh bvFAmI5lYjdyy8x0S7EtkQTVpPyFmCXlMDUTSaiov97+6F1/FeeWN258Z4Uw8KqmFK2V K8nzLLlYlK7VJXN+rPZsGWozOxhobqNyT4CseVCOm0E579/n8B031NOZePXztMd81n+l hNgkvaKW3ZGd8kxQ4LPWvU67o0eikI1OaFS5L/19px1t8Ud+2UfzVqwFcM4RzgmnVnSG zo2+pQGYs5a03txlPzvZEUSnoceVrAlZbqQ5VBseH5IqL+NHLgxKQBRI22AbtqoNEM5z gD4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KcQeKXoCKnWw4LoGTmnpc6rvmrTrMihBUZAP9LEh1Fk=; b=G1lK0z3t5lsFiUiULQQUJT0DwU4WssTrVOmsKj82jgVxdIeAM0YflJwBu1SuzhJhCc NnxMMpu76ZZfgF6iln6SHiBzNeGmrHE0JNFGYdkW+Fvg1TCPGdR95wQPUZUTALwtUR1b dOepP2PbqHzfDY0y1HgcWOnUYuWpgHPg7BXLbYGUFopxGYFaWfG6id1m/aUfubtikyYt ScqQ2V0jzwLimPmjCpQs1HxHDoDTLqTD0aqWrtvRuNTwIpaq1HuJbreWZyL8Ql7kbnpS wtDleZvST5r9fdc7BLL1vvUIcOrKMhrhtbN2HB4akifBpU0aRKMqKU8aMUCjD1uaRzoK 2Gfg== X-Gm-Message-State: AOAM532kdq3jSUC5Tb1ygvXyiYFNCXKNmdu1858/OVdICBiuAn/2Y+Gd 9K+T0Sz3mi1ZvgS3uPZCegM= X-Google-Smtp-Source: ABdhPJwRt/K469CFUgDtaSQe0abuzHHEmh3iZjVF+SqVGUt52yA9usVg+9TP5Qkg43j3Ap4+GHVrWg== X-Received: by 2002:a9d:19c8:: with SMTP id k66mr4553738otk.89.1611343804084; Fri, 22 Jan 2021 11:30:04 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:03 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson , Hillf Danton Subject: [PATCH for-next v2 2/6] RDMA/rxe: Fix misleading comments and names Date: Fri, 22 Jan 2021 13:29:39 -0600 Message-Id: <20210122192943.5538-3-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The names and comments of the 'unlocked' pool APIs are very misleading and not what was intended. This patch replaces 'rxe_xxx_nl' with 'rxe_xxx__' with comments indicating that the caller is expected to hold the rxe pool lock. Reported-by: Hillf Danton Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_mcast.c | 8 ++-- drivers/infiniband/sw/rxe/rxe_pool.c | 22 +++++------ drivers/infiniband/sw/rxe/rxe_pool.h | 55 +++++++++++++-------------- 3 files changed, 42 insertions(+), 43 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_mcast.c b/drivers/infiniband/sw/rxe/rxe_mcast.c index 5be47ce7d319..b9f06f87866e 100644 --- a/drivers/infiniband/sw/rxe/rxe_mcast.c +++ b/drivers/infiniband/sw/rxe/rxe_mcast.c @@ -15,18 +15,18 @@ static struct rxe_mc_grp *create_grp(struct rxe_dev *rxe, int err; struct rxe_mc_grp *grp; - grp = rxe_alloc_nl(&rxe->mc_grp_pool); + grp = rxe_alloc__(&rxe->mc_grp_pool); if (!grp) return ERR_PTR(-ENOMEM); INIT_LIST_HEAD(&grp->qp_list); spin_lock_init(&grp->mcg_lock); grp->rxe = rxe; - rxe_add_key_nl(grp, mgid); + rxe_add_key__(grp, mgid); err = rxe_mcast_add(rxe, mgid); if (unlikely(err)) { - rxe_drop_key_nl(grp); + rxe_drop_key__(grp); rxe_drop_ref(grp); return ERR_PTR(err); } @@ -47,7 +47,7 @@ int rxe_mcast_get_grp(struct rxe_dev *rxe, union ib_gid *mgid, write_lock_irqsave(&pool->pool_lock, flags); - grp = rxe_pool_get_key_nl(pool, mgid); + grp = rxe_pool_get_key__(pool, mgid); if (grp) goto done; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index cfcd55175572..80deec1e2924 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -266,7 +266,7 @@ static void insert_key(struct rxe_pool *pool, struct rxe_pool_entry *new) return; } -void __rxe_add_key_nl(struct rxe_pool_entry *elem, void *key) +void __rxe_add_key__(struct rxe_pool_entry *elem, void *key) { struct rxe_pool *pool = elem->pool; @@ -280,11 +280,11 @@ void __rxe_add_key(struct rxe_pool_entry *elem, void *key) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_key_nl(elem, key); + __rxe_add_key__(elem, key); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_drop_key_nl(struct rxe_pool_entry *elem) +void __rxe_drop_key__(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -297,11 +297,11 @@ void __rxe_drop_key(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_drop_key_nl(elem); + __rxe_drop_key__(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_add_index_nl(struct rxe_pool_entry *elem) +void __rxe_add_index__(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -315,11 +315,11 @@ void __rxe_add_index(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_add_index_nl(elem); + __rxe_add_index__(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void __rxe_drop_index_nl(struct rxe_pool_entry *elem) +void __rxe_drop_index__(struct rxe_pool_entry *elem) { struct rxe_pool *pool = elem->pool; @@ -333,11 +333,11 @@ void __rxe_drop_index(struct rxe_pool_entry *elem) unsigned long flags; write_lock_irqsave(&pool->pool_lock, flags); - __rxe_drop_index_nl(elem); + __rxe_drop_index__(elem); write_unlock_irqrestore(&pool->pool_lock, flags); } -void *rxe_alloc_nl(struct rxe_pool *pool) +void *rxe_alloc__(struct rxe_pool *pool) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rxe_pool_entry *elem; @@ -507,7 +507,7 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) return obj; } -void *rxe_pool_get_key_nl(struct rxe_pool *pool, void *key) +void *rxe_pool_get_key__(struct rxe_pool *pool, void *key) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; @@ -551,7 +551,7 @@ void *rxe_pool_get_key(struct rxe_pool *pool, void *key) unsigned long flags; read_lock_irqsave(&pool->pool_lock, flags); - obj = rxe_pool_get_key_nl(pool, key); + obj = rxe_pool_get_key__(pool, key); read_unlock_irqrestore(&pool->pool_lock, flags); return obj; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index 373e08554c1c..b9680f9e907c 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -106,11 +106,10 @@ int rxe_pool_init(struct rxe_dev *rxe, struct rxe_pool *pool, /* free resources from object pool */ void rxe_pool_cleanup(struct rxe_pool *pool); -/* allocate an object from pool */ -void *rxe_alloc(struct rxe_pool *pool); +/* allocate an object from pool holding and not holding the pool lock */ +void *rxe_alloc__(struct rxe_pool *pool); -/* allocate an object from pool - no lock */ -void *rxe_alloc_nl(struct rxe_pool *pool); +void *rxe_alloc(struct rxe_pool *pool); /* connect already allocated object to pool */ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem); @@ -118,60 +117,60 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem); #define rxe_add_to_pool(pool, obj) __rxe_add_to_pool(pool, &(obj)->pelem) /* assign an index to an indexed object and insert object into - * pool's rb tree with and without holding the pool_lock + * pool's rb tree holding and not holding the pool_lock */ -void __rxe_add_index(struct rxe_pool_entry *elem); +void __rxe_add_index__(struct rxe_pool_entry *elem); -#define rxe_add_index(obj) __rxe_add_index(&(obj)->pelem) +#define rxe_add_index__(obj) __rxe_add_index__(&(obj)->pelem) -void __rxe_add_index_nl(struct rxe_pool_entry *elem); +void __rxe_add_index(struct rxe_pool_entry *elem); -#define rxe_add_index_nl(obj) __rxe_add_index_nl(&(obj)->pelem) +#define rxe_add_index(obj) __rxe_add_index(&(obj)->pelem) /* drop an index and remove object from rb tree - * with and without holding the pool_lock + * holding and not holding the pool_lock */ -void __rxe_drop_index(struct rxe_pool_entry *elem); +void __rxe_drop_index__(struct rxe_pool_entry *elem); -#define rxe_drop_index(obj) __rxe_drop_index(&(obj)->pelem) +#define rxe_drop_index__(obj) __rxe_drop_index__(&(obj)->pelem) -void __rxe_drop_index_nl(struct rxe_pool_entry *elem); +void __rxe_drop_index(struct rxe_pool_entry *elem); -#define rxe_drop_index_nl(obj) __rxe_drop_index_nl(&(obj)->pelem) +#define rxe_drop_index(obj) __rxe_drop_index(&(obj)->pelem) /* assign a key to a keyed object and insert object into - * pool's rb tree with and without holding pool_lock + * pool's rb tree holding and not holding pool_lock */ +void __rxe_add_key__(struct rxe_pool_entry *elem, void *key); + +#define rxe_add_key__(obj, key) __rxe_add_key__(&(obj)->pelem, key) + void __rxe_add_key(struct rxe_pool_entry *elem, void *key); #define rxe_add_key(obj, key) __rxe_add_key(&(obj)->pelem, key) -void __rxe_add_key_nl(struct rxe_pool_entry *elem, void *key); +/* remove elem from rb tree holding and not holding the pool_lock */ +void __rxe_drop_key__(struct rxe_pool_entry *elem); -#define rxe_add_key_nl(obj, key) __rxe_add_key_nl(&(obj)->pelem, key) +#define rxe_drop_key__(obj) __rxe_drop_key__(&(obj)->pelem) -/* remove elem from rb tree with and without holding pool_lock */ void __rxe_drop_key(struct rxe_pool_entry *elem); #define rxe_drop_key(obj) __rxe_drop_key(&(obj)->pelem) -void __rxe_drop_key_nl(struct rxe_pool_entry *elem); - -#define rxe_drop_key_nl(obj) __rxe_drop_key_nl(&(obj)->pelem) - -/* lookup an indexed object from index with and without holding pool_lock. +/* lookup an indexed object from index holding and not holding the pool_lock. * takes a reference on object */ -void *rxe_pool_get_index(struct rxe_pool *pool, u32 index); +void *rxe_pool_get_index__(struct rxe_pool *pool, u32 index); -void *rxe_pool_get_index_nl(struct rxe_pool *pool, u32 index); +void *rxe_pool_get_index(struct rxe_pool *pool, u32 index); -/* lookup keyed object from key with and without holding pool_lock. +/* lookup keyed object from key holding and not holding the pool_lock. * takes a reference on the objecti */ -void *rxe_pool_get_key(struct rxe_pool *pool, void *key); +void *rxe_pool_get_key__(struct rxe_pool *pool, void *key); -void *rxe_pool_get_key_nl(struct rxe_pool *pool, void *key); +void *rxe_pool_get_key(struct rxe_pool *pool, void *key); /* cleanup an object when all references are dropped */ void rxe_elem_release(struct kref *kref); From patchwork Fri Jan 22 19:29:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42A1AC433E0 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1896323B03 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730189AbhAVTil (ORCPT ); Fri, 22 Jan 2021 14:38:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730668AbhAVTap (ORCPT ); Fri, 22 Jan 2021 14:30:45 -0500 Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A555C0613D6 for ; Fri, 22 Jan 2021 11:30:05 -0800 (PST) Received: by mail-oi1-x236.google.com with SMTP id h192so7249838oib.1 for ; Fri, 22 Jan 2021 11:30:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vyfsrg0WcwUJ4gBjz9SeNw+/uhMhz7C6FbWShe2pYj8=; b=f44++fJ2CP0tPOuOdXaDXsaW2AD1oLnV65wqR7Mfytw0QX5pKOId00iXjdpaXAaUnq iaRI5nPQXbEVg5wKzNv43khWpC7/ZhYYzUWaNvGVJ7uPMqIh52B9LRhmOF/KX+bgBfCe Qbn/lJg31aBrrGJmD5iTE+7gqnG6wXRE059+/R7t9nkIJl7woPEezunm9vjL+CJm4nro u1afrHmFxB7YrrtMYpNufINQsmtVbTrBZtbukqJIPd0RAD8ID23s2kJrfOkrtFnZrk7f /SERTSgLKul0bJKtPllwJjQqEGXcEqgMsGCyfB7u3/zKaLt1iHY+sehAVMRk/Uj5DAeS BbKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vyfsrg0WcwUJ4gBjz9SeNw+/uhMhz7C6FbWShe2pYj8=; b=Mzccy2cdNF5EGDnnupFJh0WpfeFwPCuApbR4Gv49r4BN0N0D8WxNJjZ8sDEslHw8oN 2Sr0RMqhiwi1Nip6+7fYitjGXtm9F5mnKVf8YFDcYL72wVU/meb2qQZ7xOrUe0SLNSGp mvMXvlRXkanviP5+Ffgfa4ffFL1wCpczpaiHuJeqkkIz0JbwfmR7k2a9OYKSq8N4rtST 5V+OljXibcRB3uhJ0JyTO9ZlD1sLLpXSU9rbq71I5z8tsiPkUaFP6mHffKqGELQEKqy+ K6AVX+BUsk901mffJoTabAOUq515/q551qfn9fiXLI87UH3CqPsg7fK4KA99/7qegLuv OiWA== X-Gm-Message-State: AOAM531lfvfsI97+uE7/jbv/5Y2Qswotj08LDAvpX6/wcWTdNp46PB4w q8JN5kVLO6SjwXkhYm9vADN02NLQ6TE= X-Google-Smtp-Source: ABdhPJw9QMQo7M2q5QK5J97DAf91AltH0OQ4Xv7fQ0FO3X5vXk0AmCXpXdGWhaf9XK6hZ/1p9jhXTQ== X-Received: by 2002:aca:911:: with SMTP id 17mr4262117oij.162.1611343804784; Fri, 22 Jan 2021 11:30:04 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:04 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 3/6] RDMA/rxe: Remove RXE_POOL_ATOMIC Date: Fri, 22 Jan 2021 13:29:40 -0600 Message-Id: <20210122192943.5538-4-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org rxe_alloc() used the RXE_POOL_ATOMIC flag in rxe_type_info to select GFP_ATOMIC in calls to kzalloc(). This was intended to handle cases where an object could be created in interrupt context. This no longer occurs since allocating those objects has moved into the core so this flag is not necessary. An incorrect use of this flag was still present for rxe_mc_elem objects and is removed. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 6 +----- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 80deec1e2924..7e68f99558a7 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -84,7 +84,6 @@ struct rxe_type_info rxe_type_info[RXE_NUM_TYPES] = { .name = "rxe-mc_elem", .size = sizeof(struct rxe_mc_elem), .elem_offset = offsetof(struct rxe_mc_elem, pelem), - .flags = RXE_POOL_ATOMIC, }, }; @@ -380,8 +379,6 @@ void *rxe_alloc(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - might_sleep_if(!(pool->flags & RXE_POOL_ATOMIC)); - read_lock_irqsave(&pool->pool_lock, flags); if (pool->state != RXE_POOL_STATE_VALID) { read_unlock_irqrestore(&pool->pool_lock, flags); @@ -397,8 +394,7 @@ void *rxe_alloc(struct rxe_pool *pool) if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; - obj = kzalloc(info->size, (pool->flags & RXE_POOL_ATOMIC) ? - GFP_ATOMIC : GFP_KERNEL); + obj = kzalloc(info->size, GFP_KERNEL); if (!obj) goto out_cnt; diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index b9680f9e907c..f916ad4d0406 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -11,7 +11,6 @@ #define RXE_POOL_CACHE_FLAGS (0) enum rxe_pool_flags { - RXE_POOL_ATOMIC = BIT(0), RXE_POOL_INDEX = BIT(1), RXE_POOL_KEY = BIT(2), RXE_POOL_NO_ALLOC = BIT(4), From patchwork Fri Jan 22 19:29:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D632C433E6 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3D9D823B01 for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730283AbhAVTiy (ORCPT ); Fri, 22 Jan 2021 14:38:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730671AbhAVTaq (ORCPT ); Fri, 22 Jan 2021 14:30:46 -0500 Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47606C061788 for ; Fri, 22 Jan 2021 11:30:06 -0800 (PST) Received: by mail-oi1-x22e.google.com with SMTP id q25so7200573oij.10 for ; Fri, 22 Jan 2021 11:30:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y5WqAkjvDFkAoEyL1IBjczEUhyvt4C1aoieU59QkT2M=; b=JRIoujqcG3HBMhJvxF9p1pqfTKP03h02gGmcPYPTYJpxA7zQxBXN6EeBiNp2qy5oCA tfaMC0qbnvwZgkRW9gf1F5WbBvYHW591iPiSzJdv5c52hucOsbUlH9ej8T9MP5xhDZy4 3n6RQnOYIyJD3+iVJobROnY4TeGXANIFt36sYHXP/XkdxWxBJX3uOq2yNYpQhJH8D6zT 1cJYo05K4eq56PItsaM9pmGPdB+QWs5Urcdl/y/gHDZQhEfyoiSsCoFVotKmp1pJA9p5 r4ZWS3mLifIlsOqKsk31vvAiMIYwZ/H3/lTnUhHjLNy3Ow7C2iaGmXQ2RPhhrMTTAeVU eyZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y5WqAkjvDFkAoEyL1IBjczEUhyvt4C1aoieU59QkT2M=; b=K+FtKRlwua/jzhqTotabeIeLjzIFqGuXTrxBOv9R0v5B8FarzrNH5hjIefZ+OXCKOk bmSY+4uOxltZZcQyFoCfEqPC/7H9pyTFkFSnvG1wVEPseZUNIQH6VnNXzqqfM/RhzxDl l0z6Z4Obg4/isyurd6qAQsIG13VoPaXM/E/pGhKvg6gVQmqIxOIP/k6dKFl/nS9hpc+Q 8X036nbxwd4ddzlatoMn9F7a7V4SBxx5xUZKno0vHkFTONmjIsfuJTta3KhSEn9Fn/j/ bA5YszB8IOlTTmKtx7caJdpJwjqA9aILnAYFzczQr21r55ciCUXT4PBPbgz2tMKvzyNJ vKOQ== X-Gm-Message-State: AOAM531njWRFVDfpTV8GMhhZAoF9Nx6CaIqaRE1aQDalmFIGX+Ihpjmi r/W7I3xy0gGWJyYZ49ESGk8= X-Google-Smtp-Source: ABdhPJycXMrkDsBIt1KYQvyuurfK1DhIJqhC0LtzZydTSeteSa4i/Rrrw+g2wXEoKl5LP30aQhnYZA== X-Received: by 2002:aca:f00a:: with SMTP id o10mr4445571oih.128.1611343805767; Fri, 22 Jan 2021 11:30:05 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:05 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 4/6] RDMA/rxe: Remove references to ib_device and pool Date: Fri, 22 Jan 2021 13:29:41 -0600 Message-Id: <20210122192943.5538-5-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org rxe_pool.c takes references to the pool and ib_device structs for each object allocated and also keeps an atomic num_elem count in each pool. This is more work than is needed. Pool allocation is only called from verbs APIs which already have references to ib_device and pools are only diasbled when the driver is removed so no protection of the pool addresses are needed. The elem count is used to warn if elements are still present in a pool when it is cleaned up which is useful. This patch eliminates the references to the ib_device and pool structs. The previous version only removed the ib_device reference. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 42 ++-------------------------- drivers/infiniband/sw/rxe/rxe_pool.h | 1 - 2 files changed, 2 insertions(+), 41 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 7e68f99558a7..09d8665c5343 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -142,8 +142,6 @@ int rxe_pool_init( atomic_set(&pool->num_elem, 0); - kref_init(&pool->ref_cnt); - rwlock_init(&pool->pool_lock); if (rxe_type_info[type].flags & RXE_POOL_INDEX) { @@ -165,19 +163,6 @@ int rxe_pool_init( return err; } -static void rxe_pool_release(struct kref *kref) -{ - struct rxe_pool *pool = container_of(kref, struct rxe_pool, ref_cnt); - - pool->state = RXE_POOL_STATE_INVALID; - kfree(pool->index.table); -} - -static void rxe_pool_put(struct rxe_pool *pool) -{ - kref_put(&pool->ref_cnt, rxe_pool_release); -} - void rxe_pool_cleanup(struct rxe_pool *pool) { unsigned long flags; @@ -189,7 +174,8 @@ void rxe_pool_cleanup(struct rxe_pool *pool) pool_name(pool)); write_unlock_irqrestore(&pool->pool_lock, flags); - rxe_pool_put(pool); + pool->state = RXE_POOL_STATE_INVALID; + kfree(pool->index.table); } static u32 alloc_index(struct rxe_pool *pool) @@ -345,11 +331,6 @@ void *rxe_alloc__(struct rxe_pool *pool) if (pool->state != RXE_POOL_STATE_VALID) return NULL; - kref_get(&pool->ref_cnt); - - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -366,9 +347,6 @@ void *rxe_alloc__(struct rxe_pool *pool) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return NULL; } @@ -385,12 +363,8 @@ void *rxe_alloc(struct rxe_pool *pool) return NULL; } - kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -407,9 +381,6 @@ void *rxe_alloc(struct rxe_pool *pool) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return NULL; } @@ -422,12 +393,8 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) read_unlock_irqrestore(&pool->pool_lock, flags); return -EINVAL; } - kref_get(&pool->ref_cnt); read_unlock_irqrestore(&pool->pool_lock, flags); - if (!ib_device_try_get(&pool->rxe->ib_dev)) - goto out_put_pool; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -438,9 +405,6 @@ int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) out_cnt: atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); -out_put_pool: - rxe_pool_put(pool); return -EINVAL; } @@ -461,8 +425,6 @@ void rxe_elem_release(struct kref *kref) } atomic_dec(&pool->num_elem); - ib_device_put(&pool->rxe->ib_dev); - rxe_pool_put(pool); } void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.h b/drivers/infiniband/sw/rxe/rxe_pool.h index f916ad4d0406..294839d1eed8 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.h +++ b/drivers/infiniband/sw/rxe/rxe_pool.h @@ -68,7 +68,6 @@ struct rxe_pool { struct rxe_dev *rxe; rwlock_t pool_lock; /* protects pool add/del/search */ size_t elem_size; - struct kref ref_cnt; void (*cleanup)(struct rxe_pool_entry *obj); enum rxe_pool_state state; enum rxe_pool_flags flags; From patchwork Fri Jan 22 19:29:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEBFFC4332B for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7B2C23A6A for ; Fri, 22 Jan 2021 19:40:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730370AbhAVTjP (ORCPT ); Fri, 22 Jan 2021 14:39:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730698AbhAVTbl (ORCPT ); Fri, 22 Jan 2021 14:31:41 -0500 Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E8C6C061793 for ; Fri, 22 Jan 2021 11:30:07 -0800 (PST) Received: by mail-oi1-x233.google.com with SMTP id x137so5981178oix.11 for ; Fri, 22 Jan 2021 11:30:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ibDugmfaLdpp1gYk0tRhaWa1miiO3MXc+5tI8LO/nL8=; b=C0d4KeuGBiJ3MXf4HzzL3TP0BbxQJSvh324T0PnftiJaf40L7wbvFE/hA5i7OJLGxm hIFWxLH2exEeZL3J06qGtFxkjOqvUfuw2Gii51FdezRwS0/Xd3WCTyhnhElPn/SMMdV0 nDY8y5mM0Q5NUgdGi+dFPf45abZHacWPDbBoAgGp/saFZw23bDhYq2umiMhxLjalDYnP NmHnAfdf/FhfnK1WVWHMKebhIR18IDXxd2Q4/V58IbO+9ZqW9lLsk7UdfzTdINT2er9u ImI4tIRjxQmPOhOTw9body9hTO5hy2NWMLbvT9zG0fagsa9VT44t0FSL4W9CdFwRWd+l KaiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ibDugmfaLdpp1gYk0tRhaWa1miiO3MXc+5tI8LO/nL8=; b=tlPoGrTDFclciyeGUYnttsIUFg1gg7ZV8xU9lpB/iZIve3kpd3E3iTICIeSUgxUdwC lyGTQsd4v2Rd65D3XBnQsXO1oWYwP+2PRpuQ1A83FTNkk8HZDGeGjkz9F/aSsO1mcdUP NYkrTYQy1qTSnlZElk0fFwvfVR32pWQDIu+zxx5EVng9Fl5iLyqnhKBGnOqMCeUt+6zt sd827mjAx2+5TiMjbtyucsQLt7s7z633AgWfQ0KrQi7vpuxfjqWd8fKm/1TqUakxgrYz NS5U1XNmB8gjcH86DbqMqCyPGS0w+Jrgz5ady/c0tKiCTpAM5ZZQ6nno2t+kr29o6PET hgzQ== X-Gm-Message-State: AOAM531zXtCPIwjG0W6dR2HYu90vH1kFdKZq3BJshRv3dsCV8xYQW3Ej r2/oLTUljzMcDOMsqVIl7OI= X-Google-Smtp-Source: ABdhPJycj9hUMseqiHvWRBiQPRoTZ+J5nlxtEFfbu2LWabvsYR/tVJxChS7AoMEs1H9mUpxxwaS5WQ== X-Received: by 2002:aca:d504:: with SMTP id m4mr4513645oig.158.1611343806708; Fri, 22 Jan 2021 11:30:06 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:06 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 5/6] RDMA/rxe: Remove unneeded pool->state Date: Fri, 22 Jan 2021 13:29:42 -0600 Message-Id: <20210122192943.5538-6-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org rxe_pool.c uses the field pool->state to mark a pool as invalid when it is shut down and checks it in several pool APIs to verify that the pool has not been shut dowm. This is unneeded because the pools are not marked invalid unless the entire driver is being removed at which point no functional APIs should or could be executing. This patch removes this field and associated code. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 38 +--------------------------- 1 file changed, 1 insertion(+), 37 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 09d8665c5343..7a03d49b263d 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -157,24 +157,16 @@ int rxe_pool_init( pool->key.key_size = rxe_type_info[type].key_size; } - pool->state = RXE_POOL_STATE_VALID; - out: return err; } void rxe_pool_cleanup(struct rxe_pool *pool) { - unsigned long flags; - - write_lock_irqsave(&pool->pool_lock, flags); - pool->state = RXE_POOL_STATE_INVALID; if (atomic_read(&pool->num_elem) > 0) pr_warn("%s pool destroyed with unfree'd elem\n", pool_name(pool)); - write_unlock_irqrestore(&pool->pool_lock, flags); - pool->state = RXE_POOL_STATE_INVALID; kfree(pool->index.table); } @@ -328,9 +320,6 @@ void *rxe_alloc__(struct rxe_pool *pool) struct rxe_pool_entry *elem; u8 *obj; - if (pool->state != RXE_POOL_STATE_VALID) - return NULL; - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -352,19 +341,10 @@ void *rxe_alloc__(struct rxe_pool *pool) void *rxe_alloc(struct rxe_pool *pool) { - unsigned long flags; struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rxe_pool_entry *elem; u8 *obj; - read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) { - read_unlock_irqrestore(&pool->pool_lock, flags); - return NULL; - } - - read_unlock_irqrestore(&pool->pool_lock, flags); - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -386,15 +366,6 @@ void *rxe_alloc(struct rxe_pool *pool) int __rxe_add_to_pool(struct rxe_pool *pool, struct rxe_pool_entry *elem) { - unsigned long flags; - - read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) { - read_unlock_irqrestore(&pool->pool_lock, flags); - return -EINVAL; - } - read_unlock_irqrestore(&pool->pool_lock, flags); - if (atomic_inc_return(&pool->num_elem) > pool->max_elem) goto out_cnt; @@ -437,9 +408,6 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) read_lock_irqsave(&pool->pool_lock, flags); - if (pool->state != RXE_POOL_STATE_VALID) - goto out; - node = pool->index.tree.rb_node; while (node) { @@ -460,8 +428,8 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) obj = NULL; } -out: read_unlock_irqrestore(&pool->pool_lock, flags); + return obj; } @@ -473,9 +441,6 @@ void *rxe_pool_get_key__(struct rxe_pool *pool, void *key) u8 *obj = NULL; int cmp; - if (pool->state != RXE_POOL_STATE_VALID) - goto out; - node = pool->key.tree.rb_node; while (node) { @@ -499,7 +464,6 @@ void *rxe_pool_get_key__(struct rxe_pool *pool, void *key) obj = NULL; } -out: return obj; } From patchwork Fri Jan 22 19:29:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12040829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A40AC433E0 for ; Fri, 22 Jan 2021 23:08:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 285A223B51 for ; Fri, 22 Jan 2021 23:08:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730441AbhAVTjU (ORCPT ); Fri, 22 Jan 2021 14:39:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730696AbhAVTbl (ORCPT ); Fri, 22 Jan 2021 14:31:41 -0500 Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com [IPv6:2607:f8b0:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8226FC061794 for ; Fri, 22 Jan 2021 11:30:09 -0800 (PST) Received: by mail-ot1-x32e.google.com with SMTP id d1so6172556otl.13 for ; Fri, 22 Jan 2021 11:30:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yKvKuND9nVqM8c1qCj4KjZBcq0n6sK1TXbvaI7RPjd0=; b=LC9nDPgKvB+qhAspCr8v47GFegAaLFlIBIS6kBx/IRQER86saEgZAN/4w6XPLl559f g2w9hKjdfJLDZtyL+iMIlwckO3N1NddosWB76FsRQSk4/lGju3ZWGtRC+DozDKCY8+vL 3lyqkp+53gkqvz6d2swy/VMjDF/2I+GM+L0gGuXNwMDhxDhJO+w+hfCcdIeibus51gjg 5G6WufuL52hIQR4NIVIjeG4LTC+et6vbwmaX60MwUNTgJ28IXtlKeqoYvGfWE7iTIMw3 WUPF+BFGAjr+sIMQ6znYvNfrQSFq8etkAI7rMt8WmYiflLU206p3oMJQw7L49FYz2JXU IcHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yKvKuND9nVqM8c1qCj4KjZBcq0n6sK1TXbvaI7RPjd0=; b=ujTiyvYBalfNlHFNH2ggfTmELIX+8vAv8KTNRoPQweZFYRsC/Qu/7AlFAHN6+v2itC 9CKjvGkoEzgMlVxY8SBEpAPDsormZjQ1ArG3DSuOCeQu1T5J5a7eBvHHds5YG2FXzRUH kMj/4aDxrbmOEACObCQhP7vjas+3tvPQoTR5FojCMpz/Krtj1eEziaoMHRl+9qwyYQnh KSFolQcc0O2fc7LDg/5cPvqC7EsZbcPUvxdDta/2YAUKDDrz/4CPvl+2KSQc9OyOOekc R9vbber8EbPzms7DnvqOaGvXf3hQ/QTTlZ7+DnDBK3WY0tXal7lZinCmEj3XeG8KaeTM n6Fg== X-Gm-Message-State: AOAM531z+EwfWNPnLF6KW1ijFVTc52yZX0znZUvbx7ckYGP+E2GDpR9D SKO7cysP3TWvV9Nk7X+VUr8= X-Google-Smtp-Source: ABdhPJwogg+bwnQXNrQJCwtvqv/H5z8XX8pCOtP//EOai7PrACIgCRYI6ZcboaB4kiYOWaSeV/jxrg== X-Received: by 2002:a9d:3642:: with SMTP id w60mr3567845otb.322.1611343807522; Fri, 22 Jan 2021 11:30:07 -0800 (PST) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (2603-8081-140c-1a00-7fcf-0a74-ddeb-d9b7.res6.spectrum.com. [2603:8081:140c:1a00:7fcf:a74:ddeb:d9b7]) by smtp.gmail.com with ESMTPSA id 36sm1835546oty.62.2021.01.22.11.30.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Jan 2021 11:30:07 -0800 (PST) From: Bob Pearson X-Google-Original-From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v2 6/6] RDMA/rxe: Replace missing rxe_pool_get_index__ Date: Fri, 22 Jan 2021 13:29:43 -0600 Message-Id: <20210122192943.5538-7-rpearson@hpe.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210122192943.5538-1-rpearson@hpe.com> References: <20210122192943.5538-1-rpearson@hpe.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org One of the pool APIs for when caller is holding lock was not defined but is declared in rxe_pool.h. This patch adds the definition. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_pool.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index 7a03d49b263d..3755e163f257 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -398,15 +398,12 @@ void rxe_elem_release(struct kref *kref) atomic_dec(&pool->num_elem); } -void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) +void *rxe_pool_get_index__(struct rxe_pool *pool, u32 index) { struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; struct rxe_pool_entry *elem; - u8 *obj = NULL; - unsigned long flags; - - read_lock_irqsave(&pool->pool_lock, flags); + u8 *obj; node = pool->index.tree.rb_node; @@ -428,6 +425,16 @@ void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) obj = NULL; } + return obj; +} + +void *rxe_pool_get_index(struct rxe_pool *pool, u32 index) +{ + u8 *obj; + unsigned long flags; + + read_lock_irqsave(&pool->pool_lock, flags); + obj = rxe_pool_get_index__(pool, index); read_unlock_irqrestore(&pool->pool_lock, flags); return obj; @@ -438,7 +445,7 @@ void *rxe_pool_get_key__(struct rxe_pool *pool, void *key) struct rxe_type_info *info = &rxe_type_info[pool->type]; struct rb_node *node; struct rxe_pool_entry *elem; - u8 *obj = NULL; + u8 *obj; int cmp; node = pool->key.tree.rb_node; @@ -469,7 +476,7 @@ void *rxe_pool_get_key__(struct rxe_pool *pool, void *key) void *rxe_pool_get_key(struct rxe_pool *pool, void *key) { - u8 *obj = NULL; + u8 *obj; unsigned long flags; read_lock_irqsave(&pool->pool_lock, flags);