From patchwork Mon Dec 19 05:46:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Matsuda (Fujitsu)" X-Patchwork-Id: 13076246 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E783EC4332F for ; Mon, 19 Dec 2022 05:47:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230283AbiLSFrB (ORCPT ); Mon, 19 Dec 2022 00:47:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229499AbiLSFq4 (ORCPT ); Mon, 19 Dec 2022 00:46:56 -0500 Received: from esa2.hc1455-7.c3s2.iphmx.com (esa2.hc1455-7.c3s2.iphmx.com [207.54.90.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA298958E for ; Sun, 18 Dec 2022 21:46:54 -0800 (PST) X-IronPort-AV: E=McAfee;i="6500,9779,10565"; a="100589222" X-IronPort-AV: E=Sophos;i="5.96,255,1665414000"; d="scan'208";a="100589222" Received: from unknown (HELO oym-r1.gw.nic.fujitsu.com) ([210.162.30.89]) by esa2.hc1455-7.c3s2.iphmx.com with ESMTP; 19 Dec 2022 14:46:52 +0900 Received: from oym-m1.gw.nic.fujitsu.com (oym-nat-oym-m1.gw.nic.fujitsu.com [192.168.87.58]) by oym-r1.gw.nic.fujitsu.com (Postfix) with ESMTP id 82BCED29E1 for ; Mon, 19 Dec 2022 14:46:51 +0900 (JST) Received: from m3004.s.css.fujitsu.com (m3004.s.css.fujitsu.com [10.128.233.124]) by oym-m1.gw.nic.fujitsu.com (Postfix) with ESMTP id B2385D8C03 for ; Mon, 19 Dec 2022 14:46:50 +0900 (JST) Received: from localhost.localdomain (unknown [10.19.3.107]) by m3004.s.css.fujitsu.com (Postfix) with ESMTP id 75EC1200B39D; Mon, 19 Dec 2022 14:46:50 +0900 (JST) From: Daisuke Matsuda To: linux-rdma@vger.kernel.org, leonro@nvidia.com, jgg@nvidia.com, zyjzyj2000@gmail.com Cc: rpearsonhpe@gmail.com, Rao.Shoaib@oracle.com, lizhijian@fujitsu.com, Daisuke Matsuda Subject: [PATCH v2 1/2] RDMA/rxe: Fix inaccurate constants in rxe_type_info Date: Mon, 19 Dec 2022 14:46:43 +0900 Message-Id: <20221219054644.4530-1-matsuda-daisuke@fujitsu.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org ibv_query_device() has reported incorrect device attributes, which are actually not used by the device. Make the constants correspond with the attributes shown to users. Fixes: 3ccffe8abf2f ("RDMA/rxe: Move max_elem into rxe_type_info") Fixes: 3225717f6dfa ("RDMA/rxe: Replace red-black trees by xarrays") Signed-off-by: Daisuke Matsuda Reviewed-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_pool.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index f50620f5a0a1..1151c0b5ccea 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -23,16 +23,16 @@ static const struct rxe_type_info { .size = sizeof(struct rxe_ucontext), .elem_offset = offsetof(struct rxe_ucontext, elem), .min_index = 1, - .max_index = UINT_MAX, - .max_elem = UINT_MAX, + .max_index = RXE_MAX_UCONTEXT, + .max_elem = RXE_MAX_UCONTEXT, }, [RXE_TYPE_PD] = { .name = "pd", .size = sizeof(struct rxe_pd), .elem_offset = offsetof(struct rxe_pd, elem), .min_index = 1, - .max_index = UINT_MAX, - .max_elem = UINT_MAX, + .max_index = RXE_MAX_PD, + .max_elem = RXE_MAX_PD, }, [RXE_TYPE_AH] = { .name = "ah", @@ -40,7 +40,7 @@ static const struct rxe_type_info { .elem_offset = offsetof(struct rxe_ah, elem), .min_index = RXE_MIN_AH_INDEX, .max_index = RXE_MAX_AH_INDEX, - .max_elem = RXE_MAX_AH_INDEX - RXE_MIN_AH_INDEX + 1, + .max_elem = RXE_MAX_AH, }, [RXE_TYPE_SRQ] = { .name = "srq", @@ -49,7 +49,7 @@ static const struct rxe_type_info { .cleanup = rxe_srq_cleanup, .min_index = RXE_MIN_SRQ_INDEX, .max_index = RXE_MAX_SRQ_INDEX, - .max_elem = RXE_MAX_SRQ_INDEX - RXE_MIN_SRQ_INDEX + 1, + .max_elem = RXE_MAX_SRQ, }, [RXE_TYPE_QP] = { .name = "qp", @@ -58,7 +58,7 @@ static const struct rxe_type_info { .cleanup = rxe_qp_cleanup, .min_index = RXE_MIN_QP_INDEX, .max_index = RXE_MAX_QP_INDEX, - .max_elem = RXE_MAX_QP_INDEX - RXE_MIN_QP_INDEX + 1, + .max_elem = RXE_MAX_QP, }, [RXE_TYPE_CQ] = { .name = "cq", @@ -66,8 +66,8 @@ static const struct rxe_type_info { .elem_offset = offsetof(struct rxe_cq, elem), .cleanup = rxe_cq_cleanup, .min_index = 1, - .max_index = UINT_MAX, - .max_elem = UINT_MAX, + .max_index = RXE_MAX_CQ, + .max_elem = RXE_MAX_CQ, }, [RXE_TYPE_MR] = { .name = "mr", @@ -76,7 +76,7 @@ static const struct rxe_type_info { .cleanup = rxe_mr_cleanup, .min_index = RXE_MIN_MR_INDEX, .max_index = RXE_MAX_MR_INDEX, - .max_elem = RXE_MAX_MR_INDEX - RXE_MIN_MR_INDEX + 1, + .max_elem = RXE_MAX_MR, }, [RXE_TYPE_MW] = { .name = "mw", @@ -85,7 +85,7 @@ static const struct rxe_type_info { .cleanup = rxe_mw_cleanup, .min_index = RXE_MIN_MW_INDEX, .max_index = RXE_MAX_MW_INDEX, - .max_elem = RXE_MAX_MW_INDEX - RXE_MIN_MW_INDEX + 1, + .max_elem = RXE_MAX_MW, }, }; From patchwork Mon Dec 19 05:46:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Daisuke Matsuda (Fujitsu)" X-Patchwork-Id: 13076247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8831CC4332F for ; Mon, 19 Dec 2022 05:48:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229528AbiLSFsD (ORCPT ); Mon, 19 Dec 2022 00:48:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229499AbiLSFsB (ORCPT ); Mon, 19 Dec 2022 00:48:01 -0500 X-Greylist: delayed 63 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Sun, 18 Dec 2022 21:47:59 PST Received: from esa9.hc1455-7.c3s2.iphmx.com (esa9.hc1455-7.c3s2.iphmx.com [139.138.36.223]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D3D83958E for ; Sun, 18 Dec 2022 21:47:59 -0800 (PST) X-IronPort-AV: E=McAfee;i="6500,9779,10565"; a="88778494" X-IronPort-AV: E=Sophos;i="5.96,255,1665414000"; d="scan'208";a="88778494" Received: from unknown (HELO yto-r4.gw.nic.fujitsu.com) ([218.44.52.220]) by esa9.hc1455-7.c3s2.iphmx.com with ESMTP; 19 Dec 2022 14:46:53 +0900 Received: from yto-m2.gw.nic.fujitsu.com (yto-nat-yto-m2.gw.nic.fujitsu.com [192.168.83.65]) by yto-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id DAB49D7724 for ; Mon, 19 Dec 2022 14:46:52 +0900 (JST) Received: from m3004.s.css.fujitsu.com (m3004.s.css.fujitsu.com [10.128.233.124]) by yto-m2.gw.nic.fujitsu.com (Postfix) with ESMTP id 2AF8DD355D for ; Mon, 19 Dec 2022 14:46:52 +0900 (JST) Received: from localhost.localdomain (unknown [10.19.3.107]) by m3004.s.css.fujitsu.com (Postfix) with ESMTP id 00AE8200B380; Mon, 19 Dec 2022 14:46:51 +0900 (JST) From: Daisuke Matsuda To: linux-rdma@vger.kernel.org, leonro@nvidia.com, jgg@nvidia.com, zyjzyj2000@gmail.com Cc: rpearsonhpe@gmail.com, Rao.Shoaib@oracle.com, lizhijian@fujitsu.com, Daisuke Matsuda Subject: [PATCH v2 2/2] RDMA/rxe: Prevent faulty rkey generation Date: Mon, 19 Dec 2022 14:46:44 +0900 Message-Id: <20221219054644.4530-2-matsuda-daisuke@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221219054644.4530-1-matsuda-daisuke@fujitsu.com> References: <20221219054644.4530-1-matsuda-daisuke@fujitsu.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org If you create MRs more than 0x10000 times after loading the module, responder starts to reply NAKs for RDMA/Atomic operations because of rkey violation detected in check_rkey(). The root cause is that rkeys are incremented each time a new MR is created and the value overflows into the range reserved for MWs. This commit also increases the value of RXE_MAX_MW that has been limited unlike other parameters. Fixes: 0994a1bcd5f7 ("RDMA/rxe: Bump up default maximum values used via uverbs") Signed-off-by: Daisuke Matsuda Tested-by: Li Zhijian Reviewed-by: Li Zhijian --- v1->v2: Fixed the value of RXE_MAX_MR as suggested by Li. Increased the value of RXE_MAX_MW. drivers/infiniband/sw/rxe/rxe_param.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index a754fc902e3d..9c9b5f0d929b 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -98,11 +98,11 @@ enum rxe_device_param { RXE_MAX_SRQ = DEFAULT_MAX_VALUE - RXE_MIN_SRQ_INDEX, RXE_MIN_MR_INDEX = 0x00000001, - RXE_MAX_MR_INDEX = DEFAULT_MAX_VALUE, - RXE_MAX_MR = DEFAULT_MAX_VALUE - RXE_MIN_MR_INDEX, - RXE_MIN_MW_INDEX = 0x00010001, - RXE_MAX_MW_INDEX = 0x00020000, - RXE_MAX_MW = 0x00001000, + RXE_MAX_MR_INDEX = DEFAULT_MAX_VALUE >> 1, + RXE_MAX_MR = RXE_MAX_MR_INDEX - RXE_MIN_MR_INDEX, + RXE_MIN_MW_INDEX = (DEFAULT_MAX_VALUE >> 1) + 1, + RXE_MAX_MW_INDEX = DEFAULT_MAX_VALUE, + RXE_MAX_MW = RXE_MAX_MW_INDEX - RXE_MIN_MW_INDEX, RXE_MAX_PKT_PER_ACK = 64,