From patchwork Sun Nov 13 01:08:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0247C433FE for ; Sun, 13 Nov 2022 01:08:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235073AbiKMBIm (ORCPT ); Sat, 12 Nov 2022 20:08:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230170AbiKMBIm (ORCPT ); Sat, 12 Nov 2022 20:08:42 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B76125C7 for ; Sat, 12 Nov 2022 17:08:39 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Lv032lrzlN9ChNEAvXxauoX7CEIDjAQVR47zbEM1hqA=; b=MZntfJDfQgGD+gdCYIMKlcd0gnPPaRIplLTZXKlhrRjgzYeiiDJMPjj6YT89ONae6ZYeP/ ZLkWgyaIj91as1NYbuldoIjHr6XXcl80bfyzyteGeNt/VrtkTe7v8r5rmZLAVuA7JT7kZs Kw0M9Z70j/5nyVc4+sAgLmpkJSU9VcQ= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 01/12] RDMA/rtrs-srv: Remove ib_dev_count from rtrs_srv_ib_ctx Date: Sun, 13 Nov 2022 09:08:12 +0800 Message-Id: <20221113010823.6436-2-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The ib_dev_count is supposed to track the number of added ib devices which is only used in rtrs_srv_{add,remove}_one. However we only trigger rtrs_srv_add_one from rnbd_srv_init_module -> rtrs_srv_open -> ib_register_client -> client->add which should happen only once. And so is rtrs_srv_close since it is only called by unload rnbd-server or failure case when load rnbd-server. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 16 ---------------- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 1 - 2 files changed, 17 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 22d7ba05e9fe..79504aaef0cc 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -2097,8 +2097,6 @@ static int rtrs_srv_add_one(struct ib_device *device) int ret = 0; mutex_lock(&ib_ctx.ib_dev_mutex); - if (ib_ctx.ib_dev_count) - goto out; /* * Since our CM IDs are NOT bound to any ib device we will create them @@ -2108,21 +2106,12 @@ static int rtrs_srv_add_one(struct ib_device *device) ret = rtrs_srv_rdma_init(ctx, ib_ctx.port); if (ret) { /* - * We errored out here. * According to the ib code, if we encounter an error here then the * error code is ignored, and no more calls to our ops are made. */ pr_err("Failed to initialize RDMA connection"); - goto err_out; } -out: - /* - * Keep a track on the number of ib devices added - */ - ib_ctx.ib_dev_count++; - -err_out: mutex_unlock(&ib_ctx.ib_dev_mutex); return ret; } @@ -2132,10 +2121,6 @@ static void rtrs_srv_remove_one(struct ib_device *device, void *client_data) struct rtrs_srv_ctx *ctx; mutex_lock(&ib_ctx.ib_dev_mutex); - ib_ctx.ib_dev_count--; - - if (ib_ctx.ib_dev_count) - goto out; /* * Since our CM IDs are NOT bound to any ib device we will remove them @@ -2145,7 +2130,6 @@ static void rtrs_srv_remove_one(struct ib_device *device, void *client_data) rdma_destroy_id(ctx->cm_id_ip); rdma_destroy_id(ctx->cm_id_ib); -out: mutex_unlock(&ib_ctx.ib_dev_mutex); } diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index 2f8a638e36fa..eccc432b0715 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -126,7 +126,6 @@ struct rtrs_srv_ib_ctx { struct rtrs_srv_ctx *srv_ctx; u16 port; struct mutex ib_dev_mutex; - int ib_dev_count; }; extern struct class *rtrs_dev_class; From patchwork Sun Nov 13 01:08:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6A67C43217 for ; Sun, 13 Nov 2022 01:08:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230170AbiKMBIn (ORCPT ); Sat, 12 Nov 2022 20:08:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235067AbiKMBIm (ORCPT ); Sat, 12 Nov 2022 20:08:42 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 736A4F10 for ; Sat, 12 Nov 2022 17:08:41 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sg+KPCtBI0xZlYnxlcYJguRQpWMDLjEmZ9vkTuUnmeM=; b=H3BO5yCZQc1PgmvvGkFDe/F8on27WKv3RVOJ6mLdxnFbiIvkUM5WLDZi4Mnx0LCYl309qc Un5JBMVOLGd0LwjwacOo4HbOvW+PZTxVO89K2QP7xlZ8H2RBh8JgVRCBRc1sCt+CML6MOT NV27oDji05Ra/pFM/q7SCnnCH9WvA3s= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 02/12] RDMA/rtrs-srv: Refactor rtrs_srv_rdma_cm_handler Date: Sun, 13 Nov 2022 09:08:13 +0800 Message-Id: <20221113010823.6436-3-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The RDMA_CM_EVENT_CONNECT_REQUEST is quite different to other types, let's checking it separately at the beginning of routine, then we can avoid the identation accordingly. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 79504aaef0cc..2cc8b423bcaa 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1948,24 +1948,19 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, static int rtrs_srv_rdma_cm_handler(struct rdma_cm_id *cm_id, struct rdma_cm_event *ev) { - struct rtrs_srv_path *srv_path = NULL; - struct rtrs_path *s = NULL; - - if (ev->event != RDMA_CM_EVENT_CONNECT_REQUEST) { - struct rtrs_con *c = cm_id->context; - - s = c->path; - srv_path = to_srv_path(s); - } + struct rtrs_con *c = cm_id->context; + struct rtrs_path *s = c->path; + struct rtrs_srv_path *srv_path = to_srv_path(s); - switch (ev->event) { - case RDMA_CM_EVENT_CONNECT_REQUEST: + if (ev->event == RDMA_CM_EVENT_CONNECT_REQUEST) /* * In case of error cma.c will destroy cm_id, * see cma_process_remove() */ return rtrs_rdma_connect(cm_id, ev->param.conn.private_data, ev->param.conn.private_data_len); + + switch (ev->event) { case RDMA_CM_EVENT_ESTABLISHED: /* Nothing here */ break; From patchwork Sun Nov 13 01:08:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BB99C4332F for ; Sun, 13 Nov 2022 01:08:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235076AbiKMBIo (ORCPT ); Sat, 12 Nov 2022 20:08:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235067AbiKMBIn (ORCPT ); Sat, 12 Nov 2022 20:08:43 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4027FF10 for ; Sat, 12 Nov 2022 17:08:43 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301721; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Z8HcS3vIAvBjetIAExa7JzZ0F7cUD0/OVS09XoyQLM=; b=XA7MlO0wtJ0YSJwnhiWVp3swXplIwkmpgYoYFO0UMtNJOv9rtWaORtD10CCwjy841qtez6 mfI/wnivJj3wOwWE80HWkziGKYB48lgRnB/wbfMK7uSsNSmnlJWv5ET2pg74AgCr18iGqI tV0VJhALFiHIty9JU1d3rIAG4H9AJuQ= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 03/12] RDMA/rtrs-srv: Only close srv_path if it is just allocated Date: Sun, 13 Nov 2022 09:08:14 +0800 Message-Id: <20221113010823.6436-4-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org RTRS creates several connections per nr_cpu_ids, it makes more sense to only close the path when it just allocated. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 2cc8b423bcaa..063082d29fc6 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1833,6 +1833,7 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, u16 version, con_num, cid; u16 recon_cnt; int err = -ECONNRESET; + bool alloc_path = false; if (len < sizeof(*msg)) { pr_err("Invalid RTRS connection request\n"); @@ -1906,6 +1907,7 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, pr_err("RTRS server session allocation failed: %d\n", err); goto reject_w_err; } + alloc_path = true; } err = create_con(srv_path, cm_id, cid); if (err) { @@ -1940,7 +1942,8 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, close_and_return_err: mutex_unlock(&srv->paths_mutex); - close_path(srv_path); + if (alloc_path) + close_path(srv_path); return err; } From patchwork Sun Nov 13 01:08:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2A8CC433FE for ; Sun, 13 Nov 2022 01:08:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235105AbiKMBIr (ORCPT ); Sat, 12 Nov 2022 20:08:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235078AbiKMBIp (ORCPT ); Sat, 12 Nov 2022 20:08:45 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BB9225E9 for ; Sat, 12 Nov 2022 17:08:45 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301723; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2lOdAc8lMynHl4hfQD1aIg7xEwiGyZy2SIdkL0eMrE0=; b=d83YCq7U/Ajwb3tZm3Rb2/vto1neyU7wBvn1aCD671vRh6VYWfigknyU3bGOzYq5HsfWqZ knMMOj4wL13aGvQe+NjGw73u0GBAsvr/Q2ERH63L8r2LJa/irbHDm52Zpl4UX8EsjlfDHr SVtb+qBAvPL8LzomeblEn5BPKIOBvdE= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 04/12] RDMA/rtrs-srv: refactor the handling of failure case in map_cont_bufs Date: Sun, 13 Nov 2022 09:08:15 +0800 Message-Id: <20221113010823.6436-5-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Let's call unmap_cont_bufs when failure happens, and also only update mrs_num after everything is settled which means we can remove 'mri'. Signed-off-by: Guoqing Jiang Acked-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 47 +++++++++++--------------- 1 file changed, 20 insertions(+), 27 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 063082d29fc6..88eae0dcf87f 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -561,9 +561,11 @@ static int map_cont_bufs(struct rtrs_srv_path *srv_path) { struct rtrs_srv_sess *srv = srv_path->srv; struct rtrs_path *ss = &srv_path->s; - int i, mri, err, mrs_num; + int i, err, mrs_num; unsigned int chunk_bits; int chunks_per_mr = 1; + struct ib_mr *mr; + struct sg_table *sgt; /* * Here we map queue_depth chunks to MR. Firstly we have to @@ -586,16 +588,14 @@ static int map_cont_bufs(struct rtrs_srv_path *srv_path) if (!srv_path->mrs) return -ENOMEM; - srv_path->mrs_num = mrs_num; - - for (mri = 0; mri < mrs_num; mri++) { - struct rtrs_srv_mr *srv_mr = &srv_path->mrs[mri]; - struct sg_table *sgt = &srv_mr->sgt; + for (srv_path->mrs_num = 0; srv_path->mrs_num < mrs_num; + srv_path->mrs_num++) { + struct rtrs_srv_mr *srv_mr = &srv_path->mrs[srv_path->mrs_num]; struct scatterlist *s; - struct ib_mr *mr; int nr, nr_sgt, chunks; - chunks = chunks_per_mr * mri; + sgt = &srv_mr->sgt; + chunks = chunks_per_mr * srv_path->mrs_num; if (!always_invalidate) chunks_per_mr = min_t(int, chunks_per_mr, srv->queue_depth - chunks); @@ -644,31 +644,24 @@ static int map_cont_bufs(struct rtrs_srv_path *srv_path) ib_update_fast_reg_key(mr, ib_inc_rkey(mr->rkey)); srv_mr->mr = mr; - - continue; -err: - while (mri--) { - srv_mr = &srv_path->mrs[mri]; - sgt = &srv_mr->sgt; - mr = srv_mr->mr; - rtrs_iu_free(srv_mr->iu, srv_path->s.dev->ib_dev, 1); -dereg_mr: - ib_dereg_mr(mr); -unmap_sg: - ib_dma_unmap_sg(srv_path->s.dev->ib_dev, sgt->sgl, - sgt->nents, DMA_BIDIRECTIONAL); -free_sg: - sg_free_table(sgt); - } - kfree(srv_path->mrs); - - return err; } chunk_bits = ilog2(srv->queue_depth - 1) + 1; srv_path->mem_bits = (MAX_IMM_PAYL_BITS - chunk_bits); return 0; + +dereg_mr: + ib_dereg_mr(mr); +unmap_sg: + ib_dma_unmap_sg(srv_path->s.dev->ib_dev, sgt->sgl, + sgt->nents, DMA_BIDIRECTIONAL); +free_sg: + sg_free_table(sgt); +err: + unmap_cont_bufs(srv_path); + + return err; } static void rtrs_srv_hb_err_handler(struct rtrs_con *c) From patchwork Sun Nov 13 01:08:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0ADAC4332F for ; Sun, 13 Nov 2022 01:08:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235067AbiKMBIv (ORCPT ); Sat, 12 Nov 2022 20:08:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235078AbiKMBIs (ORCPT ); Sat, 12 Nov 2022 20:08:48 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7ECC2636 for ; Sat, 12 Nov 2022 17:08:46 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9LfXALLEmKe8jXFQsk0wosHGFnkjYLt7D/WoXlEPdP0=; b=xCWPk0KiiI3K5EYg2e9XDTsaBVU6PhomTBaPlpM+PdNSYzb1qU+xuLzNYYyABlN+Pvlvnb zYd4oNtCTyBNp7noVJ1s0U+wTDggyjflkzUXxMT64fqnFAOCxsVe1XcxHwTQCFlg494ozI e69sWyXlx0AxqvHWBgYGWv4QwyVsi60= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 05/12] RDMA/rtrs-srv: Correct the checking of ib_map_mr_sg Date: Sun, 13 Nov 2022 09:08:16 +0800 Message-Id: <20221113010823.6436-6-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We should check with nr_sgt, also the only successful case is that all sg elements are mapped, so make it explict. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 88eae0dcf87f..f3bf5bbb4377 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -622,8 +622,8 @@ static int map_cont_bufs(struct rtrs_srv_path *srv_path) } nr = ib_map_mr_sg(mr, sgt->sgl, nr_sgt, NULL, max_chunk_size); - if (nr < 0 || nr < sgt->nents) { - err = nr < 0 ? nr : -EINVAL; + if (nr != nr_sgt) { + err = -EINVAL; goto dereg_mr; } From patchwork Sun Nov 13 01:08:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AB39C43217 for ; Sun, 13 Nov 2022 01:08:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235085AbiKMBIw (ORCPT ); Sat, 12 Nov 2022 20:08:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235084AbiKMBIu (ORCPT ); Sat, 12 Nov 2022 20:08:50 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C86AF10 for ; Sat, 12 Nov 2022 17:08:49 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301727; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DtlP1JN+qdU0xxBuwRmh7eGTK3TyecdbhgMfnjsW9Q=; b=uvyQnqkp+19DCKWCE7gO2R52mMYRWCsCJ0/zgO/q70R7MiI957h+biMvLNapHqA/EJ5sDs +fhAWFXm/6tchqIAhLVbPKFiUawBxFzREmIR2/ft045+8EBL5MFJH1FUfbuNchjrht1NH9 wCFSYgoHstSOzwVGLeHh+XPQOWWf6rA= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 06/12] RDMA/rtrs-clt: Correct the checking of ib_map_mr_sg Date: Sun, 13 Nov 2022 09:08:17 +0800 Message-Id: <20221113010823.6436-7-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We should check with count, also the only successful case is that all sg elements are mapped, so make it explict. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 8546b8816524..5ffc170dae8c 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1064,9 +1064,7 @@ static int rtrs_map_sg_fr(struct rtrs_clt_io_req *req, size_t count) /* Align the MR to a 4K page size to match the block virt boundary */ nr = ib_map_mr_sg(req->mr, req->sglist, count, NULL, SZ_4K); - if (nr < 0) - return nr; - if (nr < req->sg_cnt) + if (nr != count) return -EINVAL; ib_update_fast_reg_key(req->mr, ib_inc_rkey(req->mr->rkey)); From patchwork Sun Nov 13 01:08:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6969BC4332F for ; Sun, 13 Nov 2022 01:08:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235104AbiKMBIx (ORCPT ); Sat, 12 Nov 2022 20:08:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235088AbiKMBIv (ORCPT ); Sat, 12 Nov 2022 20:08:51 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78E0D25C7 for ; Sat, 12 Nov 2022 17:08:50 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PMIKurErvaKaHZgyBaYTVTZbaYHPzeiTsrjhEAmq01Y=; b=JNPyCjhKRBpOH2+Cm9GAkc6/ittCk/5msX40M29A47nB7wewGWToV5lt9ykayP6n8R8cnN kt3yVuK/9/yot+JqWnDnh6G4+RYnB26pA6AeyXjJ5ldIVKm585IMQJFkaq8pN2EgsMXskI NdIzsaJZSVFfztok4GoXGO9P7J0z97k= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 07/12] RDMA/rtrs-srv: Remove outdated comments from create_con Date: Sun, 13 Nov 2022 09:08:18 +0800 Message-Id: <20221113010823.6436-8-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Remove the orphan comments. Signed-off-by: Guoqing Jiang Acked-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index f3bf5bbb4377..4c883c57c2ef 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1671,12 +1671,6 @@ static int create_con(struct rtrs_srv_path *srv_path, srv->queue_depth * (1 + 2) + 1); max_recv_wr = srv->queue_depth + 1; - /* - * If we have all receive requests posted and - * all write requests posted and each read request - * requires an invalidate request + drain - * and qp gets into error state. - */ } cq_num = max_send_wr + max_recv_wr; atomic_set(&con->c.sq_wr_avail, max_send_wr); From patchwork Sun Nov 13 01:08:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5250BC43217 for ; Sun, 13 Nov 2022 01:08:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235123AbiKMBIz (ORCPT ); Sat, 12 Nov 2022 20:08:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235078AbiKMBIx (ORCPT ); Sat, 12 Nov 2022 20:08:53 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6030626C2 for ; Sat, 12 Nov 2022 17:08:52 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301731; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sV+CY13YgHY0UztuLmoBvu3Ttgx3cx33KGSnDdjp2sg=; b=qZY6TmgGSKWwSGqGW5ThT2eVWLBttpA6LozS6H9tYwy1weW7/NA1lTnaXEu5H6RodzaEz/ w4tIbRdtR4N7CpNfd+0ChQ0gZAunhAh49FvpnDWO8BEl/NspN3VJnOoeatCvS6rJ3dCF1k pooaX8MKi6YU78URa3faFmakZU/vyhk= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 08/12] RDMA/rtrs: Kill recon_cnt from several structs Date: Sun, 13 Nov 2022 09:08:19 +0800 Message-Id: <20221113010823.6436-9-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Seems the only relevant comment about recon_cnt is, /* * On every new session connections increase reconnect counter * to avoid clashes with previous sessions not yet closed * sessions on a server side. */ However, it is not clear how the recon_cnt avoid clashed at these places in the commit since no where checks it. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 8 -------- drivers/infiniband/ulp/rtrs/rtrs-pri.h | 3 --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 7 +------ 3 files changed, 1 insertion(+), 17 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 5ffc170dae8c..dcc8c041a141 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1802,7 +1802,6 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con) .version = cpu_to_le16(RTRS_PROTO_VER), .cid = cpu_to_le16(con->c.cid), .cid_num = cpu_to_le16(clt_path->s.con_num), - .recon_cnt = cpu_to_le16(clt_path->s.recon_cnt), }; msg.first_conn = clt_path->for_new_clt ? FIRST_CONN : 0; uuid_copy(&msg.sess_uuid, &clt_path->s.uuid); @@ -2336,13 +2335,6 @@ static int init_conns(struct rtrs_clt_path *clt_path) unsigned int cid; int err; - /* - * On every new session connections increase reconnect counter - * to avoid clashes with previous sessions not yet closed - * sessions on a server side. - */ - clt_path->s.recon_cnt++; - /* Establish all RDMA connections */ for (cid = 0; cid < clt_path->s.con_num; cid++) { err = create_con(clt_path, cid); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h index a2420eecaf5a..c4ddaeba1c59 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h @@ -109,7 +109,6 @@ struct rtrs_path { struct rtrs_con **con; unsigned int con_num; unsigned int irq_con_num; - unsigned int recon_cnt; unsigned int signal_interval; struct rtrs_ib_dev *dev; int dev_ref; @@ -177,7 +176,6 @@ struct rtrs_sg_desc { * @version: RTRS protocol version * @cid: Current connection id * @cid_num: Number of connections per session - * @recon_cnt: Reconnections counter * @sess_uuid: UUID of a session (path) * @paths_uuid: UUID of a group of sessions (paths) * @@ -196,7 +194,6 @@ struct rtrs_msg_conn_req { __le16 version; __le16 cid; __le16 cid_num; - __le16 recon_cnt; uuid_t sess_uuid; uuid_t paths_uuid; u8 first_conn : 1; diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 4c883c57c2ef..e2ea09a8def7 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1712,7 +1712,6 @@ static int create_con(struct rtrs_srv_path *srv_path, static struct rtrs_srv_path *__alloc_path(struct rtrs_srv_sess *srv, struct rdma_cm_id *cm_id, unsigned int con_num, - unsigned int recon_cnt, const uuid_t *uuid) { struct rtrs_srv_path *srv_path; @@ -1768,7 +1767,6 @@ static struct rtrs_srv_path *__alloc_path(struct rtrs_srv_sess *srv, srv_path->s.con_num = con_num; srv_path->s.irq_con_num = con_num; - srv_path->s.recon_cnt = recon_cnt; uuid_copy(&srv_path->s.uuid, uuid); spin_lock_init(&srv_path->state_lock); INIT_WORK(&srv_path->close_work, rtrs_srv_close_work); @@ -1818,7 +1816,6 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, struct rtrs_srv_sess *srv; u16 version, con_num, cid; - u16 recon_cnt; int err = -ECONNRESET; bool alloc_path = false; @@ -1848,7 +1845,6 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, pr_err("Incorrect cid: %d >= %d\n", cid, con_num); goto reject_w_err; } - recon_cnt = le16_to_cpu(msg->recon_cnt); srv = get_or_create_srv(ctx, &msg->paths_uuid, msg->first_conn); if (IS_ERR(srv)) { err = PTR_ERR(srv); @@ -1885,8 +1881,7 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, goto reject_w_err; } } else { - srv_path = __alloc_path(srv, cm_id, con_num, recon_cnt, - &msg->sess_uuid); + srv_path = __alloc_path(srv, cm_id, con_num, &msg->sess_uuid); if (IS_ERR(srv_path)) { mutex_unlock(&srv->paths_mutex); put_srv(srv); From patchwork Sun Nov 13 01:08:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEB74C433FE for ; Sun, 13 Nov 2022 01:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235088AbiKMBI5 (ORCPT ); Sat, 12 Nov 2022 20:08:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235092AbiKMBIy (ORCPT ); Sat, 12 Nov 2022 20:08:54 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D30426C2 for ; Sat, 12 Nov 2022 17:08:54 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=goPd0HJhGXGWijMgJcPFegGmgsroq4wvXv+eFdntxxo=; b=uHe4DtTvPnLcio6YFONp105Y3Q5R+ZH121XStR6B4XFYgkLLZfoiqiUVPtdQRLQ7/V5V8H ndPM3ql1/gbMpwoBrIP1JHfqkptTuFPtoEBnfesbiGBhEfLImgbFqF/ZIGyo9ZTBk/Qx55 69edO5Ur3IuJeLue7VQEMmw8/QmqDEo= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 09/12] RDMA/rtrs: Clean up rtrs_rdma_dev_pd_ops Date: Sun, 13 Nov 2022 09:08:20 +0800 Message-Id: <20221113010823.6436-10-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Let's remove them since the three members are not used. Signed-off-by: Guoqing Jiang Acked-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-pri.h | 3 --- drivers/infiniband/ulp/rtrs/rtrs.c | 22 ++++------------------ 2 files changed, 4 insertions(+), 21 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h index c4ddaeba1c59..4d15a6fd96b6 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h @@ -68,10 +68,7 @@ enum { struct rtrs_ib_dev; struct rtrs_rdma_dev_pd_ops { - struct rtrs_ib_dev *(*alloc)(void); - void (*free)(struct rtrs_ib_dev *dev); int (*init)(struct rtrs_ib_dev *dev); - void (*deinit)(struct rtrs_ib_dev *dev); }; struct rtrs_rdma_dev_pd { diff --git a/drivers/infiniband/ulp/rtrs/rtrs.c b/drivers/infiniband/ulp/rtrs/rtrs.c index ed324b47d93a..4bf9d868cc52 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs.c @@ -557,7 +557,6 @@ EXPORT_SYMBOL(rtrs_addr_to_sockaddr); void rtrs_rdma_dev_pd_init(enum ib_pd_flags pd_flags, struct rtrs_rdma_dev_pd *pool) { - WARN_ON(pool->ops && (!pool->ops->alloc ^ !pool->ops->free)); INIT_LIST_HEAD(&pool->list); mutex_init(&pool->mutex); pool->pd_flags = pd_flags; @@ -583,15 +582,8 @@ static void dev_free(struct kref *ref) list_del(&dev->entry); mutex_unlock(&pool->mutex); - if (pool->ops && pool->ops->deinit) - pool->ops->deinit(dev); - ib_dealloc_pd(dev->ib_pd); - - if (pool->ops && pool->ops->free) - pool->ops->free(dev); - else - kfree(dev); + kfree(dev); } int rtrs_ib_dev_put(struct rtrs_ib_dev *dev) @@ -618,11 +610,8 @@ rtrs_ib_dev_find_or_add(struct ib_device *ib_dev, goto out_unlock; } mutex_unlock(&pool->mutex); - if (pool->ops && pool->ops->alloc) - dev = pool->ops->alloc(); - else - dev = kzalloc(sizeof(*dev), GFP_KERNEL); - if (IS_ERR_OR_NULL(dev)) + dev = kzalloc(sizeof(*dev), GFP_KERNEL); + if (!dev) goto out_err; kref_init(&dev->ref); @@ -644,10 +633,7 @@ rtrs_ib_dev_find_or_add(struct ib_device *ib_dev, out_free_pd: ib_dealloc_pd(dev->ib_pd); out_free_dev: - if (pool->ops && pool->ops->free) - pool->ops->free(dev); - else - kfree(dev); + kfree(dev); out_err: return NULL; } From patchwork Sun Nov 13 01:08:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 763B4C4332F for ; Sun, 13 Nov 2022 01:09:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235107AbiKMBI7 (ORCPT ); Sat, 12 Nov 2022 20:08:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36638 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235127AbiKMBI5 (ORCPT ); Sat, 12 Nov 2022 20:08:57 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BB51A190 for ; Sat, 12 Nov 2022 17:08:56 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301734; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GI+N4DZ/N9r46P6gR7pi1nlnhW04QcncRt3cBrWRYbg=; b=u07R6Ks8xjMB1MPxwEuuSPaO9992bCi9eDtpCuJwr/2lHhsghWTsZ5H+r4v8hCU8Rx6/da OjCKMNcYj/GC7F3faJEZClyXBGRGDAG7gMVZJy4ByGMyy0+1XO6Ors/PyV/wOGQc1vTops ix3enZwOiJdP/SmdVyf4qM0JSIDHVmY= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 10/12] RDMA/rtrs-srv: Remove paths_num Date: Sun, 13 Nov 2022 09:08:21 +0800 Message-Id: <20221113010823.6436-11-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The paths_num is only increased by rtrs_rdma_connect -> __alloc_path which is only one time thing, so is the decreasing of it given only rtrs_srv_close_work -> del_path_from_srv, which means paths_num should always be 1. Signed-off-by: Guoqing Jiang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 8 -------- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 1 - 2 files changed, 9 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index e2ea09a8def7..400cf8ae34a3 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1437,8 +1437,6 @@ static void __add_path_to_srv(struct rtrs_srv_sess *srv, struct rtrs_srv_path *srv_path) { list_add_tail(&srv_path->s.entry, &srv->paths_list); - srv->paths_num++; - WARN_ON(srv->paths_num >= MAX_PATHS_NUM); } static void del_path_from_srv(struct rtrs_srv_path *srv_path) @@ -1450,8 +1448,6 @@ static void del_path_from_srv(struct rtrs_srv_path *srv_path) mutex_lock(&srv->paths_mutex); list_del(&srv_path->s.entry); - WARN_ON(!srv->paths_num); - srv->paths_num--; mutex_unlock(&srv->paths_mutex); } @@ -1719,10 +1715,6 @@ static struct rtrs_srv_path *__alloc_path(struct rtrs_srv_sess *srv, char str[NAME_MAX]; struct rtrs_addr path; - if (srv->paths_num >= MAX_PATHS_NUM) { - err = -ECONNRESET; - goto err; - } if (__is_path_w_addr_exists(srv, &cm_id->route.addr)) { err = -EEXIST; pr_err("Path with same addr exists\n"); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index eccc432b0715..8e4fcb578f49 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -100,7 +100,6 @@ struct rtrs_srv_sess { struct list_head paths_list; int paths_up; struct mutex paths_ev_mutex; - size_t paths_num; struct mutex paths_mutex; uuid_t paths_uuid; refcount_t refcount; From patchwork Sun Nov 13 01:08:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68FBFC433FE for ; Sun, 13 Nov 2022 01:09:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235048AbiKMBJB (ORCPT ); Sat, 12 Nov 2022 20:09:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235106AbiKMBI7 (ORCPT ); Sat, 12 Nov 2022 20:08:59 -0500 Received: from out2.migadu.com (out2.migadu.com [188.165.223.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99518BCBE for ; Sat, 12 Nov 2022 17:08:57 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Yq7ZFSi9lzPjjs62wkYKSpTdxeGjz5TAGbakjVWInAE=; b=RSFKl5HyKDJEb83bm6ZAzxFB9wR/U7E7ijr06pJ4LneM9gXyL4VPXm1hYIIiLzd/MyAsvh 9VluQOZAzBbpCbAlCKD/m7BQ9wOsWcFsiyGdQXxMaw3DdDAWiaUkyaGWHOfP+PEJR2+X7G AC1rKMhMmnaWoWH4Kelf2GnDKgQEaEw= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 11/12] RDMA/rtrs-srv: fix several issues in rtrs_srv_destroy_path_files Date: Sun, 13 Nov 2022 09:08:22 +0800 Message-Id: <20221113010823.6436-12-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org There are several issues in the function which is supposed to be paired with rtrs_srv_create_path_files. 1. rtrs_srv_stats_attr_group is not removed though it is created in rtrs_srv_create_stats_files. 2. it makes more sense to check kobj_stats.state_in_sysfs before destroy kobj_stats instead of rely on kobj.state_in_sysfs. 3. kobject_init_and_add is used for both kobjs (srv_path->kobj and srv_path->stats->kobj_stats), however we missed to call kobject_del for srv_path->kobj which was called in free_path. 4. rtrs_srv_destroy_once_sysfs_root_folders is independant of either kobj or kobj_stats. Signed-off-by: Guoqing Jiang Acked-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c index 2a3c9ac64a42..da8e205ce331 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c @@ -304,12 +304,18 @@ int rtrs_srv_create_path_files(struct rtrs_srv_path *srv_path) void rtrs_srv_destroy_path_files(struct rtrs_srv_path *srv_path) { - if (srv_path->kobj.state_in_sysfs) { + if (srv_path->stats->kobj_stats.state_in_sysfs) { + sysfs_remove_group(&srv_path->stats->kobj_stats, + &rtrs_srv_stats_attr_group); kobject_del(&srv_path->stats->kobj_stats); kobject_put(&srv_path->stats->kobj_stats); + } + + if (srv_path->kobj.state_in_sysfs) { sysfs_remove_group(&srv_path->kobj, &rtrs_srv_path_attr_group); + kobject_del(&srv_path->kobj); kobject_put(&srv_path->kobj); - - rtrs_srv_destroy_once_sysfs_root_folders(srv_path); } + + rtrs_srv_destroy_once_sysfs_root_folders(srv_path); } From patchwork Sun Nov 13 01:08:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoqing Jiang X-Patchwork-Id: 13041406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE47BC4332F for ; Sun, 13 Nov 2022 01:09:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235092AbiKMBJD (ORCPT ); Sat, 12 Nov 2022 20:09:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235127AbiKMBJB (ORCPT ); Sat, 12 Nov 2022 20:09:01 -0500 Received: from out2.migadu.com (out2.migadu.com [IPv6:2001:41d0:2:aacc::]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE8059592 for ; Sat, 12 Nov 2022 17:08:59 -0800 (PST) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1668301738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GvEkIWRlwtcZKi88qR9Qfnru6I7OFWi/eDsNysaKGsA=; b=WCQNDvT+G/o2RBU0ZLCpy+r6IgPO75cmU4wZ7RgafVKvnGsEqcWIQltUB3SayTt9i5wqAT Y5/eP1u+lnqKg5LPsNoZJO/C28WgZuL3oVVOpnQM1hqQaEOkGqiAHx9DZGdf+b23V660z/ ox+O0aEZFd9/Q8f0tIuDrYHrgQvr/AI= From: Guoqing Jiang To: haris.iqbal@ionos.com, jinpu.wang@ionos.com, jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org Subject: [PATCH RFC 12/12] RDMA/rtrs-srv: Remove kobject_del from rtrs_srv_destroy_once_sysfs_root_folders Date: Sun, 13 Nov 2022 09:08:23 +0800 Message-Id: <20221113010823.6436-13-guoqing.jiang@linux.dev> In-Reply-To: <20221113010823.6436-1-guoqing.jiang@linux.dev> References: <20221113010823.6436-1-guoqing.jiang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The kobj_paths which is created dynamically by kobject_create_and_add, and per the comment above kobject_create_and_add, we only need to call kobject_put which is not same as other kobjs such as stats->kobj_stats and srv_path->kobj. Signed-off-by: Guoqing Jiang Acked-by: Md Haris Iqbal --- drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c index da8e205ce331..c76ba29da1e2 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c @@ -203,7 +203,6 @@ rtrs_srv_destroy_once_sysfs_root_folders(struct rtrs_srv_path *srv_path) mutex_lock(&srv->paths_mutex); if (!--srv->dev_ref) { - kobject_del(srv->kobj_paths); kobject_put(srv->kobj_paths); mutex_unlock(&srv->paths_mutex); device_del(&srv->dev);