From patchwork Fri Jul 30 13:18:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6EE1C4320A for ; Fri, 30 Jul 2021 13:18:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B969B60F5C for ; Fri, 30 Jul 2021 13:18:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231147AbhG3NSl (ORCPT ); Fri, 30 Jul 2021 09:18:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238890AbhG3NSk (ORCPT ); Fri, 30 Jul 2021 09:18:40 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18AA1C06175F for ; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id b7so11305126wri.8 for ; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I23W8uRoYmeDeWltvLdj9fSm/dRKdVOIpfqaw3NOMS0=; b=DB0dzy8CBrXuhgxjNZBDYqMMAxbC/RQ28vwrShzk//CvdvgYnS+l57vaWFEFtZoNYv JHMVr+G/ZOGuEHu/6mndAoUV0KuxuTyGTuwxV1Vu4mwsr1qlzCkMb9Fi2b9GmDVus/b5 vmEuZEndwErXHQQqqJBUOjopmzy2fGrMWqOa2EHCUArQRX9P3QEDcqfOvAcG04ux0s5m mdZLB6YdwWxm379n48LeO+No3rtRHMDhT1WfklEQ2Mx3+u99o6/yD5r8Ae2/vggWSsgB zyQGyODsTcoPukFXfBPsgsrr/ip3VpNtJBR1OAnAqCfYVNl05VS1XB193PVcRiPbz/rP YfEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I23W8uRoYmeDeWltvLdj9fSm/dRKdVOIpfqaw3NOMS0=; b=mA2z3CdG1WR9qnQYOgFmDcv4GGS4lMFXGImKyqLlpKIsdaq0sQ5LWWstj5ltXvuvoQ xWwnunM+U0fpS7GysoUo1A6qK5H9DKBzlQdNwj56YjG0yJcHf/Yj500zXpbes4OGEa4C EWRr2C5SN1migAg6iKVey5edG+78BpbVM43HpKAKUCn+EucgOUiaM5OKfr9z79O6iwKz ee1LYDuSLMvxLS1AAZ0nF2xIpJvScjwYvpSQEjPZk2RLy8azPaE4nCigAfCafqTgTzjp VRQMIwZPG/JLZMOAJxsAlbbPcTPSKCPAPJaeyp7k4l2GVMCowiz30EIDcP0EPCtOWwei LbDA== X-Gm-Message-State: AOAM531JOyU1izYc54l80hWRb9uRmpaHwNhApyOE9n8+6PllAzgMwdBO jyNsqMDMQ+6/BhBGJRaLmRBAnplyVBzZ1w== X-Google-Smtp-Source: ABdhPJxjWn1dOy4kXDQRqqh4jsx+SeuGbvHSTcpO/Jjn5wbvPFjZvWUW7ptVNSE0el183t/s406lXA== X-Received: by 2002:a5d:5906:: with SMTP id v6mr3092308wrd.194.1627651114649; Fri, 30 Jul 2021 06:18:34 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:34 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com Subject: [PATCH for-next 01/10] RDMA/rtrs-clt: During add_path change for_new_clt according to path_num Date: Fri, 30 Jul 2021 15:18:23 +0200 Message-Id: <20210730131832.118865-2-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Md Haris Iqbal When all the paths are removed for a session, the addition of the first path is like a new session for the storage server. Hence, for_new_clt has to be set to 1. Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index ece3205531b8..e048bfa12755 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -3083,6 +3083,15 @@ int rtrs_clt_create_path_from_sysfs(struct rtrs_clt *clt, if (IS_ERR(sess)) return PTR_ERR(sess); + if (clt->paths_num == 0) { + /* + * When all the paths are removed for a session, + * the addition of the first path is like a new session for + * the storage server + */ + sess->for_new_clt = 1; + } + /* * It is totally safe to add path in CONNECTING state: coming * IO will never grab it. Also it is very important to add From patchwork Fri Jul 30 13:18:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D08DC432BE for ; Fri, 30 Jul 2021 13:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 65E4860F5C for ; Fri, 30 Jul 2021 13:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238912AbhG3NSm (ORCPT ); Fri, 30 Jul 2021 09:18:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238890AbhG3NSl (ORCPT ); Fri, 30 Jul 2021 09:18:41 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5F7DC06175F for ; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id k4so476784wrc.0 for ; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z+vkdV0HH1tgeL7gC1IU56T6CEYngD6yq5bmbkBJLK0=; b=cReyz2YqEOaOyTt8K8fIlWJbExcJjR9/OW7uxGNDrNvskUa1WsCM2lchm23H0wICjz 5mYa9c9o+EKtQvU49ajq9/T8HtWSn+TXiIgf/Q1PyPuWAciiQ4EInoWwNMW7XW3xQxQc CCG1H34bv6Qo2ztdLSqBfnH7W9OJGNhCYO0PkP3jgtWEP20Z6dWaF6cPciNHq3wJNfa3 e6aRB1yImN3PeegF5U2jFq+eLLq55tgyeZBa4qLf4pC9Xw0D2YJbdgCDxMzpJ+E2f6Wx mnBcoGAIHry0C+LNQATlC9TfKAy7W3qeYtVeBb5szz5jwzx8NjoDHkeiDw+b5oSVEKNs TpLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z+vkdV0HH1tgeL7gC1IU56T6CEYngD6yq5bmbkBJLK0=; b=XClpqdrs4BnZn6ymlGJpY+u4lvVUeY4HzRvKmiBaHtz43IRWwoaRae8OGX0N1LF3VS syav00mDqXe/ieaIbabRSIRf4B3aW2Orlg9VmW6SIc1qPuGX4FLVwuzJD/SLfzyhGQMz hn9xatJ/8ALMUM5cQhbWNj3knQJJV71dmq6x5oHUZg7hJqjtNwwtuqrezh4kxR9BGmHD F586gmCiQhNlt0L9PZ6dmmOlzBImOJ8xH2d88G+/7t845RM3eLiWV6njby7OT2ZiDCyE GIQy59T/GMyYDHrbCMLLJiJUkGuQUJbYArfIVg7Su/xIGSBKNeFfxTMCUEZBIg4BoFiR 5Tsg== X-Gm-Message-State: AOAM533F5+U89IJlJgFqDk/K8UdT3Huw1VzYN08G1iq8UVXmw/Jg75Qq h9mJsMWmh7Xth39NPOKh06UEsxi5fu1/WA== X-Google-Smtp-Source: ABdhPJyojkSvo0gjiOVAI2gilWd9tFGjskosd0pXWw4JVKuWPQqWmPznOHnK/ZeIlNh5Q8LkqMJHNg== X-Received: by 2002:adf:c5d2:: with SMTP id v18mr3034244wrg.386.1627651115466; Fri, 30 Jul 2021 06:18:35 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:35 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Gioh Kim Subject: [PATCH for-next 02/10] RDMA/rtrs-srv: Prevent sysfs error with path name "ctl" Date: Fri, 30 Jul 2021 15:18:24 +0200 Message-Id: <20210730131832.118865-3-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Gioh Kim If the client tries to create a path with name "ctl", the server tries to creates /sys/devices/virtual/rtrs-server/ctl/. Then server generated below error because there is already ctl directory which manages some setup of the server. sysfs: cannot create duplicate filename '/devices/virtual/rtrs-server/ctl' Workqueue: ib-comp-wq ib_cq_poll_work [ib_core] Call Trace: dump_stack+0x50/0x63 sysfs_warn_dup.cold+0x17/0x24 sysfs_create_dir_ns+0xb6/0xd0 kobject_add_internal+0xa6/0x2a0 kobject_add+0x7e/0xb0 ? _cond_resched+0x15/0x30 device_add+0x121/0x640 rtrs_srv_create_sess_files+0x18f/0x1f0 [rtrs_server] ? __alloc_pages_nodemask+0x16c/0x2b0 ? kmalloc_order+0x7c/0x90 ? kmalloc_order_trace+0x1d/0xa0 ? rtrs_iu_alloc+0x17e/0x1bf [rtrs_core] rtrs_srv_info_req_done+0x417/0x5b0 [rtrs_server] ? __switch_to_asm+0x40/0x70 __ib_process_cq+0x76/0xd0 [ib_core] ib_cq_poll_work+0x26/0x80 [ib_core] process_one_work+0x1df/0x3a0 worker_thread+0x4a/0x3c0 kthread+0xfb/0x130 ? process_one_work+0x3a0/0x3a0 ? kthread_park+0x90/0x90 ret_from_fork+0x1f/0x40 kobject_add_internal failed for ctl with -EEXIST, don't try to register things with the same name in the same directory. rtrs_server L178: device_add(): -17 This patch checks the path name and disconnect on server to prevent the kernel error. Signed-off-by: Gioh Kim Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index cd9a4ccf4c28..b814a6052cf1 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -758,6 +758,14 @@ static bool exist_sessname(struct rtrs_srv_ctx *ctx, struct rtrs_srv_sess *sess; bool found = false; + /* + * Session name "ct" is not allowed because + * /sys/devices/virtual/rtrs-server/ctl already exists + * for setup management. + */ + if (!strcmp(sessname, "ctl")) + return true; + mutex_lock(&ctx->srv_mutex); list_for_each_entry(srv, &ctx->srv_list, ctx_list) { mutex_lock(&srv->paths_mutex); From patchwork Fri Jul 30 13:18:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 289E2C43214 for ; Fri, 30 Jul 2021 13:18:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 125C860F5C for ; Fri, 30 Jul 2021 13:18:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238923AbhG3NSm (ORCPT ); Fri, 30 Jul 2021 09:18:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238890AbhG3NSm (ORCPT ); Fri, 30 Jul 2021 09:18:42 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91F5EC06175F for ; Fri, 30 Jul 2021 06:18:37 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id d8so11288611wrm.4 for ; Fri, 30 Jul 2021 06:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Loqr2AeYIZuZk6MiTZIgumKJU6aYenJOQapXQ2RPPUY=; b=YrVSWZsMVciWuwmdyAv9Q3gX32Fis/mLcu/7ocJn1aRQxSvqpzhUCGf24z+Lir4wif K3iXxgOrDYo740Cl6GLVPwcQfwSWrbTzFb+QtXgdxo2Sck9ft6VtZuj5r2De2tHlUOUv hTl2vTlH4T+Q10dGYYCIcqbdTmmOMORBJmutQ1rg6FCtXh5a3H+o+RMeWlkI090PN1ru gfUg2vs+iYh1lcL5R6de02iyGWS7KlOad6TK8h8xbM1pVJ6Kvwxy9Bignv4UGk4ZyY+H StWpW+ItZ8aFpAAzenb+R5+OXktEJ2mbvFStLDcUj1xA12Ib52vkxXI8el+QktmNqkD5 DVPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Loqr2AeYIZuZk6MiTZIgumKJU6aYenJOQapXQ2RPPUY=; b=WPhjIpgJyPIZLYHxw744tyfOE5mSmK5FW/i6o88KYmuG0rJytpD6p3wUCWrNwxHL+m TrozS801X9Fj9BtooG2VUCsiSQtpCoAUfnTNgtY6RB3QGRKUJhi/u9rSXUB9WQww3MPz kaUTCgl83YIX4drTgSDR0q8isx8HKp9f1xR/UM/fmOpxIVAwid1Z7ZMMOmfoYjbbGRN2 tb1UcwrlVlrvlIJZZBG7kaJO9lvPYhWhuHkHC6VuKANqjsCqmty1n7Hsz/CUbD2Y7A2P I6mXGHuPVX854zBa0B2uiL2s/2fLUwk0D4pwL0raY91foCyPa6SGbub5EqNUDf6LVw+h Hfrg== X-Gm-Message-State: AOAM531qk1WeWz+IATJpp2uLGfyL3F3IMBOKSHmljN4i7AO7qeQdR012 q/FiZ/ecWn+FOwuhdmnVEBRYh2zjIqcmPg== X-Google-Smtp-Source: ABdhPJyvo/K17KY0HBs5AjTIrNkmugyFgOOV9MDb9MUEudmLdt2NQo3hnJ0s9XAhRUPBoQmHschygA== X-Received: by 2002:a5d:68cc:: with SMTP id p12mr3091082wrw.161.1627651116096; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:35 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com Subject: [PATCH for-next 03/10] RDMA/rtrs: Use sysfs_emit instead of s*printf function for sysfs show Date: Fri, 30 Jul 2021 15:18:25 +0200 Message-Id: <20210730131832.118865-4-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Md Haris Iqbal sysfs_emit function was added to be aware of the PAGE_SIZE maximum of the temporary buffer used for outputting sysfs content, so there is no possible overruns. So replace the uses of any s*printf functions for the sysfs show functions with sysfs_emit. Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 24 +++++++++----------- drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c | 2 +- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c index 26bbe5d6dff5..c5c047aa45a4 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c @@ -45,24 +45,23 @@ int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, size_t used; int cpu; - used = scnprintf(buf, len, " "); + used = sysfs_emit(buf, " "); for_each_possible_cpu(cpu) - used += scnprintf(buf + used, len - used, " CPU%u", cpu); + used += sysfs_emit_at(buf, used, " CPU%u", cpu); - used += scnprintf(buf + used, len - used, "\nfrom:"); + used += sysfs_emit_at(buf, used, "\nfrom:"); for_each_possible_cpu(cpu) { s = per_cpu_ptr(stats->pcpu_stats, cpu); - used += scnprintf(buf + used, len - used, " %d", + used += sysfs_emit_at(buf, used, " %d", atomic_read(&s->cpu_migr.from)); } - used += scnprintf(buf + used, len - used, "\nto :"); + used += sysfs_emit_at(buf, used, "\nto :"); for_each_possible_cpu(cpu) { s = per_cpu_ptr(stats->pcpu_stats, cpu); - used += scnprintf(buf + used, len - used, " %d", - s->cpu_migr.to); + used += sysfs_emit_at(buf, used, " %d", s->cpu_migr.to); } - used += scnprintf(buf + used, len - used, "\n"); + used += sysfs_emit_at(buf, used, "\n"); return used; } @@ -70,9 +69,8 @@ int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf, size_t len) { - return scnprintf(buf, len, "%d %d\n", - stats->reconnects.successful_cnt, - stats->reconnects.fail_cnt); + return sysfs_emit(buf, "%d %d\n", stats->reconnects.successful_cnt, + stats->reconnects.fail_cnt); } ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, @@ -94,7 +92,7 @@ ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, sum.failover_cnt += r->failover_cnt; } - return scnprintf(page, len, "%llu %llu %llu %llu %u %llu\n", + return sysfs_emit(page, "%llu %llu %llu %llu %u %llu\n", sum.dir[READ].cnt, sum.dir[READ].size_total, sum.dir[WRITE].cnt, sum.dir[WRITE].size_total, atomic_read(&stats->inflight), sum.failover_cnt); @@ -103,7 +101,7 @@ ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, ssize_t rtrs_clt_reset_all_help(struct rtrs_clt_stats *s, char *page, size_t len) { - return scnprintf(page, len, "echo 1 to reset all statistics\n"); + return sysfs_emit(page, "echo 1 to reset all statistics\n"); } int rtrs_clt_reset_rdma_stats(struct rtrs_clt_stats *stats, bool enable) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c index 20efd44297fb..9c43ce5ba1c1 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c @@ -102,7 +102,7 @@ static ssize_t rtrs_srv_src_addr_show(struct kobject *kobj, sess = container_of(kobj, struct rtrs_srv_sess, kobj); cnt = sockaddr_to_str((struct sockaddr *)&sess->s.dst_addr, page, PAGE_SIZE); - return cnt + scnprintf(page + cnt, PAGE_SIZE - cnt, "\n"); + return cnt + sysfs_emit_at(page, cnt, "\n"); } static struct kobj_attribute rtrs_srv_src_addr_attr = From patchwork Fri Jul 30 13:18:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4A1DC4338F for ; Fri, 30 Jul 2021 13:18:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF7F960F5C for ; Fri, 30 Jul 2021 13:18:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238929AbhG3NSn (ORCPT ); Fri, 30 Jul 2021 09:18:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238937AbhG3NSm (ORCPT ); Fri, 30 Jul 2021 09:18:42 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 331F1C06175F for ; Fri, 30 Jul 2021 06:18:38 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id k4so5966085wms.3 for ; Fri, 30 Jul 2021 06:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WoPUXUz948c94A+csjEJkfSs5ZrNmAzBhOOAfwUgLvE=; b=iBHCX6ptNT0pvn+H7CJXUE0cIeYn9bCCmEsu590sJfAaai98qE9bDPS//qkXZkl3q6 2dyq2xUoxCwnQuYBqkEmZ0d4yASJkN8z8PDIQV8q8T+Ymf0N96PXj3nN2rn77lV+6JAf KHnDVUUMf/49RkU3r/HP/XlmyW/P1vzrH2N9N78QVmN+IuJfxK0wIbTz4u7J+WWF2hVf e9EqumsGwdgvh5wBRme4T6GckOFCUKHfhIp0siZqMZNxJ00k7pj3XqQi6VRZRUTz+e1V 2mtcUz0UstEO0KL5tVeFxgu4YSrqwkYf19yq213KuR74Y/LV9GovJE1WLu9bgBzbdKsd xoOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WoPUXUz948c94A+csjEJkfSs5ZrNmAzBhOOAfwUgLvE=; b=T8MezzCsgn/K6o4f+STTyBatexYpoeGc+nRNKdvL4m09Mt7Owgf/sDPqp2OlN0eUKJ 1QqXLlQvMx5h7tYtz8hEqdIu6hU57+EhOCBvFmzmOgwIxlMIC2Y+ntj+CR6a0aQJadaS vCvQbn3JhP1Wh6YFYsRnw/xyYo7eBcEvztcEnGf2hXm7dpUnbgmVF80y0En9K9CuOtAg 03h/aDYWl2881hYGsPzwdxZa/4Dgho3nr7jlPSYgJilpbIc67Cyj19C3emo/aQWYCFyz Ty8JK4/GOU1lFcnHk/sCU/UXThCyum8IBtz7uyb/irncp88rrsF+6g9l6waMRv4Jf3YZ 7bSg== X-Gm-Message-State: AOAM533aunMfKVgXW2WKIiwJb8iU8ZSbZIxzHE/09u94HXHrxOpYoZHh 25k18gvsKRdEq2D1MB1BUUrGjQTUNtN19g== X-Google-Smtp-Source: ABdhPJwmo1pb2DLhtiZsdZGYqHjsJlRd+X9t246lNza96PAci5Yzzqrnvi6eN145ekkJAjKvRUckdQ== X-Received: by 2002:a05:600c:3510:: with SMTP id h16mr2914779wmq.65.1627651116690; Fri, 30 Jul 2021 06:18:36 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:36 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com Subject: [PATCH for-next 04/10] RDMA/rtrs: Remove unused functions Date: Fri, 30 Jul 2021 15:18:26 +0200 Message-Id: <20210730131832.118865-5-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The two functions are unused, so just remove them. Signed-off-by: Jack Wang Reviewed-by: Md Haris Iqbal Reviewed-by: Leon Romanovsky --- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 5 +---- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 4 ---- 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h index 3c3ff094588c..72f9136e3c24 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h @@ -229,10 +229,7 @@ int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, char *buf, size_t len); int rtrs_clt_reset_reconnects_stat(struct rtrs_clt_stats *stats, bool enable); int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf, - size_t len); -int rtrs_clt_reset_wc_comp_stats(struct rtrs_clt_stats *stats, bool enable); -int rtrs_clt_stats_wc_completion_to_str(struct rtrs_clt_stats *stats, char *buf, - size_t len); + size_t len); int rtrs_clt_reset_rdma_stats(struct rtrs_clt_stats *stats, bool enable); ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, char *page, size_t len); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index e81774f5acd3..9d8d2a91a235 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -138,10 +138,6 @@ static inline void rtrs_srv_update_rdma_stats(struct rtrs_srv_stats *s, int rtrs_srv_reset_rdma_stats(struct rtrs_srv_stats *stats, bool enable); ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, char *page, size_t len); -int rtrs_srv_reset_wc_completion_stats(struct rtrs_srv_stats *stats, - bool enable); -int rtrs_srv_stats_wc_completion_to_str(struct rtrs_srv_stats *stats, char *buf, - size_t len); int rtrs_srv_reset_all_stats(struct rtrs_srv_stats *stats, bool enable); ssize_t rtrs_srv_reset_all_help(struct rtrs_srv_stats *stats, char *page, size_t len); From patchwork Fri Jul 30 13:18:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C39CC4320E for ; Fri, 30 Jul 2021 13:18:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83A4260EFD for ; Fri, 30 Jul 2021 13:18:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238937AbhG3NSo (ORCPT ); Fri, 30 Jul 2021 09:18:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238890AbhG3NSn (ORCPT ); Fri, 30 Jul 2021 09:18:43 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBEE0C06175F for ; Fri, 30 Jul 2021 06:18:38 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id m19so5990519wms.0 for ; Fri, 30 Jul 2021 06:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gipMW1mQKUh3sQRjOfvvLSct1hvDgV+kPX6qBnraHXA=; b=g2XDAkJ3ynaNz9H0/KsAwvTdmT1ETjajYUfof+EtQ57x/1ha/n2lr54rEGvtmmqgsa GzcReVGGFtieeBbAsQwq8iFUSIBLNcCdnU3ZLwx2qBy76SaWWTUcHDPCZZSxlrgfa9hK GYe5rFLnjDw8by9nDUXiOWq/Q5zUVD7gdXXVeuKbqyKq8Shsc7353QmTr009AdhgneMl HX203rQHgY8vm3PZwGgssS0Ou4iwp3MsBZ57DWFEDIOe2f9DEqfEwCTdijxNSa3Sauug 9GkhtZALzBaOwo50NX9GTgBoXv7m/KIGOAnjeczyYjZa3cLQgIxGzNZylwyF7xEGpACo E7qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gipMW1mQKUh3sQRjOfvvLSct1hvDgV+kPX6qBnraHXA=; b=Mb/27Vv1NeZxJr4cRXzmxoMiJ7B7luLaV5Ax9RWR8s5eRydnsfl/6wKSEi69ZxGswE ZdIziZLfSNRH2q4fhxxXPUmOWAc4/qX6j7cY6jTDun7alCnQCv7PsmQsKitqbU8/Ac+8 7kc438Pw49rojFKb2bwcOWz0QMC+E9+WFHwWczoGrW5/Gq2sg5tFB6rt23836XJ/aqN1 n+nZ/i/8V+PV6I4DxO7LCIPAKQQCXipXyPj8I64QuQ9sO5AuJMK/tfzlI73fMqD/xuXm BgbVAvnCu3IIoXj8fYS/m3rBnBCHRWyfpnxlKfAGCM89Sl/+JACfLH5zB30eNmhMmdZY URTw== X-Gm-Message-State: AOAM530s4doft2J7m8I7XQ30EXZE/YdW+OVXR+ZfyJgMGolH6oBhnkqS xgFDI2OYOlOtSZub9I1V92/WqqP6lDbOpA== X-Google-Smtp-Source: ABdhPJws8cYC4VHvFoGyQduhlHB4e+ZTY/jTOF9lqSiVNO58ER0lB1HAv6iNYSxSdBK1tg/xzMYRbA== X-Received: by 2002:a05:600c:2302:: with SMTP id 2mr1090297wmo.169.1627651117425; Fri, 30 Jul 2021 06:18:37 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:37 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Gioh Kim Subject: [PATCH for-next 05/10] RDMA/rtrs: Fix warning when use poll mode Date: Fri, 30 Jul 2021 15:18:27 +0200 Message-Id: <20210730131832.118865-6-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org when test with poll mode, it will fail and lead to warning below: echo "sessname=bla path=gid:fe80::2:c903:4e:d0b3@gid:fe80::2:c903:8:ca17 device_path=/dev/nullb2 nr_poll_queues=-1" | sudo tee /sys/devices/virtual/rnbd-client/ctl/map_device rnbd_client L597: Mapping device /dev/nullb2 on session bla, (access_mode: rw, nr_poll_queues: 8) WARNING: CPU: 3 PID: 9886 at drivers/infiniband/core/cq.c:447 ib_cq_pool_get+0x26f/0x2a0 [ib_core] The problem is in case of poll queue, we need to still call ib_alloc_cq/ib_free_cq, we can't use cq_poll api for poll queue. Signed-off-by: Jack Wang Reviewed-by: Gioh Kim --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 1 + drivers/infiniband/ulp/rtrs/rtrs.c | 17 ++++++++++++++--- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index b814a6052cf1..5969a74d45a0 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1776,6 +1776,7 @@ static struct rtrs_srv_sess *__alloc_sess(struct rtrs_srv *srv, strscpy(sess->s.sessname, str, sizeof(sess->s.sessname)); sess->s.con_num = con_num; + sess->s.irq_con_num = con_num; sess->s.recon_cnt = recon_cnt; uuid_copy(&sess->s.uuid, uuid); spin_lock_init(&sess->state_lock); diff --git a/drivers/infiniband/ulp/rtrs/rtrs.c b/drivers/infiniband/ulp/rtrs/rtrs.c index ca542e477d38..9bc323490ce3 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs.c +++ b/drivers/infiniband/ulp/rtrs/rtrs.c @@ -228,7 +228,12 @@ static int create_cq(struct rtrs_con *con, int cq_vector, int nr_cqe, struct rdma_cm_id *cm_id = con->cm_id; struct ib_cq *cq; - cq = ib_cq_pool_get(cm_id->device, nr_cqe, cq_vector, poll_ctx); + if (con->cid >= con->sess->irq_con_num) + cq = ib_alloc_cq(cm_id->device, con, nr_cqe, cq_vector, + poll_ctx); + else + cq = ib_cq_pool_get(cm_id->device, nr_cqe, cq_vector, poll_ctx); + if (IS_ERR(cq)) { rtrs_err(con->sess, "Creating completion queue failed, errno: %ld\n", PTR_ERR(cq)); @@ -283,7 +288,10 @@ int rtrs_cq_qp_create(struct rtrs_sess *sess, struct rtrs_con *con, err = create_qp(con, sess->dev->ib_pd, max_send_wr, max_recv_wr, max_send_sge); if (err) { - ib_cq_pool_put(con->cq, con->nr_cqe); + if (con->cid >= con->sess->irq_con_num) + ib_free_cq(con->cq); + else + ib_cq_pool_put(con->cq, con->nr_cqe); con->cq = NULL; return err; } @@ -300,7 +308,10 @@ void rtrs_cq_qp_destroy(struct rtrs_con *con) con->qp = NULL; } if (con->cq) { - ib_cq_pool_put(con->cq, con->nr_cqe); + if (con->cid >= con->sess->irq_con_num) + ib_free_cq(con->cq); + else + ib_cq_pool_put(con->cq, con->nr_cqe); con->cq = NULL; } } From patchwork Fri Jul 30 13:18:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39E2DC4338F for ; Fri, 30 Jul 2021 13:18:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 21C8A60F94 for ; Fri, 30 Jul 2021 13:18:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238964AbhG3NSp (ORCPT ); Fri, 30 Jul 2021 09:18:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238958AbhG3NSp (ORCPT ); Fri, 30 Jul 2021 09:18:45 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A062AC06175F for ; Fri, 30 Jul 2021 06:18:39 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id f14-20020a05600c154eb02902519e4abe10so9163221wmg.4 for ; Fri, 30 Jul 2021 06:18:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cglNi5sbjH/UruPM4dmVzfGLRlJu/atFoMDKVgS8CUk=; b=TR7M/9LDs0YFBhpjT/ENr9KM6a/d7ngqwVrQxu1fKBTi4FoGptfr+YJ8Wk4MctKr2C z0/rZJQZtN4gGRWHb2BinO+YelG6ilrZbkFoRlgenhjtWui9DHLZWKNf0+euUzyE5dFL j/qBVKD//qR5rA3WehgEcF4pyvyTnKlQcuTxMM5h/yamKAu3s27cXY1ZKCDy9YV/QJy8 D8/m1wUwzbJDe86vnz3rXy3AvH8Qz1bYkXQg6pR9KajjO1wYBnA8RCY74njR5Sv22Gga Ft3fldoiT/bjzH77azjAZbyqMxDMiyh31bq25s3V/MxXHsuW35H6oM4Etd7CcZyvGWlE AhzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cglNi5sbjH/UruPM4dmVzfGLRlJu/atFoMDKVgS8CUk=; b=hWO+qo+IlpfCxzoT1p/SIg0xqdk3C6weBK4Om71pZjUhwpx8PqvPK2rvlqM60PdjBa 3GPcFzRC8p0UD/LdAIwurHAzNlmJxlkuTLF4DIa3JKW2Zt7twD7+Wfb+NtdNrrrFVNXh XiP2Q0P7aCWz+QWGtsF3jBAvenmaKMOEpZDJ3eSNAIUbjRIkme+V+Jnqww5Gq+89jw0p m0Dcd2yvOAlvZpozCDBrLcONMlIt7XIutYDJXv/pIQoUU8abTW/pmpbOqfhZ9H+FgTAS xQMohwTVY6Bx4AEJcIWro28eVPLd8adt4p/Wj1GIAqYimRIq/mJd/BlQysDI6n255az7 qqVw== X-Gm-Message-State: AOAM530HKOUlp6FfSP4YWqC+oEVcEaYx+LuiKAQ3Fy/tk02ksNtXbgUd Gr6G+ZiCL9lvMheiR3T4IrURZ5kozSf51A== X-Google-Smtp-Source: ABdhPJzb3bpw99fWjhwsg/0J5NlkmIyV/r0Y8K344/DmeAy3D2xA4l8lAk3JzHJE1cJrKm+i5912dg== X-Received: by 2002:a1c:7709:: with SMTP id t9mr3004689wmi.161.1627651118118; Fri, 30 Jul 2021 06:18:38 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:37 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com Subject: [PATCH for-next 06/10] RDMA/rtrs: Remove len parameter from helper print functions of sysfs Date: Fri, 30 Jul 2021 15:18:28 +0200 Message-Id: <20210730131832.118865-7-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Md Haris Iqbal Since we have changed all sysfs show functions to use sysfs_emit, we do not require the len (PAGE_SIZE) in our helper print functions. So remove it from the function parameter. Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 12 ++++-------- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 12 +++++------- drivers/infiniband/ulp/rtrs/rtrs-pri.h | 2 +- drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c | 3 +-- drivers/infiniband/ulp/rtrs/rtrs-srv.h | 3 +-- 5 files changed, 12 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c index c5c047aa45a4..585f83cd7ed1 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c @@ -37,8 +37,7 @@ void rtrs_clt_inc_failover_cnt(struct rtrs_clt_stats *stats) s->rdma.failover_cnt++; } -int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, - char *buf, size_t len) +int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, char *buf) { struct rtrs_clt_stats_pcpu *s; @@ -66,15 +65,13 @@ int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, return used; } -int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf, - size_t len) +int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf) { return sysfs_emit(buf, "%d %d\n", stats->reconnects.successful_cnt, stats->reconnects.fail_cnt); } -ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, - char *page, size_t len) +ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, char *page) { struct rtrs_clt_stats_rdma sum; struct rtrs_clt_stats_rdma *r; @@ -98,8 +95,7 @@ ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, atomic_read(&stats->inflight), sum.failover_cnt); } -ssize_t rtrs_clt_reset_all_help(struct rtrs_clt_stats *s, - char *page, size_t len) +ssize_t rtrs_clt_reset_all_help(struct rtrs_clt_stats *s, char *page) { return sysfs_emit(page, "echo 1 to reset all statistics\n"); } diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h index 72f9136e3c24..c31f920e6d10 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h @@ -223,19 +223,17 @@ void rtrs_clt_update_all_stats(struct rtrs_clt_io_req *req, int dir); int rtrs_clt_reset_rdma_lat_distr_stats(struct rtrs_clt_stats *stats, bool enable); ssize_t rtrs_clt_stats_rdma_lat_distr_to_str(struct rtrs_clt_stats *stats, - char *page, size_t len); + char *page); int rtrs_clt_reset_cpu_migr_stats(struct rtrs_clt_stats *stats, bool enable); -int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, char *buf, - size_t len); +int rtrs_clt_stats_migration_cnt_to_str(struct rtrs_clt_stats *stats, char *buf); int rtrs_clt_reset_reconnects_stat(struct rtrs_clt_stats *stats, bool enable); -int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf, - size_t len); +int rtrs_clt_stats_reconnects_to_str(struct rtrs_clt_stats *stats, char *buf); int rtrs_clt_reset_rdma_stats(struct rtrs_clt_stats *stats, bool enable); ssize_t rtrs_clt_stats_rdma_to_str(struct rtrs_clt_stats *stats, - char *page, size_t len); + char *page); int rtrs_clt_reset_all_stats(struct rtrs_clt_stats *stats, bool enable); ssize_t rtrs_clt_reset_all_help(struct rtrs_clt_stats *stats, - char *page, size_t len); + char *page); /* rtrs-clt-sysfs.c */ diff --git a/drivers/infiniband/ulp/rtrs/rtrs-pri.h b/drivers/infiniband/ulp/rtrs/rtrs-pri.h index d12ddfa50747..78eac9a4f703 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-pri.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-pri.h @@ -398,7 +398,7 @@ static ssize_t get_value##_show(struct kobject *kobj, \ { \ type *stats = container_of(kobj, type, kobj_stats); \ \ - return print(stats, page, PAGE_SIZE); \ + return print(stats, page); \ } #define STAT_ATTR(type, stat, print, reset) \ diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c index 12c374b5eb6e..44b1c1652131 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv-stats.c @@ -23,8 +23,7 @@ int rtrs_srv_reset_rdma_stats(struct rtrs_srv_stats *stats, bool enable) return -EINVAL; } -ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, - char *page, size_t len) +ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, char *page) { struct rtrs_srv_stats_rdma_stats *r = &stats->rdma_stats; diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.h b/drivers/infiniband/ulp/rtrs/rtrs-srv.h index 9d8d2a91a235..7d403c12faf3 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.h @@ -136,8 +136,7 @@ static inline void rtrs_srv_update_rdma_stats(struct rtrs_srv_stats *s, /* functions which are implemented in rtrs-srv-stats.c */ int rtrs_srv_reset_rdma_stats(struct rtrs_srv_stats *stats, bool enable); -ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, - char *page, size_t len); +ssize_t rtrs_srv_stats_rdma_to_str(struct rtrs_srv_stats *stats, char *page); int rtrs_srv_reset_all_stats(struct rtrs_srv_stats *stats, bool enable); ssize_t rtrs_srv_reset_all_help(struct rtrs_srv_stats *stats, char *page, size_t len); From patchwork Fri Jul 30 13:18:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C730C432BE for ; Fri, 30 Jul 2021 13:18:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7318360EFD for ; Fri, 30 Jul 2021 13:18:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238979AbhG3NSr (ORCPT ); Fri, 30 Jul 2021 09:18:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50586 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238971AbhG3NSq (ORCPT ); Fri, 30 Jul 2021 09:18:46 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6DFDC061799 for ; Fri, 30 Jul 2021 06:18:40 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id h14so11267714wrx.10 for ; Fri, 30 Jul 2021 06:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=srcX7bsGAXceZVqa1Kk5LYuDbolnujrpC54EAFKdSWs=; b=N6LjtYoHqkZZSFwoHJ30pJ5X8BBZ8kayipNivmIwadDlzP8xj6S8zj5Di/8yctk2je 5JfH1TXBHs9D0GRF+ppvhcpi3oRq29iXSKfmOGt0fjl8lncZenuhLGaJr1dMhW0wSLrH 3DdDO/MApizGHf6MY8cjnNEyEGijwQAey07kBhG16SbHSPoQwFo5HkAIlNGTF8ziXlEg 9Cv8T6ekKqRSfB1vOJAYg+6cszFo/6oPXdZChDJkoNSITxQjLO8jZMAuC0dFvhKKgxru gsmM5r0Bhm19SdlyT/Q2sbX5jUkzxR0tfZn6b9H69GKr4Lc2hHkTlD/oHNx4OcTPYQv/ 1ZZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=srcX7bsGAXceZVqa1Kk5LYuDbolnujrpC54EAFKdSWs=; b=EZ8Ci5aYb80L3GcIoYr9VkdU8s6Zn7NI1O1GVmUnrAY7YQ3XOak0Qvdpg7yePkUDxa Eu5Kjwmcw0kV2UHm0pcUczCOFe9F46YsPvhc95HcoirRWCSqetCdKTnaEEqwmBAmhNkQ HRXbEspAeBXaarU1oqZd8m5Ao8hLaaDquQq++HXtygsRQ86jna+sDUAdVDYfLoeOHOzN oF/b/5MqBjQRXkoP3S9aCGYxavHGvR/7GQxv51zc615CZZeLpKRg7rTjfJQqG5HgoYC+ OEdYAYtZfoyXwGK0ecbMq5DBu1Id2OBRk0PcF5XVdihOhtY0uBp/KzQdHMmBIyurVc5b zCOw== X-Gm-Message-State: AOAM531+st4Hm/SGXWgO9TA4OKhQgggRMcaSHJIN4ByDjOyV6lkrXntJ nDoWaE6KlOO98JnPFS9a8HLhNDPfXhPv4A== X-Google-Smtp-Source: ABdhPJwp2yW6Y9UroTOet1waYlrGTEiyudX9nxwQPuhtpNH7x57mrzpzwJoPO/Yz1FXY7cbApTCOeA== X-Received: by 2002:a5d:4a49:: with SMTP id v9mr1070408wrs.4.1627651119107; Fri, 30 Jul 2021 06:18:39 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:38 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Gioh Kim Subject: [PATCH for-next 07/10] RDMA/rtrs: Remove all likely and unlikely Date: Fri, 30 Jul 2021 15:18:29 +0200 Message-Id: <20210730131832.118865-8-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Gioh Kim The IO performance test with fio after swapping the likely and unlikely macros in all if-statement shows no difference. They do not help for the performance of rtrs. Thanks to Haakon Bugge for the test scenario. The fio test did random read on 32 rnbd devices and 64 processes. Test environment: - Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz - 376G memory - kernel version: 5.4.86 - gcc version: gcc (Debian 8.3.0-6) 8.3.0 - Infiniband controller: Mellanox Technologies MT27800 Family [ConnectX-5] Test result: - before swapping: IOPS=829k, BW=3239MiB/s - after swapping: IOPS=829k, BW=3238MiB/s - remove all (un)likely: IOPS=829k, BW=3238MiB/s Signed-off-by: Gioh Kim Signed-off-by: Jack Wang Reviewed-by: Leon Romanovsky --- drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 2 +- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 126 +++++++++---------- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 74 ++++++----- 3 files changed, 99 insertions(+), 103 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c index 585f83cd7ed1..4f1980a3608a 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c @@ -20,7 +20,7 @@ void rtrs_clt_update_wc_stats(struct rtrs_clt_con *con) cpu = raw_smp_processor_id(); s = this_cpu_ptr(stats->pcpu_stats); - if (unlikely(con->cpu != cpu)) { + if (con->cpu != cpu) { s->cpu_migr.to++; /* Careful here, override s pointer */ diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index e048bfa12755..3cf7118a1c00 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -75,9 +75,9 @@ __rtrs_get_permit(struct rtrs_clt *clt, enum rtrs_clt_con_type con_type) */ do { bit = find_first_zero_bit(clt->permits_map, max_depth); - if (unlikely(bit >= max_depth)) + if (bit >= max_depth) return NULL; - } while (unlikely(test_and_set_bit_lock(bit, clt->permits_map))); + } while (test_and_set_bit_lock(bit, clt->permits_map)); permit = get_permit(clt, bit); WARN_ON(permit->mem_id != bit); @@ -115,14 +115,14 @@ struct rtrs_permit *rtrs_clt_get_permit(struct rtrs_clt *clt, DEFINE_WAIT(wait); permit = __rtrs_get_permit(clt, con_type); - if (likely(permit) || !can_wait) + if (permit || !can_wait) return permit; do { prepare_to_wait(&clt->permits_wait, &wait, TASK_UNINTERRUPTIBLE); permit = __rtrs_get_permit(clt, con_type); - if (likely(permit)) + if (permit) break; io_schedule(); @@ -175,7 +175,7 @@ struct rtrs_clt_con *rtrs_permit_to_clt_con(struct rtrs_clt_sess *sess, { int id = 0; - if (likely(permit->con_type == RTRS_IO_CON)) + if (permit->con_type == RTRS_IO_CON) id = (permit->cpu_id % (sess->s.irq_con_num - 1)) + 1; return to_clt_con(sess->s.con[id]); @@ -329,7 +329,7 @@ static void rtrs_clt_fast_reg_done(struct ib_cq *cq, struct ib_wc *wc) { struct rtrs_clt_con *con = to_clt_con(wc->qp->qp_context); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(con->c.sess, "Failed IB_WR_REG_MR: %s\n", ib_wc_status_msg(wc->status)); rtrs_rdma_error_recovery(con); @@ -349,13 +349,13 @@ static void rtrs_clt_inv_rkey_done(struct ib_cq *cq, struct ib_wc *wc) container_of(wc->wr_cqe, typeof(*req), inv_cqe); struct rtrs_clt_con *con = to_clt_con(wc->qp->qp_context); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(con->c.sess, "Failed IB_WR_LOCAL_INV: %s\n", ib_wc_status_msg(wc->status)); rtrs_rdma_error_recovery(con); } req->need_inv = false; - if (likely(req->need_inv_comp)) + if (req->need_inv_comp) complete(&req->inv_comp); else /* Complete request from INV callback */ @@ -390,7 +390,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, sess = to_clt_sess(con->c.sess); if (req->sg_cnt) { - if (unlikely(req->dir == DMA_FROM_DEVICE && req->need_inv)) { + if (req->dir == DMA_FROM_DEVICE && req->need_inv) { /* * We are here to invalidate read requests * ourselves. In normal scenario server should @@ -405,7 +405,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, * should do that ourselves. */ - if (likely(can_wait)) { + if (can_wait) { req->need_inv_comp = true; } else { /* This should be IO path, so always notify */ @@ -416,10 +416,10 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, refcount_inc(&req->ref); err = rtrs_inv_rkey(req); - if (unlikely(err)) { + if (err) { rtrs_err(con->c.sess, "Send INV WR key=%#x: %d\n", req->mr->rkey, err); - } else if (likely(can_wait)) { + } else if (can_wait) { wait_for_completion(&req->inv_comp); } else { /* @@ -463,7 +463,7 @@ static int rtrs_post_send_rdma(struct rtrs_clt_con *con, enum ib_send_flags flags; struct ib_sge sge; - if (unlikely(!req->sg_size)) { + if (!req->sg_size) { rtrs_wrn(con->c.sess, "Doing RDMA Write failed, no data supplied\n"); return -EINVAL; @@ -513,7 +513,7 @@ static void rtrs_clt_recv_done(struct rtrs_clt_con *con, struct ib_wc *wc) iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); err = rtrs_iu_post_recv(&con->c, iu); - if (unlikely(err)) { + if (err) { rtrs_err(con->c.sess, "post iu failed %d\n", err); rtrs_rdma_error_recovery(con); } @@ -533,7 +533,7 @@ static void rtrs_clt_rkey_rsp_done(struct rtrs_clt_con *con, struct ib_wc *wc) iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); - if (unlikely(wc->byte_len < sizeof(*msg))) { + if (wc->byte_len < sizeof(*msg)) { rtrs_err(con->c.sess, "rkey response is malformed: size %d\n", wc->byte_len); goto out; @@ -541,7 +541,7 @@ static void rtrs_clt_rkey_rsp_done(struct rtrs_clt_con *con, struct ib_wc *wc) ib_dma_sync_single_for_cpu(sess->s.dev->ib_dev, iu->dma_addr, iu->size, DMA_FROM_DEVICE); msg = iu->buf; - if (unlikely(le16_to_cpu(msg->type) != RTRS_MSG_RKEY_RSP)) { + if (le16_to_cpu(msg->type) != RTRS_MSG_RKEY_RSP) { rtrs_err(sess->clt, "rkey response is malformed: type %d\n", le16_to_cpu(msg->type)); goto out; @@ -551,8 +551,8 @@ static void rtrs_clt_rkey_rsp_done(struct rtrs_clt_con *con, struct ib_wc *wc) goto out; rtrs_from_imm(be32_to_cpu(wc->ex.imm_data), &imm_type, &imm_payload); - if (likely(imm_type == RTRS_IO_RSP_IMM || - imm_type == RTRS_IO_RSP_W_INV_IMM)) { + if (imm_type == RTRS_IO_RSP_IMM || + imm_type == RTRS_IO_RSP_W_INV_IMM) { u32 msg_id; w_inval = (imm_type == RTRS_IO_RSP_W_INV_IMM); @@ -605,7 +605,7 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) bool w_inval = false; int err; - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { if (wc->status != IB_WC_WR_FLUSH_ERR) { rtrs_err(sess->clt, "RDMA failed: %s\n", ib_wc_status_msg(wc->status)); @@ -625,8 +625,8 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) return; rtrs_from_imm(be32_to_cpu(wc->ex.imm_data), &imm_type, &imm_payload); - if (likely(imm_type == RTRS_IO_RSP_IMM || - imm_type == RTRS_IO_RSP_W_INV_IMM)) { + if (imm_type == RTRS_IO_RSP_IMM || + imm_type == RTRS_IO_RSP_W_INV_IMM) { u32 msg_id; w_inval = (imm_type == RTRS_IO_RSP_W_INV_IMM); @@ -657,7 +657,7 @@ static void rtrs_clt_rdma_done(struct ib_cq *cq, struct ib_wc *wc) err = rtrs_post_recv_empty_x2(&con->c, &io_comp_cqe); else err = rtrs_post_recv_empty(&con->c, &io_comp_cqe); - if (unlikely(err)) { + if (err) { rtrs_err(con->c.sess, "rtrs_post_recv_empty(): %d\n", err); rtrs_rdma_error_recovery(con); @@ -703,7 +703,7 @@ static int post_recv_io(struct rtrs_clt_con *con, size_t q_size) } else { err = rtrs_post_recv_empty(&con->c, &io_comp_cqe); } - if (unlikely(err)) + if (err) return err; } @@ -728,7 +728,7 @@ static int post_recv_sess(struct rtrs_clt_sess *sess) q_size *= 2; err = post_recv_io(to_clt_con(sess->s.con[cid]), q_size); - if (unlikely(err)) { + if (err) { rtrs_err(sess->clt, "post_recv_io(), err: %d\n", err); return err; } @@ -789,7 +789,7 @@ static struct rtrs_clt_sess *get_next_path_rr(struct path_it *it) ppcpu_path = this_cpu_ptr(clt->pcpu_path); path = rcu_dereference(*ppcpu_path); - if (unlikely(!path)) + if (!path) path = list_first_or_null_rcu(&clt->paths_list, typeof(*path), s.entry); else @@ -820,10 +820,10 @@ static struct rtrs_clt_sess *get_next_path_min_inflight(struct path_it *it) int inflight; list_for_each_entry_rcu(sess, &clt->paths_list, s.entry) { - if (unlikely(READ_ONCE(sess->state) != RTRS_CLT_CONNECTED)) + if (READ_ONCE(sess->state) != RTRS_CLT_CONNECTED) continue; - if (unlikely(!list_empty(raw_cpu_ptr(sess->mp_skip_entry)))) + if (!list_empty(raw_cpu_ptr(sess->mp_skip_entry))) continue; inflight = atomic_read(&sess->stats->inflight); @@ -871,10 +871,10 @@ static struct rtrs_clt_sess *get_next_path_min_latency(struct path_it *it) ktime_t latency; list_for_each_entry_rcu(sess, &clt->paths_list, s.entry) { - if (unlikely(READ_ONCE(sess->state) != RTRS_CLT_CONNECTED)) + if (READ_ONCE(sess->state) != RTRS_CLT_CONNECTED) continue; - if (unlikely(!list_empty(raw_cpu_ptr(sess->mp_skip_entry)))) + if (!list_empty(raw_cpu_ptr(sess->mp_skip_entry))) continue; latency = sess->s.hb_cur_latency; @@ -1063,7 +1063,7 @@ static int rtrs_map_sg_fr(struct rtrs_clt_io_req *req, size_t count) nr = ib_map_mr_sg(req->mr, req->sglist, count, NULL, SZ_4K); if (nr < 0) return nr; - if (unlikely(nr < req->sg_cnt)) + if (nr < req->sg_cnt) return -EINVAL; ib_update_fast_reg_key(req->mr, ib_inc_rkey(req->mr->rkey)); @@ -1087,7 +1087,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) const size_t tsize = sizeof(*msg) + req->data_len + req->usr_len; - if (unlikely(tsize > sess->chunk_size)) { + if (tsize > sess->chunk_size) { rtrs_wrn(s, "Write request failed, size too big %zu > %d\n", tsize, sess->chunk_size); return -EMSGSIZE; @@ -1095,7 +1095,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) if (req->sg_cnt) { count = ib_dma_map_sg(sess->s.dev->ib_dev, req->sglist, req->sg_cnt, req->dir); - if (unlikely(!count)) { + if (!count) { rtrs_wrn(s, "Write request failed, map failed\n"); return -EINVAL; } @@ -1149,7 +1149,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) ret = rtrs_post_rdma_write_sg(req->con, req, rbuf, fr_en, req->usr_len + sizeof(*msg), imm, wr, &inv_wr); - if (unlikely(ret)) { + if (ret) { rtrs_err_rl(s, "Write request failed: error=%d path=%s [%s:%u]\n", ret, kobject_name(&sess->kobj), sess->hca_name, @@ -1180,7 +1180,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) const size_t tsize = sizeof(*msg) + req->data_len + req->usr_len; - if (unlikely(tsize > sess->chunk_size)) { + if (tsize > sess->chunk_size) { rtrs_wrn(s, "Read request failed, message size is %zu, bigger than CHUNK_SIZE %d\n", tsize, sess->chunk_size); @@ -1190,7 +1190,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) if (req->sg_cnt) { count = ib_dma_map_sg(dev->ib_dev, req->sglist, req->sg_cnt, req->dir); - if (unlikely(!count)) { + if (!count) { rtrs_wrn(s, "Read request failed, dma map failed\n"); return -EINVAL; @@ -1255,7 +1255,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) ret = rtrs_post_send_rdma(req->con, req, &sess->rbufs[buf_id], req->data_len, imm, wr); - if (unlikely(ret)) { + if (ret) { rtrs_err_rl(s, "Read request failed: error=%d path=%s [%s:%u]\n", ret, kobject_name(&sess->kobj), sess->hca_name, @@ -1288,15 +1288,14 @@ static int rtrs_clt_failover_req(struct rtrs_clt *clt, for (path_it_init(&it, clt); (alive_sess = it.next_path(&it)) && it.i < it.clt->paths_num; it.i++) { - if (unlikely(READ_ONCE(alive_sess->state) != - RTRS_CLT_CONNECTED)) + if (READ_ONCE(alive_sess->state) != RTRS_CLT_CONNECTED) continue; req = rtrs_clt_get_copy_req(alive_sess, fail_req); if (req->dir == DMA_TO_DEVICE) err = rtrs_clt_write_req(req); else err = rtrs_clt_read_req(req); - if (unlikely(err)) { + if (err) { req->in_use = false; continue; } @@ -1331,7 +1330,7 @@ static void fail_all_outstanding_reqs(struct rtrs_clt_sess *sess) complete_rdma_req(req, -ECONNABORTED, false, true); err = rtrs_clt_failover_req(clt, req); - if (unlikely(err)) + if (err) /* Failover failed, notify anyway */ req->conf(req->priv, err); } @@ -1963,7 +1962,7 @@ static int rtrs_clt_rdma_cm_handler(struct rdma_cm_id *cm_id, break; case RDMA_CM_EVENT_ESTABLISHED: cm_err = rtrs_rdma_conn_established(con, ev); - if (likely(!cm_err)) { + if (!cm_err) { /* * Report success and wake up. Here we abuse state_wq, * i.e. wake up without state change, but we set cm_err. @@ -2382,7 +2381,7 @@ static void rtrs_clt_info_req_done(struct ib_cq *cq, struct ib_wc *wc) iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); rtrs_iu_free(iu, sess->s.dev->ib_dev, 1); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(sess->clt, "Sess info request send failed: %s\n", ib_wc_status_msg(wc->status)); rtrs_clt_change_state_get_old(sess, RTRS_CLT_CONNECTING_ERR, NULL); @@ -2399,7 +2398,7 @@ static int process_info_rsp(struct rtrs_clt_sess *sess, int i, sgi; sg_cnt = le16_to_cpu(msg->sg_cnt); - if (unlikely(!sg_cnt || (sess->queue_depth % sg_cnt))) { + if (!sg_cnt || (sess->queue_depth % sg_cnt)) { rtrs_err(sess->clt, "Incorrect sg_cnt %d, is not multiple\n", sg_cnt); return -EINVAL; @@ -2409,9 +2408,8 @@ static int process_info_rsp(struct rtrs_clt_sess *sess, * Check if IB immediate data size is enough to hold the mem_id and * the offset inside the memory chunk. */ - if (unlikely((ilog2(sg_cnt - 1) + 1) + - (ilog2(sess->chunk_size - 1) + 1) > - MAX_IMM_PAYL_BITS)) { + if ((ilog2(sg_cnt - 1) + 1) + (ilog2(sess->chunk_size - 1) + 1) > + MAX_IMM_PAYL_BITS) { rtrs_err(sess->clt, "RDMA immediate size (%db) not enough to encode %d buffers of size %dB\n", MAX_IMM_PAYL_BITS, sg_cnt, sess->chunk_size); @@ -2429,7 +2427,7 @@ static int process_info_rsp(struct rtrs_clt_sess *sess, total_len += len; - if (unlikely(!len || (len % sess->chunk_size))) { + if (!len || (len % sess->chunk_size)) { rtrs_err(sess->clt, "Incorrect [%d].len %d\n", sgi, len); return -EINVAL; @@ -2443,11 +2441,11 @@ static int process_info_rsp(struct rtrs_clt_sess *sess, } } /* Sanity check */ - if (unlikely(sgi != sg_cnt || i != sess->queue_depth)) { + if (sgi != sg_cnt || i != sess->queue_depth) { rtrs_err(sess->clt, "Incorrect sg vector, not fully mapped\n"); return -EINVAL; } - if (unlikely(total_len != sess->chunk_size * sess->queue_depth)) { + if (total_len != sess->chunk_size * sess->queue_depth) { rtrs_err(sess->clt, "Incorrect total_len %d\n", total_len); return -EINVAL; } @@ -2469,14 +2467,14 @@ static void rtrs_clt_info_rsp_done(struct ib_cq *cq, struct ib_wc *wc) WARN_ON(con->c.cid); iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(sess->clt, "Sess info response recv failed: %s\n", ib_wc_status_msg(wc->status)); goto out; } WARN_ON(wc->opcode != IB_WC_RECV); - if (unlikely(wc->byte_len < sizeof(*msg))) { + if (wc->byte_len < sizeof(*msg)) { rtrs_err(sess->clt, "Sess info response is malformed: size %d\n", wc->byte_len); goto out; @@ -2484,24 +2482,24 @@ static void rtrs_clt_info_rsp_done(struct ib_cq *cq, struct ib_wc *wc) ib_dma_sync_single_for_cpu(sess->s.dev->ib_dev, iu->dma_addr, iu->size, DMA_FROM_DEVICE); msg = iu->buf; - if (unlikely(le16_to_cpu(msg->type) != RTRS_MSG_INFO_RSP)) { + if (le16_to_cpu(msg->type) != RTRS_MSG_INFO_RSP) { rtrs_err(sess->clt, "Sess info response is malformed: type %d\n", le16_to_cpu(msg->type)); goto out; } rx_sz = sizeof(*msg); rx_sz += sizeof(msg->desc[0]) * le16_to_cpu(msg->sg_cnt); - if (unlikely(wc->byte_len < rx_sz)) { + if (wc->byte_len < rx_sz) { rtrs_err(sess->clt, "Sess info response is malformed: size %d\n", wc->byte_len); goto out; } err = process_info_rsp(sess, msg); - if (unlikely(err)) + if (err) goto out; err = post_recv_sess(sess); - if (unlikely(err)) + if (err) goto out; state = RTRS_CLT_CONNECTED; @@ -2528,13 +2526,13 @@ static int rtrs_send_sess_info(struct rtrs_clt_sess *sess) rtrs_clt_info_req_done); rx_iu = rtrs_iu_alloc(1, rx_sz, GFP_KERNEL, sess->s.dev->ib_dev, DMA_FROM_DEVICE, rtrs_clt_info_rsp_done); - if (unlikely(!tx_iu || !rx_iu)) { + if (!tx_iu || !rx_iu) { err = -ENOMEM; goto out; } /* Prepare for getting info response */ err = rtrs_iu_post_recv(&usr_con->c, rx_iu); - if (unlikely(err)) { + if (err) { rtrs_err(sess->clt, "rtrs_iu_post_recv(), err: %d\n", err); goto out; } @@ -2549,7 +2547,7 @@ static int rtrs_send_sess_info(struct rtrs_clt_sess *sess) /* Send info request */ err = rtrs_iu_post_send(&usr_con->c, tx_iu, sizeof(*msg), NULL); - if (unlikely(err)) { + if (err) { rtrs_err(sess->clt, "rtrs_iu_post_send(), err: %d\n", err); goto out; } @@ -2560,7 +2558,7 @@ static int rtrs_send_sess_info(struct rtrs_clt_sess *sess) sess->state != RTRS_CLT_CONNECTING, msecs_to_jiffies( RTRS_CONNECT_TIMEOUT_MS)); - if (unlikely(READ_ONCE(sess->state) != RTRS_CLT_CONNECTED)) { + if (READ_ONCE(sess->state) != RTRS_CLT_CONNECTED) { if (READ_ONCE(sess->state) == RTRS_CLT_CONNECTING_ERR) err = -ECONNRESET; else @@ -2572,7 +2570,7 @@ static int rtrs_send_sess_info(struct rtrs_clt_sess *sess) rtrs_iu_free(tx_iu, sess->s.dev->ib_dev, 1); if (rx_iu) rtrs_iu_free(rx_iu, sess->s.dev->ib_dev, 1); - if (unlikely(err)) + if (err) /* If we've never taken async path because of malloc problems */ rtrs_clt_change_state_get_old(sess, RTRS_CLT_CONNECTING_ERR, NULL); @@ -2920,7 +2918,7 @@ int rtrs_clt_remove_path_from_sysfs(struct rtrs_clt_sess *sess, &old_state); } while (!changed && old_state != RTRS_CLT_DEAD); - if (likely(changed)) { + if (changed) { rtrs_clt_remove_path_from_arr(sess); rtrs_clt_destroy_sess_files(sess, sysfs_self); kobject_put(&sess->kobj); @@ -2992,10 +2990,10 @@ int rtrs_clt_request(int dir, struct rtrs_clt_req_ops *ops, rcu_read_lock(); for (path_it_init(&it, clt); (sess = it.next_path(&it)) && it.i < it.clt->paths_num; it.i++) { - if (unlikely(READ_ONCE(sess->state) != RTRS_CLT_CONNECTED)) + if (READ_ONCE(sess->state) != RTRS_CLT_CONNECTED) continue; - if (unlikely(usr_len + hdr_len > sess->max_hdr_size)) { + if (usr_len + hdr_len > sess->max_hdr_size) { rtrs_wrn_rl(sess->clt, "%s request failed, user message size is %zu and header length %zu, but max size is %u\n", dir == READ ? "Read" : "Write", @@ -3010,7 +3008,7 @@ int rtrs_clt_request(int dir, struct rtrs_clt_req_ops *ops, err = rtrs_clt_read_req(req); else err = rtrs_clt_write_req(req); - if (unlikely(err)) { + if (err) { req->in_use = false; continue; } diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 5969a74d45a0..cc65cffdc65a 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -183,7 +183,7 @@ static void rtrs_srv_reg_mr_done(struct ib_cq *cq, struct ib_wc *wc) struct rtrs_sess *s = con->c.sess; struct rtrs_srv_sess *sess = to_srv_sess(s); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(s, "REG MR failed: %s\n", ib_wc_status_msg(wc->status)); close_sess(sess); @@ -215,7 +215,7 @@ static int rdma_write_sg(struct rtrs_srv_op *id) sg_cnt = le16_to_cpu(id->rd_msg->sg_cnt); need_inval = le16_to_cpu(id->rd_msg->flags) & RTRS_MSG_NEED_INVAL_F; - if (unlikely(sg_cnt != 1)) + if (sg_cnt != 1) return -EINVAL; offset = 0; @@ -228,7 +228,7 @@ static int rdma_write_sg(struct rtrs_srv_op *id) /* WR will fail with length error * if this is 0 */ - if (unlikely(plist->length == 0)) { + if (plist->length == 0) { rtrs_err(s, "Invalid RDMA-Write sg list length 0\n"); return -EINVAL; } @@ -321,7 +321,7 @@ static int rdma_write_sg(struct rtrs_srv_op *id) offset, DMA_BIDIRECTIONAL); err = ib_post_send(id->con->c.qp, &id->tx_wr.wr, NULL); - if (unlikely(err)) + if (err) rtrs_err(s, "Posting RDMA-Write-Request to QP failed, err: %d\n", err); @@ -361,7 +361,7 @@ static int send_io_resp_imm(struct rtrs_srv_con *con, struct rtrs_srv_op *id, sg_cnt = le16_to_cpu(rd_msg->sg_cnt); if (need_inval) { - if (likely(sg_cnt)) { + if (sg_cnt) { inv_wr.wr_cqe = &io_comp_cqe; inv_wr.sg_list = NULL; inv_wr.num_sge = 0; @@ -437,7 +437,7 @@ static int send_io_resp_imm(struct rtrs_srv_con *con, struct rtrs_srv_op *id, imm_wr.wr.ex.imm_data = cpu_to_be32(imm); err = ib_post_send(id->con->c.qp, wr, NULL); - if (unlikely(err)) + if (err) rtrs_err_rl(s, "Posting RDMA-Reply to QP failed, err: %d\n", err); @@ -494,7 +494,7 @@ bool rtrs_srv_resp_rdma(struct rtrs_srv_op *id, int status) id->status = status; - if (unlikely(sess->state != RTRS_SRV_CONNECTED)) { + if (sess->state != RTRS_SRV_CONNECTED) { rtrs_err_rl(s, "Sending I/O response failed, session %s is disconnected, sess state %s\n", kobject_name(&sess->kobj), @@ -506,8 +506,7 @@ bool rtrs_srv_resp_rdma(struct rtrs_srv_op *id, int status) ib_update_fast_reg_key(mr->mr, ib_inc_rkey(mr->mr->rkey)); } - if (unlikely(atomic_sub_return(1, - &con->c.sq_wr_avail) < 0)) { + if (atomic_sub_return(1, &con->c.sq_wr_avail) < 0) { rtrs_err(s, "IB send queue full: sess=%s cid=%d\n", kobject_name(&sess->kobj), con->c.cid); @@ -523,7 +522,7 @@ bool rtrs_srv_resp_rdma(struct rtrs_srv_op *id, int status) else err = rdma_write_sg(id); - if (unlikely(err)) { + if (err) { rtrs_err_rl(s, "IO response failed: %d: sess=%s\n", err, kobject_name(&sess->kobj)); close_sess(sess); @@ -710,7 +709,7 @@ static void rtrs_srv_info_rsp_done(struct ib_cq *cq, struct ib_wc *wc) iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); rtrs_iu_free(iu, sess->s.dev->ib_dev, 1); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(s, "Sess info response send failed: %s\n", ib_wc_status_msg(wc->status)); close_sess(sess); @@ -807,7 +806,7 @@ static int process_info_req(struct rtrs_srv_con *con, size_t tx_sz; err = post_recv_sess(sess); - if (unlikely(err)) { + if (err) { rtrs_err(s, "post_recv_sess(), err: %d\n", err); return err; } @@ -820,14 +819,14 @@ static int process_info_req(struct rtrs_srv_con *con, strscpy(sess->s.sessname, msg->sessname, sizeof(sess->s.sessname)); rwr = kcalloc(sess->mrs_num, sizeof(*rwr), GFP_KERNEL); - if (unlikely(!rwr)) + if (!rwr) return -ENOMEM; tx_sz = sizeof(*rsp); tx_sz += sizeof(rsp->desc[0]) * sess->mrs_num; tx_iu = rtrs_iu_alloc(1, tx_sz, GFP_KERNEL, sess->s.dev->ib_dev, DMA_TO_DEVICE, rtrs_srv_info_rsp_done); - if (unlikely(!tx_iu)) { + if (!tx_iu) { err = -ENOMEM; goto rwr_free; } @@ -859,7 +858,7 @@ static int process_info_req(struct rtrs_srv_con *con, } err = rtrs_srv_create_sess_files(sess); - if (unlikely(err)) + if (err) goto iu_free; kobject_get(&sess->kobj); get_device(&sess->srv->dev); @@ -879,7 +878,7 @@ static int process_info_req(struct rtrs_srv_con *con, /* Send info response */ err = rtrs_iu_post_send(&con->c, tx_iu, tx_sz, reg_wr); - if (unlikely(err)) { + if (err) { rtrs_err(s, "rtrs_iu_post_send(), err: %d\n", err); iu_free: rtrs_iu_free(tx_iu, sess->s.dev->ib_dev, 1); @@ -902,14 +901,14 @@ static void rtrs_srv_info_req_done(struct ib_cq *cq, struct ib_wc *wc) WARN_ON(con->c.cid); iu = container_of(wc->wr_cqe, struct rtrs_iu, cqe); - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(s, "Sess info request receive failed: %s\n", ib_wc_status_msg(wc->status)); goto close; } WARN_ON(wc->opcode != IB_WC_RECV); - if (unlikely(wc->byte_len < sizeof(*msg))) { + if (wc->byte_len < sizeof(*msg)) { rtrs_err(s, "Sess info request is malformed: size %d\n", wc->byte_len); goto close; @@ -917,13 +916,13 @@ static void rtrs_srv_info_req_done(struct ib_cq *cq, struct ib_wc *wc) ib_dma_sync_single_for_cpu(sess->s.dev->ib_dev, iu->dma_addr, iu->size, DMA_FROM_DEVICE); msg = iu->buf; - if (unlikely(le16_to_cpu(msg->type) != RTRS_MSG_INFO_REQ)) { + if (le16_to_cpu(msg->type) != RTRS_MSG_INFO_REQ) { rtrs_err(s, "Sess info request is malformed: type %d\n", le16_to_cpu(msg->type)); goto close; } err = process_info_req(con, msg); - if (unlikely(err)) + if (err) goto close; out: @@ -944,11 +943,11 @@ static int post_recv_info_req(struct rtrs_srv_con *con) rx_iu = rtrs_iu_alloc(1, sizeof(struct rtrs_msg_info_req), GFP_KERNEL, sess->s.dev->ib_dev, DMA_FROM_DEVICE, rtrs_srv_info_req_done); - if (unlikely(!rx_iu)) + if (!rx_iu) return -ENOMEM; /* Prepare for getting info response */ err = rtrs_iu_post_recv(&con->c, rx_iu); - if (unlikely(err)) { + if (err) { rtrs_err(s, "rtrs_iu_post_recv(), err: %d\n", err); rtrs_iu_free(rx_iu, sess->s.dev->ib_dev, 1); return err; @@ -963,7 +962,7 @@ static int post_recv_io(struct rtrs_srv_con *con, size_t q_size) for (i = 0; i < q_size; i++) { err = rtrs_post_recv_empty(&con->c, &io_comp_cqe); - if (unlikely(err)) + if (err) return err; } @@ -984,7 +983,7 @@ static int post_recv_sess(struct rtrs_srv_sess *sess) q_size = srv->queue_depth; err = post_recv_io(to_srv_con(sess->s.con[cid]), q_size); - if (unlikely(err)) { + if (err) { rtrs_err(s, "post_recv_io(), err: %d\n", err); return err; } @@ -1007,13 +1006,13 @@ static void process_read(struct rtrs_srv_con *con, void *data; int ret; - if (unlikely(sess->state != RTRS_SRV_CONNECTED)) { + if (sess->state != RTRS_SRV_CONNECTED) { rtrs_err_rl(s, "Processing read request failed, session is disconnected, sess state %s\n", rtrs_srv_state_str(sess->state)); return; } - if (unlikely(msg->sg_cnt != 1 && msg->sg_cnt != 0)) { + if (msg->sg_cnt != 1 && msg->sg_cnt != 0) { rtrs_err_rl(s, "Processing read request failed, invalid message\n"); return; @@ -1031,7 +1030,7 @@ static void process_read(struct rtrs_srv_con *con, ret = ctx->ops.rdma_ev(srv->priv, id, READ, data, data_len, data + data_len, usr_len); - if (unlikely(ret)) { + if (ret) { rtrs_err_rl(s, "Processing read request failed, user module cb reported for msg_id %d, err: %d\n", buf_id, ret); @@ -1065,7 +1064,7 @@ static void process_write(struct rtrs_srv_con *con, void *data; int ret; - if (unlikely(sess->state != RTRS_SRV_CONNECTED)) { + if (sess->state != RTRS_SRV_CONNECTED) { rtrs_err_rl(s, "Processing write request failed, session is disconnected, sess state %s\n", rtrs_srv_state_str(sess->state)); @@ -1082,8 +1081,8 @@ static void process_write(struct rtrs_srv_con *con, data_len = off - usr_len; data = page_address(srv->chunks[buf_id]); ret = ctx->ops.rdma_ev(srv->priv, id, WRITE, data, data_len, - data + data_len, usr_len); - if (unlikely(ret)) { + data + data_len, usr_len); + if (ret) { rtrs_err_rl(s, "Processing write request failed, user module callback reports err: %d\n", ret); @@ -1147,7 +1146,7 @@ static void rtrs_srv_inv_rkey_done(struct ib_cq *cq, struct ib_wc *wc) u32 msg_id, off; void *data; - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { rtrs_err(s, "Failed IB_WR_LOCAL_INV: %s\n", ib_wc_status_msg(wc->status)); close_sess(sess); @@ -1204,7 +1203,7 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc) u32 imm_type, imm_payload; int err; - if (unlikely(wc->status != IB_WC_SUCCESS)) { + if (wc->status != IB_WC_SUCCESS) { if (wc->status != IB_WC_WR_FLUSH_ERR) { rtrs_err(s, "%s (wr_cqe: %p, type: %d, vendor_err: 0x%x, len: %u)\n", @@ -1224,21 +1223,20 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc) if (WARN_ON(wc->wr_cqe != &io_comp_cqe)) return; err = rtrs_post_recv_empty(&con->c, &io_comp_cqe); - if (unlikely(err)) { + if (err) { rtrs_err(s, "rtrs_post_recv(), err: %d\n", err); close_sess(sess); break; } rtrs_from_imm(be32_to_cpu(wc->ex.imm_data), &imm_type, &imm_payload); - if (likely(imm_type == RTRS_IO_REQ_IMM)) { + if (imm_type == RTRS_IO_REQ_IMM) { u32 msg_id, off; void *data; msg_id = imm_payload >> sess->mem_bits; off = imm_payload & ((1 << sess->mem_bits) - 1); - if (unlikely(msg_id >= srv->queue_depth || - off >= max_chunk_size)) { + if (msg_id >= srv->queue_depth || off >= max_chunk_size) { rtrs_err(s, "Wrong msg_id %u, off %u\n", msg_id, off); close_sess(sess); @@ -1250,7 +1248,7 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc) mr->msg_off = off; mr->msg_id = msg_id; err = rtrs_srv_inv_rkey(con, mr); - if (unlikely(err)) { + if (err) { rtrs_err(s, "rtrs_post_recv(), err: %d\n", err); close_sess(sess); @@ -1278,7 +1276,7 @@ static void rtrs_srv_rdma_done(struct ib_cq *cq, struct ib_wc *wc) */ atomic_add(s->signal_interval, &con->c.sq_wr_avail); - if (unlikely(!list_empty_careful(&con->rsp_wr_wait_list))) + if (!list_empty_careful(&con->rsp_wr_wait_list)) rtrs_rdma_process_wr_wait_list(con); break; From patchwork Fri Jul 30 13:18:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1421C4320E for ; Fri, 30 Jul 2021 13:18:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C899D60EFD for ; Fri, 30 Jul 2021 13:18:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238958AbhG3NSr (ORCPT ); Fri, 30 Jul 2021 09:18:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238947AbhG3NSq (ORCPT ); Fri, 30 Jul 2021 09:18:46 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80B2BC0613D3 for ; Fri, 30 Jul 2021 06:18:41 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id b13so433063wrs.3 for ; Fri, 30 Jul 2021 06:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lqPYdNt3Gv7bWSp6dNQq+BRHzI0K63w0pHIcUAXk5rI=; b=feJtNpxhED1R1cf0aha3h82MtAB+vMlHhIb7nJ5pOriM1sc3IFsCf0L0tn4Hu+CRBg UKfqQFf/p2Yg+mBHHCoveSUx5HHFyY+Ktfodyo25/POt2BP4NuonRaHGEZcZPAx9ytkU k8lVsjDno60waMjFLldMShqQR0v5Fvi0/QEZ39tTQFdjjO7r7SPcXAhkuWvWZmb7Yxkk wwuHl++TTCj5KryG+bJxe07ENLKgfhJWKwpkntgcMiQzUqUTEhi/qfHd7xwRcrPi/vSR 4gmwV+JhmE8BRhfKpQfzJN8OQDzn/h1w0jO9zuovCQSGxNanU+T8Rb45RKqpDcfnv3Bl GihA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lqPYdNt3Gv7bWSp6dNQq+BRHzI0K63w0pHIcUAXk5rI=; b=Fj+bCW79sbnd4SMG2DDC9Kx4XZWhu8zENa+58Hyae+6E5Cc8Fw8cmza4eCMp4tVRK4 umiHswoTd3sXnD7wU69MOKo+hT5hlUcUoKfLoeu+38VGGjYl/jynkgQJpZkKwDlfkOxi +0DkU2dQtoOVnFcwBJjK0f3JN8Lh3f46m1oEIHdk62YwdO98OLgh+st5G2/K6QKnu8nB xIa6CGQ+PRoDhzs+KLj+4mKdZbni1BD90kirEiFMlcUWnGjMrrhiYI/K3iSVzFko90sN aEMJA1tJQBgrQSNwWZ+O7OLgm2cUzTXamCVF5r5WZ0l7rXh88i4MWGWn0ONna2SGnzSj iKuw== X-Gm-Message-State: AOAM531EPp1Y9zZqd4T30KeMe7vrJVzzAP0PeFSRMbRGOCvyfJgCIYDf PwJoD6ywlrqMRu2RJtDjice72vRiSTiRrw== X-Google-Smtp-Source: ABdhPJwVnZSNaTdn1EHR7wL6cqaqR6BJG5gRw+XBoDXyt5e8CX0AoXEpzkGiATVnIZjzXuMxSgwnQg== X-Received: by 2002:a5d:6991:: with SMTP id g17mr3078606wru.253.1627651119944; Fri, 30 Jul 2021 06:18:39 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:39 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Gioh Kim Subject: [PATCH for-next 08/10] RDMA/rtrs-clt: Fix counting inflight IO Date: Fri, 30 Jul 2021 15:18:30 +0200 Message-Id: <20210730131832.118865-9-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Gioh Kim There are mis-match at counting inflight IO after changing the multipath policy. For example, we started fio test with round-robin policy and then we changed the policy to min-inflight. IOs created under the RR policy is finished under the min-inflight policy and inflight counter only decreased. So the counter would be negative value. And also we started fio test with min-inflight policy and changed the policy to the round-robin. IOs created under the min-inflight policy increased the inflight IO counter but the inflight IO counter was not decreased because the policy was the round-robin when IO was finished. So it should count IOs only if the IO is created under the min-inflight policy. It should not care the policy when the IO is finished. This patch adds a field mp_policy in struct rtrs_clt_io_req and stores the multipath policy when an object of rtrs_clt_io_req is created. Then rtrs-clt checks the mp_policy of only struct rtrs_clt_io_req instead of the struct rtrs_clt. Signed-off-by: Gioh Kim Signed-off-by: Jack Wang Reviewed-by: Leon Romanovsky --- drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c | 2 +- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 7 ++++--- drivers/infiniband/ulp/rtrs/rtrs-clt.h | 1 + 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c index 4f1980a3608a..61d5e0018392 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt-stats.c @@ -174,7 +174,7 @@ void rtrs_clt_update_all_stats(struct rtrs_clt_io_req *req, int dir) len = req->usr_len + req->data_len; rtrs_clt_update_rdma_stats(stats, len, dir); - if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT) + if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_inc(&stats->inflight); } diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 3cf7118a1c00..5cce727abca0 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -438,7 +438,7 @@ static void complete_rdma_req(struct rtrs_clt_io_req *req, int errno, } if (!refcount_dec_and_test(&req->ref)) return; - if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT) + if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&sess->stats->inflight); req->in_use = false; @@ -964,6 +964,7 @@ static void rtrs_clt_init_req(struct rtrs_clt_io_req *req, req->need_inv_comp = false; req->inv_errno = 0; refcount_set(&req->ref, 1); + req->mp_policy = sess->clt->mp_policy; iov_iter_kvec(&iter, READ, vec, 1, usr_len); len = _copy_from_iter(req->iu->buf, usr_len, &iter); @@ -1154,7 +1155,7 @@ static int rtrs_clt_write_req(struct rtrs_clt_io_req *req) "Write request failed: error=%d path=%s [%s:%u]\n", ret, kobject_name(&sess->kobj), sess->hca_name, sess->hca_port); - if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT) + if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&sess->stats->inflight); if (req->sg_cnt) ib_dma_unmap_sg(sess->s.dev->ib_dev, req->sglist, @@ -1260,7 +1261,7 @@ static int rtrs_clt_read_req(struct rtrs_clt_io_req *req) "Read request failed: error=%d path=%s [%s:%u]\n", ret, kobject_name(&sess->kobj), sess->hca_name, sess->hca_port); - if (sess->clt->mp_policy == MP_POLICY_MIN_INFLIGHT) + if (req->mp_policy == MP_POLICY_MIN_INFLIGHT) atomic_dec(&sess->stats->inflight); req->need_inv = false; if (req->sg_cnt) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.h b/drivers/infiniband/ulp/rtrs/rtrs-clt.h index c31f920e6d10..6d81aae53df4 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.h +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.h @@ -101,6 +101,7 @@ struct rtrs_clt_io_req { unsigned int usr_len; void *priv; bool in_use; + enum rtrs_mp_policy mp_policy; struct rtrs_clt_con *con; struct rtrs_sg_desc *desc; struct ib_sge *sge; From patchwork Fri Jul 30 13:18:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A4FDC4320A for ; Fri, 30 Jul 2021 13:18:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3B66860F5C for ; Fri, 30 Jul 2021 13:18:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238957AbhG3NSr (ORCPT ); Fri, 30 Jul 2021 09:18:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238960AbhG3NSr (ORCPT ); Fri, 30 Jul 2021 09:18:47 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52AF1C061765 for ; Fri, 30 Jul 2021 06:18:42 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id z4so11301741wrv.11 for ; Fri, 30 Jul 2021 06:18:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1I8y8GbTfIcLOGxJC+GhalbEZDN5CcEiTKmEbgcwKwY=; b=d8egKxtti9H3tqnaC55zcFNmddIpfKUsvlDqDAwntacFKaVLZjGa96DQAnOicwqEjk J3nJY90UPDZotvL2R6ICUtpHqsVvs+vWCfc0punQhVMOHDg6QPJcoK8Yrcf44z+FagCr p1BCJsJufg+kbBPq62pOuCWdcrkBHLnrzQN1fLsAGTv0W7SssCm+bUFAKXLPqGKJW8h/ zmSR7LEL3n9BvvYdf+STl/aoB8/pWqgy42RoUOCXixm8jARnC7F1c05xH4VmSsneycHe vjrz1sCn3jwEP7osAMDUxCMaboC4pv43LvnC84+plmDuoQOTHrcCI21otLFBUr9XfR5T 4htQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1I8y8GbTfIcLOGxJC+GhalbEZDN5CcEiTKmEbgcwKwY=; b=tJRgCXWYNGR+xBErYI6AJawS3iFLP1fTsVfeuaQKITLYg0YhsC7vShCYdM/0V+sK4D e6S3No7GiBG0jvdz32X6Ayjf09vwkE0uG3uaVgyQvd1Y//huz+DID6SfJNJ6F3hH9EpC BWfmJ3mYOPCZVq9sKCTWl+QNjgT97baDI9m8P+Pf7DCjOQf+bUyHJ2SG4VmyaFidiFNi 6MLFagpUK2txUFjrhZIWaEoSd2b8KLcWFtDupny+1Q6k1uq2xuIt6xkoBWTS4SyoH9jn iaFD0zANoY5UT5K2OoQNhhA9NBheSNwxj0fn3EnswgJAt7/A8gLzfZMdo3ZbR9AiLqYt 4x7g== X-Gm-Message-State: AOAM533/5M60iKxf39MQFRrw7Y9L95p8fEvKLGDBWsximj1Kb2bUZftK R7jgHZA5hL1hmntWPFTZ9hw8dUc9T2JorQ== X-Google-Smtp-Source: ABdhPJyBu2wHcHCVr0Za615l8Dc+tvB1/xcj41P8rUHIwdcZtPlzzArFCZA8UI0PcbFprLpF5/LR8g== X-Received: by 2002:adf:f64b:: with SMTP id x11mr3025520wrp.155.1627651120721; Fri, 30 Jul 2021 06:18:40 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:40 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Md Haris Iqbal , Gioh Kim Subject: [PATCH for-next 09/10] RDMA/rtrs: Add support to disable an IB port on the storage side Date: Fri, 30 Jul 2021 15:18:31 +0200 Message-Id: <20210730131832.118865-10-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Md Haris Iqbal This commit adds support to reject connection on a specific IB port which can be specified in the added sysfs entry for the rtrs-server module. Example, $ echo "mlx4_0 1" > /sys/class/rtrs-server/ctl/disable_port When a connection request is received on the above IB port, rtrs_srv rejects the connection and notifies the client to disable reconnection attempts. A manual reconnect has to be triggerred in such a case. A manual reconnect can be triggered by doing the following, echo 1 > /sys/class/rtrs-client/blya/paths//reconnect Signed-off-by: Md Haris Iqbal Reviewed-by: Gioh Kim Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 10 ++++ drivers/infiniband/ulp/rtrs/rtrs-srv.c | 82 +++++++++++++++++++++++++- 2 files changed, 90 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 5cce727abca0..21001818e607 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1898,6 +1898,7 @@ static int rtrs_rdma_conn_rejected(struct rtrs_clt_con *con, struct rdma_cm_event *ev) { struct rtrs_sess *s = con->c.sess; + struct rtrs_clt_sess *sess = to_clt_sess(s); const struct rtrs_msg_conn_rsp *msg; const char *rej_msg; int status, errno; @@ -1916,6 +1917,15 @@ static int rtrs_rdma_conn_rejected(struct rtrs_clt_con *con, rtrs_err(s, "Connect rejected: status %d (%s), rtrs errno %d\n", status, rej_msg, errno); + + if (errno == -ENETDOWN) { + /* + * Stop reconnection attempts + */ + sess->reconnect_attempts = -1; + rtrs_err(s, + "Disabling auto-reconnect. Trigger a manual reconnect after issue is resolved\n"); + } } else { rtrs_err(s, "Connect rejected but with malformed message: status %d (%s)\n", diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index cc65cffdc65a..90d833041ccf 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -32,7 +32,9 @@ MODULE_LICENSE("GPL"); static struct rtrs_rdma_dev_pd dev_pd; static mempool_t *chunk_pool; struct class *rtrs_dev_class; +static struct device *rtrs_dev; static struct rtrs_srv_ib_ctx ib_ctx; +static char disabled_port[NAME_MAX]; static int __read_mostly max_chunk_size = DEFAULT_MAX_CHUNK_SIZE; static int __read_mostly sess_queue_depth = DEFAULT_SESS_QUEUE_DEPTH; @@ -1826,6 +1828,20 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, u16 recon_cnt; int err = -ECONNRESET; + if (strnlen(disabled_port, NAME_MAX) > 0) { + char ib_device[NAME_MAX]; + + snprintf(ib_device, NAME_MAX, "%s %u", cm_id->device->name, cm_id->port_num); + if (strncmp(disabled_port, ib_device, NAME_MAX) == 0) { + /* + * Reject connection attempt on disabled port + */ + pr_err("Error: Connection request on a disabled port"); + err = -ENETDOWN; + goto reject_w_err; + } + } + if (len < sizeof(*msg)) { pr_err("Invalid RTRS connection request\n"); goto reject_w_err; @@ -2242,6 +2258,56 @@ static int check_module_params(void) return 0; } +static ssize_t disable_port_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *page) +{ + return sysfs_emit(page, "%s\n", disabled_port); +} + +static ssize_t disable_port_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + int ret, len; + + if (count > 1 && strnlen(disabled_port, NAME_MAX) > 0) { + pr_err("A disabled port already exists.\n"); + return -EINVAL; + } + + ret = strscpy(disabled_port, buf, NAME_MAX); + if (ret == -E2BIG) { + pr_err("String too big.\n"); + disabled_port[0] = '\0'; + return ret; + } + + len = strlen(disabled_port); + if (len > 0 && disabled_port[len-1] == '\n') + disabled_port[len-1] = '\0'; + + return ret; +} + +static struct kobj_attribute rtrs_srv_disable_port_device_attr = + __ATTR(disable_port, 0644, + disable_port_show, disable_port_store); + +static struct attribute *default_attrs[] = { + &rtrs_srv_disable_port_device_attr.attr, + NULL, +}; + +static struct attribute_group default_attr_group = { + .attrs = default_attrs, +}; + +static const struct attribute_group *default_attr_groups[] = { + &default_attr_group, + NULL, +}; + static int __init rtrs_server_init(void) { int err; @@ -2268,15 +2334,26 @@ static int __init rtrs_server_init(void) err = PTR_ERR(rtrs_dev_class); goto out_chunk_pool; } + + rtrs_dev = device_create_with_groups(rtrs_dev_class, NULL, + MKDEV(0, 0), NULL, + default_attr_groups, "ctl"); + if (IS_ERR(rtrs_dev)) { + err = PTR_ERR(rtrs_dev); + goto out_class; + } + rtrs_wq = alloc_workqueue("rtrs_server_wq", 0, 0); if (!rtrs_wq) { err = -ENOMEM; - goto out_dev_class; + goto out_dev; } return 0; -out_dev_class: +out_dev: + device_destroy(rtrs_dev_class, MKDEV(0, 0)); +out_class: class_destroy(rtrs_dev_class); out_chunk_pool: mempool_destroy(chunk_pool); @@ -2287,6 +2364,7 @@ static int __init rtrs_server_init(void) static void __exit rtrs_server_exit(void) { destroy_workqueue(rtrs_wq); + device_destroy(rtrs_dev_class, MKDEV(0, 0)); class_destroy(rtrs_dev_class); mempool_destroy(chunk_pool); rtrs_rdma_dev_pd_deinit(&dev_pd); From patchwork Fri Jul 30 13:18:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 12411249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A574C4338F for ; Fri, 30 Jul 2021 13:18:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 167C860EFD for ; Fri, 30 Jul 2021 13:18:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238974AbhG3NSs (ORCPT ); Fri, 30 Jul 2021 09:18:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238968AbhG3NSr (ORCPT ); Fri, 30 Jul 2021 09:18:47 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 210A7C06175F for ; Fri, 30 Jul 2021 06:18:43 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id r2so11342023wrl.1 for ; Fri, 30 Jul 2021 06:18:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FiDrtsKqJ+K23w0j/JTQkz/7+Ff/23dlUuakahMZLKM=; b=MalitphDqQA608tDuDJ9JDEwHR6O0GF2ey+qF+y51VV7q0WpSClD3/I1RTg2Xk7k9r J/ibM6JiTNybKdVUZA3VA6qBudCwFJsp3H1lqCo3njrKrW9j2+42T+qAgy9X0pB3r55K 97Fe8iwpWdPDPQxHB+KhrryH/8496yqgeGfWfTGKIAZn9RwiV8emSYAZ34bLIhTdTmed i80lOFG0wWxI4zF7p/KXVnptdUY6o6a+o9/C4vDoIfbcoeedMcNUACJUXXCNvxH+Hx8n BjRE4N5TK5j/mkk6quN3/VSSLHekQ3uJUK5bcthYCWAw+ajoI2S2KhJWnXxsYD6bPgkC J2uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FiDrtsKqJ+K23w0j/JTQkz/7+Ff/23dlUuakahMZLKM=; b=YeHVfAbuJi2qHQhTQU2D0GMW+mimLSE37G7TMJMH9XSLRdkZ81Jk9Vh3RrgFyLHrJk I0al/x+UfL4hC6IKV6A2h+abmtsjU97tDzDHxI2gzcnl0pv1O5h0vUtf1LRbDCcHivFS eNXIje1EGkkqMmbeyU070iYhMO2HcZi67zlCjTRqTEw+POL+I5TJpcXSjCL2SyGw8DWE XDTjBy5kGqDKXsSN2IUou3HuH6rqVSRIQpH252AkOCQx38tH2aN9qj8klxwEAnlQl01X hMxpct7kvKf9ciXA2RLqJICoAk8Ie5WPKlh8DrVpMz+HZrhzFsxrxvYdsEfLxjjxhk7N u77g== X-Gm-Message-State: AOAM533v8z3o3/IMoUQ9L8qyYPNoqDdK3wnIsA9ouX/3A71jxFgOzM52 9+rz/t/uqsiLUX8oOiEQ/2tvogC6467+lw== X-Google-Smtp-Source: ABdhPJzasaHcMHSCGNMj07op6jexJ8E5LEIsOwSpTOpYXD0Ke4+e2U0I8nS5F8pGAVkwrOiFyydW0g== X-Received: by 2002:a05:6000:1818:: with SMTP id m24mr3040878wrh.49.1627651121678; Fri, 30 Jul 2021 06:18:41 -0700 (PDT) Received: from jwang-Latitude-5491.fritz.box ([2001:16b8:496a:8500:4512:4a6e:16f3:2377]) by smtp.gmail.com with ESMTPSA id z5sm1626012wmp.26.2021.07.30.06.18.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Jul 2021 06:18:41 -0700 (PDT) From: Jack Wang To: linux-rdma@vger.kernel.org Cc: bvanassche@acm.org, leon@kernel.org, dledford@redhat.com, jgg@ziepe.ca, haris.iqbal@ionos.com, jinpu.wang@ionos.com, Gioh Kim Subject: [PATCH for-next 10/10] RDMA/rtrs: remove (void) casting for functions Date: Fri, 30 Jul 2021 15:18:32 +0200 Message-Id: <20210730131832.118865-11-jinpu.wang@ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210730131832.118865-1-jinpu.wang@ionos.com> References: <20210730131832.118865-1-jinpu.wang@ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Gioh Kim As recommended by Leon https://www.spinics.net/lists/linux-rdma/msg102200.html this patch removes (void) casting because that does nothing. Signed-off-by: Gioh Kim Reviewed-by: Md Haris Iqbal Signed-off-by: Jack Wang Signed-off-by: Gioh Kim Reviewed-by: Md Haris Iqbal Signed-off-by: Jack Wang --- drivers/infiniband/ulp/rtrs/rtrs-srv.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/rtrs/rtrs-srv.c b/drivers/infiniband/ulp/rtrs/rtrs-srv.c index 90d833041ccf..7d6df83eb636 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-srv.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-srv.c @@ -1918,7 +1918,7 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, err = create_con(sess, cm_id, cid); if (err) { rtrs_err((&sess->s), "create_con(), error %d\n", err); - (void)rtrs_rdma_do_reject(cm_id, err); + rtrs_rdma_do_reject(cm_id, err); /* * Since session has other connections we follow normal way * through workqueue, but still return an error to tell cma.c @@ -1929,7 +1929,7 @@ static int rtrs_rdma_connect(struct rdma_cm_id *cm_id, err = rtrs_rdma_do_accept(sess, cm_id); if (err) { rtrs_err((&sess->s), "rtrs_rdma_do_accept(), error %d\n", err); - (void)rtrs_rdma_do_reject(cm_id, err); + rtrs_rdma_do_reject(cm_id, err); /* * Since current connection was successfully added to the * session we follow normal way through workqueue to close the