From patchwork Wed Dec 9 08:20:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B603CC433FE for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5EA8D23BAC for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726278AbgLIIVg (ORCPT ); Wed, 9 Dec 2020 03:21:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726231AbgLIIVg (ORCPT ); Wed, 9 Dec 2020 03:21:36 -0500 Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6976C0613D6 for ; Wed, 9 Dec 2020 00:20:55 -0800 (PST) Received: by mail-ej1-x643.google.com with SMTP id qw4so809820ejb.12 for ; Wed, 09 Dec 2020 00:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i0es42Aa/rvDu/nGYWkHzt0D8sik7Ir6dCcdmicZQ30=; b=VVVQ1FiRl/gjQscUtCkjwwp27lRRUtrYr5i3uN+hpS9/aCg4Em3eKFQ8tCJUbzVnUC z9Pp6nsrG3NR08oHgpXSBAcOqjheLlvzzZpb/JQZEtW5zvokYM4RVMdT0i5lpJ36LCuy iNW0/mbIW9Gpkt4mNsnjzJQpFXY735jgoRq6IT7AW6i7GEernVwzV7iOjWdlMgNazAYQ q5pd08q8UVJw0yduADu2qTT2WCtpoevvXO2bTUDBJr/vAeAr2Lu7lDnBuiTGb3z4q2a3 2dsZRbwoIyanQDfRnfy50gr2iny0u/GgMOWDDsZHfklz1qmZwUjfaMHX+JrNrax9euB2 dg0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i0es42Aa/rvDu/nGYWkHzt0D8sik7Ir6dCcdmicZQ30=; b=aZCBAwYOo57+zZ92KW1NrU3gO4mZ+LCM/4TtEDMG1/N0XZxan3A/AnjFrbKpYia26R gDGubm8TLjirHA2K1bkZ1NuZnH2ffRd2hhFrzwkZKLfiNxC/056zuwkODi3lmYj64IJ/ pjpL5DGreLlmlJknK0+uToeYhTtn3SdmPy15TA8f93uAz5p3sVSeBGgj/meYN9l0v7ru mEd/GhLfds/a2D7LVKr+XrP7wjcp/9TY+zCKdinn3npaDRK7ZNDECmj9sflCBKs/53Fv rxnDlzeBZIGJswfwvKa0bc295SkmBgmpFYqfDbLAkfcYnsr8mg1q4HpdNpywdwe/Z7Ex Ss7w== X-Gm-Message-State: AOAM533xodSwjetUII/6gxVDWOkMUM3cIB1ivMzdfWnUjM/bbgAEMPj0 qvflATKAD70nUr4uXlcS5LzBlMOy70ujHA== X-Google-Smtp-Source: ABdhPJx1Etru+OzCMNCiuB6rWpCjveRpilZxhtD6r7Ud2IFGI442XjoObaRGYGDR541HmNkblL8uuA== X-Received: by 2002:a17:906:76cd:: with SMTP id q13mr610934ejn.67.1607502054182; Wed, 09 Dec 2020 00:20:54 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:53 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Md Haris Iqbal , kernel test robot Subject: [PATCH for-next 1/7] block/rnbd-clt: Get rid of warning regarding size argument in strlcpy Date: Wed, 9 Dec 2020 09:20:45 +0100 Message-Id: <20201209082051.12306-2-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Md Haris Iqbal The kernel test robot triggerred the following warning, >> drivers/block/rnbd/rnbd-clt.c:1397:42: warning: size argument in 'strlcpy' call appears to be size of the source; expected the size of the destination [-Wstrlcpy-strlcat-size] strlcpy(dev->pathname, pathname, strlen(pathname) + 1); ~~~~~~~^~~~~~~~~~~~~ To get rid of the above warning, use a len variable for doing kzalloc and then strlcpy. Fixes: 64e8a6ece1a5 ("block/rnbd-clt: Dynamically alloc buffer for pathname & blk_symlink_name") Reported-by: kernel test robot Signed-off-by: Md Haris Iqbal Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-clt.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index a199b190c73d..62b77b5dc061 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -1365,7 +1365,7 @@ static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess, const char *pathname) { struct rnbd_clt_dev *dev; - int ret; + int len, ret; dev = kzalloc_node(sizeof(*dev), GFP_KERNEL, NUMA_NO_NODE); if (!dev) @@ -1388,12 +1388,13 @@ static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess, goto out_queues; } - dev->pathname = kzalloc(strlen(pathname) + 1, GFP_KERNEL); + len = strlen(pathname) + 1; + dev->pathname = kzalloc(len, GFP_KERNEL); if (!dev->pathname) { ret = -ENOMEM; goto out_queues; } - strlcpy(dev->pathname, pathname, strlen(pathname) + 1); + strlcpy(dev->pathname, pathname, len); dev->clt_device_id = ret; dev->sess = sess; From patchwork Wed Dec 9 08:20:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3073C4167B for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CD1023BAF for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726281AbgLIIVh (ORCPT ); Wed, 9 Dec 2020 03:21:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726231AbgLIIVg (ORCPT ); Wed, 9 Dec 2020 03:21:36 -0500 Received: from mail-ej1-x642.google.com (mail-ej1-x642.google.com [IPv6:2a00:1450:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74E0EC061793 for ; Wed, 9 Dec 2020 00:20:56 -0800 (PST) Received: by mail-ej1-x642.google.com with SMTP id x16so831537ejj.7 for ; Wed, 09 Dec 2020 00:20:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fwUB98OMU5vN+dBpGv0EWnnHA2hhEx1MlU32S5W8EnI=; b=Y7iAr8W9bTfDF+GBMnJgiqr+SqswugJViBgz+7cr1bw2XVu4V16CQAENB3Q7wBbAz7 NFYLuzpQmGxVgVPTpLW7sCypvlj+y1ybi1YYOkcbdsGzI0dCurYUI9QBhAaiInm+B1WZ jPS9SKy+lxtp8nICT8rWvP7LgaONnrpP1SZvApUFW/Fv7n6YN0vRTw4IYuKZJDxoBe1J l268QuEXKUzw6PgBfhoHkzyiPLN4mT8zQ/3HBGmyJ9NW9iUXO9v2Cfmc19PC0S+SpHyw ZO/DOak7P3gR9PKJhF1dvoisLRhasPltTfgFUL2uXh5l5w0/tgFNq9r/oJ3f4twwF3tZ VuxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fwUB98OMU5vN+dBpGv0EWnnHA2hhEx1MlU32S5W8EnI=; b=FAOEOBdADRVwvFObgnum+eyaOjiQu7FTQ5T327SfJgtMuGmUUF9E5zRQ8RZQKEcsjQ FYaVMiBDMgUxfBvp3L9r3Ui5QVI97lhyuQm7IYF/PYxhZ4ppFurCKAfEvYPnxTrl7Iwt 55ZWOcuOj1LUeROcLCGmygSDLMTm4ni2taWmzY2UZcXGRkVS0SLsDi6j+q6JC9vCQa4Y mOAXdjAPm6eleEbImv91Wqh778jfgXMQYBKNACJbTnkIdlD+lK9aEoU2wMQKoQemgmVZ JQkUdMZHqMKvU5d4XfGZWKXhRevrp7VqacCkeKxE8TmoRQR/SbGmesJ7/1a/H0jVtts+ RNmQ== X-Gm-Message-State: AOAM531fKxN3QP3wwe/TrAeoMEbfdgpghnyy/wHVi8Iqb05IDd30zB5J QNZN+GPFENXei17MYz3z8up0Q9Za7qOqaA== X-Google-Smtp-Source: ABdhPJxmMFgDEnqtKjdXG4Wk75pBmlC+sPXVEOEbJkKy4oxIAiMat73fsjasSujS/E764NNjdbXOYQ== X-Received: by 2002:a17:906:60c8:: with SMTP id f8mr1085979ejk.14.1607502055098; Wed, 09 Dec 2020 00:20:55 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:54 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Md Haris Iqbal Subject: [PATCH for-next 2/7] block/rnbd-clt: Fix possible memleak Date: Wed, 9 Dec 2020 09:20:46 +0100 Message-Id: <20201209082051.12306-3-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org In error case, we do not free the memory for blk_symlink_name. Do it by free the memory in error case, and set to NULL afterwards. Also fix the condition in rnbd_clt_remove_dev_symlink. Fixes: 64e8a6ece1a5 ("block/rnbd-clt: Dynamically alloc buffer for pathname & blk_symlink_name") Signed-off-by: Jack Wang Reviewed-by: Md Haris Iqbal --- drivers/block/rnbd/rnbd-clt-sysfs.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt-sysfs.c b/drivers/block/rnbd/rnbd-clt-sysfs.c index a7caeedeb198..d4aa6bfc9555 100644 --- a/drivers/block/rnbd/rnbd-clt-sysfs.c +++ b/drivers/block/rnbd/rnbd-clt-sysfs.c @@ -432,7 +432,7 @@ void rnbd_clt_remove_dev_symlink(struct rnbd_clt_dev *dev) * i.e. rnbd_clt_unmap_dev_store() leading to a sysfs warning because * of sysfs link already was removed already. */ - if (strlen(dev->blk_symlink_name) && try_module_get(THIS_MODULE)) { + if (dev->blk_symlink_name && try_module_get(THIS_MODULE)) { sysfs_remove_link(rnbd_devs_kobj, dev->blk_symlink_name); kfree(dev->blk_symlink_name); module_put(THIS_MODULE); @@ -521,7 +521,8 @@ static int rnbd_clt_add_dev_symlink(struct rnbd_clt_dev *dev) return 0; out_err: - dev->blk_symlink_name[0] = '\0'; + kfree(dev->blk_symlink_name); + dev->blk_symlink_name = NULL ; return ret; } From patchwork Wed Dec 9 08:20:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08B4AC19425 for ; Wed, 9 Dec 2020 08:21:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B3F7B23B99 for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726297AbgLIIVi (ORCPT ); Wed, 9 Dec 2020 03:21:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726231AbgLIIVh (ORCPT ); Wed, 9 Dec 2020 03:21:37 -0500 Received: from mail-ed1-x543.google.com (mail-ed1-x543.google.com [IPv6:2a00:1450:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 735EEC061794 for ; Wed, 9 Dec 2020 00:20:57 -0800 (PST) Received: by mail-ed1-x543.google.com with SMTP id c7so585299edv.6 for ; Wed, 09 Dec 2020 00:20:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I1nkDS+t71lcZcyKCx1EdDkNSPvYcn8wLT9ZWpiLudM=; b=LRdITsDERTgvDHMgQpQKgIs34CL9uWsDYndYAFxesZIQtK9dEoQPV0OjxiFl/fGMHF F24OpbAq6FMA9ypVJxkl5PVT4eMKbBd2/cCefII1kV+WBZWw5EhgVSO5VV8Mx8pKWaI/ He6dCS0YRfHXioBYgXnngJGqdumW/KTZFnhwE5HEVkWjBvH0rOn2hLwWoaeRsVHU+n5u /fyxSo+B2HpfhdvTnPcH59brgyj4ZxEee3zotzKERhAZclxybh7t4jpJCTgKSAYhHgF5 1H2EJYcLj8whNpzwcUoDpudlK4ZctgRjniGd+q3dvRNiJ7ROT6JxZcKTjo7G4CO0j7TM 9f6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I1nkDS+t71lcZcyKCx1EdDkNSPvYcn8wLT9ZWpiLudM=; b=DgG34e5QmOjEva3ZXoN2srPhGpbJfrpNfZyv3DYWPJfCl6dc0pYpCkEPchGW0xcJsX tv9GRgIRDRhtF3sA2AUrG24FvM1THxPkJ1/rz7/ZR9/ZKpd8ziuoAJwieWPHUdKyVX/F bbhjSfGMR05ZRln88RDIiofWEYglwrzFFH+K1BC+EKytFFN6tlpaVkq+HggY9e5SEUD4 gWlnCoTT3ieqCSvwis5fdpcyeZtLnk2BOzA2K9e/6k9R02ovYHG14cBQwGGaKr/MduQ+ kOjMHPSW+tHsuiDbqAjUBGIVS5jsvCr4MB+bYF4McCd1epwDmgOCHLHHgMeuFwiPXFGx S53A== X-Gm-Message-State: AOAM532ahkc60y1cicg4II1703Ns7lbXAo0HFCocyhiyHF8ZmmL/axX0 tnvrUJUeXcGCQhSGHB9Fp4R8T6K2wwX2kw== X-Google-Smtp-Source: ABdhPJxtGAVLmA2bo9FrsyfjmZb3lmCvDri6XFBZOXp3X8XOqgji9Gi3rMUJXl9FJjPwQUQxvbuUdQ== X-Received: by 2002:a50:ed17:: with SMTP id j23mr963804eds.218.1607502056062; Wed, 09 Dec 2020 00:20:56 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:55 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Md Haris Iqbal , Guoqing Jiang Subject: [PATCH for-next 3/7] block/rnbd-srv: Protect dev session sysfs removal Date: Wed, 9 Dec 2020 09:20:47 +0100 Message-Id: <20201209082051.12306-4-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Md Haris Iqbal Since the removal of the session sysfs can also be called from the function destroy_sess, there is a need to protect the call from the function rnbd_srv_sess_dev_force_close Fixes: 786998050cbc ("block/rnbd-srv: close a mapped device from server side.") Signed-off-by: Md Haris Iqbal Reviewed-by: Guoqing Jiang Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-srv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/block/rnbd/rnbd-srv.c b/drivers/block/rnbd/rnbd-srv.c index d1ee72ed8384..066411cce5e2 100644 --- a/drivers/block/rnbd/rnbd-srv.c +++ b/drivers/block/rnbd/rnbd-srv.c @@ -338,9 +338,10 @@ static int rnbd_srv_link_ev(struct rtrs_srv *rtrs, void rnbd_srv_sess_dev_force_close(struct rnbd_srv_sess_dev *sess_dev) { + mutex_lock(&sess_dev->sess->lock); rnbd_srv_destroy_dev_session_sysfs(sess_dev); + mutex_unlock(&sess_dev->sess->lock); sess_dev->keep_id = true; - } static int process_msg_close(struct rtrs_srv *rtrs, From patchwork Wed Dec 9 08:20:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DAFC2BB40 for ; Wed, 9 Dec 2020 08:21:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D951B23B87 for ; Wed, 9 Dec 2020 08:21:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726307AbgLIIVi (ORCPT ); Wed, 9 Dec 2020 03:21:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36044 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726231AbgLIIVi (ORCPT ); Wed, 9 Dec 2020 03:21:38 -0500 Received: from mail-ej1-x643.google.com (mail-ej1-x643.google.com [IPv6:2a00:1450:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46223C06179C for ; Wed, 9 Dec 2020 00:20:58 -0800 (PST) Received: by mail-ej1-x643.google.com with SMTP id bo9so804232ejb.13 for ; Wed, 09 Dec 2020 00:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J9910OhrigqXRatu8UjPkLXZju51Dv3N9YERCZWMqnE=; b=bZhZoWri7UFBH3VYaLU8Kb/qN0oyDS3EEriWhlC/pDgQ71zFXm8YYQ7qKxk5uNfUAe TsWQmmLs/Mngp49fWEPYnMoZffJEHQCReXpHabbbDLgqG01pXAOr6qLCx+ym/9BeuPap 5hSY+F70pDWuRHJP9G/bQZ/NjndqnNvCHGWCZ8OIrrbv2mu/YsLDVOf+PbfOTv1qa7py lkRRo6yosSLisIZ0HqZJTCfcZuShVbbyOV6eXRUjdPlyPnqhVF6rC/eXsWCyUYRCvStZ xVVXQX8ML6w3RzEboREy2ViUkrj+1IMsj6ShF8Y53FhbBg6YyUioapg60EGUdQN4yilA 6SEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J9910OhrigqXRatu8UjPkLXZju51Dv3N9YERCZWMqnE=; b=HecbuHmGjilKLBfN6LZuBMRjsVqYoo9osY4gqvSmP/18BaK2ZILWi5ZVHdhDSRCcCj IIdkIjAo5F+HYQQthxu1n6BsHIkkvfpQba9VHp3Oh9qHSYtYX3fceFcxfQhrVPatQ+j1 lf6O9mQ1n6GXN819wubNi80baaeaEAPlDdJvmRMfy43ZGeWRJBjpQbzweKYAF9dSU7tW fv6mUiusWeR+pMQYixRX6WRXhLy8TMTgpAfhJ95tWtpRIUqHOqWx1aevQ9wOBkIS+w/G kMsKs8CN1Ge0pE5ZgB64P8/sTxo+hiPmUtKO3f3moRmO9fHuUoCt7h1+02Qd4oS27l5r c+nw== X-Gm-Message-State: AOAM532v9+hDvBhZDISJ2Wu5QZn2Qheg8hA1otcPLnlAmmMVCEN6Na3s 39BI8fvXXMRsqsTPWR4L+aiD169ffOmJcQ== X-Google-Smtp-Source: ABdhPJwzR6mVIjY8Bdi9/o2qMI+WjFmxUlF7szby/BoTER2Jn6fpka099e5PVD94hbRnxM8K6drhuw== X-Received: by 2002:a17:906:ec9:: with SMTP id u9mr1096878eji.400.1607502056889; Wed, 09 Dec 2020 00:20:56 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:56 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com Subject: [PATCH for-next 4/7] block/rnbd: Fix typos Date: Wed, 9 Dec 2020 09:20:48 +0100 Message-Id: <20201209082051.12306-5-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-clt.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index 62b77b5dc061..9641afa17095 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -359,7 +359,7 @@ static struct rnbd_iu *rnbd_get_iu(struct rnbd_clt_session *sess, * 2nd reference is dropped after confirmation with the response is * returned. * 1st and 2nd can happen in any order, so the rnbd_iu should be - * released (rtrs_permit returned to ibbtrs) only leased after both + * released (rtrs_permit returned to rtrs) only after both * are finished. */ atomic_set(&iu->refcount, 2); @@ -803,7 +803,7 @@ static struct rnbd_clt_session *alloc_sess(const char *sessname) rnbd_init_cpu_qlists(sess->cpu_queues); /* - * That is simple percpu variable which stores cpu indeces, which are + * That is simple percpu variable which stores cpu indices, which are * incremented on each access. We need that for the sake of fairness * to wake up queues in a round-robin manner. */ @@ -1668,7 +1668,7 @@ static void rnbd_destroy_sessions(void) /* * Here at this point there is no any concurrent access to sessions * list and devices list: - * 1. New session or device can'be be created - session sysfs files + * 1. New session or device can't be created - session sysfs files * are removed. * 2. Device or session can't be removed - module reference is taken * into account in unmap device sysfs callback. From patchwork Wed Dec 9 08:20:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FE04C433FE for ; Wed, 9 Dec 2020 08:22:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1421923B97 for ; Wed, 9 Dec 2020 08:22:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725997AbgLIIWP (ORCPT ); Wed, 9 Dec 2020 03:22:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725816AbgLIIWP (ORCPT ); Wed, 9 Dec 2020 03:22:15 -0500 Received: from mail-ej1-x644.google.com (mail-ej1-x644.google.com [IPv6:2a00:1450:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 320C5C0617A6 for ; Wed, 9 Dec 2020 00:20:59 -0800 (PST) Received: by mail-ej1-x644.google.com with SMTP id d17so825060ejy.9 for ; Wed, 09 Dec 2020 00:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CA2w/lh8ZkqeTZgM/LEMkBOMEtxjc9/IiyW2kEytwok=; b=IzIoKlPKjezOrSWhINuQARrCH4wL2jNfYKv1y2X9NCvQI+KEIhXKObFz+0IqsXCzda LIhldRUEKuQ/F8ENOzmchr2/qqKLMtZG6Vf0ZcewZtUqC47R7ppPabcuw87NUWMGak9Y KNHgWVI+nx4ikZihv+QFD1WHGK3MD4i4w7q4BH31d4NZs6ehFKyq4LzBxAHP1l+OFPK0 9rcZhsf2jJs9ro8HOta/7vaH2beq5Eo3EyAmx0kNp8FMY4bdOVuFKpu+eKZex1kYvOjt H1SItUo4vvN3ICHYLEgX89wbiCDBCwlmGZC3tV1hHlSQ5X93mewLM4DV0F3BORVDRJO6 iqhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CA2w/lh8ZkqeTZgM/LEMkBOMEtxjc9/IiyW2kEytwok=; b=Y6UOlVVHzb7r72cVJS17xY/DvwU6Uk65b7g0bGT/9pChLf++R0+JTtyi8zIB15Kq88 p5wZO/yQWxfE//ies44izGeyF4RZsWOOXyueS/u3njPeOVflUn3D/Dsl7OIT79+ZLn8A BvfHgOQrp3E8/OTgCiPN7t5SjjzocCXg44dxsuCzkHVv+mq2DybCg+BxkXNI29URrOMs siKFdQGl2OZiZth4OFL7hITS0AS54stXF6ogE2ZNDksSk9CCNfocpTxgyr9RZofDICOA mWCTGV96qGaLUS14yt+EziYIchXvOFPWxt/+k2/LveeSs6NubZ6nJ+y4mMgodaesP2ZW FkjQ== X-Gm-Message-State: AOAM531tsbiWxdDfI156UmqFlqZ7nR3sseVONdINnipEilxFNr2vPpXg RYq+UAfKYbIg9YWA5ybLvWsxDCV3Dd0MfQ== X-Google-Smtp-Source: ABdhPJzztrAF4g2a5Zo2pZGeaqNTXNMozVoMrUZ6ZPGtYXA3vGmfKyhGf/u91ZnXoSWnUSHu4rk8rw== X-Received: by 2002:a17:906:2857:: with SMTP id s23mr1082751ejc.218.1607502057771; Wed, 09 Dec 2020 00:20:57 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:57 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Gioh Kim Subject: [PATCH for-next 5/7] block/rnbd: Set write-back cache and fua same to the target device Date: Wed, 9 Dec 2020 09:20:49 +0100 Message-Id: <20201209082051.12306-6-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Gioh Kim The rnbd-client always sets the write-back cache and fua attributes of the rnbd device queue regardless of the target device on the server. That generates IO hang issue when the target device does not support both of write-back cacne and fua. This patch adds more fields for the cache policy and fua into the device opening message. The rnbd-server sends the information if the target device supports the write-back cache and fua and rnbd-client recevives it and set the device queue accordingly. Signed-off-by: Gioh Kim [jwang: some minor change, rename a few varables, remove unrelated comments.] Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-clt.c | 8 +++++--- drivers/block/rnbd/rnbd-clt.h | 2 ++ drivers/block/rnbd/rnbd-proto.h | 9 ++++++++- drivers/block/rnbd/rnbd-srv.c | 9 +++++++-- 4 files changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index 9641afa17095..fe8c738a1797 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -88,6 +88,8 @@ static int rnbd_clt_set_dev_attr(struct rnbd_clt_dev *dev, dev->discard_alignment = le32_to_cpu(rsp->discard_alignment); dev->secure_discard = le16_to_cpu(rsp->secure_discard); dev->rotational = rsp->rotational; + dev->wc = !!(rsp->cache_policy & RNBD_WRITEBACK); + dev->fua = !!(rsp->cache_policy & RNBD_FUA); dev->max_hw_sectors = sess->max_io_size / SECTOR_SIZE; dev->max_segments = BMAX_SEGMENTS; @@ -1305,7 +1307,7 @@ static void setup_request_queue(struct rnbd_clt_dev *dev) blk_queue_max_segments(dev->queue, dev->max_segments); blk_queue_io_opt(dev->queue, dev->sess->max_io_size); blk_queue_virt_boundary(dev->queue, SZ_4K - 1); - blk_queue_write_cache(dev->queue, true, true); + blk_queue_write_cache(dev->queue, dev->wc, dev->fua); dev->queue->queuedata = dev; } @@ -1530,13 +1532,13 @@ struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname, } rnbd_clt_info(dev, - "map_device: Device mapped as %s (nsectors: %zu, logical_block_size: %d, physical_block_size: %d, max_write_same_sectors: %d, max_discard_sectors: %d, discard_granularity: %d, discard_alignment: %d, secure_discard: %d, max_segments: %d, max_hw_sectors: %d, rotational: %d)\n", + "map_device: Device mapped as %s (nsectors: %zu, logical_block_size: %d, physical_block_size: %d, max_write_same_sectors: %d, max_discard_sectors: %d, discard_granularity: %d, discard_alignment: %d, secure_discard: %d, max_segments: %d, max_hw_sectors: %d, rotational: %d, wc: %d, fua: %d)\n", dev->gd->disk_name, dev->nsectors, dev->logical_block_size, dev->physical_block_size, dev->max_write_same_sectors, dev->max_discard_sectors, dev->discard_granularity, dev->discard_alignment, dev->secure_discard, dev->max_segments, - dev->max_hw_sectors, dev->rotational); + dev->max_hw_sectors, dev->rotational, dev->wc, dev->fua); mutex_unlock(&dev->lock); diff --git a/drivers/block/rnbd/rnbd-clt.h b/drivers/block/rnbd/rnbd-clt.h index b193d5904050..efd67ae286ca 100644 --- a/drivers/block/rnbd/rnbd-clt.h +++ b/drivers/block/rnbd/rnbd-clt.h @@ -112,6 +112,8 @@ struct rnbd_clt_dev { enum rnbd_access_mode access_mode; bool read_only; bool rotational; + bool wc; + bool fua; u32 max_hw_sectors; u32 max_write_same_sectors; u32 max_discard_sectors; diff --git a/drivers/block/rnbd/rnbd-proto.h b/drivers/block/rnbd/rnbd-proto.h index ca166241452c..c1bc5c0fef71 100644 --- a/drivers/block/rnbd/rnbd-proto.h +++ b/drivers/block/rnbd/rnbd-proto.h @@ -108,6 +108,11 @@ struct rnbd_msg_close { __le32 device_id; }; +enum rnbd_cache_policy { + RNBD_FUA = 1 << 0, + RNBD_WRITEBACK = 1 << 1, +}; + /** * struct rnbd_msg_open_rsp - response message to RNBD_MSG_OPEN * @hdr: message header @@ -124,6 +129,7 @@ struct rnbd_msg_close { * @max_segments: max segments hardware support in one transfer * @secure_discard: supports secure discard * @rotation: is a rotational disc? + * @cache_policy: support write-back caching or FUA? */ struct rnbd_msg_open_rsp { struct rnbd_msg_hdr hdr; @@ -139,7 +145,8 @@ struct rnbd_msg_open_rsp { __le16 max_segments; __le16 secure_discard; u8 rotational; - u8 reserved[11]; + u8 cache_policy; + u8 reserved[10]; }; /** diff --git a/drivers/block/rnbd/rnbd-srv.c b/drivers/block/rnbd/rnbd-srv.c index 066411cce5e2..b8e44331e494 100644 --- a/drivers/block/rnbd/rnbd-srv.c +++ b/drivers/block/rnbd/rnbd-srv.c @@ -550,6 +550,7 @@ static void rnbd_srv_fill_msg_open_rsp(struct rnbd_msg_open_rsp *rsp, struct rnbd_srv_sess_dev *sess_dev) { struct rnbd_dev *rnbd_dev = sess_dev->rnbd_dev; + struct request_queue *q = bdev_get_queue(rnbd_dev->bdev); rsp->hdr.type = cpu_to_le16(RNBD_MSG_OPEN_RSP); rsp->device_id = @@ -574,8 +575,12 @@ static void rnbd_srv_fill_msg_open_rsp(struct rnbd_msg_open_rsp *rsp, cpu_to_le32(rnbd_dev_get_discard_alignment(rnbd_dev)); rsp->secure_discard = cpu_to_le16(rnbd_dev_get_secure_discard(rnbd_dev)); - rsp->rotational = - !blk_queue_nonrot(bdev_get_queue(rnbd_dev->bdev)); + rsp->rotational = !blk_queue_nonrot(q); + rsp->cache_policy = 0; + if (test_bit(QUEUE_FLAG_WC, &q->queue_flags)) + rsp->cache_policy |= RNBD_WRITEBACK; + if (blk_queue_fua(q)) + rsp->cache_policy |= RNBD_FUA; } static struct rnbd_srv_sess_dev * From patchwork Wed Dec 9 08:20:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C09D9C4361B for ; Wed, 9 Dec 2020 08:22:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 73D7723B87 for ; Wed, 9 Dec 2020 08:22:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725816AbgLIIWP (ORCPT ); Wed, 9 Dec 2020 03:22:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725936AbgLIIWP (ORCPT ); Wed, 9 Dec 2020 03:22:15 -0500 Received: from mail-ej1-x642.google.com (mail-ej1-x642.google.com [IPv6:2a00:1450:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47C46C0617A7 for ; Wed, 9 Dec 2020 00:21:00 -0800 (PST) Received: by mail-ej1-x642.google.com with SMTP id d17so825132ejy.9 for ; Wed, 09 Dec 2020 00:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gFGSiFO4+XIY4m2BFSEVedlYeaCeUpAVPL4LKBEVyCs=; b=AZybXq4lzpiTpD/2OxziM75tArl4j2SqHiNG1J/gksGJ1HM68jE39W0FRzGp3uixBb Tm1KNIDfYrDv1VBlKmC4b5Mq7gQA3PlY5aNdddWyWD2vxrjBvQHgPgsbzy6DzIOgKVPz 5jps4N/d+3SS4wU/ToqZ7PNMqCa1+qGzhr2yA4vEmTFIV4IRF/1nmy3J4I0b0S/o7V4p A5c54weqGWlBanU1kFA7Aio1NeHb6bpl4eqhTJ7ZpY5PQvwQKKzWOxoIJcLPr6etBTrt +FXkzKVPcqdJVTgs2e0sirEzUwHadlkFwbJKiJB4COQhRZtFGN/BpghsyUSKc6UZ91tm vLbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gFGSiFO4+XIY4m2BFSEVedlYeaCeUpAVPL4LKBEVyCs=; b=c/LmusUPkJTixOS7QOBa/3bQ1B1I9P7FXavQs4ALdT0etEhzOmPqES5FNTPCm8+/3Q WtpUOVU20HoX7ItQ2no8ycEKJQQqALkwK+hjzdWdUY+M2Xlfuh2oCUZZtpvZycsqhe7e in+gxx2sIr4OKHOxdAY96QRakdpeOCGVmK9Ih2j8N2D063KBhhGeg9CPXx8nlTZeTQbj OEb1xNmpEZA9SpK4tsTAgK0aOPqwAFWbnMRmc61UXLspmv1qEHdeSjUuOwRSV2UEt46E O/bgzXjvcW7l4lf5q84xaOSDQJAObkiTFriYV4fEqKRK9iypCjn2kkN+X3JWiPPzDHyD w5Og== X-Gm-Message-State: AOAM530vOd89xew77R2IFTxewH3P8M2ydC9MxxYLTkUG6IfTf/gVH2dB 29T9I4NqFq/KjrVftG9eecNT7C1J+QR/8g== X-Google-Smtp-Source: ABdhPJx7sFKlEZWEjgZ1aX6LJaLfXMXiy7P4iT2FhP4Np2NQ258kHEjml0VcTnMuMElBs8BPXVsLeQ== X-Received: by 2002:a17:906:fa88:: with SMTP id lt8mr1106751ejb.408.1607502058798; Wed, 09 Dec 2020 00:20:58 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:58 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Gioh Kim Subject: [PATCH for-next 6/7] block/rnbd-clt: Dynamically allocate sglist for rnbd_iu Date: Wed, 9 Dec 2020 09:20:50 +0100 Message-Id: <20201209082051.12306-7-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Gioh Kim The BMAX_SEGMENT static array for scatterlist is embedded in rnbd_iu structure to avoid memory allocation in hot IO path. In many cases, we do need only several sg entries because many IOs have only several segments. This patch change rnbd_iu to check the number of segments in the request and allocate sglist dynamically. For io path, use sg_alloc_table_chained to allocate sg list faster. First it makes two sg entries after pdu of request. The sg_alloc_table_chained uses the pre-allocated sg entries if the number of segments of the request is less than two. So it reduces the number of memory allocation. Signed-off-by: Gioh Kim Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-clt.c | 58 +++++++++++++++++++---------------- drivers/block/rnbd/rnbd-clt.h | 10 +++++- 2 files changed, 41 insertions(+), 27 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index fe8c738a1797..c37d94181ff4 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -384,6 +384,7 @@ static void rnbd_softirq_done_fn(struct request *rq) struct rnbd_iu *iu; iu = blk_mq_rq_to_pdu(rq); + sg_free_table_chained(&iu->sgt, RNBD_INLINE_SG_CNT); rnbd_put_permit(sess, iu->permit); blk_mq_end_request(rq, errno_to_blk_status(iu->errno)); } @@ -477,7 +478,7 @@ static int send_msg_close(struct rnbd_clt_dev *dev, u32 device_id, bool wait) iu->buf = NULL; iu->dev = dev; - sg_mark_end(&iu->sglist[0]); + sg_alloc_table(&iu->sgt, 1, GFP_KERNEL); msg.hdr.type = cpu_to_le16(RNBD_MSG_CLOSE); msg.device_id = cpu_to_le32(device_id); @@ -492,6 +493,7 @@ static int send_msg_close(struct rnbd_clt_dev *dev, u32 device_id, bool wait) err = errno; } + sg_free_table(&iu->sgt); rnbd_put_iu(sess, iu); return err; } @@ -564,7 +566,8 @@ static int send_msg_open(struct rnbd_clt_dev *dev, bool wait) iu->buf = rsp; iu->dev = dev; - sg_init_one(iu->sglist, rsp, sizeof(*rsp)); + sg_alloc_table(&iu->sgt, 1, GFP_KERNEL); + sg_init_one(iu->sgt.sgl, rsp, sizeof(*rsp)); msg.hdr.type = cpu_to_le16(RNBD_MSG_OPEN); msg.access_mode = dev->access_mode; @@ -572,7 +575,7 @@ static int send_msg_open(struct rnbd_clt_dev *dev, bool wait) WARN_ON(!rnbd_clt_get_dev(dev)); err = send_usr_msg(sess->rtrs, READ, iu, - &vec, sizeof(*rsp), iu->sglist, 1, + &vec, sizeof(*rsp), iu->sgt.sgl, 1, msg_open_conf, &errno, wait); if (err) { rnbd_clt_put_dev(dev); @@ -582,6 +585,7 @@ static int send_msg_open(struct rnbd_clt_dev *dev, bool wait) err = errno; } + sg_free_table(&iu->sgt); rnbd_put_iu(sess, iu); return err; } @@ -610,7 +614,8 @@ static int send_msg_sess_info(struct rnbd_clt_session *sess, bool wait) iu->buf = rsp; iu->sess = sess; - sg_init_one(iu->sglist, rsp, sizeof(*rsp)); + sg_alloc_table(&iu->sgt, 1, GFP_KERNEL); + sg_init_one(iu->sgt.sgl, rsp, sizeof(*rsp)); msg.hdr.type = cpu_to_le16(RNBD_MSG_SESS_INFO); msg.ver = RNBD_PROTO_VER_MAJOR; @@ -626,7 +631,7 @@ static int send_msg_sess_info(struct rnbd_clt_session *sess, bool wait) goto put_iu; } err = send_usr_msg(sess->rtrs, READ, iu, - &vec, sizeof(*rsp), iu->sglist, 1, + &vec, sizeof(*rsp), iu->sgt.sgl, 1, msg_sess_info_conf, &errno, wait); if (err) { rnbd_clt_put_sess(sess); @@ -636,7 +641,7 @@ static int send_msg_sess_info(struct rnbd_clt_session *sess, bool wait) } else { err = errno; } - + sg_free_table(&iu->sgt); rnbd_put_iu(sess, iu); return err; } @@ -1016,11 +1021,10 @@ static int rnbd_client_xfer_request(struct rnbd_clt_dev *dev, * See queue limits. */ if (req_op(rq) != REQ_OP_DISCARD) - sg_cnt = blk_rq_map_sg(dev->queue, rq, iu->sglist); + sg_cnt = blk_rq_map_sg(dev->queue, rq, iu->sgt.sgl); if (sg_cnt == 0) - /* Do not forget to mark the end */ - sg_mark_end(&iu->sglist[0]); + sg_mark_end(&iu->sgt.sgl[0]); msg.hdr.type = cpu_to_le16(RNBD_MSG_IO); msg.device_id = cpu_to_le32(dev->device_id); @@ -1029,13 +1033,13 @@ static int rnbd_client_xfer_request(struct rnbd_clt_dev *dev, .iov_base = &msg, .iov_len = sizeof(msg) }; - size = rnbd_clt_get_sg_size(iu->sglist, sg_cnt); + size = rnbd_clt_get_sg_size(iu->sgt.sgl, sg_cnt); req_ops = (struct rtrs_clt_req_ops) { .priv = iu, .conf_fn = msg_io_conf, }; err = rtrs_clt_request(rq_data_dir(rq), &req_ops, rtrs, permit, - &vec, 1, size, iu->sglist, sg_cnt); + &vec, 1, size, iu->sgt.sgl, sg_cnt); if (unlikely(err)) { rnbd_clt_err_rl(dev, "RTRS failed to transfer IO, err: %d\n", err); @@ -1122,6 +1126,7 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx, struct rnbd_clt_dev *dev = rq->rq_disk->private_data; struct rnbd_iu *iu = blk_mq_rq_to_pdu(rq); int err; + blk_status_t ret = BLK_STS_IOERR; if (unlikely(dev->dev_state != DEV_STATE_MAPPED)) return BLK_STS_IOERR; @@ -1133,32 +1138,33 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx, return BLK_STS_RESOURCE; } + iu->sgt.sgl = iu->first_sgl; + err = sg_alloc_table_chained(&iu->sgt, + /* Even-if the request has no segment, + * sglist must have one entry at least */ + blk_rq_nr_phys_segments(rq) ? : 1, + iu->sgt.sgl, + RNBD_INLINE_SG_CNT); + if (err) { + rnbd_clt_err_rl(dev, "sg_alloc_table_chained ret=%d\n", err); + return BLK_STS_IOERR; + } + blk_mq_start_request(rq); err = rnbd_client_xfer_request(dev, rq, iu); if (likely(err == 0)) return BLK_STS_OK; if (unlikely(err == -EAGAIN || err == -ENOMEM)) { rnbd_clt_dev_kick_mq_queue(dev, hctx, 10/*ms*/); - rnbd_put_permit(dev->sess, iu->permit); - return BLK_STS_RESOURCE; + ret = BLK_STS_RESOURCE; } - + sg_free_table_chained(&iu->sgt, RNBD_INLINE_SG_CNT); rnbd_put_permit(dev->sess, iu->permit); - return BLK_STS_IOERR; -} - -static int rnbd_init_request(struct blk_mq_tag_set *set, struct request *rq, - unsigned int hctx_idx, unsigned int numa_node) -{ - struct rnbd_iu *iu = blk_mq_rq_to_pdu(rq); - - sg_init_table(iu->sglist, BMAX_SEGMENTS); - return 0; + return ret; } static struct blk_mq_ops rnbd_mq_ops = { .queue_rq = rnbd_queue_rq, - .init_request = rnbd_init_request, .complete = rnbd_softirq_done_fn, }; @@ -1172,7 +1178,7 @@ static int setup_mq_tags(struct rnbd_clt_session *sess) tag_set->numa_node = NUMA_NO_NODE; tag_set->flags = BLK_MQ_F_SHOULD_MERGE | BLK_MQ_F_TAG_QUEUE_SHARED; - tag_set->cmd_size = sizeof(struct rnbd_iu); + tag_set->cmd_size = sizeof(struct rnbd_iu) + RNBD_RDMA_SGL_SIZE; tag_set->nr_hw_queues = num_online_cpus(); return blk_mq_alloc_tag_set(tag_set); diff --git a/drivers/block/rnbd/rnbd-clt.h b/drivers/block/rnbd/rnbd-clt.h index efd67ae286ca..44b116028ff6 100644 --- a/drivers/block/rnbd/rnbd-clt.h +++ b/drivers/block/rnbd/rnbd-clt.h @@ -44,6 +44,13 @@ struct rnbd_iu_comp { int errno; }; +#ifdef CONFIG_ARCH_NO_SG_CHAIN +#define RNBD_INLINE_SG_CNT 0 +#else +#define RNBD_INLINE_SG_CNT 2 +#endif +#define RNBD_RDMA_SGL_SIZE (sizeof(struct scatterlist) * RNBD_INLINE_SG_CNT) + struct rnbd_iu { union { struct request *rq; /* for block io */ @@ -56,11 +63,12 @@ struct rnbd_iu { /* use to send msg associated with a sess */ struct rnbd_clt_session *sess; }; - struct scatterlist sglist[BMAX_SEGMENTS]; + struct sg_table sgt; struct work_struct work; int errno; struct rnbd_iu_comp comp; atomic_t refcount; + struct scatterlist first_sgl[]; }; struct rnbd_cpu_qlist { From patchwork Wed Dec 9 08:20:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jinpu Wang X-Patchwork-Id: 11960889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4439FC19425 for ; Wed, 9 Dec 2020 08:22:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 01C5923B97 for ; Wed, 9 Dec 2020 08:22:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726354AbgLIIWQ (ORCPT ); Wed, 9 Dec 2020 03:22:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725936AbgLIIWQ (ORCPT ); Wed, 9 Dec 2020 03:22:16 -0500 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4853BC0617B0 for ; Wed, 9 Dec 2020 00:21:01 -0800 (PST) Received: by mail-ed1-x542.google.com with SMTP id v22so574568edt.9 for ; Wed, 09 Dec 2020 00:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.ionos.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ywpjMsHON3M/dUVBVN/LPgCWwxFLN82iOCwXIqn+k/A=; b=iDdzn6b86ed3KJmzMOcoeo19Fj16OaXmaqzSmrLVAb5nkASf+K1TQ34LPiji7L9LTK 6Z7yPmmEEHjBTXScdHdQevk5ZmtbrZVa9Y2Nu7kJZopsxzK5Qbdo7huPQ5FbXF2MzSay yOg50CXlb20MVgMp0UZK9Dl/cD1oD/LES8XYn+JR+Vxjomm1OLN0XLCdX1Op07uBdPnb EASUhTVEvT0gzkQnp06iEUtWGF5WNha0p4VZCc+Rx8KmVKxMNDtNCKG6c0yvkaWx/74J fB5+J8URhpc4yUmutW5aOP5woBPsztIIQai89mUPmg7/8iGp959RmNe5VQOHQQ93C7a3 Fy5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ywpjMsHON3M/dUVBVN/LPgCWwxFLN82iOCwXIqn+k/A=; b=twNgAMRGieKep1UzSLFfl8ZYZBTL987zFUMMmyiRG8fgD8uLttLgXz6M4XNt4VVUCb 8L9/1r1x7ao1Q9jrfJ2RTw3c2+is90l0vIQwJUk3GMjwMuMBOuo4InWHl4IMDWPKne1S kOlCkGdRS5BsVvWf6NjiINmBI9IDP47nP8kPOTYi9z3/VTW/wupGJrJx72q+2P8mhpt6 jeCsJpN5V9Dzmao1lVNLcNX1lwUhOUj0GkEoogRKD7cLG6RWH52JpBZk5kGYLQmcVht6 YGCuqOWqiJD5uOBLlSIstwvKrq7aBLDZa7OxRBKiQ5rX11c+d2S791e0f0kB4mHeZNb8 5Drg== X-Gm-Message-State: AOAM5329w63Q/9XQgNtW/LUlwIsyQn/QEhDjAqTIo6x463zaP8TJW0J7 Voa/v+4ODwUN5qK+BzvWM+vF37jh9lDTNg== X-Google-Smtp-Source: ABdhPJzlyhaBnNQqNaBh7L69fdAJvLtaOVlYbABJGxK8FlD9KqrqfCXzFuWAqwV759BNzJ+ETZz7nA== X-Received: by 2002:a50:b002:: with SMTP id i2mr914864edd.99.1607502059894; Wed, 09 Dec 2020 00:20:59 -0800 (PST) Received: from jwang-Latitude-5491.fkb.profitbricks.net ([2001:16b8:49e0:2500:1d14:118d:b29c:98ec]) by smtp.gmail.com with ESMTPSA id cf17sm823225edb.16.2020.12.09.00.20.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Dec 2020 00:20:59 -0800 (PST) From: Jack Wang To: linux-block@vger.kernel.org Cc: axboe@kernel.dk, hch@infradead.org, sagi@grimberg.me, bvanassche@acm.org, danil.kipnis@cloud.ionos.com, Gioh Kim Subject: [PATCH for-next 7/7] block/rnbd-clt: Does not request pdu to rtrs-clt Date: Wed, 9 Dec 2020 09:20:51 +0100 Message-Id: <20201209082051.12306-8-jinpu.wang@cloud.ionos.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> References: <20201209082051.12306-1-jinpu.wang@cloud.ionos.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org From: Gioh Kim Previously the rnbd client requested the rtrs to allocate rnbd_iu just after the rtrs_iu. So the rnbd client passes the size of rnbd_iu for rtrs_clt_open() and rtrs creates an array of rnbd_iu and rtrs_iu. For IO handling, rnbd_iu exists after the request because we pass the size of rnbd_iu when setting the tag-set. Therefore we do not use the rnbd_iu allocated by rtrs for IO handling. We only use the rnbd_iu allocated by rtrs when doing session initialization. Almost all rnbd_iu allocated by rtrs are wasted. By this patch the rnbd client does not request rnbd_iu allocation to rtrs but allocate it for itself when doing session initialization. Also remove unused rtrs_permit_to_pdu from rtrs. Signed-off-by: Gioh Kim Signed-off-by: Jack Wang --- drivers/block/rnbd/rnbd-clt.c | 17 +++++++++++++---- drivers/infiniband/ulp/rtrs/rtrs-clt.c | 6 ------ drivers/infiniband/ulp/rtrs/rtrs.h | 7 ------- 3 files changed, 13 insertions(+), 17 deletions(-) diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c index c37d94181ff4..9165e70bee0c 100644 --- a/drivers/block/rnbd/rnbd-clt.c +++ b/drivers/block/rnbd/rnbd-clt.c @@ -349,12 +349,19 @@ static struct rnbd_iu *rnbd_get_iu(struct rnbd_clt_session *sess, struct rnbd_iu *iu; struct rtrs_permit *permit; + iu = kzalloc(sizeof(*iu), GFP_KERNEL); + if (!iu) { + return NULL; + } + permit = rnbd_get_permit(sess, con_type, wait ? RTRS_PERMIT_WAIT : RTRS_PERMIT_NOWAIT); - if (unlikely(!permit)) + if (unlikely(!permit)) { + kfree(iu); return NULL; - iu = rtrs_permit_to_pdu(permit); + } + iu->permit = permit; /* * 1st reference is dropped after finishing sending a "user" message, @@ -373,8 +380,10 @@ static struct rnbd_iu *rnbd_get_iu(struct rnbd_clt_session *sess, static void rnbd_put_iu(struct rnbd_clt_session *sess, struct rnbd_iu *iu) { - if (atomic_dec_and_test(&iu->refcount)) + if (atomic_dec_and_test(&iu->refcount)) { rnbd_put_permit(sess, iu->permit); + kfree(iu); + } } static void rnbd_softirq_done_fn(struct request *rq) @@ -1216,7 +1225,7 @@ find_and_get_or_create_sess(const char *sessname, */ sess->rtrs = rtrs_clt_open(&rtrs_ops, sessname, paths, path_cnt, port_nr, - sizeof(struct rnbd_iu), + 0, /* Do not use pdu of rtrs */ RECONNECT_DELAY, BMAX_SEGMENTS, BLK_MAX_SEGMENT_SIZE, MAX_RECONNECTS); diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index bcfa6258a7af..7644c3f627ca 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -157,12 +157,6 @@ void rtrs_clt_put_permit(struct rtrs_clt *clt, struct rtrs_permit *permit) } EXPORT_SYMBOL(rtrs_clt_put_permit); -void *rtrs_permit_to_pdu(struct rtrs_permit *permit) -{ - return permit + 1; -} -EXPORT_SYMBOL(rtrs_permit_to_pdu); - /** * rtrs_permit_to_clt_con() - returns RDMA connection pointer by the permit * @sess: client session pointer diff --git a/drivers/infiniband/ulp/rtrs/rtrs.h b/drivers/infiniband/ulp/rtrs/rtrs.h index 9af750f4d783..8738e90e715a 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs.h +++ b/drivers/infiniband/ulp/rtrs/rtrs.h @@ -63,13 +63,6 @@ struct rtrs_clt *rtrs_clt_open(struct rtrs_clt_ops *ops, void rtrs_clt_close(struct rtrs_clt *sess); -/** - * rtrs_permit_to_pdu() - converts rtrs_permit to opaque pdu pointer - * @permit: RTRS permit pointer, it associates the memory allocation for future - * RDMA operation. - */ -void *rtrs_permit_to_pdu(struct rtrs_permit *permit); - enum { RTRS_PERMIT_NOWAIT = 0, RTRS_PERMIT_WAIT = 1,