From patchwork Mon Feb 14 07:59:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84391C433F5 for ; Mon, 14 Feb 2022 08:45:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242389AbiBNIph (ORCPT ); Mon, 14 Feb 2022 03:45:37 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:45692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242329AbiBNIpc (ORCPT ); Mon, 14 Feb 2022 03:45:32 -0500 X-Greylist: delayed 598 seconds by postgrey-1.37 at lindbergh.monkeyblade.net; Mon, 14 Feb 2022 00:45:24 PST Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26A015046E; Mon, 14 Feb 2022 00:45:24 -0800 (PST) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220214083521epoutp025ba97d9fe8a39ef44d248309d2a63c94~TmhkvZKHH1820318203epoutp021; Mon, 14 Feb 2022 08:35:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220214083521epoutp025ba97d9fe8a39ef44d248309d2a63c94~TmhkvZKHH1820318203epoutp021 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827721; bh=vIdJ/mHzkG/ncG862yZ99yroqi17ftw/JCJp3El+0s8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jgUd0kBcEvXAr4+t8qpQSXf8ReGsnJEkNHnrLWYpWUqaCa1dIMoo92p5nVwekEybG ervYHZqlxUjr8jvkMNqC1NsZJyICI2j0AQG01aY683eNgH4oz+vz5yLSMx1myTC9Qe DXILqW+EJAA4CUa8116+Ufaao1i2v9k6q63fWaJM= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220214083521epcas5p3faa13564d4db2b93ad37de4a2fde889d~TmhkZ4JD40865408654epcas5p3Q; Mon, 14 Feb 2022 08:35:21 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.176]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JxyG94kpTz4x9QG; Mon, 14 Feb 2022 08:35:13 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 97.16.05590.1441A026; Mon, 14 Feb 2022 17:35:13 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20220214080558epcas5p17c1fb3b659b956908ff7215a61bcc0c9~TmH6th4YZ3176231762epcas5p1S; Mon, 14 Feb 2022 08:05:58 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080558epsmtrp2c5a3f5d650b84129c4a6aabcc9512707~TmH6q_H_O2569425694epsmtrp2M; Mon, 14 Feb 2022 08:05:58 +0000 (GMT) X-AuditID: b6c32a4b-723ff700000015d6-f1-620a1441c60b Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id D6.C1.29871.66D0A026; Mon, 14 Feb 2022 17:05:58 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080553epsmtip2d84b9b4e078e403260c79d7d89473125~TmH2M4tpN2253222532epsmtip2t; Mon, 14 Feb 2022 08:05:53 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, SelvaKumar S , Nitesh Shetty , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 01/10] block: make bio_map_kern() non static Date: Mon, 14 Feb 2022 13:29:51 +0530 Message-Id: <20220214080002.18381-2-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjOuff2tqAlV1A5sGzWEkQkfHSWelAQtxlzM0RBzLYYBS9wBQb9 WEsViW7FThe+FHBjUA0iigQ0giiOIcWCYYYPUVPpBhuCWiaxEURBcPIxysXNf8/7nOd5n/O+ J0eAO3fx3QVJilRWrWBSxKQjcf2Wt7fvJ0sdYwNMVhGqbv8NR409r3joYt8JEhW+eIOjkeYn PFRwooiPzFYnZBw+xUP3JjMw9KR2FkONZQUYqrzYiqGnFecAunF2FEOZHfcwNPVIglpnn5Oo oMUC0GC3AUPGXh/UaGwjkLnhNInOXBjko+zf60nUZDPiqOL2DIa6DFMkqrdmAHT97Rkc3XrY TaDLthEC2SbbSHTsyjhAR3Pe8NHd6du8TR60+UEYbei/Q9L5+mE+/auhj0/ffXiFoPWlfxG0 +Y6Wrq3KJOmr57+jT/5RAegbPTqSPtLZitNFL8dIOlc/TNKjg70EPdLUTUYs35UcnMgy8axa xCrilPFJioQQcVhUzGcxgbIAia8kCK0TixSMnA0Rb94a4bslKWVuiWLRfiZFO0dFMBqN2H9j sFqpTWVFiUpNaoiYVcWnqKQqPw0j12gVCX4KNnW9JCDg48A54d7kxOc9RTxV3uK0oqZsTAdy HLOAgwBSUnhcdx63Y2fqBoD5XVuzgOMcfglg/c0igiteA5g1NcTPAoJ5R2GuF8cbAXz55xCP cx/FYEOv0q4hKR/YMSuw00spAlZOTMz3wakSPqxrs/HtBy5UKOw8aZpPJihPOPrcQtqxkFoP R55OAO52HvDso+b5/g7UBmiyVeCcZglsK7YSdoxTK6C+7hRuD4DUMUf4z7NCnDNvhjebDQvY BT67fY3PYXf4athIcoZsACc7+zGuKAJQn6cnOVUovN84jdnHwSlvWN3gz9Efwp/aL2NcshPM fWvFOF4I60veYQ94qbp0oY0btExkLGAaNvRPLKz0OIA2wzSRB0SG9yYyvDeR4f/oUoBXATdW pZEnsJpA1VoFe+C/V45TymvB/MdaE1YPHg+88GsBmAC0ACjAxUuF0XccYp2F8czBdFatjFFr U1hNCwicW3k+7r4sTjn3MxWpMRJpUIBUJpNJg9bKJGJXYXtCDeNMJTCpbDLLqlj1Ox8mcHDX YfJ9MNqysytz2+uomeBX+dF/twnLPhqNHMqb4Zn1Xoqu3SmqQxti08J7TW4hP+yr+DQ8Mm1F 50RGybeZw1VST9H2hi8+4P3sZOm96Htgj1oQtsTV33db9umsGnlnR3mUeYd2XVj6+K7PQ7I9 Q/sre5Lc0woYL9OsbFNd8jdSn4GvL11ycXXSjfw49njPor3Du/mtZdcekEd3rk5f5LY/2Vps Ta9p6zZPrxo9N1RVz3+0oxys1g0eGXtR++XKqhrH5gbhV6b26T45fnCWWTxYPjU+ELVsyz3R 3u1BvjWrLuvDc7ot9xOW/0KXPzFHHi7229h6yGvl1dmRSg+Z+PAF18bvpzska8WEJpGRrMHV GuZf/OP5VOEEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmphleLIzCtJLcpLzFFi42LZdlhJXjeNlyvJ4FafksX6U8eYLfbc/Mxq sfpuP5vFtA8/mS3eH3zMajGpfwa7xeUnfBZ7381mtbjwo5HJ4vGm/0wWexZNYrJYufook8Xz 5YsZLXYv/Mhk0Xn6ApPFn4eGFkf/v2WzmHToGqPF06uzmCz23tK22LP3JIvF5V1z2CzmL3vK btF9fQebxb7Xe5ktlh//x2RxbtYfNosdTxoZLbb9ns9scfjeVRaLda/fs1i8/nGSzaJt41dG i9aen+wW5/8eZ3VQ9rh8xdtj1v2zbB4Tm9+xe+ycdZfd4/y9jSwezQvusHhcPlvqsWlVJ5vH 5iX1HpNvLGf02H2zgc2j6cxRZo8Zn76wefQ2v2Pz+Pj0FovH+31X2QLEorhsUlJzMstSi/Tt Ergy3t6cwVowgadixr5upgbGHq4uRg4OCQETiWm96l2MXBxCArsZJf717WbpYuQEiktKLPt7 hBnCFpZY+e85O0RRM5PE5I5jzCDNbALaEqf/c4DUiAiwSKz8/p0FpIZZ4AS7xPmWQ+wgCWEB e4kzkw+ADWIRUJX4+PYaG4jNK2Al8f75d0aIBcoSCx8eZAWxOQWsJQ68Xg5WLwRUM+MqxBxe AUGJkzOfgB3HLCAv0bx1NvMERoFZSFKzkKQWMDKtYpRMLSjOTc8tNiwwzEst1ytOzC0uzUvX S87P3cQITihamjsYt6/6oHeIkYmD8RCjBAezkghv3FnOJCHelMTKqtSi/Pii0pzU4kOM0hws SuK8F7pOxgsJpCeWpGanphakFsFkmTg4pRqY2nac05xrOOOM0YoDzzrm9l1w4CpIPpEW7B9U 8+r47F8J+3McLi49qdy4ZFud6epX+x7qvDO8JhPknaa9M6Fg8olO2e8fmphFwjaFC1Z9Upr1 e/9Wz/VPwyoNrPc1+hl+07k/z2p1pAqTU5rBSyWhrvnWXe6vxBVcX8hvKft/kyPl8f9WmfNM ce2t2rXmjaIrtn47+HT7zsprIt6q9bnFbPvPOvp1H6tj1hTY2PizZXW+HMt+x/WVaS6lPyt3 R5+eU743t3BpQUL8olLx6wuebP2RPeXvrRz+WTzf8lUXqivv1/6iInnUdDoj3+vmSy9nMzCu rd6n/sE7rshMaE37uunTF5t98F14PXHv5KXlSizFGYmGWsxFxYkAWynKtZcDAAA= X-CMS-MailID: 20220214080558epcas5p17c1fb3b659b956908ff7215a61bcc0c9 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080558epcas5p17c1fb3b659b956908ff7215a61bcc0c9 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: SelvaKumar S Make bio_map_kern() non static Signed-off-by: SelvaKumar S Signed-off-by: Nitesh Shetty --- block/blk-map.c | 2 +- include/linux/blkdev.h | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/block/blk-map.c b/block/blk-map.c index 4526adde0156..c110205df0d5 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -336,7 +336,7 @@ static void bio_map_kern_endio(struct bio *bio) * Map the kernel address into a bio suitable for io to a block * device. Returns an error pointer in case of error. */ -static struct bio *bio_map_kern(struct request_queue *q, void *data, +struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len, gfp_t gfp_mask) { unsigned long kaddr = (unsigned long)data; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 3bfc75a2a450..efed3820cbf7 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1106,6 +1106,8 @@ extern int blkdev_issue_discard(struct block_device *bdev, sector_t sector, extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, int flags, struct bio **biop); +struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len, + gfp_t gfp_mask); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Mon Feb 14 07:59:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F912C4332F for ; Mon, 14 Feb 2022 08:45:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242346AbiBNIpe (ORCPT ); Mon, 14 Feb 2022 03:45:34 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:45698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242333AbiBNIpc (ORCPT ); Mon, 14 Feb 2022 03:45:32 -0500 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BDB64EA0F; Mon, 14 Feb 2022 00:45:24 -0800 (PST) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220214083529epoutp02d6e136aa7429ff0b0a8a0ed3613ef704~TmhrkvpxX1820318203epoutp023; Mon, 14 Feb 2022 08:35:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220214083529epoutp02d6e136aa7429ff0b0a8a0ed3613ef704~TmhrkvpxX1820318203epoutp023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827729; bh=oLVwwrvhR9cVBwUQ0x3VRa7cDAkLrCyx65XBVMW9AB8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JhdBfWf3NhUgvYBeUdLYtk7leX4H1pkhCKALYSXDFpyiXAcASWATQYO2qO+gk5rfI AgryfZ0WRCxBpRQvyhm6hmt/XQgqSAIyjIll/js7cnxbSMV4QXUJff8EisBiV2M1bJ f2UFV8KxPKlubWQK8M1OHKIILxAtkndV36nG1nEI= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220214083528epcas5p3105dbe18e923ff93842fa3be100ba12e~TmhrGbIC40862708627epcas5p36; Mon, 14 Feb 2022 08:35:28 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.180]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JxyGL6J9dz4x9QC; Mon, 14 Feb 2022 08:35:22 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 89.C6.06423.8441A026; Mon, 14 Feb 2022 17:35:20 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20220214080605epcas5p16868dae515a6355cf9fecf22df4f3c3d~TmIBV6EUB3178831788epcas5p1f; Mon, 14 Feb 2022 08:06:05 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080605epsmtrp2264952eef19bcd049835c1054d6f7113~TmIBTUD1r2568925689epsmtrp2C; Mon, 14 Feb 2022 08:06:05 +0000 (GMT) X-AuditID: b6c32a49-b01ff70000001917-e4-620a1448fcce Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id CC.C1.29871.D6D0A026; Mon, 14 Feb 2022 17:06:05 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080600epsmtip2f599380620ecf7167a64c45a95d05894~TmH8yHySr2250822508epsmtip2r; Mon, 14 Feb 2022 08:06:00 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , SelvaKumar S , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 02/10] block: Introduce queue limits for copy-offload support Date: Mon, 14 Feb 2022 13:29:52 +0530 Message-Id: <20220214080002.18381-3-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02TezBcdxTH53fv3btLylxL2h9JJ7omHZGQ3YTtT0TTEe3cNGaq0kmHtvSG W9Ram92VViedYks9Kl7VsMRKZIJICfFmE4+iXiWjktKKGEsN45FIg7Gi2E2a/87r8/uec35z eDj/LteGFypVsnIpIxGQpkRN2z57R9rK9IywIdMOlXd34KhpeImDSkdTSfTz4iqOFlomOCgj NZuLBnXmSDufy0EDKzEYmqjcwFDTlQwMlZS2Y+ifokKAGi8/wlBizwCG9OMi1L4xR6KM1nsA TQ6pMaQd2Y+atF0EGmzII5Hm2iQXJd+vI9HtWS2OijqfYeh3tZ5EdboYgGrWNDhqezBEoLLZ BQLNrnSRKL7iX4Diflzlov71Ts47dvTgHydp9VgfSaer5rl0vXqUS/c/qCBoVcHfBD3YF0lX Xk8k6VtXv6Mz/ywCdONwNEnH9rbjdPbjJySdopon6UeTIwS9cHuI9H7VL+xoCMsEsXJbVhoY ERQqDXYXnDwVcDzARSwUOYpc0VsCWykTzroLPL28Hd8LlWwuUWB7jpFEboa8GYVCcPDto/KI SCVrGxKhULoLWFmQROYsc1Iw4YpIabCTlFUeEQmFh1w2Cz8PC+mYWuXKLn729f2RHBANaj9I AiY8SDnDiZlaMgmY8vhUI4CanhmuwXkM4FBlLcfgPAVwNV/DeY6sVWUThoQWwN7iTiMfh8Ef SorxJMDjkdR+2LPB2wKsKAKWLC9vAziVz4Uld/RgK2FJ+cCNn/KJLZug9sKOynx8yzajjsBL iWuYQc0OXh5v2VY2odxg82yRscYCduXotlmc2gNV1bn4lgCkYk1hlv4i1wB7Qt2VIdJgW8KZ zipj3AYuzWtJA5AM4ErvGGZwsgFUpamMxDF4t2kd2xoHp/bB8oaDhvDrMKu7DDMom8OUNZ2x UzNYl6970fWN8gLjM9bw3nKM0aZh5+C0cV0XAKyY7gZpwFb90kTqlyZS/y9dAPDrwJqVKcKD WYWLTCRlv3rx0YER4ZVg+7YcTtSB0YeLTq0A44FWAHm4wMrMv8/kDN8siIn6hpVHBMgjJayi Fbhsrjwdt9kZGLF5nFJlgMjZVegsFoudXQ+LRYLXzHqCbzJ8KphRsmEsK2PlzzmMZ2ITjR1/ 6js/mQth/Fnf9DH74Y/nRydjl3Y2ya2nBuqb9TtWbnncaGh+kuPV0PW9+kBf5qfZebhzVL32 WkXn++dSWW5Fdc15r8JTH/rf+fJSkL2r2+mCXTaHoqZ9kru1IxlTKn3dAnOsdtdC268F017x XsWHmz0eul7wJCyrlcll39b3B7rcfONsqZhetDjRL0nzH3jzF/7eOI+Pyjyvmujhs3e5eabV /rN/ERr5bt8E6UpJlWtItF9epOdg2NHdCdW/RY/7ua15BZZ9ITAvmguVJATt2SiNLrM/UC/U AFGhTlh6+hOL8xvlS69UZDklT7rwm9drbBxaxovqTXYwPvEpc2LZAl9AKEIYkQMuVzD/ATSM zYHkBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02SfUwTdxyH87u73l1ruhwF9NA5khqNhdlJXLavzi0sanZzWSTLEhPJxDpu SNYCa6XCHIJ2GN5Gx8sUygBBQ6EqSJnABkWpUugYgwwKA8WNUOxIGRUF5UVglmaJ/32S58nz 14fGJd8KNtKxcSd5dZxCKSVFROMdafAOlVh0fOdkoQzqfrXh0Dr8RABXR/UkXHi0gIOnfVwA +foiCvqdr4BlukQAffNnMRg3r2LQWpmPQc3VDgxcxssIWipmMMjs7sPg+VgYdKz+S0K+dRDB hMOAgWUkFFotdgL6f/mRhPKqCQqyh5pJaHNbcDB2rmDwu+E5Cc3Oswgal8pxuPPAQUCt20OA e95Owvn6OQTpOQsU9C53CsK3cP0DH3GGv3pILk83TXE/G0YprvdBPcHpLt0nuP6eRM5syiS5 hiupXMGfRsS1DKeR3LnfOnCu6PEsyX2nmya5mYkRgvO0OciI9UdEe6N5ZayWV7/x3jHRCdvD BSrh4mdJQyPFKA01HcpCQppl3mSXfioispCIljAtiL1weZrygSC2avku7tv+bM2Ki/JJOox1 jdW9ADRNMqFs9yrtdQIYgq159mwthDNdFOspv0Z5HX8mgp27Feh1CGYrazOXrTXFzB62NHMJ 8/W3sBVj7QLvFjLvsLfdxjVH8sIpclgpn+/H2oudhHfjTDCru1mCf48Yw0vI8BK6hDATCuIT NKoYlSYsISyOPyXXKFSaxLgY+efxKjNaO0qIrBk1mR7JrQijkRWxNC4NEB/tER6XiKMVyV/z 6vgodaKS11jRJpqQbhD3ZdmjJEyM4iT/Jc8n8Or/KUYLN6Zhr0fvfpKy3/j0k9F/qFpTdF7l jXjZweHcP/x22a47c4pf2xF7TNhV2rA81ZAS2n14cVNGgAyL2h2SI0gTHrmX2tWtLZsKMgde e5jeGN6X+8PUHuXTv79quLi/QB4YfC73bv16t2n29oGJ+V3bzB9XFiulGClq15cu6t4OqL2f fXp83cwh8enqK2e4zcJ1g/aalMn3bfIyz05tOkVnOCo1FMomD2y4ldx5ODIqaSC8sCO0jljZ +2nnu/FD2raBueTqQb/8D2a+uSk583h2smrf1rdcr1ruERXVTQe3uQrOO/ct6wu7pk71uv2/ SE/dHllSfSNPH2He3pok4zcvajPIyKTVD6WE5oQiLARXaxT/AWTOCwmXAwAA X-CMS-MailID: 20220214080605epcas5p16868dae515a6355cf9fecf22df4f3c3d X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080605epcas5p16868dae515a6355cf9fecf22df4f3c3d References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add device limits as sysfs entries, - copy_offload (RW) - copy_max_bytes (RW) - copy_max_hw_bytes (RO) - copy_max_range_bytes (RW) - copy_max_range_hw_bytes (RO) - copy_max_nr_ranges (RW) - copy_max_nr_ranges_hw (RO) Above limits help to split the copy payload in block layer. copy_offload, used for setting copy offload(1) or emulation(0). copy_max_bytes: maximum total length of copy in single payload. copy_max_range_bytes: maximum length in a single entry. copy_max_nr_ranges: maximum number of entries in a payload. copy_max_*_hw_*: Reflects the device supported maximum limits. Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S Signed-off-by: Kanchan Joshi --- block/blk-settings.c | 59 ++++++++++++++++++ block/blk-sysfs.c | 138 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 13 ++++ 3 files changed, 210 insertions(+) diff --git a/block/blk-settings.c b/block/blk-settings.c index b880c70e22e4..4baccc93a294 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -57,6 +57,12 @@ void blk_set_default_limits(struct queue_limits *lim) lim->misaligned = 0; lim->zoned = BLK_ZONED_NONE; lim->zone_write_granularity = 0; + lim->max_hw_copy_sectors = 0; + lim->max_copy_sectors = 0; + lim->max_hw_copy_nr_ranges = 0; + lim->max_copy_nr_ranges = 0; + lim->max_hw_copy_range_sectors = 0; + lim->max_copy_range_sectors = 0; } EXPORT_SYMBOL(blk_set_default_limits); @@ -82,6 +88,12 @@ void blk_set_stacking_limits(struct queue_limits *lim) lim->max_write_same_sectors = UINT_MAX; lim->max_write_zeroes_sectors = UINT_MAX; lim->max_zone_append_sectors = UINT_MAX; + lim->max_hw_copy_sectors = ULONG_MAX; + lim->max_copy_sectors = ULONG_MAX; + lim->max_hw_copy_range_sectors = UINT_MAX; + lim->max_copy_range_sectors = UINT_MAX; + lim->max_hw_copy_nr_ranges = USHRT_MAX; + lim->max_copy_nr_ranges = USHRT_MAX; } EXPORT_SYMBOL(blk_set_stacking_limits); @@ -178,6 +190,45 @@ void blk_queue_max_discard_sectors(struct request_queue *q, } EXPORT_SYMBOL(blk_queue_max_discard_sectors); +/** + * blk_queue_max_copy_sectors - set max sectors for a single copy payload + * @q: the request queue for the device + * @max_copy_sectors: maximum number of sectors to copy + **/ +void blk_queue_max_copy_sectors(struct request_queue *q, + unsigned int max_copy_sectors) +{ + q->limits.max_hw_copy_sectors = max_copy_sectors; + q->limits.max_copy_sectors = max_copy_sectors; +} +EXPORT_SYMBOL(blk_queue_max_copy_sectors); + +/** + * blk_queue_max_copy_range_sectors - set max sectors for a single range, in a copy payload + * @q: the request queue for the device + * @max_copy_range_sectors: maximum number of sectors to copy in a single range + **/ +void blk_queue_max_copy_range_sectors(struct request_queue *q, + unsigned int max_copy_range_sectors) +{ + q->limits.max_hw_copy_range_sectors = max_copy_range_sectors; + q->limits.max_copy_range_sectors = max_copy_range_sectors; +} +EXPORT_SYMBOL(blk_queue_max_copy_range_sectors); + +/** + * blk_queue_max_copy_nr_ranges - set max number of ranges, in a copy payload + * @q: the request queue for the device + * @max_copy_nr_ranges: maximum number of ranges + **/ +void blk_queue_max_copy_nr_ranges(struct request_queue *q, + unsigned int max_copy_nr_ranges) +{ + q->limits.max_hw_copy_nr_ranges = max_copy_nr_ranges; + q->limits.max_copy_nr_ranges = max_copy_nr_ranges; +} +EXPORT_SYMBOL(blk_queue_max_copy_nr_ranges); + /** * blk_queue_max_write_same_sectors - set max sectors for a single write same * @q: the request queue for the device @@ -541,6 +592,14 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->max_segment_size = min_not_zero(t->max_segment_size, b->max_segment_size); + t->max_copy_sectors = min(t->max_copy_sectors, b->max_copy_sectors); + t->max_hw_copy_sectors = min(t->max_hw_copy_sectors, b->max_hw_copy_sectors); + t->max_copy_range_sectors = min(t->max_copy_range_sectors, b->max_copy_range_sectors); + t->max_hw_copy_range_sectors = min(t->max_hw_copy_range_sectors, + b->max_hw_copy_range_sectors); + t->max_copy_nr_ranges = min(t->max_copy_nr_ranges, b->max_copy_nr_ranges); + t->max_hw_copy_nr_ranges = min(t->max_hw_copy_nr_ranges, b->max_hw_copy_nr_ranges); + t->misaligned |= b->misaligned; alignment = queue_limit_alignment_offset(b, start); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 9f32882ceb2f..9ddd07f142d9 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -212,6 +212,129 @@ static ssize_t queue_discard_zeroes_data_show(struct request_queue *q, char *pag return queue_var_show(0, page); } +static ssize_t queue_copy_offload_show(struct request_queue *q, char *page) +{ + return queue_var_show(blk_queue_copy(q), page); +} + +static ssize_t queue_copy_offload_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long copy_offload; + ssize_t ret = queue_var_store(©_offload, page, count); + + if (ret < 0) + return ret; + + if (copy_offload && !q->limits.max_hw_copy_sectors) + return -EINVAL; + + if (copy_offload) + blk_queue_flag_set(QUEUE_FLAG_COPY, q); + else + blk_queue_flag_clear(QUEUE_FLAG_COPY, q); + + return ret; +} + +static ssize_t queue_copy_max_hw_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", + (unsigned long long)q->limits.max_hw_copy_sectors << 9); +} + +static ssize_t queue_copy_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", + (unsigned long long)q->limits.max_copy_sectors << 9); +} + +static ssize_t queue_copy_max_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long max_copy; + ssize_t ret = queue_var_store(&max_copy, page, count); + + if (ret < 0) + return ret; + + if (max_copy & (queue_logical_block_size(q) - 1)) + return -EINVAL; + + max_copy >>= 9; + if (max_copy > q->limits.max_hw_copy_sectors) + max_copy = q->limits.max_hw_copy_sectors; + + q->limits.max_copy_sectors = max_copy; + return ret; +} + +static ssize_t queue_copy_range_max_hw_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", + (unsigned long long)q->limits.max_hw_copy_range_sectors << 9); +} + +static ssize_t queue_copy_range_max_show(struct request_queue *q, + char *page) +{ + return sprintf(page, "%llu\n", + (unsigned long long)q->limits.max_copy_range_sectors << 9); +} + +static ssize_t queue_copy_range_max_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long max_copy; + ssize_t ret = queue_var_store(&max_copy, page, count); + + if (ret < 0) + return ret; + + if (max_copy & (queue_logical_block_size(q) - 1)) + return -EINVAL; + + max_copy >>= 9; + if (max_copy > UINT_MAX) + return -EINVAL; + + if (max_copy > q->limits.max_hw_copy_range_sectors) + max_copy = q->limits.max_hw_copy_range_sectors; + + q->limits.max_copy_range_sectors = max_copy; + return ret; +} + +static ssize_t queue_copy_nr_ranges_max_hw_show(struct request_queue *q, char *page) +{ + return queue_var_show(q->limits.max_hw_copy_nr_ranges, page); +} + +static ssize_t queue_copy_nr_ranges_max_show(struct request_queue *q, + char *page) +{ + return queue_var_show(q->limits.max_copy_nr_ranges, page); +} + +static ssize_t queue_copy_nr_ranges_max_store(struct request_queue *q, + const char *page, size_t count) +{ + unsigned long max_nr; + ssize_t ret = queue_var_store(&max_nr, page, count); + + if (ret < 0) + return ret; + + if (max_nr > USHRT_MAX) + return -EINVAL; + + if (max_nr > q->limits.max_hw_copy_nr_ranges) + max_nr = q->limits.max_hw_copy_nr_ranges; + + q->limits.max_copy_nr_ranges = max_nr; + return ret; +} + static ssize_t queue_write_same_max_show(struct request_queue *q, char *page) { return sprintf(page, "%llu\n", @@ -597,6 +720,14 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); +QUEUE_RW_ENTRY(queue_copy_offload, "copy_offload"); +QUEUE_RO_ENTRY(queue_copy_max_hw, "copy_max_hw_bytes"); +QUEUE_RW_ENTRY(queue_copy_max, "copy_max_bytes"); +QUEUE_RO_ENTRY(queue_copy_range_max_hw, "copy_max_range_hw_bytes"); +QUEUE_RW_ENTRY(queue_copy_range_max, "copy_max_range_bytes"); +QUEUE_RO_ENTRY(queue_copy_nr_ranges_max_hw, "copy_max_nr_ranges_hw"); +QUEUE_RW_ENTRY(queue_copy_nr_ranges_max, "copy_max_nr_ranges"); + QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); @@ -643,6 +774,13 @@ static struct attribute *queue_attrs[] = { &queue_discard_max_entry.attr, &queue_discard_max_hw_entry.attr, &queue_discard_zeroes_data_entry.attr, + &queue_copy_offload_entry.attr, + &queue_copy_max_hw_entry.attr, + &queue_copy_max_entry.attr, + &queue_copy_range_max_hw_entry.attr, + &queue_copy_range_max_entry.attr, + &queue_copy_nr_ranges_max_hw_entry.attr, + &queue_copy_nr_ranges_max_entry.attr, &queue_write_same_max_entry.attr, &queue_write_zeroes_max_entry.attr, &queue_zone_append_max_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index efed3820cbf7..792e6d556589 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -254,6 +254,13 @@ struct queue_limits { unsigned int discard_alignment; unsigned int zone_write_granularity; + unsigned long max_hw_copy_sectors; + unsigned long max_copy_sectors; + unsigned int max_hw_copy_range_sectors; + unsigned int max_copy_range_sectors; + unsigned short max_hw_copy_nr_ranges; + unsigned short max_copy_nr_ranges; + unsigned short max_segments; unsigned short max_integrity_segments; unsigned short max_discard_segments; @@ -562,6 +569,7 @@ struct request_queue { #define QUEUE_FLAG_RQ_ALLOC_TIME 27 /* record rq->alloc_time_ns */ #define QUEUE_FLAG_HCTX_ACTIVE 28 /* at least one blk-mq hctx is active */ #define QUEUE_FLAG_NOWAIT 29 /* device supports NOWAIT */ +#define QUEUE_FLAG_COPY 30 /* supports copy offload */ #define QUEUE_FLAG_MQ_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_SAME_COMP) | \ @@ -585,6 +593,7 @@ bool blk_queue_flag_test_and_set(unsigned int flag, struct request_queue *q); #define blk_queue_io_stat(q) test_bit(QUEUE_FLAG_IO_STAT, &(q)->queue_flags) #define blk_queue_add_random(q) test_bit(QUEUE_FLAG_ADD_RANDOM, &(q)->queue_flags) #define blk_queue_discard(q) test_bit(QUEUE_FLAG_DISCARD, &(q)->queue_flags) +#define blk_queue_copy(q) test_bit(QUEUE_FLAG_COPY, &(q)->queue_flags) #define blk_queue_zone_resetall(q) \ test_bit(QUEUE_FLAG_ZONE_RESETALL, &(q)->queue_flags) #define blk_queue_secure_erase(q) \ @@ -958,6 +967,10 @@ extern void blk_queue_max_discard_segments(struct request_queue *, extern void blk_queue_max_segment_size(struct request_queue *, unsigned int); extern void blk_queue_max_discard_sectors(struct request_queue *q, unsigned int max_discard_sectors); +extern void blk_queue_max_copy_sectors(struct request_queue *q, unsigned int max_copy_sectors); +extern void blk_queue_max_copy_range_sectors(struct request_queue *q, + unsigned int max_copy_range_sectors); +extern void blk_queue_max_copy_nr_ranges(struct request_queue *q, unsigned int max_copy_nr_ranges); extern void blk_queue_max_write_same_sectors(struct request_queue *q, unsigned int max_write_same_sectors); extern void blk_queue_max_write_zeroes_sectors(struct request_queue *q, From patchwork Mon Feb 14 07:59:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 020F3C433F5 for ; Mon, 14 Feb 2022 08:57:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242505AbiBNI5Q (ORCPT ); Mon, 14 Feb 2022 03:57:16 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:52348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231379AbiBNI5P (ORCPT ); Mon, 14 Feb 2022 03:57:15 -0500 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E2735F8F8; Mon, 14 Feb 2022 00:57:07 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220214085705epoutp028949dbcb2b9a5f51b53fb31f6de78f90~Tm0igiQe10681406814epoutp02K; Mon, 14 Feb 2022 08:57:05 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220214085705epoutp028949dbcb2b9a5f51b53fb31f6de78f90~Tm0igiQe10681406814epoutp02K DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644829025; bh=Ex53mHj+W2Mk+5SobOV1KuNT3KeVZaZm5yu1z7E0s2Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=o0kPwWCZewBYOtYhT5yUH6hpi/sZ5jvjW1pfO8aO0wpIHFdop7xVZoxsIxKBC7Oah bgQnQSKWYjVn3HTtrIwdSMBoueBrzbzBee6uU9PIyx+WDsBVXOI0dBtr0S9hKX5UYv SZzSEyy5asZPbFBhtMQkq5pU8pQNvCtauklxQeVo= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20220214085704epcas5p433fe0ae635ce463ec33978f2d52c11fc~Tm0h5kZTU2118621186epcas5p48; Mon, 14 Feb 2022 08:57:04 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.175]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JxylC1ZzFz4x9Q9; Mon, 14 Feb 2022 08:56:55 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 57.3D.06423.6591A026; Mon, 14 Feb 2022 17:56:55 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20220214080612epcas5p2228606969011ce88a94d3b1be30d0614~TmIH9V4h72142721427epcas5p2N; Mon, 14 Feb 2022 08:06:12 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080612epsmtrp21dcf801d7107998893711767c2b140d0~TmIH7gD1L2568725687epsmtrp2f; Mon, 14 Feb 2022 08:06:12 +0000 (GMT) X-AuditID: b6c32a49-31b55a8000001917-52-620a195679c1 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 70.D1.29871.47D0A026; Mon, 14 Feb 2022 17:06:12 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080608epsmtip2df35a273e58e2cb2ae994af95e1eaa21~TmIDdugLd2250822508epsmtip2u; Mon, 14 Feb 2022 08:06:08 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , SelvaKumar S , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 03/10] block: Add copy offload support infrastructure Date: Mon, 14 Feb 2022 13:29:53 +0530 Message-Id: <20220214080002.18381-4-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjm3NveXtjQu6Ls0ITYFQzjUzqgHhCmi0CuYjImyxZJFCtcgQGl a4tsM9uEBvxgKh+CUIbUiVZ0AYHJykeZYNChVjAVRhFQBsQojg9RAQFZ4cLmv+d5zvOe9zzv yUvi/Ls8ARkvUzEKmTRRRNhwaq+7unl+6WCzz/tRFoYqb93AUaN5kosu950kUMH4DI7Gmge5 KPdkIQ+ZhlYhw2gxF3VMp2FosHoBQ42/5GKo/HIrhh7rzgHUcHYCQ0dvd2BobkCMWhf+IVBu SxdAw50aDBl63FGjoY2DTPU/E6j0wjAPZf2lJ1DTiAFHuptvMHRXM0cg/VAaQLWzpTi63t/J QRUjYxw0Mt1GoMyqlwBl/DTDQ+3zN7lbnGjT/TBa89BI0DnqUR5dp+nj0e39VRxare3l0CZj Cl196ShB15T9SOd16wDdYD5E0Ol3WnG68PkLgj6uHiXoieEeDj3W1EmE20cmBMYx0hhGIWRk 0ckx8bLYIFFYRNTWKD+Jt9hT7I82ioQyaRITJAreEe4ZGp9oGaJIeECamGKRwqVKpWjDx4GK 5BQVI4xLVqqCRIw8JlHuK/dSSpOUKbJYLxmjChB7e3/kZzHuTYhraTfz5M+++Kbn3jz3ENCH HgPWJKR8YU3PG+IYsCH5VAOA9/VVGEueA1jR3MdZdPGpVwBmPA5ZqSjpasJZkwHAtMEFDksy MJhfV2QhJElQ7vD2ArlYsIbiwPKpqSUPTp3hwfI/5sDigR1Fw4oXvfgi5lDrYXH54JJuSwXA 7pK/AdvNCZ4daOYuYmtqE7w2osNZz3uwrWho6XU4tQ6qrxbjrD/dBo5X7GdxMCytXVi+xw4+ vfkbj8UCODlqWMoMqSwAp+88xFhSCKA6W02wrs3wXuM8tpgGp1xhZf0GVnaE+bcqMLbxKnh8 dghjdVuoP7OCneCvldrlaxxg11TaMqZhs7qSy470BIAdD3ZmA6HmrTyat/Jo/u+sBfgl4MDI lUmxjNJPLpYxqf/9cnRyUjVYWiy3bXrQ92jcqwVgJGgBkMRFa2z3GK338W1jpN9+xyiSoxQp iYyyBfhZBp6DC9ZGJ1s2U6aKEvv6e/tKJBJffx+JWPS+7e3YK1I+FStVMQkMI2cUK3UYaS04 hJ24MjliH3BqxrnecXWq18U8gbOH+QfBPlm9+PuSq7OB8dXNblRuubw5VUuFnH71xNFwWF0k mAk4fzDH2vnTPx9szdx1vsgcPXZEItFlCnGFR7pUG2ma36yNsH+5Ov6inY/Lhzv4V7up4K+g JGzL542rgrvP7dZecx8W1F0L1JV9HVdmPJKXPWFl3eHe7xBYYKrv3YPymjJmhwcu9PvVhmRa 7axdGA85bQzT5H+QN7ZfYzYerPHr7KPmT/kwAR57Uw98Frw71PXs2j2R6+KYTe1mrdX09rJP nugPZ+4S9pFtox4Fz8aN5PobES661tdkkcvv7/rzVRuf1gbYv97+DqnsShRxlHFSsRuuUEr/ BdfCf+XhBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02SbUxTZxiGfc93m9UdC5SD07DUNOIHaFWyN5sjhBF9Iy7ZfswtDMQ6zwqT lqYtMjVGsFjXsk2Hm2KBoXOjUgkORKgWmBSYq1jAYKFlgEEguHXy4UAgCmyFLfHflVxX7ufP w+Dik+RKJk2t57VqRbqUEhI1TdLwSL1IuH/z074t8NrdX3FY5/ubhFf7TlPw3PgsDscaB0mY f7qAhp1Dy2H9aCEJO2ZyMDhYtYDBuh/yMVh2tQWDI9bLADouTWDQ1NqBwRcDctiy8ISC+c4u AIc9FgzW92yAdfUuAnbeKqJgSekwDfO67RRs8Nfj0HpnHoNtlhcUtA/lAFjzvASHTf0eAlb4 xwjon3FR0Fg5BeDJL2dp2D53h4xdgzofJCDLQzeFvjGM0uimpY9G7f2VBDJc7CVQpzsTVdlM FLr+43F01msFyOHLptCJey04Kng6SaGvDKMUmhjuIdBYg4d6T5Io3H6AT087xGs3xewTpjrb fbTmrz2f99yfI7OBfYcZCBiO3cYVdzXgZiBkxKwDcL1WH74kwrjSueb/OIgrmx+hlyIDxplq ZykzYBiK3cC1LjCBJpgluLLpaSLQ4OxvNDdWUk4HRBCLuIrJ3sUhgpVxhWWDIMAi9k3OW/wI LB1Yw10aaCQDLGDf4m77rYu9+N+mwOOkl/oVnOvCEBFgnA3nDDcK8TOAtbykLC+piwCzgTBe o1MpVTq5Rq7ms6J0CpUuU62M+iRDVQUWP2X9OjuotY1HOQHGACfgGFwaLNrrFuwXiw4oDh/h tRkp2sx0XucErzGENFTUYXaliFmlQs8f5HkNr/3fYoxgZTaWGl4TtK7I97p/16dHx1tVBtOf dJ47Wdbf/bXmUK1x5+9hs/zxPEGo/QrWdOrwnubYFcaiIxPdPdac7UR16XcPNv+ye/Xbabcm +Y9DvMtylO+kSG58oGnSflH0SkxXwpWkkJrz+tuW4mR126uiVSHHJLEpDcgzsFW2fCKD/Wnq gmR3W7Aqtys/9Ki+WpkceTO62LDMuzraXT4j2WhzVc4ffKPPyL4r+z6k8KO45izmYUd2TO6p qj/IEYfy2lRjfFLW4/hnkunIXNuHe983VehVhtjG2p2X196LSJx0RCcmGX+OSPxWtulRXP9n z50JO7aI5Gvjy737zjwuNVeHRazaZcbJOCmhS1XI1+NaneIfIKwiXZgDAAA= X-CMS-MailID: 20220214080612epcas5p2228606969011ce88a94d3b1be30d0614 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080612epcas5p2228606969011ce88a94d3b1be30d0614 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Introduce blkdev_issue_copy which supports source and destination bdevs, and an array of (source, destination and copy length) tuples. Introduce REQ_COPY copy offload operation flag. Create a read-write bio pair with a token as payload and submitted to the device in order. Read request populates token with source specific information which is then passed with write request. This design is courtesy Mikulas Patocka's token based copy Larger copy will be divided, based on max_copy_sectors, max_copy_range_sector limits. Signed-off-by: Nitesh Shetty Signed-off-by: SelvaKumar S Signed-off-by: Arnav Dawn --- block/blk-lib.c | 227 ++++++++++++++++++++++++++++++++++++++ block/blk.h | 2 + include/linux/blk_types.h | 21 ++++ include/linux/blkdev.h | 2 + include/uapi/linux/fs.h | 14 +++ 5 files changed, 266 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 1b8ced45e4e5..efa7e2a5fab7 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -135,6 +135,233 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, } EXPORT_SYMBOL(blkdev_issue_discard); +/* + * Wait on and process all in-flight BIOs. This must only be called once + * all bios have been issued so that the refcount can only decrease. + * This just waits for all bios to make it through bio_copy_end_io. IO + * errors are propagated through cio->io_error. + */ +static int cio_await_completion(struct cio *cio) +{ + int ret = 0; + unsigned long flags; + + spin_lock_irqsave(&cio->lock, flags); + if (cio->refcount) { + cio->waiter = current; + __set_current_state(TASK_UNINTERRUPTIBLE); + spin_unlock_irqrestore(&cio->lock, flags); + blk_io_schedule(); + /* wake up sets us TASK_RUNNING */ + spin_lock_irqsave(&cio->lock, flags); + cio->waiter = NULL; + ret = cio->io_err; + } + spin_unlock_irqrestore(&cio->lock, flags); + kvfree(cio); + + return ret; +} + +static void bio_copy_end_io(struct bio *bio) +{ + struct copy_ctx *ctx = bio->bi_private; + struct cio *cio = ctx->cio; + sector_t clen; + int ri = ctx->range_idx; + unsigned long flags; + + if (bio->bi_status) { + cio->io_err = bio->bi_status; + clen = (bio->bi_iter.bi_sector - ctx->start_sec) << SECTOR_SHIFT; + cio->rlist[ri].comp_len = min_t(sector_t, clen, cio->rlist[ri].comp_len); + } + __free_page(bio->bi_io_vec[0].bv_page); + kfree(ctx); + bio_put(bio); + + spin_lock_irqsave(&cio->lock, flags); + if (!--cio->refcount && cio->waiter) + wake_up_process(cio->waiter); + spin_unlock_irqrestore(&cio->lock, flags); +} + +/* + * blk_copy_offload - Use device's native copy offload feature + * Go through user provide payload, prepare new payload based on device's copy offload limits. + */ +int blk_copy_offload(struct block_device *src_bdev, int nr_srcs, + struct range_entry *rlist, struct block_device *dst_bdev, gfp_t gfp_mask) +{ + struct request_queue *sq = bdev_get_queue(src_bdev); + struct request_queue *dq = bdev_get_queue(dst_bdev); + struct bio *read_bio, *write_bio; + struct copy_ctx *ctx; + struct cio *cio; + struct page *token; + sector_t src_blk, copy_len, dst_blk; + sector_t remaining, max_copy_len = LONG_MAX; + unsigned long flags; + int ri = 0, ret = 0; + + cio = kzalloc(sizeof(struct cio), GFP_KERNEL); + if (!cio) + return -ENOMEM; + cio->rlist = rlist; + spin_lock_init(&cio->lock); + + max_copy_len = min_t(sector_t, sq->limits.max_copy_sectors, dq->limits.max_copy_sectors); + max_copy_len = min3(max_copy_len, (sector_t)sq->limits.max_copy_range_sectors, + (sector_t)dq->limits.max_copy_range_sectors) << SECTOR_SHIFT; + + for (ri = 0; ri < nr_srcs; ri++) { + cio->rlist[ri].comp_len = rlist[ri].len; + src_blk = rlist[ri].src; + dst_blk = rlist[ri].dst; + for (remaining = rlist[ri].len; remaining > 0; remaining -= copy_len) { + copy_len = min(remaining, max_copy_len); + + token = alloc_page(gfp_mask); + if (unlikely(!token)) { + ret = -ENOMEM; + goto err_token; + } + + ctx = kzalloc(sizeof(struct copy_ctx), gfp_mask); + if (!ctx) { + ret = -ENOMEM; + goto err_ctx; + } + ctx->cio = cio; + ctx->range_idx = ri; + ctx->start_sec = rlist[ri].src; + + read_bio = bio_alloc(src_bdev, 1, REQ_OP_READ | REQ_COPY | REQ_NOMERGE, + gfp_mask); + if (!read_bio) { + ret = -ENOMEM; + goto err_read_bio; + } + read_bio->bi_iter.bi_sector = src_blk >> SECTOR_SHIFT; + read_bio->bi_iter.bi_size = copy_len; + __bio_add_page(read_bio, token, PAGE_SIZE, 0); + ret = submit_bio_wait(read_bio); + bio_put(read_bio); + if (ret) + goto err_read_bio; + + write_bio = bio_alloc(dst_bdev, 1, REQ_OP_WRITE | REQ_COPY | REQ_NOMERGE, + gfp_mask); + if (!write_bio) { + ret = -ENOMEM; + goto err_read_bio; + } + write_bio->bi_iter.bi_sector = dst_blk >> SECTOR_SHIFT; + write_bio->bi_iter.bi_size = copy_len; + __bio_add_page(write_bio, token, PAGE_SIZE, 0); + write_bio->bi_end_io = bio_copy_end_io; + write_bio->bi_private = ctx; + + spin_lock_irqsave(&cio->lock, flags); + ++cio->refcount; + spin_unlock_irqrestore(&cio->lock, flags); + + submit_bio(write_bio); + src_blk += copy_len; + dst_blk += copy_len; + } + } + + /* Wait for completion of all IO's*/ + return cio_await_completion(cio); + +err_read_bio: + kfree(ctx); +err_ctx: + __free_page(token); +err_token: + rlist[ri].comp_len = min_t(sector_t, rlist[ri].comp_len, (rlist[ri].len - remaining)); + + cio->io_err = ret; + return cio_await_completion(cio); +} + +static inline int blk_copy_sanity_check(struct block_device *src_bdev, + struct block_device *dst_bdev, struct range_entry *rlist, int nr) +{ + unsigned int align_mask = max( + bdev_logical_block_size(dst_bdev), bdev_logical_block_size(src_bdev)) - 1; + sector_t len = 0; + int i; + + for (i = 0; i < nr; i++) { + if (rlist[i].len) + len += rlist[i].len; + else + return -EINVAL; + if ((rlist[i].dst & align_mask) || (rlist[i].src & align_mask) || + (rlist[i].len & align_mask)) + return -EINVAL; + rlist[i].comp_len = 0; + } + + if (len && len >= MAX_COPY_TOTAL_LENGTH) + return -EINVAL; + + return 0; +} + +static inline bool blk_check_copy_offload(struct request_queue *src_q, + struct request_queue *dest_q) +{ + if (blk_queue_copy(dest_q) && blk_queue_copy(src_q)) + return true; + + return false; +} + +/* + * blkdev_issue_copy - queue a copy + * @src_bdev: source block device + * @nr_srcs: number of source ranges to copy + * @rlist: array of source/dest/len + * @dest_bdev: destination block device + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * Description: + * Copy source ranges from source block device to destination block device. + * length of a source range cannot be zero. + */ +int blkdev_issue_copy(struct block_device *src_bdev, int nr, + struct range_entry *rlist, struct block_device *dest_bdev, gfp_t gfp_mask) +{ + struct request_queue *src_q = bdev_get_queue(src_bdev); + struct request_queue *dest_q = bdev_get_queue(dest_bdev); + int ret = -EINVAL; + + if (!src_q || !dest_q) + return -ENXIO; + + if (!nr) + return -EINVAL; + + if (nr >= MAX_COPY_NR_RANGE) + return -EINVAL; + + if (bdev_read_only(dest_bdev)) + return -EPERM; + + ret = blk_copy_sanity_check(src_bdev, dest_bdev, rlist, nr); + if (ret) + return ret; + + if (blk_check_copy_offload(src_q, dest_q)) + ret = blk_copy_offload(src_bdev, nr, rlist, dest_bdev, gfp_mask); + + return ret; +} +EXPORT_SYMBOL(blkdev_issue_copy); + /** * __blkdev_issue_write_same - generate number of bios with same page * @bdev: target blockdev diff --git a/block/blk.h b/block/blk.h index abb663a2a147..94d2b055750b 100644 --- a/block/blk.h +++ b/block/blk.h @@ -292,6 +292,8 @@ static inline bool blk_may_split(struct request_queue *q, struct bio *bio) break; } + if (unlikely(op_is_copy(bio->bi_opf))) + return false; /* * All drivers must accept single-segments bios that are <= PAGE_SIZE. * This is a quick and dirty check that relies on the fact that diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 5561e58d158a..e5c967c9f174 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -418,6 +418,7 @@ enum req_flag_bits { /* for driver use */ __REQ_DRV, __REQ_SWAP, /* swapping request. */ + __REQ_COPY, /* copy request*/ __REQ_NR_BITS, /* stops here */ }; @@ -442,6 +443,7 @@ enum req_flag_bits { #define REQ_DRV (1ULL << __REQ_DRV) #define REQ_SWAP (1ULL << __REQ_SWAP) +#define REQ_COPY (1ULL << __REQ_COPY) #define REQ_FAILFAST_MASK \ (REQ_FAILFAST_DEV | REQ_FAILFAST_TRANSPORT | REQ_FAILFAST_DRIVER) @@ -498,6 +500,11 @@ static inline bool op_is_discard(unsigned int op) return (op & REQ_OP_MASK) == REQ_OP_DISCARD; } +static inline bool op_is_copy(unsigned int op) +{ + return (op & REQ_COPY); +} + /* * Check if a bio or request operation is a zone management operation, with * the exception of REQ_OP_ZONE_RESET_ALL which is treated as a special case @@ -532,4 +539,18 @@ struct blk_rq_stat { u64 batch; }; +struct cio { + struct range_entry *rlist; + struct task_struct *waiter; /* waiting task (NULL if none) */ + spinlock_t lock; /* protects refcount and waiter */ + int refcount; + blk_status_t io_err; +}; + +struct copy_ctx { + int range_idx; + sector_t start_sec; + struct cio *cio; +}; + #endif /* __LINUX_BLK_TYPES_H */ diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 792e6d556589..1c2f65f1f143 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1121,6 +1121,8 @@ extern int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, struct bio **biop); struct bio *bio_map_kern(struct request_queue *q, void *data, unsigned int len, gfp_t gfp_mask); +int blkdev_issue_copy(struct block_device *src_bdev, int nr_srcs, + struct range_entry *src_rlist, struct block_device *dest_bdev, gfp_t gfp_mask); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index bdf7b404b3e7..55bca8f6e8ed 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -64,6 +64,20 @@ struct fstrim_range { __u64 minlen; }; +/* Maximum no of entries supported */ +#define MAX_COPY_NR_RANGE (1 << 12) + +/* maximum total copy length */ +#define MAX_COPY_TOTAL_LENGTH (1 << 21) + +/* Source range entry for copy */ +struct range_entry { + __u64 src; + __u64 dst; + __u64 len; + __u64 comp_len; +}; + /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */ #define FILE_DEDUPE_RANGE_SAME 0 #define FILE_DEDUPE_RANGE_DIFFERS 1 From patchwork Mon Feb 14 07:59:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33EF5C433FE for ; Mon, 14 Feb 2022 08:35:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242118AbiBNIgB (ORCPT ); Mon, 14 Feb 2022 03:36:01 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233030AbiBNIgA (ORCPT ); Mon, 14 Feb 2022 03:36:00 -0500 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DFA125C7A; Mon, 14 Feb 2022 00:35:53 -0800 (PST) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20220214083549epoutp0166504a6ec6c246ead4f88719420bb918~Tmh_qGFES1136711367epoutp01b; Mon, 14 Feb 2022 08:35:49 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20220214083549epoutp0166504a6ec6c246ead4f88719420bb918~Tmh_qGFES1136711367epoutp01b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827749; bh=yjC6zWenkLUSwu5qpAM0es03QhJaWyze1YcezW23Qd8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OIszECcYVZDIIuHq3HCXWK/DLpBHZ4JviqIUxDOXpNhDZwTWDGk+99eBNkJD75YHJ 3fzZG4ZK8M7ilhpJUItC5y1DnFqhqKdCyXdE1W5kpEqwZdre+rcIU8Wf6vBeYl1tVn Ma4CGOROIAkxCwPIu2mdr+KWNpAa6CXUrzFkwsRc= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20220214083549epcas5p2748eb531c83cc8bcc76037d15ed7a202~Tmh_JVqcQ0070900709epcas5p2Z; Mon, 14 Feb 2022 08:35:49 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.176]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JxyGl2mDgz4x9QY; Mon, 14 Feb 2022 08:35:43 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 0D.D6.06423.B541A026; Mon, 14 Feb 2022 17:35:39 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20220214080620epcas5p364a6e5bbf5ade9d115945d7b0cb1b947~TmIOhqSJ31726917269epcas5p36; Mon, 14 Feb 2022 08:06:20 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080620epsmtrp2b82da63aa6f748b02e74cd5c9c23f0c3~TmIOfljmA2569425694epsmtrp2U; Mon, 14 Feb 2022 08:06:20 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-31-620a145becb2 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 97.94.08738.B7D0A026; Mon, 14 Feb 2022 17:06:19 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080615epsmtip2a6c26a2edbfcb401039983a9d28e574e~TmIKDKmgP2389323893epsmtip2Q; Mon, 14 Feb 2022 08:06:15 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , =?utf-8?q?Javier_Gonz=C3=A1lez?= , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 04/10] block: Introduce a new ioctl for copy Date: Mon, 14 Feb 2022 13:29:54 +0530 Message-Id: <20220214080002.18381-5-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTZxT2vfdyW0iKd4DdC4uxK1kGGoXO0r0wEJOhu37gHASz4ARKuRZC abt+TNzUIQiLKIIwCBSdoAhRUQQ2RKCgODCAfCzlSwWVAS6KIgIKpgJrvbD57znneZ5z3nPe HC7u0MFx4cYodYxGKVUISTui6pb7p2t3O9lFenakrUJlrc04qrs7ZYMuDaaTKGfiDY5e3By2 QZnpuRxkGrFHxvF8G9Q1exhDwxULGKo7m4mhC5eaMPRPyTmAagtfWoiBKQ462taFobdDItS0 8JxEmY29AI32GDBkvLcG1RlbCGSqOUWiM8WjHHSsr5pE9WNGHJXcnsdQh+EtiapHDgNUZT6D o1sPegh0ZewFgVLKXwGUfPwNB3XO3bbZ6EqburfRhoftJH0yaZxDXzcMcujOB+UEnVQwQNCm dj1dcfEoSVcW/Uxn9ZcAuvZuAkkn3mnC6dzJaZJOSxon6Zej9wj6RX0PuZMfGusbzUijGI2A UcpUUTFKuZ9wW3D4l+FeEk/RWpE3+lwoUErjGD9hwPadazfHKCxLFAp+kCr0ltROqVYr9Njg q1HpdYwgWqXV+QkZdZRCLVav00rjtHqlfJ2S0fmIPD0/87III2KjZ6/lEOr+j+NP9JwmE0D9 R6nAlgspMTRXjhGpwI7rQNUCmHxjHLMSDtQkgEUZtiwxBeCz0cv4kqM78Q1giRoAU8xmkg2S MTjRkm5huFySWgPbFrhWgxNFwAszM4QV41QlB9blh1qxI+UPzXfLbayYoD6BHYZfgRXzKB/Y 9uDPxWausHDo5juNLfUFvDFWgrOaD2BL3shizVUw6Y983PoGSGXbwbyrmTasOQAeqVngsNgR Pr39+yJ2gVPjRpI1HANw9s5DjA1yAUzKSCJZlT/8q24Os06DU+6wrMaDTa+E2a1XMLazPUwz j2Bsngerf1vCrrC0rGCxjDPsnTm8iGlYntNMsPs9AWDDc0UGEBjeG8jw3kCG/zsXAPwicGbU 2jg5o/VSi5TMvv9+WaaKqwDvDmv1lmow+GhiXSPAuKARQC4udOKFtdtGOvCipPt/ZDSqcI1e wWgbgZdl4ydxlxUyleUylbpwkdjbUyyRSMTe6yUi4Ye8NvlVqQMll+qYWIZRM5olH8a1dUnA fLc2yfLsmydr97Tu7ig9EOzSfPBRll1DxfWggCf8gu/2HTLtKr4ZkbbstHEgtf/x0Pe/pIVs uu8f+TbeTcF3W75Hk3W2qUgzXDYndjqXH7uyN/j19P2WV6WCyJCrHv57V8TNizrT09f7DsXL gr2e+G5KiBRnycNcVQe/DTWJAq8suyf6W+Ie87QkfPbrZgWYSHzMf8rr3HvQiS4sZo5cTg9y fF06Hfa4vrZj+3nmfHDg8j6gL/IhGjc+/Olad9dxQV+hNsZPIDkQT58+VL91R+gO/vzykMTJ r+wbnrk5e0y7JxfF7KkaQru+ma8IlGX0TmzYHHFqTYrt2AC/r7tyY/b+AWOQkNBGS0WrcY1W +i+oMGJD4QQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA03SbUxTZxQH8D333t57aVK8VMVHjeiqBoIZk0mXJ7iBLzCfbGQxapzZkrlq rwVGS9daHRqzYrcZcU5FZHDxBV+AgC/YziHSFrAVTS0FJnZSR3FGGNMKBXEKUWQUYua3f87/ l3O+HJaUlohmsRmarbxOo8iS0WKqximb+85OiXjT4jKrHFXfvE4im29IhM76D9CocGCERMGr D0Qo/0ARg9q7w5G9v0SE2oZzCfTAMkYg26l8AlWebSJQb8VpgKwnB8eLziEG7XW3Eejl/XjU NNZHo3zHHwD1eAUC2e8uQja7i0LtdUdpdKK8h0H77tTSqD5gJ1HFjVcEahFe0qi2Oxegmhcn SOTs8lLoQiBIoR/N/wL0w08jDGodvSFaNh+33/4EC/c8ND5k6mfwFcHP4NYuM4VNpZ0UbvcY sKVqL41/PfMdPtxRAbDVZ6Tx7uYmEhc9eUrj/aZ+Gg/23KVwsN5Lr478XPyBks/K2Mbr3k36 Spw+fLmQ0na8/e3P3mO0EdTPzgNhLOQS4O3dIyAPiFkpVwvgrUETNVnMhOWj18jJPBVWvupl JpGJgIXujvGCZWluEXSPsSEzjaNg5fPnVMiQnJ+B5y9ZJhZN5ZLhC59ZFMoUtxC2CAUglCVc InR3vT4wH568f3XChHFLYWOgYmIuHTdFXgcz6SOgq7ibCt0luWhYfVwaGpPcXGj6rYQ8CCKE N5TwvxLeUKWArAIzea1erVLr47XvafjtcXqFWm/QqOI2Z6stYOJ/YmNrga1qIM4BCBY4AGRJ 2TTJl56wTVKJUpGzg9dlb9QZsni9A8xmKdkMSVuea6OUUym28l/zvJbXvW4JNmyWkXCW1F0L H854ttYpT9QEoj3aDn9kIGXOLzhdt74gMb741PX3z1XtW11dtP1yi5Ky1qTU/b3Y2fBxapLl iWrts4up0w+Otu7YkPo0uiX/yCXDwz2bM8vvaPJMsoSRNLvL2HAh5eGCoTLjUt9RzzZXUqRK cFu/2OWzNczwh7vtrd2Zhs+Kg47S6GVT7tE6/OcS65qYdSuPHBqzM8kr6h8dtxUsjPmnLnmd +aNVt/oWwK6c8u/NW1TsORP6K8exR+0/NrBkXlpjc7N8Dm78NKr5wz7lzrSyrLf8Db1bOlfK RuU3s6N+P/ONPJgbY7uo1JIRG3atiMKZHm4goBCtP+9YnnD4sYzSpyviY0mdXvEft6Oxdq4D AAA= X-CMS-MailID: 20220214080620epcas5p364a6e5bbf5ade9d115945d7b0cb1b947 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080620epcas5p364a6e5bbf5ade9d115945d7b0cb1b947 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add new BLKCOPY ioctl that offloads copying of one or more sources ranges to one or more destination in a device. COPY ioctl accepts a 'copy_range' structure that contains no of range, a reserved field , followed by an array of ranges. Each source range is represented by 'range_entry' that contains source start offset, destination start offset and length of source ranges (in bytes) MAX_COPY_NR_RANGE, limits the number of entries for the IOCTL and MAX_COPY_TOTAL_LENGTH limits the total copy length, IOCTL can handle. Example code, to issue BLKCOPY: /* Sample example to copy three entries with [dest,src,len], * [32768, 0, 4096] [36864, 4096, 4096] [40960,8192,4096] on same device */ int main(void) { int i, ret, fd; unsigned long src = 0, dst = 32768, len = 4096; struct copy_range *cr; cr = (struct copy_range *)malloc(sizeof(*cr)+ (sizeof(struct range_entry)*3)); cr->nr_range = 3; cr->reserved = 0; for (i = 0; i< cr->nr_range; i++, src += len, dst += len) { cr->range_list[i].dst = dst; cr->range_list[i].src = src; cr->range_list[i].len = len; cr->range_list[i].comp_len = 0; } fd = open("/dev/nvme0n1", O_RDWR); if (fd < 0) return 1; ret = ioctl(fd, BLKCOPY, cr); if (ret != 0) printf("copy failed, ret= %d\n", ret); for (i=0; i< cr->nr_range; i++) if (cr->range_list[i].len != cr->range_list[i].comp_len) printf("Partial copy for entry %d: requested %llu, completed %llu\n", i, cr->range_list[i].len, cr->range_list[i].comp_len); close(fd); free(cr); return ret; } Signed-off-by: Nitesh Shetty Signed-off-by: Javier González Signed-off-by: Arnav Dawn --- block/ioctl.c | 32 ++++++++++++++++++++++++++++++++ include/uapi/linux/fs.h | 9 +++++++++ 2 files changed, 41 insertions(+) diff --git a/block/ioctl.c b/block/ioctl.c index 4a86340133e4..a2dc2cfbae6d 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -124,6 +124,36 @@ static int blk_ioctl_discard(struct block_device *bdev, fmode_t mode, return err; } +static int blk_ioctl_copy(struct block_device *bdev, fmode_t mode, + unsigned long arg) +{ + struct copy_range crange, *ranges = NULL; + size_t payload_size = 0; + int ret; + + if (!(mode & FMODE_WRITE)) + return -EBADF; + + if (copy_from_user(&crange, (void __user *)arg, sizeof(crange))) + return -EFAULT; + + if (unlikely(!crange.nr_range || crange.reserved || crange.nr_range >= MAX_COPY_NR_RANGE)) + return -EINVAL; + + payload_size = (crange.nr_range * sizeof(struct range_entry)) + sizeof(crange); + + ranges = memdup_user((void __user *)arg, payload_size); + if (IS_ERR(ranges)) + return PTR_ERR(ranges); + + ret = blkdev_issue_copy(bdev, ranges->nr_range, ranges->range_list, bdev, GFP_KERNEL); + if (copy_to_user((void __user *)arg, ranges, payload_size)) + ret = -EFAULT; + + kfree(ranges); + return ret; +} + static int blk_ioctl_zeroout(struct block_device *bdev, fmode_t mode, unsigned long arg) { @@ -455,6 +485,8 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode, case BLKSECDISCARD: return blk_ioctl_discard(bdev, mode, arg, BLKDEV_DISCARD_SECURE); + case BLKCOPY: + return blk_ioctl_copy(bdev, mode, arg); case BLKZEROOUT: return blk_ioctl_zeroout(bdev, mode, arg); case BLKGETDISKSEQ: diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 55bca8f6e8ed..190911ea4311 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -78,6 +78,14 @@ struct range_entry { __u64 comp_len; }; +struct copy_range { + __u64 nr_range; + __u64 reserved; + + /* Range_list always must be at the end */ + struct range_entry range_list[]; +}; + /* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */ #define FILE_DEDUPE_RANGE_SAME 0 #define FILE_DEDUPE_RANGE_DIFFERS 1 @@ -199,6 +207,7 @@ struct fsxattr { #define BLKROTATIONAL _IO(0x12,126) #define BLKZEROOUT _IO(0x12,127) #define BLKGETDISKSEQ _IOR(0x12,128,__u64) +#define BLKCOPY _IOWR(0x12, 129, struct copy_range) /* * A jump here: 130-136 are reserved for zoned block devices * (see uapi/linux/blkzoned.h) From patchwork Mon Feb 14 07:59:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE41CC433F5 for ; Mon, 14 Feb 2022 08:45:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242375AbiBNIpf (ORCPT ); Mon, 14 Feb 2022 03:45:35 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:45700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242337AbiBNIpc (ORCPT ); Mon, 14 Feb 2022 03:45:32 -0500 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8619D50B23; Mon, 14 Feb 2022 00:45:24 -0800 (PST) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220214083556epoutp024dddc4b9d802bf82dc7061e2d9370414~TmiEs-XB41892418924epoutp02l; Mon, 14 Feb 2022 08:35:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220214083556epoutp024dddc4b9d802bf82dc7061e2d9370414~TmiEs-XB41892418924epoutp02l DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827756; bh=Jk1XBysYVAeAktqEvqRFtYvZSeJ08EcHL4/tvVzQpYk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gOH5w7EqZ4fXJqNZp4iW6ntkI/evicJpHCUMaE9jI6p/TRhdsdN371m1yFOy/zEz9 qTt5wChRi5SPKzdM+5+/kEqs5Qy0hlpDKYRX/OqRI01NMGp2K5guCL76uZWQkThBgM jFXQdsDt44aGkmklTl7Y46SttooWa9oNv+MzoEPM= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20220214083555epcas5p424b251263923f65aed745ef628764e64~TmiEW4rq60059600596epcas5p4R; Mon, 14 Feb 2022 08:35:55 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.175]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JxyGs4hCfz4x9Q8; Mon, 14 Feb 2022 08:35:49 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 00.F6.06423.1641A026; Mon, 14 Feb 2022 17:35:45 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20220214080627epcas5p11a8aef1b6aae05f61c7d241477bd11a3~TmIVGw70B0153001530epcas5p1J; Mon, 14 Feb 2022 08:06:27 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220214080627epsmtrp123429468577eabaf6365018f3ea1987c~TmIVFYbve0046200462epsmtrp1R; Mon, 14 Feb 2022 08:06:27 +0000 (GMT) X-AuditID: b6c32a49-b01ff70000001917-4c-620a146142fe Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id DE.D1.29871.28D0A026; Mon, 14 Feb 2022 17:06:26 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080622epsmtip28823f62221122673c71d380c4a49e123~TmIQp-CxL2250822508epsmtip24; Mon, 14 Feb 2022 08:06:22 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , Vincent Fu , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 05/10] block: add emulation for copy Date: Mon, 14 Feb 2022 13:29:55 +0530 Message-Id: <20220214080002.18381-6-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TaVBbVRjtfe/lJaFSnhTsLajQoIwUoURZLhTqQqfzoJ0OUmecOlgM8IY9 ySSh2jpa9qFQtiAOBgtUOkCLsomIbEoAKYS1bLayylIKIwFECEKNQIL23znnfuc78313Pg5u 3Mc244QKZYxEKIjgkQZETYvNK3YBJgYBDi2+qLzzFxw1PPiThUrH0kn0xfImjtTN0ywkT89h o4GZQ6hxKZeF+jQxGJqu0mKo4Ws5hu6UtmHoUXEhQPW3VjB0XdWHoe0pPmrT/kEiuXIYoNkh BYYaH9qihsYOAg3UfUWi/KJZNkoZqSVR02Ijjorb/8FQj2KbRLUzMQDVbOXjqGV8iEBli2oC JVb+BVDCjU02mk5JAqj3STvrTSt6YPAsrZjoJunMuCU2/aNijE33jlcSdFzBKEEPdEfRVXev k/R3t6/RWb8WA7r+QTRJx3a14XTO6hpJp8YtkfTK7EOCVjcNkT7PvR/uHsIIghiJJSMMFAWF CoM9eGcv+Hv6Ozk78O34rsiFZykURDIevNPnfOzOhEbs7JBneVkQEbUj+QikUt6JU+4SUZSM sQwRSWUePEYcFCF2FNtLBZHSKGGwvZCRufEdHF5z2in8MDwkuX+SJf6B93FMayeIBj+ZJwMu B1KOsKixhZ0MDDjGVD2Aaz2xLB1ZBVDbPg50ZB3AWM0quW9RTU7oLY0ApkyV6UkCBjfjf9+x cDgkZQtVWs6uwYQi4J2NDWK3BqcUbHhz63Ni9+Ew5QLbtDN7XQnqZfibZp61iw0pN5g9kqZP s4K3ppr3dC51Ev68WIzrap6FHV/O7PXBKQsY930uvhsAqUQDOLw+ytaZT8OlsixMhw/DhfZq vW4GH6cnsnWGFAA1XROYjuQAGJcRp49+A/Y3PMF2x8EpG1hed0InvwCzO8swXfIhmLo1ow8w hLV5+9gKflNeoG9zFA5vxOgxDScGlfp1pQG4pB0EGcBS8dREiqcmUvwfXQDwu+AoI5ZGBjNS JzFfyHz030cHiiKrwN5pHfeqBWOTy/ZKgHGAEkAOzjMxvNTNDTA2DBJcucpIRP6SqAhGqgRO OyvPxM1MA0U7tymU+fMdXR0cnZ2dHV1fd+bzjhiqgisExlSwQMaEM4yYkez7MA7XLBqLzH4r o2jN/bOMeKcFBeWZs1kSdOB5o/lP5mxG/UyzuFur3lYLBkn27fdC1YPn8l6yG8+q6L95Vf5t 55WhA67yiu0Q60DnIsxhjmu94FdwLHPEC9X4JXTFF4+VBxLnJxMtyCn0rrq2x9u45v7tkvww z9no9VWPD0Jerct9+7FvT+uL92SeqX62wrmLl7JUR5Lw0vDtT1vd5GqR9tijsBRvTe5SqzV2 odBCQ2ouelptVGb4v3OoR+leecro/hTuc9AryjA23F5krlq5Nv53NTqTWXQ5zCS3GfRyLWPn h33ZRoXGyhtNeeaBY5rNkyXLDKjucBmaP5/znkqdJn/m4BZmyiOkIQL+cVwiFfwLDbxU+eME AAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmplleLIzCtJLcpLzFFi42LZdlhJXreJlyvJYN5Ga4v1p44xW+y5+ZnV YvXdfjaLaR9+Mlu8P/iY1WJS/wx2i8tP+Cz2vpvNanHhRyOTxeNN/5ks9iyaxGSxcvVRJovn yxczWuxe+JHJovP0BSaLPw8NLY7+f8tmMenQNUaLp1dnMVnsvaVtsWfvSRaLy7vmsFnMX/aU 3aL7+g42i32v9zJbLD/+j8ni3Kw/bBY7njQyWmz7PZ/Z4vC9qywW616/Z7Fo2/iV0aK15ye7 xePuDkaL83+Pszooe1y+4u0x6/5ZNo+Jze/YPXbOusvucf7eRhaP5gV3WDwuny312LSqk81j 85J6j8k3ljN67L7ZwObRdOYos8eMT1/YPHqb37F5fHx6i8Xj/b6rbAFiUVw2Kak5mWWpRfp2 CVwZXRcfsBZsV6poPHKKsYFxv3QXIyeHhICJxOkH99m7GLk4hAR2M0os+fafGSIhKbHs7xEo W1hi5b/nUEXNTBLr3r9j6WLk4GAT0JY4/Z8DpEZEgEVi5ffvLCA1zAJH2CUOnVzEBJIQFjCX OPr/CRuIzSKgKnH7xwtWEJtXwEpi6vU+NogFyhILHx4Ei3MKWEsceL0cbLEQUM2Mq4fYIeoF JU7OfMICYjMLyEs0b53NPIFRYBaS1CwkqQWMTKsYJVMLinPTc4sNCwzzUsv1ihNzi0vz0vWS 83M3MYJTipbmDsbtqz7oHWJk4mA8xCjBwawkwht3ljNJiDclsbIqtSg/vqg0J7X4EKM0B4uS OO+FrpPxQgLpiSWp2ampBalFMFkmDk6pBqYV06+2Cysz/524mdN13+y8HYoJutsb6s/1xyS6 18ueWr9/7ansd75VU9yFg08mLv1yvG858961K8TyNe9bcpx08o/30w7/3tH//MHutgsOztsf Hd+wQzF44b5f9u+kZbbobTf6m/Ayaq3fJrfPmm+y3eesmR/RE1WifcVhTU3Es1K3UP2LXk15 fn27/GJ0Fc1cru5x7u45snnz5SViVWGSB6LCbb9+39qyfs2cl3tY1rrybTjdw3t5RalsT/ED zq/NPz8mPb37bcWh68Zn1j2L2N7HauT4q11k5qJZT+WdfZ75Jp5RVnP78kv72wKjzIcVfl9D cmP4r1zTk1Nc4ruRfYPMT0mNr7fTtVeeNn6kxFKckWioxVxUnAgAW7qeGZgDAAA= X-CMS-MailID: 20220214080627epcas5p11a8aef1b6aae05f61c7d241477bd11a3 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080627epcas5p11a8aef1b6aae05f61c7d241477bd11a3 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org For the devices which does not support copy, copy emulation is added. Copy-emulation is implemented by reading from source ranges into memory and writing to the corresponding destination synchronously. Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu --- block/blk-lib.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index efa7e2a5fab7..59c770814e72 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -286,6 +286,65 @@ int blk_copy_offload(struct block_device *src_bdev, int nr_srcs, return cio_await_completion(cio); } +int blk_submit_rw_buf(struct block_device *bdev, void *buf, sector_t buf_len, + sector_t sector, unsigned int op, gfp_t gfp_mask) +{ + struct request_queue *q = bdev_get_queue(bdev); + struct bio *bio, *parent = NULL; + sector_t max_hw_len = min_t(unsigned int, queue_max_hw_sectors(q), + queue_max_segments(q) << (PAGE_SHIFT - SECTOR_SHIFT)) << SECTOR_SHIFT; + sector_t len, remaining; + int ret; + + for (remaining = buf_len; remaining > 0; remaining -= len) { + len = min_t(int, max_hw_len, remaining); +retry: + bio = bio_map_kern(q, buf, len, gfp_mask); + if (IS_ERR(bio)) { + len >>= 1; + if (len) + goto retry; + return PTR_ERR(bio); + } + + bio->bi_iter.bi_sector = sector >> SECTOR_SHIFT; + bio->bi_opf = op; + bio_set_dev(bio, bdev); + bio->bi_end_io = NULL; + bio->bi_private = NULL; + + if (parent) { + bio_chain(parent, bio); + submit_bio(parent); + } + parent = bio; + sector += len; + buf = (char *) buf + len; + } + ret = submit_bio_wait(bio); + bio_put(bio); + + return ret; +} + +static void *blk_alloc_buf(sector_t req_size, sector_t *alloc_size, gfp_t gfp_mask) +{ + int min_size = PAGE_SIZE; + void *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp_mask); + if (buf) { + *alloc_size = req_size; + return buf; + } + /* retry half the requested size */ + req_size >>= 1; + } + + return NULL; +} + static inline int blk_copy_sanity_check(struct block_device *src_bdev, struct block_device *dst_bdev, struct range_entry *rlist, int nr) { @@ -311,6 +370,64 @@ static inline int blk_copy_sanity_check(struct block_device *src_bdev, return 0; } +static inline sector_t blk_copy_max_range(struct range_entry *rlist, int nr, sector_t *max_len) +{ + int i; + sector_t len = 0; + + *max_len = 0; + for (i = 0; i < nr; i++) { + *max_len = max(*max_len, rlist[i].len); + len += rlist[i].len; + } + + return len; +} + +/* + * If native copy offload feature is absent, this function tries to emulate, + * by copying data from source to a temporary buffer and from buffer to + * destination device. + */ +static int blk_copy_emulate(struct block_device *src_bdev, int nr, + struct range_entry *rlist, struct block_device *dest_bdev, gfp_t gfp_mask) +{ + void *buf = NULL; + int ret, nr_i = 0; + sector_t src, dst, copy_len, buf_len, read_len, copied_len, max_len = 0, remaining = 0; + + copy_len = blk_copy_max_range(rlist, nr, &max_len); + buf = blk_alloc_buf(max_len, &buf_len, gfp_mask); + if (!buf) + return -ENOMEM; + + for (copied_len = 0; copied_len < copy_len; copied_len += read_len) { + if (!remaining) { + rlist[nr_i].comp_len = 0; + src = rlist[nr_i].src; + dst = rlist[nr_i].dst; + remaining = rlist[nr_i++].len; + } + + read_len = min_t(sector_t, remaining, buf_len); + ret = blk_submit_rw_buf(src_bdev, buf, read_len, src, REQ_OP_READ, gfp_mask); + if (ret) + goto out; + src += read_len; + remaining -= read_len; + ret = blk_submit_rw_buf(dest_bdev, buf, read_len, dst, REQ_OP_WRITE, + gfp_mask); + if (ret) + goto out; + else + rlist[nr_i - 1].comp_len += read_len; + dst += read_len; + } +out: + kvfree(buf); + return ret; +} + static inline bool blk_check_copy_offload(struct request_queue *src_q, struct request_queue *dest_q) { @@ -357,6 +474,8 @@ int blkdev_issue_copy(struct block_device *src_bdev, int nr, if (blk_check_copy_offload(src_q, dest_q)) ret = blk_copy_offload(src_bdev, nr, rlist, dest_bdev, gfp_mask); + else + ret = blk_copy_emulate(src_bdev, nr, rlist, dest_bdev, gfp_mask); return ret; } From patchwork Mon Feb 14 07:59:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02AD0C4332F for ; Mon, 14 Feb 2022 08:36:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232622AbiBNIgP (ORCPT ); Mon, 14 Feb 2022 03:36:15 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242076AbiBNIgM (ORCPT ); Mon, 14 Feb 2022 03:36:12 -0500 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7188F5FF0B; Mon, 14 Feb 2022 00:36:03 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20220214083601epoutp03cb7838a422ed0b459923802b93a7704a~TmiJ9miDc0925309253epoutp03I; Mon, 14 Feb 2022 08:36:01 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20220214083601epoutp03cb7838a422ed0b459923802b93a7704a~TmiJ9miDc0925309253epoutp03I DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827761; bh=ZvNkb0zuJHNi8XxnkFtd8J/B7UB6bfOt+vOmPP9/8M4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l9SU+rybvacOePWrTuWT5NhPSZX2rvEOTuo+AL5RCHodsYFmrYt/BqK4b+E9rNZkd SNACwkFcp2DCInyIyC0cV3J/EP0PzLLdpdIkEVCGVPrqF8IJnqnllzzEWkzAtOCs0t Yow7tGqPtT1c9AcGPAPOc+3CkAnr9dOVR8yt4dgU= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20220214083601epcas5p15c66ffc22cd51e1cb1cfb18d2985b538~TmiJciE400265302653epcas5p1G; Mon, 14 Feb 2022 08:36:01 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.181]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4JxyGx1Gn9z4x9QQ; Mon, 14 Feb 2022 08:35:53 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id CC.C7.46822.AC31A026; Mon, 14 Feb 2022 17:33:14 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20220214080634epcas5p40b2c60e959b89355a25db7c69219d039~TmIcJwx7A1303713037epcas5p4k; Mon, 14 Feb 2022 08:06:34 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080634epsmtrp250835e480bc2ea21d3ba017bccabe385~TmIcIb8Iw2568325683epsmtrp29; Mon, 14 Feb 2022 08:06:34 +0000 (GMT) X-AuditID: b6c32a4a-de5ff7000000b6e6-b8-620a13ca091c Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id CE.E1.29871.A8D0A026; Mon, 14 Feb 2022 17:06:34 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080629epsmtip2b4be61fccf0cdbc69ebf8b410989e239~TmIXUfyE72320123201epsmtip2o; Mon, 14 Feb 2022 08:06:29 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, SelvaKumar S , Nitesh Shetty , =?utf-8?q?Javier_Gonz=C3=A1lez?= , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 06/10] nvme: add copy offload support Date: Mon, 14 Feb 2022 13:29:56 +0530 Message-Id: <20220214080002.18381-7-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02TeVCUdRjH5/e+y7sLufQKmD9hqp2lHJE4lgB/CEqpxTvglA02NTYFr/AG juwxe6Q5kxw7oHLEVQSLAaKGHInLNVzLIIQIuAGBXHEOEBQjl3IFtu2yS/nf5zm+z/N7nt88 HNyql23LOSeSM1IRHcEnLFhVzQ4OTu3WFmddx5Z8UGn7AxzVDz41Q8UjyQTKWNzA0cL9STOU lpzJRj1Tlkgzn22GutajMTRZpsNQfX4ahgqLWzA0U3AToLobS/rA8FM2utbRhaGtCQFq0T0h UFpTH0DTj1UY0gw5onpNGwv11F4nUO5P02yU0F9NoIY5DY4KWv/B0K+qLQJVT0UDVLWZi6Pm 0ccsdHdugYXm1tsIFKdeASg2cYONOp+3mr3zBtXTG0CpxrQElaqcZ1M1qhE21TmqZlHKvGEW 1aNVUGVF1wiq/FYklT5QAKi6wSiCinnUglOZy88IKkk5T1BL00MsaqHhMXHqlTPnfcIZOpSR 8hhRiDj0nCjsCD8gMOh4kIenq8BJ4IUO8XkiWsgc4Z84ecrp/XMR+k3yeV/REQq96xQtk/Fd jvpIxQo5wwsXy+RH+IwkNELiLnGW0UKZQhTmLGLkhwWurm4e+sTg8+Hp6hlcsiK+WJEzCKLA 5pl4YM6BpDv8WVuCG9iKrAOw96FbPLDQ8zKAWpUSGI1VAJVxC+wdRaKunTAGNACOVlaZGY1Y DPatleizOByCdIQdOo5BYEOyYOHaGsvAODnKhvmphIGtSQTju38zMzCLfBNmZOi2mUsehv1x faZm9vDGxP1tvznpDRvnCnBjzm7YljVlqvk6VFZm44Y3QDLLAkY3LBFG8Qn4y80/MCNbw79a K0xFbeHTeQ1hFCQAuP5oDDMamfo5U5QmtS/srn+OGabBSQdYWutidL8Kv2+/ixk7W8KkzSlT Ay6sztlhe1hSmmcqs0+/lGgTU7Cpdg0zbutbALM2y4kUwFO9MJHqhYlU/7fOA3gR2MdIZMIw RuYhcRMxF/775hCxsAxsn9dB/2owMb7o3AQwDmgCkIPzbbhfaM3PWnFD6a8vMVJxkFQRwcia gId+5am47Z4Qsf4+RfIggbuXq7unp6e719ueAv5ebkfYPdqKDKPlzHmGkTDSHR3GMbeNwi67 FfuFfaDz8LualDAm2G3XKqRuf9f80pOuko/wQyXt5eWXtwobq6veu3PcPzi9RBtpyWsZd2j0 3tunthnKX69mF9X6lH7I764v5Vmbcy8NfnpRsal+a/+uurWte9y0K7ETs/uTf7SdXg1KHvj4 ZqGYvj57IffAJhbpfNoqcvwH+Z4UbZpdLzv9c3XnMzK45qjL6YBjKq7jZwL3hsyEb0YqYl6z K1Q9HIkJFPo8J4Sf/LlxoIFJ+TJ74N2YQIsHih5/oqr8b/txgQXdPxKzarncNflyZahNTuLV FWtLVnSNn2Y4de5kiG/5SMIddfYu8xVnqe+s8+3f7Y95XPG2m5nTZdxa5LNk4bTgIC6V0f8C dY2/yecEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA02Sf1CTdRzH+z7Ps2fPpsvHQfUd3oFO/J3QrrIPgiWn5aPmHWV/2dmc8ITW frUJKV0xXGJMCBnEzeEJGscCPAgoRAYmI/Dm+BmiYiB4gOTtAgU5IX8sxq7L/96f9+t19/7n w5DSIkEIc1B7iDdoVWo5LaZqm+Vh6y0S8f7Xim2hUHm1lYSGvikBlA9k05B/f5aEiaZhAViz bULoGXkRGscLBNA1k0bAcLWPgIZzVgJKy1sIGHP8iMB59sEc6J8SQoani4AndxTQ4vubBqvr OoLRXjsBjbfWQUOjm4Ke+tM0FJaMCuHEjToaLnkbSXBceUZAh/0JDXUjaQhqHxeS0Hy7l4IK 7wQF3hk3DelV0wiOZc4KofPpFcHmcK7n2k7OPthOcznmcSF30T4g5DpvV1Gcuaif4nrak7jq sgyaqylO5XJvOhDn7DPR3NG2FpKzTT6kuSzzOM09GL1FcROXeum4l/eIYxJ49cFk3hD59j7x gdyqMVI/rTv8y5k+ZEKP91iQiMHsGzjTd5W2IDEjZZ0IZ3dXEgEgwyVPfycDOQiXPhsTBiQz gWeGJuYOhqHZddjjY/xOMEvh0kePKL9Dst8xuNMxjPwgiAVs6f5D4M8UuwLn5/vms4TdiG+k XxcGBpbjs3ea5nsRG40vex3zw9I5x9brEgb8xdh9aoTy75LsKlx5RuqvSTYMm38tIE+ixfbn LPv/lv05qwiRZUjG642aRI1RoVdo+S8jjCqNMUmbGBGv01Sj+Sdau6YOXSi7H+FCBINcCDOk PFjySbtov1SSoDqSwht0SkOSmje60BKGkr8i6bK4lVI2UXWI/5zn9bzhP0owohATkbL56Pt/ /WbC6bG7v/W+FbFs7w+ChLsLD6dsNcs+5gvfCz25+txSzdiRDVMyz+vimC8indWJ94ZQYUrW Al3TYIL6+MXwhbMhqlc7oo6N197cy47fbR2cyfloW8KHP9Wpc995Nya444NJremr0JJdmV9X uEc9+5Tecktxf+rKvCi5/MLutoH6Jc6O9TnpMtPk+U+jCvJPfBb9MGMlDNmbFNZrS7ktsfUL arlNtjWrfo6Pxot2eNpemk4eTm6W6WLDlG/uisoKt0S2Lftme8UmZ8z57Xl/itIuF+Vt2LnC fS8O8qy5g/GymqGc6VPfdwfXbAxq2UYsSlX+c1r5guD46v5WfdzWULecMh5QKdaSBqPqXxM/ KmezAwAA X-CMS-MailID: 20220214080634epcas5p40b2c60e959b89355a25db7c69219d039 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080634epcas5p40b2c60e959b89355a25db7c69219d039 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: SelvaKumar S For device supporting native copy, nvme driver receives read and write request with BLK_COPY op flags. For read request the nvme driver populates the payload with source information. For write request the driver converts it to nvme copy command using the source information in the payload and submits to the device. current design only supports single source range. This design is courtesy Mikulas Patocka's token based copy trace event support for nvme_copy_cmd. Set the device copy limits to queue limits. Signed-off-by: SelvaKumar S Signed-off-by: Kanchan Joshi Signed-off-by: Nitesh Shetty Signed-off-by: Javier González Signed-off-by: Arnav Dawn --- drivers/nvme/host/core.c | 119 +++++++++++++++++++++++++++++++++++++- drivers/nvme/host/fc.c | 4 ++ drivers/nvme/host/nvme.h | 7 +++ drivers/nvme/host/pci.c | 9 +++ drivers/nvme/host/rdma.c | 6 ++ drivers/nvme/host/tcp.c | 8 +++ drivers/nvme/host/trace.c | 19 ++++++ include/linux/nvme.h | 43 +++++++++++++- 8 files changed, 210 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 961a5f8a44d2..731a091f4bc3 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -821,6 +821,90 @@ static inline void nvme_setup_flush(struct nvme_ns *ns, cmnd->common.nsid = cpu_to_le32(ns->head->ns_id); } +static inline blk_status_t nvme_setup_copy_read(struct nvme_ns *ns, struct request *req) +{ + struct bio *bio = req->bio; + struct nvme_copy_token *token = bvec_kmap_local(&bio->bi_io_vec[0]); + + memcpy(token->subsys, "nvme", 4); + token->ns = ns; + token->src_sector = bio->bi_iter.bi_sector; + token->sectors = bio->bi_iter.bi_size >> 9; + + return 0; +} + +static inline blk_status_t nvme_setup_copy_write(struct nvme_ns *ns, + struct request *req, struct nvme_command *cmnd) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct nvme_copy_range *range = NULL; + struct bio *bio = req->bio; + struct nvme_copy_token *token = bvec_kmap_local(&bio->bi_io_vec[0]); + sector_t src_sector, dst_sector, n_sectors; + u64 src_lba, dst_lba, n_lba; + unsigned short nr_range = 1; + u16 control = 0; + u32 dsmgmt = 0; + + if (unlikely(memcmp(token->subsys, "nvme", 4))) + return BLK_STS_NOTSUPP; + if (unlikely(token->ns != ns)) + return BLK_STS_NOTSUPP; + + src_sector = token->src_sector; + dst_sector = bio->bi_iter.bi_sector; + n_sectors = token->sectors; + if (WARN_ON(n_sectors != bio->bi_iter.bi_size >> 9)) + return BLK_STS_NOTSUPP; + + src_lba = nvme_sect_to_lba(ns, src_sector); + dst_lba = nvme_sect_to_lba(ns, dst_sector); + n_lba = nvme_sect_to_lba(ns, n_sectors); + + if (unlikely(nvme_lba_to_sect(ns, src_lba) != src_sector) || + unlikely(nvme_lba_to_sect(ns, dst_lba) != dst_sector) || + unlikely(nvme_lba_to_sect(ns, n_lba) != n_sectors)) + return BLK_STS_NOTSUPP; + + if (WARN_ON(!n_lba)) + return BLK_STS_NOTSUPP; + + if (req->cmd_flags & REQ_FUA) + control |= NVME_RW_FUA; + + if (req->cmd_flags & REQ_FAILFAST_DEV) + control |= NVME_RW_LR; + + memset(cmnd, 0, sizeof(*cmnd)); + cmnd->copy.opcode = nvme_cmd_copy; + cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id); + cmnd->copy.sdlba = cpu_to_le64(blk_rq_pos(req) >> (ns->lba_shift - 9)); + + range = kmalloc_array(nr_range, sizeof(*range), + GFP_ATOMIC | __GFP_NOWARN); + if (!range) + return BLK_STS_RESOURCE; + + range[0].slba = cpu_to_le64(src_lba); + range[0].nlb = cpu_to_le16(n_lba - 1); + + cmnd->copy.nr_range = 0; + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * nr_range; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + if (ctrl->nr_streams) + nvme_assign_write_stream(ctrl, req, &control, &dsmgmt); + + cmnd->copy.control = cpu_to_le16(control); + cmnd->copy.dspec = cpu_to_le32(dsmgmt); + + return BLK_STS_OK; +} + static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req, struct nvme_command *cmnd) { @@ -1024,10 +1108,16 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req) ret = nvme_setup_discard(ns, req, cmd); break; case REQ_OP_READ: - ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_read); + if (unlikely(req->cmd_flags & REQ_COPY)) + ret = nvme_setup_copy_read(ns, req); + else + ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_read); break; case REQ_OP_WRITE: - ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_write); + if (unlikely(req->cmd_flags & REQ_COPY)) + ret = nvme_setup_copy_write(ns, req, cmd); + else + ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_write); break; case REQ_OP_ZONE_APPEND: ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_zone_append); @@ -1682,6 +1772,29 @@ static void nvme_config_discard(struct gendisk *disk, struct nvme_ns *ns) blk_queue_max_write_zeroes_sectors(queue, UINT_MAX); } +static void nvme_config_copy(struct gendisk *disk, struct nvme_ns *ns, + struct nvme_id_ns *id) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct request_queue *q = disk->queue; + + if (!(ctrl->oncs & NVME_CTRL_ONCS_COPY)) { + blk_queue_max_copy_sectors(q, 0); + blk_queue_max_copy_range_sectors(q, 0); + blk_queue_max_copy_nr_ranges(q, 0); + blk_queue_flag_clear(QUEUE_FLAG_COPY, q); + return; + } + + /* setting copy limits */ + if (blk_queue_flag_test_and_set(QUEUE_FLAG_COPY, q)) + return; + + blk_queue_max_copy_sectors(q, nvme_lba_to_sect(ns, le32_to_cpu(id->mcl))); + blk_queue_max_copy_range_sectors(q, nvme_lba_to_sect(ns, le16_to_cpu(id->mssrl))); + blk_queue_max_copy_nr_ranges(q, id->msrc + 1); +} + static bool nvme_ns_ids_valid(struct nvme_ns_ids *ids) { return !uuid_is_null(&ids->uuid) || @@ -1864,6 +1977,7 @@ static void nvme_update_disk_info(struct gendisk *disk, nvme_config_discard(disk, ns); blk_queue_max_write_zeroes_sectors(disk->queue, ns->ctrl->max_zeroes_sectors); + nvme_config_copy(disk, ns, id); set_disk_ro(disk, (id->nsattr & NVME_NS_ATTR_RO) || test_bit(NVME_NS_FORCE_RO, &ns->flags)); @@ -4728,6 +4842,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64); BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64); + BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64); BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64); diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 71b3108c22f0..5057ab1a1875 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2788,6 +2788,10 @@ nvme_fc_queue_rq(struct blk_mq_hw_ctx *hctx, if (ret) return ret; + if (unlikely((rq->cmd_flags & REQ_COPY) && (req_op(rq) == REQ_OP_READ))) { + blk_mq_end_request(rq, BLK_STS_OK); + return BLK_STS_OK; + } /* * nvme core doesn't quite treat the rq opaquely. Commands such * as WRITE ZEROES will return a non-zero rq payload_bytes yet diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index a162f6c6da6e..117658a8cf5f 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -474,6 +474,13 @@ struct nvme_ns { }; +struct nvme_copy_token { + char subsys[4]; + struct nvme_ns *ns; + u64 src_sector; + u64 sectors; +}; + /* NVMe ns supports metadata actions by the controller (generate/strip) */ static inline bool nvme_ns_has_pi(struct nvme_ns *ns) { diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 6a99ed680915..a7b0f129a19d 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -916,6 +916,11 @@ static blk_status_t nvme_prep_rq(struct nvme_dev *dev, struct request *req) if (ret) return ret; + if (unlikely((req->cmd_flags & REQ_COPY) && (req_op(req) == REQ_OP_READ))) { + blk_mq_end_request(req, BLK_STS_OK); + return BLK_STS_OK; + } + if (blk_rq_nr_phys_segments(req)) { ret = nvme_map_data(dev, req, &iod->cmd); if (ret) @@ -929,6 +934,7 @@ static blk_status_t nvme_prep_rq(struct nvme_dev *dev, struct request *req) } blk_mq_start_request(req); + return BLK_STS_OK; out_unmap_data: nvme_unmap_data(dev, req); @@ -962,6 +968,9 @@ static blk_status_t nvme_queue_rq(struct blk_mq_hw_ctx *hctx, ret = nvme_prep_rq(dev, req); if (unlikely(ret)) return ret; + if (unlikely((req->cmd_flags & REQ_COPY) && (req_op(req) == REQ_OP_READ))) + return ret; + spin_lock(&nvmeq->sq_lock); nvme_sq_copy_cmd(nvmeq, &iod->cmd); nvme_write_sq_db(nvmeq, bd->last); diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 9c55e4be8a39..060abf310fb8 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -2070,6 +2070,12 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, if (ret) goto unmap_qe; + if (unlikely((rq->cmd_flags & REQ_COPY) && (req_op(rq) == REQ_OP_READ))) { + blk_mq_end_request(rq, BLK_STS_OK); + ret = BLK_STS_OK; + goto unmap_qe; + } + blk_mq_start_request(rq); if (IS_ENABLED(CONFIG_BLK_DEV_INTEGRITY) && diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index 01e24b5703db..c36b727384a8 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -2315,6 +2315,11 @@ static blk_status_t nvme_tcp_setup_cmd_pdu(struct nvme_ns *ns, if (ret) return ret; + if (unlikely((rq->cmd_flags & REQ_COPY) && (req_op(rq) == REQ_OP_READ))) { + blk_mq_end_request(rq, BLK_STS_OK); + return BLK_STS_OK; + } + req->state = NVME_TCP_SEND_CMD_PDU; req->status = cpu_to_le16(NVME_SC_SUCCESS); req->offset = 0; @@ -2380,6 +2385,9 @@ static blk_status_t nvme_tcp_queue_rq(struct blk_mq_hw_ctx *hctx, if (unlikely(ret)) return ret; + if (unlikely((rq->cmd_flags & REQ_COPY) && (req_op(rq) == REQ_OP_READ))) + return ret; + blk_mq_start_request(rq); nvme_tcp_queue_request(req, true, bd->last); diff --git a/drivers/nvme/host/trace.c b/drivers/nvme/host/trace.c index 2a89c5aa0790..ab72bf546a13 100644 --- a/drivers/nvme/host/trace.c +++ b/drivers/nvme/host/trace.c @@ -150,6 +150,23 @@ static const char *nvme_trace_read_write(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvme_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 slba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "slba=%llu, nr_range=%u, ctrl=0x%x, dsmgmt=%u, reftag=%u", + slba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvme_trace_dsm(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -243,6 +260,8 @@ const char *nvme_trace_parse_nvm_cmd(struct trace_seq *p, return nvme_trace_zone_mgmt_send(p, cdw10); case nvme_cmd_zone_mgmt_recv: return nvme_trace_zone_mgmt_recv(p, cdw10); + case nvme_cmd_copy: + return nvme_trace_copy(p, cdw10); default: return nvme_trace_common(p, cdw10); } diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 855dd9b3e84b..7ed966058f4c 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -309,7 +309,7 @@ struct nvme_id_ctrl { __u8 nvscc; __u8 nwpc; __le16 acwu; - __u8 rsvd534[2]; + __le16 ocfs; __le32 sgls; __le32 mnan; __u8 rsvd544[224]; @@ -335,6 +335,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_ONCS_RESERVATIONS = 1 << 5, NVME_CTRL_ONCS_TIMESTAMP = 1 << 6, + NVME_CTRL_ONCS_COPY = 1 << 8, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, NVME_CTRL_OACS_DIRECTIVES = 1 << 5, @@ -383,7 +384,10 @@ struct nvme_id_ns { __le16 npdg; __le16 npda; __le16 nows; - __u8 rsvd74[18]; + __le16 mssrl; + __le32 mcl; + __u8 msrc; + __u8 rsvd91[11]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -704,6 +708,7 @@ enum nvme_opcode { nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, nvme_cmd_resv_release = 0x15, + nvme_cmd_copy = 0x19, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, nvme_cmd_zone_append = 0x7d, @@ -725,7 +730,8 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ - nvme_opcode_name(nvme_cmd_zone_append)) + nvme_opcode_name(nvme_cmd_zone_append), \ + nvme_opcode_name(nvme_cmd_copy)) @@ -898,6 +904,36 @@ struct nvme_dsm_range { __le64 slba; }; +struct nvme_copy_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2; + __le64 metadata; + union nvme_data_ptr dptr; + __le64 sdlba; + __u8 nr_range; + __u8 rsvd12; + __le16 control; + __le16 rsvd13; + __le16 dspec; + __le32 ilbrt; + __le16 lbat; + __le16 lbatm; +}; + +struct nvme_copy_range { + __le64 rsvd0; + __le64 slba; + __le16 nlb; + __le16 rsvd18; + __le32 rsvd20; + __le32 eilbrt; + __le16 elbat; + __le16 elbatm; +}; + struct nvme_write_zeroes_cmd { __u8 opcode; __u8 flags; @@ -1449,6 +1485,7 @@ struct nvme_command { struct nvme_download_firmware dlfw; struct nvme_format_cmd format; struct nvme_dsm_cmd dsm; + struct nvme_copy_command copy; struct nvme_write_zeroes_cmd write_zeroes; struct nvme_zone_mgmt_send_cmd zms; struct nvme_zone_mgmt_recv_cmd zmr; From patchwork Mon Feb 14 07:59:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC8A8C433FE for ; Mon, 14 Feb 2022 08:36:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242128AbiBNIgS (ORCPT ); Mon, 14 Feb 2022 03:36:18 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242059AbiBNIgP (ORCPT ); Mon, 14 Feb 2022 03:36:15 -0500 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25A1C25C7F; Mon, 14 Feb 2022 00:36:08 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20220214083606epoutp039281e2ae9164ad7dc8fe15768b7721ce~TmiOXmiKu0830708307epoutp03l; Mon, 14 Feb 2022 08:36:06 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20220214083606epoutp039281e2ae9164ad7dc8fe15768b7721ce~TmiOXmiKu0830708307epoutp03l DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827766; bh=Z9Nyys2UP6FAeY8io8HElJ9II7fdwdaF85FhVinN+aE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BHL0Zc8+sFoTXtvQXuRrNTTwq9Ym+GmzpryJ9ItD4dLE7o0PA+ePB8qH3N3rtCGlQ /bCY5WSUwmFYuoebXOB/cmeKDfdaPvOpLc4uDyaDW+FKalINzRHhtn2I0ut147I+C+ mcEm5MK5SiGr2j/VFVUEs+ozZ4MUk0ozaMk9jtU0= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20220214083606epcas5p4e61537cf556f825454c8696b45416828~TmiN73zh02392823928epcas5p4z; Mon, 14 Feb 2022 08:36:06 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JxyH50264z4x9QT; Mon, 14 Feb 2022 08:36:01 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 7C.46.05590.E641A026; Mon, 14 Feb 2022 17:35:58 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20220214080641epcas5p4662e8d0c86f93d525032067cc039c7af~TmIix_KqB1267812678epcas5p4L; Mon, 14 Feb 2022 08:06:41 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220214080641epsmtrp1fac2e9de9dda32a0897d456a1ecab2ca~TmIixC6dL3265432654epsmtrp1P; Mon, 14 Feb 2022 08:06:41 +0000 (GMT) X-AuditID: b6c32a4b-739ff700000015d6-b4-620a146e47e5 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 02.F1.29871.19D0A026; Mon, 14 Feb 2022 17:06:41 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080636epsmtip2516464898b02ee0a0722eb6af56da7cb~TmIeTvagw2207122071epsmtip2b; Mon, 14 Feb 2022 08:06:36 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 07/10] nvmet: add copy command support for bdev and file ns Date: Mon, 14 Feb 2022 13:29:57 +0530 Message-Id: <20220214080002.18381-8-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TaVBTVxj1vvfyXsIU5hFQL1RbGsYfbCHRBC8C1lrbeVNah0oHOs60NMAb QEOSJqF0mSlbgbLJ0qIxMGy2ZUTLJiqVgIKlbIJSDAgjULaqZQoIlX0pIdj673zfPedbzp2P i/PbKHtuhELLqhUyuYC0IK7ddnJ2U9haBItaz1Goov03HBn65zjo0mAmic7OLOFounGMg3Iy dRTqGbdC9VN5HHRvMQ5DY9UbGDKU5GDo4qVmDD0qvQBQXfFTDKV03MPQ6ogYNW/8TaKcpl6A Jox6DNUPuCBDfRuBem7kk6jwpwkKpfXVkqhhsh5HpS3rGOrSr5KodjwOoGsrhTi6PWQkUPnk NIGSqp4BlJi+RKG7ay2cIwKm574vox/uJJnshCmK+UU/SDF3h6oIJqHoIcH0dEYx1WUpJHPl hxjmuwelgKnrjyWZ+DvNOKOb/YdkMhKmSObpxADBTDcYSb9dJ097h7OyUFbtwCpClKERijAf ga9/0JtBUg+R2E3siQ4KHBSySNZHcOxdP7e3I+Sb/gkcPpPJozZTfjKNRuB+2FutjNKyDuFK jdZHwKpC5SqJSqiRRWqiFGFCBas9JBaJ9ks3iZ+cDn+gM2CqZvR5ycMkMhbEuqcCHhfSEljV vkSlAgsun64DMMswsx3MArhcnrcdzAO4Hr9MpALulqRrVWRS8+l6AH8sOmzmJGKw0zCPmTgk 7QI7Nrgmji1NwIsLC4SJg9NzJGy9vIKZHmxoP3h2ZgiY+AS9D27MWpvSlvQhmF7Yi5unc4TF I40cE+bRXvDWZClu5ljDtvPjhAnj9Ksw4WoebqoP6XUeXC8s45jFx2CT7jFpxjbwr5Yayozt 4ZPMJMosSANw8c4wZg50ACZkJWwrXofdhrWtbXDaCVbc2PZrL8xtL8fMna1gxso4Zs5bwtqC 59gRXq4o2i5jB3sX4rYxA1PapzGzW2cAXHm0RmUBB/0LG+lf2Ej/f+sigJcBO1aliQxjNVLV AQUb/d8vhygjq8HWTTn71oLRP2aETQDjgiYAubjA1vLjTl4w3zJU9sWXrFoZpI6Ss5omIN10 PBu33xmi3DxKhTZILPEUSTw8PCSeBzzEgt2W7WGVMj4dJtOyp1lWxaqf6zAuzz4WO5dcMKbw 5v2edD5gR55d2oXv73/06ZnK6517ONf9vYKMfL834oKv3vwwv8VmeG7eIv9kn9Ao/XntzwIB vtY+F/9sYV9c1/GvOl0wF4k8O+be5I3dy6FRrmX7p2PmR6yTuwkv/+5TRwffGxFmnBh8P/CV mAp5vJNTs2//qQIdle9uDR3LjcG/RqBm7h7nxqQa6VuK3NG+hcBiZZPVS6uB/Y+Nw+NdFTW3 PuC97J9e8FpGMWzYeURU6rrr4DvflI5Gh2sle68vduTmq6sjM3U1A99ODgiT4YkAvntF8c0r cvViZXWQ6/HZ5fTlDlFx71G/6AZL6knq1wGt6zlCrESayB/ZMSkgNOEysTOu1sj+BarE1/3c BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Sf0wTdxjG/d5d747O6lGcHCjBtHObGgtNzPZma4hhsN1CdIvGZYoTC9yq WwtNC7Iti1abmVDZEBpNV1RQHB2YzQAb4qCNlBVEKeBqKYo/QtqOuAJFGHFs0M7Klvjfk+fz ed6/XhoXR4hk+mBRCa8rUqolpJBo65akbq4SCfPTz/a8DJdv9ODQeWdWAJfuV5Jwenoeh3CX XwDVlRYKPIEVYJ+qEcDQX0cx8LdEMei8UI1B4yUXBuO2egQd5x9jUH5zCIOFMTm4opMkVDuH EQS9VgzsdzdBp72PAM8vZ0iobQhScMLXToIjZMfB1hvBYMC6QEJ74CiCtn9qceh+4CXgx1CY gOPNcwi+qpinYHCxV7BVwnlu53DWh26SqzJOUdxV632KG3zQTHDGunsE53GXci1N5STXevEI Zx6xIa7jjoHkjvW7cM4y8yfJfW2cIrnHwbsEF3Z4yfdX7xEqCnn1wUO8Li1jv/DAiKUT07rg swv3jpMGZEgzIZpmmS3swEK6CQlpMdOB2Ed1bZgJxT3tk9iGxV/xpZzANkbGqSXJiLEz5h+w 2JhkNrE3o3TMWcUQbOOTJ0TMwZmzFNsf6qdiIIHZzi6af0cxn2DWs9GZ+FgtYt5gK2qH/7sv Zc+PdQliOY55k70Wsj3rxU8di9dJLfnxbN+3ASKWcSaVNf5cg59EjPU5ZH0O1SGsCSXxWr1G pdHLtfIivkymV2r0pUUqWUGxpgU9e5GNG9rRlaZpmRNhNHIilsYlq0T73HH5YlGh8vMveF1x nq5UzeudaA1NSBJFQ6a+PDGjUpbwn/K8ltf9TzE6LtmAnTLkwLV3LX8k1q/J/d5ZUKnrojIH opt3jbs/woZXz7/30jnTbNb60bd0BaorJx4OHjaH1oU25Obv/HJlSnyaY4tvbRntfPGVlMnM Vs9oYjjv48EjjX5bMDJHLZMaXl++2/uCipe6R65Pjmzf31pf9SgQLHitZPK7rPZ1/mBSCAve +O2Dd5QTY9yMVqQ+LPPIPxErak65WoqvR5oNb+f8tGJZctc2/7Gdy31lubd93bOwUmp2nFzr zgxPZKSl1/elZGdIHbpRX8eub3YXV/w9NZyg6W34MBvfpx4/NLdXlmXP3nE10pp3KyAeMhY2 T29T7N060dOtuLVQMletUqTqXy2XEPoDSvlGXKdX/guGWMx7kQMAAA== X-CMS-MailID: 20220214080641epcas5p4662e8d0c86f93d525032067cc039c7af X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080641epcas5p4662e8d0c86f93d525032067cc039c7af References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Arnav Dawn Add support for handling target command on target. For bdev-ns we call into blkdev_issue_copy, which the block layer completes by a offloaded copy request to backend bdev or by emulating the request. For file-ns we call vfs_copy_file_range to service our request. Currently target always shows copy capability by setting NVME_CTRL_ONCS_COPY in controller ONCS. Signed-off-by: Arnav Dawn Signed-off-by: Nitesh Shetty --- drivers/nvme/target/admin-cmd.c | 8 +++- drivers/nvme/target/io-cmd-bdev.c | 65 +++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-file.c | 48 +++++++++++++++++++++++ 3 files changed, 119 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 6fb24746de06..3577e8af8003 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -431,8 +431,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) id->nn = cpu_to_le32(NVMET_MAX_NAMESPACES); id->mnan = cpu_to_le32(NVMET_MAX_NAMESPACES); id->oncs = cpu_to_le16(NVME_CTRL_ONCS_DSM | - NVME_CTRL_ONCS_WRITE_ZEROES); - + NVME_CTRL_ONCS_WRITE_ZEROES | NVME_CTRL_ONCS_COPY); /* XXX: don't report vwc if the underlying device is write through */ id->vwc = NVME_CTRL_VWC_PRESENT; @@ -530,6 +529,11 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req) if (req->ns->bdev) nvmet_bdev_set_limits(req->ns->bdev, id); + else { + id->msrc = to0based(BIO_MAX_VECS); + id->mssrl = cpu_to_le16(BIO_MAX_VECS << (PAGE_SHIFT - SECTOR_SHIFT)); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl) * BIO_MAX_VECS); + } /* * We just provide a single LBA format that matches what the diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index 95c2bbb0b2f5..47504aec20ce 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -46,6 +46,30 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg; /* NOWS = Namespace Optimal Write Size */ id->nows = to0based(ql->io_opt / ql->logical_block_size); + + /*Copy limits*/ + if (ql->max_copy_sectors) { + id->mcl = cpu_to_le32((ql->max_copy_sectors << 9) / ql->logical_block_size); + id->mssrl = cpu_to_le16((ql->max_copy_range_sectors << 9) / + ql->logical_block_size); + id->msrc = to0based(ql->max_copy_nr_ranges); + } else { + if (ql->zoned == BLK_ZONED_NONE) { + id->msrc = to0based(BIO_MAX_VECS); + id->mssrl = cpu_to_le16( + (BIO_MAX_VECS << PAGE_SHIFT) / ql->logical_block_size); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl) * BIO_MAX_VECS); +#ifdef CONFIG_BLK_DEV_ZONED + } else { + /* TODO: get right values for zoned device */ + id->msrc = to0based(BIO_MAX_VECS); + id->mssrl = cpu_to_le16(min((BIO_MAX_VECS << PAGE_SHIFT), + ql->chunk_sectors) / ql->logical_block_size); + id->mcl = cpu_to_le32(min(le16_to_cpu(id->mssrl) * BIO_MAX_VECS, + ql->chunk_sectors)); +#endif + } + } } void nvmet_bdev_ns_disable(struct nvmet_ns *ns) @@ -433,6 +457,43 @@ static void nvmet_bdev_execute_write_zeroes(struct nvmet_req *req) } } +static void nvmet_bdev_execute_copy(struct nvmet_req *req) +{ + struct nvme_copy_range range; + struct range_entry *rlist; + struct nvme_command *cmnd = req->cmd; + sector_t dest, dest_off = 0; + int ret, id, nr_range; + + nr_range = cmnd->copy.nr_range + 1; + dest = le64_to_cpu(cmnd->copy.sdlba) << req->ns->blksize_shift; + rlist = kmalloc_array(nr_range, sizeof(*rlist), GFP_KERNEL); + + for (id = 0 ; id < nr_range; id++) { + ret = nvmet_copy_from_sgl(req, id * sizeof(range), &range, sizeof(range)); + if (ret) + goto out; + + rlist[id].dst = dest + dest_off; + rlist[id].src = le64_to_cpu(range.slba) << req->ns->blksize_shift; + rlist[id].len = (le16_to_cpu(range.nlb) + 1) << req->ns->blksize_shift; + rlist[id].comp_len = 0; + dest_off += rlist[id].len; + } + ret = blkdev_issue_copy(req->ns->bdev, nr_range, rlist, req->ns->bdev, GFP_KERNEL); + if (ret) { + for (id = 0 ; id < nr_range; id++) { + if (rlist[id].len != rlist[id].comp_len) { + req->cqe->result.u32 = cpu_to_le32(id); + break; + } + } + } +out: + kfree(rlist); + nvmet_req_complete(req, errno_to_nvme_status(req, ret)); +} + u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) { switch (req->cmd->common.opcode) { @@ -451,6 +512,10 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_bdev_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_bdev_execute_copy; + return 0; + default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c index 6be6e59d273b..cf51169cd71d 100644 --- a/drivers/nvme/target/io-cmd-file.c +++ b/drivers/nvme/target/io-cmd-file.c @@ -347,6 +347,46 @@ static void nvmet_file_dsm_work(struct work_struct *w) } } +static void nvmet_file_copy_work(struct work_struct *w) +{ + struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); + int nr_range; + loff_t pos; + struct nvme_command *cmnd = req->cmd; + int ret = 0, len = 0, src, id; + + nr_range = cmnd->copy.nr_range + 1; + pos = le64_to_cpu(req->cmd->copy.sdlba) << req->ns->blksize_shift; + if (unlikely(pos + req->transfer_len > req->ns->size)) { + nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC)); + return; + } + + for (id = 0 ; id < nr_range; id++) { + struct nvme_copy_range range; + + ret = nvmet_copy_from_sgl(req, id * sizeof(range), &range, + sizeof(range)); + if (ret) + goto out; + + len = (le16_to_cpu(range.nlb) + 1) << (req->ns->blksize_shift); + src = (le64_to_cpu(range.slba) << (req->ns->blksize_shift)); + ret = vfs_copy_file_range(req->ns->file, src, req->ns->file, pos, len, 0); +out: + if (ret != len) { + pos += ret; + req->cqe->result.u32 = cpu_to_le32(id); + nvmet_req_complete(req, ret < 0 ? errno_to_nvme_status(req, ret) : + errno_to_nvme_status(req, -EIO)); + return; + + } else + pos += len; +} + nvmet_req_complete(req, ret); + +} static void nvmet_file_execute_dsm(struct nvmet_req *req) { if (!nvmet_check_data_len_lte(req, nvmet_dsm_len(req))) @@ -355,6 +395,11 @@ static void nvmet_file_execute_dsm(struct nvmet_req *req) schedule_work(&req->f.work); } +static void nvmet_file_execute_copy(struct nvmet_req *req) +{ + INIT_WORK(&req->f.work, nvmet_file_copy_work); + schedule_work(&req->f.work); +} static void nvmet_file_write_zeroes_work(struct work_struct *w) { struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); @@ -401,6 +446,9 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_file_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_file_execute_copy; + return 0; default: return nvmet_report_invalid_opcode(req); } From patchwork Mon Feb 14 07:59:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 035E5C4332F for ; Mon, 14 Feb 2022 08:36:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242157AbiBNIgo (ORCPT ); Mon, 14 Feb 2022 03:36:44 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:58042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242149AbiBNIgl (ORCPT ); Mon, 14 Feb 2022 03:36:41 -0500 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD4A94F472; Mon, 14 Feb 2022 00:36:11 -0800 (PST) Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20220214083609epoutp01607522428af862a444d322516ac30c74~TmiRjM7mR1004510045epoutp01e; Mon, 14 Feb 2022 08:36:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20220214083609epoutp01607522428af862a444d322516ac30c74~TmiRjM7mR1004510045epoutp01e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827770; bh=l8y9YIQatSjWKA1VT3EIziglsZq7xzC4RJS+MLkmrv8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K+XdKpjwhi4oERZotVvXzndFIqfjfbeuYNhw/4x2hq6PXjgvhCxn7+vAPL18SGn+Y ZOyf7a6tk2bCPGNsAVJ2C8mvRBFMn9xYBoahJkCUreVYIv15qGbUW8kPuOt8JRyYei BT5VT7cQ+LSZKF7avooOjMj1ELXmGRTDj6oPGVcE= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220214083609epcas5p3b54754d3fa83cb27a2fceddb96a6586b~TmiRK3TtR2552125521epcas5p3b; Mon, 14 Feb 2022 08:36:09 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.177]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4JxyH74RDkz4x9Pr; Mon, 14 Feb 2022 08:36:03 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 6C.07.06423.3741A026; Mon, 14 Feb 2022 17:36:03 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20220214080649epcas5p36ab21e7d33b99eac1963e637389c8be4~TmIp4Xabc2239022390epcas5p3K; Mon, 14 Feb 2022 08:06:49 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220214080649epsmtrp1d094b9eacc8993b8a46c37c4150afd95~TmIp3i47q0046200462epsmtrp1V; Mon, 14 Feb 2022 08:06:49 +0000 (GMT) X-AuditID: b6c32a49-b13ff70000001917-91-620a14730caf Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 16.F1.29871.99D0A026; Mon, 14 Feb 2022 17:06:49 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080644epsmtip251c38a47e899471516f26528f3ac9dc3~TmIk8FVhf2207122071epsmtip2c; Mon, 14 Feb 2022 08:06:43 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 08/10] dm: Add support for copy offload. Date: Mon, 14 Feb 2022 13:29:58 +0530 Message-Id: <20220214080002.18381-9-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta1ATVxidu7vZJHRCl4f0Sm2NsbYCAomG9GJVHMV2UTtDdZzWjiOusCZI SNIktNY6UyCVEeShSB2IbY3UMfIoKBVKkVDEoTyCqEOhkhERgVLFBjVVQ0Vo4krrv3POPd/5 7vfduQLcv40fLEjWGFm9hlFLSB+i7mJIaLgh0Gen1JY9H1V3/oqjxn4XD1UMFJDo6L1JHE1c GOahwoJiPuoZ8UU25zEeuuLOwNBwzQyGGksLMVRW0YqhMev3AJ0/cR9D2fYrGJoakqHWmb9I VNjSB9BorxlDNkcYarR1EKin4RsSHT81ykcHf68nUdO4DUfWtmkMdZunSFQ/kgFQ3ZPjOLp4 o5dAVeMTBMo6+xCg/bmTfHT5aRtvtYTu+W0DbR68RNKHTU4+/bN5gE9fvnGWoE2W6wTdcymN rinPJukfT35JH7lmBfT5/nSSzuxqxeniB3+TdJ7JSdL3Rx0EPdHUS8YHfZyyQsUySaxezGoS tUnJGuVKyYbNCWsTohRSWbgsGr0tEWuYVHalJHZjfPi7yWrP/iTiTxl1mkeKZwwGSeSqFXpt mpEVq7QG40oJq0tS6+S6CAOTakjTKCM0rHG5TCpdGuUx7khRNV0b4ukevLrHVZZBpIOxoBwg EEBKDo8NEDnAR+BPnQcw86dTgCMPAHSUnvMQoYc8ArA4N3K2wPXUyHlsAFp7b/E4sh+Dednd pNdEUmHQPiPw1gZSBCx7/PhZB5xykbC98gnmPQig3oHO6juE109Qi+Af5VqvLKKWQ1NTEe7F kFoITwxd4Hmx0GNvHrfinMcPdpSMEF6MU/OhqfYY7s2H1LQQHnF0E9xFY+FkJcHlBMA7bef4 HA6Gtwuy+Jz/IIDurkGMI8UAmg6ZSM4VA682PsW8QTgVAqsbIjn5Nfh1ZxXGNfaFeU9GME4X wfrvZvFCWFlteR4zF/Y9zniOadj81STOLSsfwB9KMolDQGx+YSDzCwOZ/29tAXg5mMvqDKlK 1hClk2nYz/5740Rtag149qNC4+rBwM17ES0AE4AWAAW4JFC0/ZJwp78oifl8L6vXJujT1Kyh BUR5Fn4YD56TqPV8SY0xQSaPlsoVCoU8eplCJnlFZFeeYfwpJWNkU1hWx+pn6zCBMDgdW+to zpJ1Fq0+vH5ZsDtuzTQbNHjqtL7z5CP3J31BbW+NYXH7lrZ3zyv1+faotMjSV97uJ+n9U3y9 LmiXrWuvcrf0F0Un69ix2Y0zqtQDxemu2G3vrftn1/CcdnZCUVK2u/n9rs6N6tDpmDOblNqA mhz7R84l64edJcKKkObX5dYiWnpNs+LNxVlTi0Sj87etCdz30pb8rdKYD4uGQsX49JBc+MaC kCPT9ti7eduzAzT2qwfm7Mm/W87zRf2Bp80uyzljfm5cjmWRv+lh1p6qDxi/RzM9DfC+uKz2 9rwvUu5W+XYsdodsWmKNKKwL24XJo+/cutm0KmnHy1u3dKiWmy2166okhEHFyEJxvYH5F+aG uCvaBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrHIsWRmVeSWpSXmKPExsWy7bCSvO5MXq4kg5/X+S3WnzrGbLHn5mdW i9V3+9kspn34yWzx/uBjVotJ/TPYLS4/4bPY+242q8WFH41MFo83/Wey2LNoEpPFytVHmSye L1/MaLF74Ucmi87TF5gs/jw0tDj6/y2bxaRD1xgtnl6dxWSx95a2xZ69J1ksLu+aw2Yxf9lT dovu6zvYLPa93stssfz4PyaLc7P+sFnseNLIaLHt93xmi8P3rrJYrHv9nsWibeNXRovWnp/s Fuf/Hmd1UPK4fMXbY9b9s2weE5vfsXvsnHWX3eP8vY0sHs0L7rB4XD5b6rFpVSebx+Yl9R6T byxn9Nh9s4HNo+nMUWaPGZ++sHn0Nr9j8/j49BaLx/t9V9kCxKK4bFJSczLLUov07RK4Mvbd eMha8Em64vPKRpYGxudiXYwcHBICJhKf/5Z0MXJxCAnsZpRofDOPsYuREyguKbHs7xFmCFtY YuW/5+wQRc1MEm1vfrCDNLMJaEuc/s8BUiMiwCKx8vt3FpAaZoG57BJnXp9hB0kIC1hLvFv/ igWknkVAVeLZqnyQMK+AlUTzvilQ85UlFj48yApicwKVH3i9HCwuBFQz4+ohdoh6QYmTM5+w gNjMAvISzVtnM09gFJiFJDULSWoBI9MqRsnUguLc9NxiwwLDvNRyveLE3OLSvHS95PzcTYzg JKKluYNx+6oPeocYmTgYDzFKcDArifDGneVMEuJNSaysSi3Kjy8qzUktPsQozcGiJM57oetk vJBAemJJanZqakFqEUyWiYNTqoHJp07ar2qOwYSmHZ2P/+kWr1vJmFfjNyvrz/FOfaHGl/u+ /U5Y7LVyc/iX6HmL9TLSoqqqliarvVKWb8k8U8V96MmpHp0QIaOFjPV7fvc1rfx3bXM/4z/7 J4fj5aw1XlafuzM5pXLOlsp5ndzyMi+CP59JyjbOXta6/IfN1XXJi8LMdjXdX7I4OMU5/dHm A3uCJRmnMR15VvNqRnPl9P9xdu9sPEtKGw/vTFu+8/VRW6lDoa7Mb+Tn3Tr05OfbSJ5SEY6u SfU+hgtmXH7E5u8uVyhuPMG9c+eFW49MnvdO2bmLnVl9WmuKzOddFbOL+l9uXrXq7sX20k67 4md/3ToYDi99JG75dXpkQ/jHGLPjSizFGYmGWsxFxYkAtXupRZEDAAA= X-CMS-MailID: 20220214080649epcas5p36ab21e7d33b99eac1963e637389c8be4 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080649epcas5p36ab21e7d33b99eac1963e637389c8be4 References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Before enabling copy for dm target, check if underlying devices and dm target support copy. Avoid split happening inside dm target. Fail early if the request needs split, currently splitting copy request is not supported. Signed-off-by: Nitesh Shetty --- drivers/md/dm-table.c | 45 +++++++++++++++++++++++++++++++++++ drivers/md/dm.c | 6 +++++ include/linux/device-mapper.h | 5 ++++ 3 files changed, 56 insertions(+) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index e43096cfe9e2..8dc9ae6a6a86 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1903,6 +1903,38 @@ static bool dm_table_supports_nowait(struct dm_table *t) return true; } +static int device_not_copy_capable(struct dm_target *ti, struct dm_dev *dev, + sector_t start, sector_t len, void *data) +{ + struct request_queue *q = bdev_get_queue(dev->bdev); + + return !blk_queue_copy(q); +} + +static bool dm_table_supports_copy(struct dm_table *t) +{ + struct dm_target *ti; + unsigned int i; + + for (i = 0; i < dm_table_get_num_targets(t); i++) { + ti = dm_table_get_target(t, i); + + if (!ti->copy_supported) + return false; + + /* + * target provides copy support (as implied by setting + * 'copy_supported') and it relies on _all_ data devices having copy support. + */ + if (ti->copy_supported && + (!ti->type->iterate_devices || + ti->type->iterate_devices(ti, device_not_copy_capable, NULL))) + return false; + } + + return true; +} + static int device_not_discard_capable(struct dm_target *ti, struct dm_dev *dev, sector_t start, sector_t len, void *data) { @@ -2000,6 +2032,19 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, } else blk_queue_flag_set(QUEUE_FLAG_DISCARD, q); + if (!dm_table_supports_copy(t)) { + blk_queue_flag_clear(QUEUE_FLAG_COPY, q); + /* Must also clear discard limits... */ + q->limits.max_copy_sectors = 0; + q->limits.max_hw_copy_sectors = 0; + q->limits.max_copy_range_sectors = 0; + q->limits.max_hw_copy_range_sectors = 0; + q->limits.max_copy_nr_ranges = 0; + q->limits.max_hw_copy_nr_ranges = 0; + } else { + blk_queue_flag_set(QUEUE_FLAG_COPY, q); + } + if (dm_table_supports_secure_erase(t)) blk_queue_flag_set(QUEUE_FLAG_SECERASE, q); diff --git a/drivers/md/dm.c b/drivers/md/dm.c index ab9cc91931f9..3b4cd49c489d 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1372,6 +1372,12 @@ static int __split_and_process_non_flush(struct clone_info *ci) if (__process_abnormal_io(ci, ti, &r)) return r; + if ((unlikely(op_is_copy(ci->bio->bi_opf)) && + max_io_len(ti, ci->sector) < ci->sector_count)) { + DMERR("%s: Error IO size(%u) is greater than maximum target size(%llu)\n", + __func__, ci->sector_count, max_io_len(ti, ci->sector)); + return -EIO; + } len = min_t(sector_t, max_io_len(ti, ci->sector), ci->sector_count); r = __clone_and_map_data_bio(ci, ti, ci->sector, &len); diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index b26fecf6c8e8..acfd4018125a 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -362,6 +362,11 @@ struct dm_target { * zone append operations using regular writes. */ bool emulate_zone_append:1; + + /* + * copy offload is supported + */ + bool copy_supported:1; }; void *dm_per_bio_data(struct bio *bio, size_t data_size); From patchwork Mon Feb 14 07:59:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63ED3C4332F for ; Mon, 14 Feb 2022 08:46:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242488AbiBNIqJ (ORCPT ); Mon, 14 Feb 2022 03:46:09 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:46410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242552AbiBNIqF (ORCPT ); Mon, 14 Feb 2022 03:46:05 -0500 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20C1E606C2; Mon, 14 Feb 2022 00:45:37 -0800 (PST) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20220214083616epoutp04bebac9c91502b22ec389367e354046aa~TmiXeG4Dy0136801368epoutp04Z; Mon, 14 Feb 2022 08:36:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20220214083616epoutp04bebac9c91502b22ec389367e354046aa~TmiXeG4Dy0136801368epoutp04Z DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827776; bh=ngGeLKh/cNFBqZ2ebzcJUg1MB4xMAkdY6dg48NrWGfk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T+xy6Tv4m9DEzueG2vKkPQax2Qnf/Cu9x/SRWm3SVQYzMofaa0HWWIGQKvmyz/ems 1EN7mxX+PlcpPXYxXQqbhoQ9wzTtbNuAt4yNPOv6FaDvFeGCDa5DN+MDPn6MjxtCgp LCFwhlcmAmEvh6fX0YlkJECV/uyP80Tg0LPEGwxc= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20220214083615epcas5p139fdb7d151237f990c3306d6d6a1642a~TmiXJFcUW1332613326epcas5p1l; Mon, 14 Feb 2022 08:36:15 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.174]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JxyHC4TqLz4x9QH; Mon, 14 Feb 2022 08:36:07 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id C5.17.06423.7741A026; Mon, 14 Feb 2022 17:36:07 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20220214080656epcas5p31c80cce4f9638bccdf2bc225b339c37e~TmIwwT_j10612006120epcas5p3K; Mon, 14 Feb 2022 08:06:56 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220214080656epsmtrp13385868747ae487e254764d8bab6138b~TmIwve7fn0045300453epsmtrp1g; Mon, 14 Feb 2022 08:06:56 +0000 (GMT) X-AuditID: b6c32a49-b01ff70000001917-a2-620a147714cb Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id D9.B4.08738.0AD0A026; Mon, 14 Feb 2022 17:06:56 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080651epsmtip233c40a25e259e947cf3374c16faf4684~TmIsK1UTr2418624186epsmtip2k; Mon, 14 Feb 2022 08:06:51 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, Nitesh Shetty , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 09/10] dm: Enable copy offload for dm-linear target Date: Mon, 14 Feb 2022 13:29:59 +0530 Message-Id: <20220214080002.18381-10-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTZxTfd+/ltmXBXbCTD2Sx6WYWMGDrWvYhMnCiXqJsGLMtumVwgTsg QNu0ZTr9A1hHIzDk4VBWioAOkPdEZYzXEIKM12BDqBqesUSF8J4B5NFRis7/ft85v9/5nXO+ HC5u185x5EbI1KxSxkQJSWuiusXZxfUM3zpYZCwUoMqOeziqf7hghUqHUkl0eXYZRzN3H1uh jNQsDuozbkMN09lWqHcpHkOPq0wYqr+WgaHi0lYMPSm6DlBd/hyGEjt7MbQ6JkatpikSZTQP ADTer8NQw6M9qL6hnUB9tXoS5RaOc1CyoYZEjZMNOCpqW8fQX7pVEtUY4wGqXsnFUctwP4Eq JmcIpL35HKCEH5c5qGetzcpHSPfdP0brRrpJOl0zzaF/1w1x6J7hmwStyRsk6L7uGLqqJJGk b/0SS196UATouodxJP19VytOZ83/S9IpmmmSnht/RNAzjf1kwI7TkQfCWSaUVQpYWYg8NEIW 5iU8djLwUKDUXSR2FXugD4UCGRPNegl9jwe4HomI2tifUPAtExWzEQpgVCrh3o8OKOUxalYQ LlepvYSsIjRKIVG4qZhoVYwszE3GqveLRaJ90g1iUGT42HwzoXhKnM1suoLHAROeBHhcSElg 3twASALWXDuqDsDlpDQrc8KOmgdwJmmnJbEAYMf9XPKlIj1xaotUC+B6HNdCSsBg8Y2LWBLg cklqD+w0cc0cPkXA4sVFwszBqQUS/lm2ssnZTh2BlQufmDkEtRv+ZpgDZmxDeUJtyWXM4vUu zB+7u+nF24g3TRbhFo4tbP/ZSJgxTu2CmjvZuLk+pF7wYHtjj5VF7AtHV3sIC94OJ9pucyzY ET5L1XIsgmQAl7pGMMsjC0BNmmZrTG/4d/3aZqc45Qwra/dawu/AzI4KzOK8DaasGLc6tYE1 V42vui6rzNsq4wAHFuO3MA2fG8o4lm1dBLBhaolIAwLdaxPpXptI9791HsBLgAOrUEWHsSqp Qixjz7z65RB5dBXYvCkXvxowNDrr1gwwLmgGkIsL+TZfd/OC7WxCme/OsUp5oDImilU1A+nG ytNxx7dD5BtHKVMHiiUeIom7u7vE4wN3sdDepjPsV8aOCmPUbCTLKljlSx3G5TnGYTeu+686 HVqvCPEuLvk8Uz9SfuqNhKGnyW/Kzg9PV8r0balfGtKKP5X25aQzef31hHRKqo3WHz3n4pm7 FKxyMvIlvCj7/Mbyfe+ZDhZ0n/fGg3zeOhh44fSE9VjJFxqvItuGlUuDV78KERX/ZND8MIg3 qjvjXhy+XflEUcZ2iCYiffX21zLD5NkSJ+2KIaes3bdavXt0/pTqZE5rStqVVoeaiPeXTuzw t/1Y45Xr5/NZk+lZRcIMOlrgzzs733ln3Fj+wL2lcOi4diX+j9gC7Uh+262u+HTx7D+lfoJ7 zOqunYOeojEUf6I3NClW7/RNd7CWFfIjhp0Tg5CEX3Jhv25NSKjCGbELrlQx/wHDR1On3AQA AA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Sf1CTdRzH7/t9nj3Ps3Wzx0HwaJzEkBQ4wHV2fSKyrjx9qtN+/FUeJ458 DjkH0rORGlciK5JBh8yEuaHgj4MYAQk5STZvjCaHQtAtfqZ2S5B0OZiaQDlXE7vzv9d9Xq+7 9z8fhlCEyOVMbr5OEPPVGiUlI209ytiUerkse03976ug7eIFAuzjdyTQfKWSgurZBQJmuq9J wFhposEzuQQcfosEhub3Y7jWHsJgP2HE0NTsxjDdeBJB1/EAhrJLQxjue1XgDt2iwOgaQTA1 bMbgmEgGu6OPBM+5WgrqGqZoKB/tpOC8z0FAY+8DDD+Z71PQObkfge2fOgJ6rg6T0OqbIaH0 9F8IvqhYoGEw2Ct5Vcl7fnmLN/82QPFVej/N/2C+QvODV0+TvL7+Msl7Bgr5dmsZxXec2scf GmtEfNd4McWX9LsJ3nT7LsV/pfdTfGBqguRnzg9T70RtkWVsFzS5Hwti2rptsh3e2y6y4A9y z2FnDVGMQoQBSRmOXctVld2SGJCMUbCdiKt0B/CiWMY1BH98FEVwTQ+m6cVIj7numqOkATEM xSZzl0JMuIlkSa5pbo4MNwR7lOb6ff10uIlgN3BtdzaHG5JN4M6OBlCY5exLXKm1+tFWPHfc 2y0Js/S/u9PX+HBXwaZzpmEXvdgv5fqOTJJhJthYTn/GQhxErPkxZX5M1SNsRcuEAm1eTp5W VfBcvrA7VavO0xbm56R+uCuvHT38kaSkTmS3zqa6EGaQC3EMoYyUbx2QZivk29V7PxHEXVli oUbQutDTDKmMlg8Z+rIUbI5aJ+wUhAJB/N9iRrq8GL9tm+5+ofp6/8+HR8tft39UEhHo6Aie 8woLNbrEwK8V358tjXvj5fSFZ++9yfdpPyeKlq7F6S1pDdS7iau/qa07mLTu+WaNUVzd5ivf F7eyOBpfyI2afdJ0stZ8zFM0NqaWPlV98bOthzKX3MxIcyQ4/ZVzcQLoeluiYv789sZAW7rh zIqWmA3uTclHvsvIdJR88ETXdX9wVc/G9SnxRTszDCa3bv6ZxDT3zXnvi+YKyxpn66fj5Cnj MVVUlcWqcji34KwRcWMwsuLrcVvsyN5t7pQvYyYGdyfcoHxgMd9bn9W6Z1r62nvvrzzQdbmB yZ6dFK3xLlv0gSC+G9Ss+PuVzZlKUrtDrUoiRK36X2mdGvKSAwAA X-CMS-MailID: 20220214080656epcas5p31c80cce4f9638bccdf2bc225b339c37e X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080656epcas5p31c80cce4f9638bccdf2bc225b339c37e References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Setting copy_supported flag to enable offload. Signed-off-by: Nitesh Shetty --- drivers/md/dm-linear.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 1b97a11d7151..8910728bc8df 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -62,6 +62,7 @@ static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_secure_erase_bios = 1; ti->num_write_same_bios = 1; ti->num_write_zeroes_bios = 1; + ti->copy_supported = 1; ti->private = lc; return 0; From patchwork Mon Feb 14 08:00:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 12745192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8423BC4332F for ; Mon, 14 Feb 2022 08:36:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242172AbiBNIg5 (ORCPT ); Mon, 14 Feb 2022 03:36:57 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:58704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242145AbiBNIgw (ORCPT ); Mon, 14 Feb 2022 03:36:52 -0500 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0078102; Mon, 14 Feb 2022 00:36:21 -0800 (PST) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20220214083619epoutp03fdafbcfeb68da98e253b27a5749ed248~TmiahyoXG0835308353epoutp03p; Mon, 14 Feb 2022 08:36:19 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20220214083619epoutp03fdafbcfeb68da98e253b27a5749ed248~TmiahyoXG0835308353epoutp03p DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1644827779; bh=NovAt/GDDE66tSp8rEA4A5MVhKoxVSbI0akDxNWa7S4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nRS0yc0uhlu13N3mmjD6tBcluK1GX6VhfvjKzNwgGBbL5c8t0O6aHl6a3EHCw/Tnv rWoF5/lOTmNjsZl+2cMGHeESqYK9Yb5AUu2OIWS+QznPyu8jgKw5WN3nFmspDvfXTQ LoUqrcf1AzqurMIMvi9StxEe0HIGeNxFRNyy/RUU= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20220214083619epcas5p2c99d2183476cd8fa4403752a5619e589~TmiaKIGT70375203752epcas5p2x; Mon, 14 Feb 2022 08:36:19 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.180]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4JxyHK2H6Gz4x9QR; Mon, 14 Feb 2022 08:36:13 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id EA.66.05590.D741A026; Mon, 14 Feb 2022 17:36:13 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20220214080703epcas5p2980d814681e2f3328490824710c8fded~TmI3USuY90791707917epcas5p2u; Mon, 14 Feb 2022 08:07:03 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220214080703epsmtrp249b45a8cd1222a254a2ef47f35286212~TmI3SuWDK2569225692epsmtrp2X; Mon, 14 Feb 2022 08:07:03 +0000 (GMT) X-AuditID: b6c32a4b-739ff700000015d6-f2-620a147d56dd Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 31.C4.08738.7AD0A026; Mon, 14 Feb 2022 17:07:03 +0900 (KST) Received: from test-zns.sa.corp.samsungelectronics.net (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220214080658epsmtip21e180a2b3b2f84a24af9a6880292d0cf~TmIyx1Bi52320123201epsmtip2t; Mon, 14 Feb 2022 08:06:58 +0000 (GMT) From: Nitesh Shetty Cc: javier@javigon.com, chaitanyak@nvidia.com, linux-block@vger.kernel.org, linux-scsi@vger.kernel.org, dm-devel@redhat.com, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, axboe@kernel.dk, msnitzer@redhat.com, bvanassche@acm.org, martin.petersen@oracle.com, hare@suse.de, kbusch@kernel.org, hch@lst.de, Frederick.Knight@netapp.com, osandov@fb.com, lsf-pc@lists.linux-foundation.org, djwong@kernel.org, josef@toxicpanda.com, clm@fb.com, dsterba@suse.com, tytso@mit.edu, jack@suse.com, joshi.k@samsung.com, arnav.dawn@samsung.com, nitheshshetty@gmail.com, SelvaKumar S , Nitesh Shetty , Alasdair Kergon , Mike Snitzer , Sagi Grimberg , James Smart , Chaitanya Kulkarni , Alexander Viro , linux-kernel@vger.kernel.org Subject: [PATCH v3 10/10] dm kcopyd: use copy offload support Date: Mon, 14 Feb 2022 13:30:00 +0530 Message-Id: <20220214080002.18381-11-nj.shetty@samsung.com> X-Mailer: git-send-email 2.30.0-rc0 In-Reply-To: <20220214080002.18381-1-nj.shetty@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Te0xTZxTfd+/tbcvEXXnto9ONlc35CEhHwQ8VJdO4ayCRZDqdy4YFroVR StcW3ZxuhUaNYAEhTiyEhzAqorycphbqLA4JIMhS0cF4iNC4iONtAHmt5eLmf7/zO7/fd75z Tg4Pd2nlCnixcjWjlEtkQtKJuHFn7Rqf425OkX4vJ5ahiqa7OKrtGOegsu50Ev08Mo2jYUs/ B2WmZ3ORdWA5Mg/lcFDbVBKG+qsXMFR7MRNDpWX1GHpqKAKopnAUQ6eb2zA02ydC9Qv/kCiz 7iFAtnY9hsyd61GtuZFAVlMuifJLbFyU+shIoluDZhwZGuYx1KqfJZFxIAmgGzP5OLrT006g 8sFhAg1ONZLoZNULgE6cmeai+3MNnBBv2voglNb3tpD0We0Ql76p7+bS93uqCFpb0EXQ1pZE uvryaZK+VvwTnfWnAdA1HRqSTr5Xj9PZYxMkrdMOkfSorZOgh2+1k+EeB+K2xDCSaEbpxcij EqJj5dJgYehnEdsjAgL9RD6iILRR6CWXxDPBwh1h4T47Y2X2IQq9DktkiXYqXKJSCTds3aJM SFQzXjEJKnWwkFFEyxRiha9KEq9KlEt95Yx6k8jP7+MAu/BgXIzpUieh6PL67mZbGkcD5gQp gM+DlBhWXtURKcCJ50LVAKgraeewwRiAz4vnMTYYBzCjaYZ8ZWmpuE2yCROAp4pal/wnMGjT 9HFTAI9HUuth8wLPYXCjCFg6Obmowak8LrzeOMh1JFypYGgpaiUdeoL6EOpqpQ7amdoMTdeM OFvMGxb2WTgOzLfztwcNOKtZARsvDBAOjFPvQe31nCX9PB8WX/2IxTtgiX3BLHaFzxp+5bJY AMeHzIsNQCoVwKl7vRgbZAOozdAuObbBP2rnMMfncGotrDBtYOlV8FxTOcYWXg51MwMYyztD Y94r7A2vVBQsPeMJH04mLfYIKRo2J0N2VmkA3n3QjWUAL/1r/ehf60f/f+UCgF8GnoxCFS9l VAEKfzlz5L8tRyXEV4PFw1oXagRPHo/41gGMB+oA5OFCN+evW/iRLs7Rku+PMsqECGWijFHV gQD7vM/iAveoBPtlytURInGQnzgwMFAc5B8oEr7t3CStlLhQUomaiWMYBaN85cN4fIEGO7z/ 3PLIBo/I8N7NWStL0/xXXwrbd5SYeWpJfVca/HlAfkyNopU+dObxNfNJrejgijX8D4z9HH83 2dTqvv4XnuXj7i+422y9mrmSVIPLrMDj+PiJJFfrXnWuR5j5Td9DG5sWOPtXLYDKJ8nuv4Ws vT7y6SPe7POvclfm/1gUFlnZFfMsnJfanlX6IGVsuqN52ewvGeXa8xf27gnRzPwdceVUbJvt i6jRN+Z3VaVXGiwdx9aHbtr6ztBb90M1O3Iv3owzFGFpgr7tu5utZ/a0ytyPje+cSC/Me5+Q ZgJZtGuZUfrDkcLbB0K/+d0ic/oy+ZOV3wY15fRs2IXO79ttkuv4Ti/d/hILCVWMRLQOV6ok /wLNPg7W4QQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Sf0yUdRzH931+362rpwP0C7RgZ7QJSuBYfoxQ22o8y3RWjqU1z0MeTwbc 0d2BmW1iV25ASpEanhlyFhdHkwGJBxyXPIR68uvceVwSmgaEdeOQ0yaoJ3lRm/+99n6/9t7n jw9HKs10HJevM4kGnaZQxcipth5VwnKbQp6b9sPBFGi6eI4E55XbNDRerWLgyK05Eqa7x2io rqphwTv+JHQFj9Hgmd1HwFjLPAFOazUBDY29BEzaTiLorJshoLzPQ8CDG+nQOz/FQLU0jGDC ZyGgayQFnF1uCrwdXzNQWz/BQqXfwYAr0EWC7fxDAgYtDxhwjO9D0Ha/loSeaz4KTgWmKQjM uhnY3/w3gk8/m2NhKHyeXrtE8F5eJ1h+G2CEL8xBVmi3XGWFoWvNlGA+MUoJ3oESocVezgit 3+4VvvzFhoTOK2WM8HF/LynUhO4wwgFzkBFmJkYoYdrlYzYu2iJ/OU8szC8VDS+s3ibf2fH9 CFU8mvhBu+cgXYbCcRVIxmE+Aw80nWUirOQdCLeFNyzksbg+/DO5wFG44eEkW4HkjxwzgQPe flSBOI7hU3DfPBdxonkKN9y9S0Uckr/A4qFPJDZSRPFZuPvkIBPxKT4JH3BqI7GCz8QdrY7/ 9pfguhvddIRlj/KzARu5cM9LuMa3MKPgn8buo+NUhEk+AZtPHyM/R7zlscryWHUCEXYUKxYb i7RFxvTiFTpxV6pRU2Qs0WlTt+uLWtC/f5Kc7EBO+61UCREckhDmSFW0YuuALFepyNPs/lA0 6NWGkkLRKKF4jlItVngq3Golr9WYxAJRLBYN/7cEJ4srI1ZefiWR6bmp/em129mhpxBSxO5J bGuS6y9k5lVlOSCB3VZnXRTu3LAKjt9j4sv75kKq3d/1bQ+uz6pMUy+T1LYdMUux0mL/asaM D52beKLZt3ksp/mdtKD7ee9bwR0N9b9bc0yLoVZ/r9y+f3POj67nxJqhP9+350S/ZzHpNN+M uwa1IUNo8M6KI+30JtNwUjVND0d/tPZt9eH03DdX7bn0bkHGbP/K/KP+qcqL1uWt8Wck72pK FnX9maYkf6l7o29076+Xetdlv+gafWPpXFX2+teVszfTkjadEktpv39L5l/DjYf+eHbXmVev l249PillxHiscWsa9VJhQWJ7zFSdTEUZd2rSk0mDUfMP3I9B1JYDAAA= X-CMS-MailID: 20220214080703epcas5p2980d814681e2f3328490824710c8fded X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220214080703epcas5p2980d814681e2f3328490824710c8fded References: <20220214080002.18381-1-nj.shetty@samsung.com> To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: SelvaKumar S Introduce copy_jobs to use copy-offload, if supported by underlying devices otherwise fall back to existing method. run_copy_jobs() calls block layer copy offload API, if both source and destination request queue are same and support copy offload. On successful completion, destination regions copied count is made zero, failed regions are processed via existing method. Signed-off-by: SelvaKumar S Signed-off-by: Arnav Dawn Signed-off-by: Nitesh Shetty --- drivers/md/dm-kcopyd.c | 55 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 49 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-kcopyd.c b/drivers/md/dm-kcopyd.c index 37b03ab7e5c9..214fadd6d71f 100644 --- a/drivers/md/dm-kcopyd.c +++ b/drivers/md/dm-kcopyd.c @@ -74,18 +74,20 @@ struct dm_kcopyd_client { atomic_t nr_jobs; /* - * We maintain four lists of jobs: + * We maintain five lists of jobs: * - * i) jobs waiting for pages - * ii) jobs that have pages, and are waiting for the io to be issued. - * iii) jobs that don't need to do any IO and just run a callback - * iv) jobs that have completed. + * i) jobs waiting to try copy offload + * ii) jobs waiting for pages + * iii) jobs that have pages, and are waiting for the io to be issued. + * iv) jobs that don't need to do any IO and just run a callback + * v) jobs that have completed. * - * All four of these are protected by job_lock. + * All five of these are protected by job_lock. */ spinlock_t job_lock; struct list_head callback_jobs; struct list_head complete_jobs; + struct list_head copy_jobs; struct list_head io_jobs; struct list_head pages_jobs; }; @@ -579,6 +581,42 @@ static int run_io_job(struct kcopyd_job *job) return r; } +static int run_copy_job(struct kcopyd_job *job) +{ + int r, i, count = 0; + struct range_entry range; + + struct request_queue *src_q, *dest_q; + + for (i = 0; i < job->num_dests; i++) { + range.dst = job->dests[i].sector << SECTOR_SHIFT; + range.src = job->source.sector << SECTOR_SHIFT; + range.len = job->source.count << SECTOR_SHIFT; + + src_q = bdev_get_queue(job->source.bdev); + dest_q = bdev_get_queue(job->dests[i].bdev); + + if (src_q != dest_q || !blk_queue_copy(src_q)) + break; + + r = blkdev_issue_copy(job->source.bdev, 1, &range, job->dests[i].bdev, GFP_KERNEL); + if (r) + break; + + job->dests[i].count = 0; + count++; + } + + if (count == job->num_dests) { + push(&job->kc->complete_jobs, job); + } else { + push(&job->kc->pages_jobs, job); + r = 0; + } + + return r; +} + static int run_pages_job(struct kcopyd_job *job) { int r; @@ -659,6 +697,7 @@ static void do_work(struct work_struct *work) spin_unlock_irq(&kc->job_lock); blk_start_plug(&plug); + process_jobs(&kc->copy_jobs, kc, run_copy_job); process_jobs(&kc->complete_jobs, kc, run_complete_job); process_jobs(&kc->pages_jobs, kc, run_pages_job); process_jobs(&kc->io_jobs, kc, run_io_job); @@ -676,6 +715,8 @@ static void dispatch_job(struct kcopyd_job *job) atomic_inc(&kc->nr_jobs); if (unlikely(!job->source.count)) push(&kc->callback_jobs, job); + else if (job->source.bdev->bd_disk == job->dests[0].bdev->bd_disk) + push(&kc->copy_jobs, job); else if (job->pages == &zero_page_list) push(&kc->io_jobs, job); else @@ -916,6 +957,7 @@ struct dm_kcopyd_client *dm_kcopyd_client_create(struct dm_kcopyd_throttle *thro spin_lock_init(&kc->job_lock); INIT_LIST_HEAD(&kc->callback_jobs); INIT_LIST_HEAD(&kc->complete_jobs); + INIT_LIST_HEAD(&kc->copy_jobs); INIT_LIST_HEAD(&kc->io_jobs); INIT_LIST_HEAD(&kc->pages_jobs); kc->throttle = throttle; @@ -971,6 +1013,7 @@ void dm_kcopyd_client_destroy(struct dm_kcopyd_client *kc) BUG_ON(!list_empty(&kc->callback_jobs)); BUG_ON(!list_empty(&kc->complete_jobs)); + WARN_ON(!list_empty(&kc->copy_jobs)); BUG_ON(!list_empty(&kc->io_jobs)); BUG_ON(!list_empty(&kc->pages_jobs)); destroy_workqueue(kc->kcopyd_wq);