From patchwork Fri Aug 5 15:42:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12937490 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5608BC00140 for ; Fri, 5 Aug 2022 15:55:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241432AbiHEPzR (ORCPT ); Fri, 5 Aug 2022 11:55:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241475AbiHEPy4 (ORCPT ); Fri, 5 Aug 2022 11:54:56 -0400 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8EE478207 for ; Fri, 5 Aug 2022 08:53:25 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20220805155308epoutp024f926f04129fe4de61fcc22eb2255908~Ifc5rGoeX1590515905epoutp023 for ; Fri, 5 Aug 2022 15:53:08 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20220805155308epoutp024f926f04129fe4de61fcc22eb2255908~Ifc5rGoeX1590515905epoutp023 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1659714788; bh=lOaVuemJcvzVqvhmC9wH8jBWzJSDY06o4HSKqGiM8oo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZakMqwxC5Pm1/VO6o7e4U6t+nRdNPho7hnl4aQaQ612xzmtgCrx+IHWilRPJMY46t L4Cgt5ld6mwZ+mTJUSQ7bTHqWhMQ3/bsz3UpHhxEgoY21GhmdSwhsX2oNC+17/lLcO oCX52cGLFBcqGpfMK/siVdTvn62CXyZVbQEUgV9c= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220805155306epcas5p3d8fbce6a79f02a294b7a787a987b8e50~Ifc4V12580910109101epcas5p3t; Fri, 5 Aug 2022 15:53:06 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.178]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4Lzqr06dB5z4x9Pq; Fri, 5 Aug 2022 15:53:04 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 12.4A.09662.0EC3DE26; Sat, 6 Aug 2022 00:53:04 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20220805155304epcas5p1bb687a8f9b25317af39def01696626e8~Ifc2T7P2I2222122221epcas5p1l; Fri, 5 Aug 2022 15:53:04 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220805155304epsmtrp1ce7d0923a98420122d44a8d1b0cd35ca~Ifc2TLlrM0820308203epsmtrp1c; Fri, 5 Aug 2022 15:53:04 +0000 (GMT) X-AuditID: b6c32a49-86fff700000025be-5a-62ed3ce00915 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id E3.B9.08905.0EC3DE26; Sat, 6 Aug 2022 00:53:04 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220805155303epsmtip2d0a9a4c3e09c5280320d566a091e40c6~Ifc0-PsF71887518875epsmtip2O; Fri, 5 Aug 2022 15:53:03 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, joshiiitr@gmail.com, gost.dev@samsung.com, Kanchan Joshi Subject: [PATCH 1/4] fs: add file_operations->uring_cmd_iopoll Date: Fri, 5 Aug 2022 21:12:23 +0530 Message-Id: <20220805154226.155008-2-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220805154226.155008-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCJsWRmVeSWpSXmKPExsWy7bCmpu4Dm7dJBvfXG1msvtvPZnHzwE4m i5WrjzJZvGs9x2Jx9P9bNovzbw8zWey9pW0xf9lTdotDk5uZHDg9ds66y+5x+Wypx+Yl9R67 bzawebzfd5XNo2/LKkaPz5vkAtijsm0yUhNTUosUUvOS81My89JtlbyD453jTc0MDHUNLS3M lRTyEnNTbZVcfAJ03TJzgO5SUihLzCkFCgUkFhcr6dvZFOWXlqQqZOQXl9gqpRak5BSYFOgV J+YWl+al6+WlllgZGhgYmQIVJmRnbD/9mL1gL2vF9Vsn2RoYj7F0MXJySAiYSMy5/Imti5GL Q0hgN6PE14etrCAJIYFPjBLzZqtAJL4xStyevIkdpmPavpOsEIm9jBIv9l5kgXA+M0rcWbUc aBYHB5uApsSFyaUgDSIC8hJfbq8Fq2EWOMQo8XzJK2aQhLCAncTau9vAprIIqEqsXnMIzOYV sJRY3DOdCWKbvMTMS9/B4pwCVhKth7cwQ9QISpyc+QTsB2agmuats5lBFkgItHJIvJ97GOo5 F4mln26xQdjCEq+Ob4F6QUriZX8blJ0scWnmOahlJRKP9xyEsu0lWk/1M4M8wwz0zPpd+hC7 +CR6fz9hAglLCPBKdLQJQVQrStyb9JQVwhaXeDhjCZTtIbGj7Tk7JHx6GSUeTz7HPIFRfhaS F2YheWEWwrYFjMyrGCVTC4pz01OLTQsM81LL4RGbnJ+7iRGcPLU8dzDeffBB7xAjEwcjMGw5 mJVEeH/ueJ0kxJuSWFmVWpQfX1Sak1p8iNEUGMYTmaVEk/OB6TuvJN7QxNLAxMzMzMTS2MxQ SZzX6+qmJCGB9MSS1OzU1ILUIpg+Jg5OqQamRdLbghTKX4QsW7aVaZWf+I6eOe5rP85RU5fq 0N/KdGYe40O5OUUfjidGpn8x55XK5+biYvS6tJ+1gHuXavSeoNZ3YhL3/xg31q0Med3VqJfW sNRzn5rXyanqySoM9wIKnustZlQ4q/I3tEZ3bmHE43+FoksOPN2qp/2+X3fZUrceZ+sZJ3uP yZT86Pi0LHOV+yHhL1UGVrm3Pp20OFYpURP8wMimwyDhRc3P5DeXrtjZKDxTjzKMvJtreyW0 ZtvmvAnzSvv0XeUvNs2+uVv89z7BB4/C2lxDtz7d9tbQs1Hg4INXTkYHDVaujDptO0PKMXCS X2BW2pf3MqE9lQH7bD97aB+KNXv50MPytoQSS3FGoqEWc1FxIgAYCx8EJwQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrOLMWRmVeSWpSXmKPExsWy7bCSvO4Dm7dJBo83y1msvtvPZnHzwE4m i5WrjzJZvGs9x2Jx9P9bNovzbw8zWey9pW0xf9lTdotDk5uZHDg9ds66y+5x+Wypx+Yl9R67 bzawebzfd5XNo2/LKkaPz5vkAtijuGxSUnMyy1KL9O0SuDK2n37MXrCXteL6rZNsDYzHWLoY OTkkBEwkpu07ydrFyMUhJLCbUWL2ojnMEAlxieZrP9ghbGGJlf+es0MUfWSU2H31C2MXIwcH m4CmxIXJpSA1IgKKEhs/NjGC1DALnGKUeH/0DdgGYQE7ibV3t4ENYhFQlVi95hCYzStgKbG4 ZzoTxAJ5iZmXvoPFOQWsJFoPb2EGmS8EVDP/DhdEuaDEyZlPwEYyA5U3b53NPIFRYBaS1Cwk qQWMTKsYJVMLinPTc4sNCwzzUsv1ihNzi0vz0vWS83M3MYLDXktzB+P2VR/0DjEycTAeYpTg YFYS4f2543WSEG9KYmVValF+fFFpTmrxIUZpDhYlcd4LXSfjhQTSE0tSs1NTC1KLYLJMHJxS DUwTOFhSeXxeCuzKFN5jKqnddev+rzkJSnwff22c+jC24K2oreK6DZtM5JwYlQR0UxbEb29s cJd1e7aN61LAk+lLRR58OpM6+2CUWedfXutjlkmLvCZk1n+7fe0ky854xz+7Qrh0WOK+9sj1 l1r++fGjO2KfxJympx9YFbc8Smk+I9Dzvi/15MWzZyzPO/SeDX99f+6GzacLBBelFM9o7HE+ fMlvXaBinfK1oz8v6zpmOWzSPlmTKqKcyihza0l6kKypdMLHubsk34oJXtoR+Z337/+oDs/E x9pLmq5crp8rV7BZcsbhzvjPKVcdZhyJC3Vh/Hr0zudDD5nnfOq5NX+O7b5dSZmSv/4In9nN dn65EktxRqKhFnNRcSIAf9tb1eoCAAA= X-CMS-MailID: 20220805155304epcas5p1bb687a8f9b25317af39def01696626e8 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220805155304epcas5p1bb687a8f9b25317af39def01696626e8 References: <20220805154226.155008-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org io_uring will trigger this to do completion polling on uring-cmd operations. Signed-off-by: Kanchan Joshi --- include/linux/fs.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 9f131e559d05..449941f99f50 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2134,6 +2134,7 @@ struct file_operations { loff_t len, unsigned int remap_flags); int (*fadvise)(struct file *, loff_t, loff_t, int); int (*uring_cmd)(struct io_uring_cmd *ioucmd, unsigned int issue_flags); + int (*uring_cmd_iopoll)(struct io_uring_cmd *ioucmd); } __randomize_layout; struct inode_operations { From patchwork Fri Aug 5 15:42:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12937491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FF9CC00140 for ; Fri, 5 Aug 2022 15:55:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241418AbiHEPzW (ORCPT ); Fri, 5 Aug 2022 11:55:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241072AbiHEPzE (ORCPT ); Fri, 5 Aug 2022 11:55:04 -0400 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70E457D7BB for ; Fri, 5 Aug 2022 08:53:28 -0700 (PDT) Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20220805155311epoutp012097db5e177de01aba74d8b33f9bb1e8~Ifc8gsROe0170701707epoutp01f for ; Fri, 5 Aug 2022 15:53:11 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20220805155311epoutp012097db5e177de01aba74d8b33f9bb1e8~Ifc8gsROe0170701707epoutp01f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1659714791; bh=rdPsAxxvz7Zcyw0iMWU8XUYENn+3wN4ROlvI+GUBGXg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A1HjEbucBXyFdofe2M+NhuqpyC6AF8LVCaRhlxXZShIy2tSAwYTxlCEsrr8YAhP5d unLXcW+LlW9vpFe1UoSUhEuS0kN5of0HonIeKIMBo8naoD6n7uJ9VRdaiQ2D0HqQoO c3vUknUAjPE0t3gftGr1rHAXBhWL093jluAcR8Q8= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220805155309epcas5p384a0ee735459ccd2d8a9ce175a29962d~Ifc7NWRYn0910509105epcas5p3f; Fri, 5 Aug 2022 15:53:09 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.176]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4Lzqr41mS4z4x9Pq; Fri, 5 Aug 2022 15:53:08 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id D3.4A.09662.4EC3DE26; Sat, 6 Aug 2022 00:53:08 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c~Ifc5YwS1t2379723797epcas5p4v; Fri, 5 Aug 2022 15:53:07 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220805155307epsmtrp1e83fbf0cf5317a6c905583bb8c121099~Ifc5YDy1q0820308203epsmtrp1e; Fri, 5 Aug 2022 15:53:07 +0000 (GMT) X-AuditID: b6c32a49-86fff700000025be-62-62ed3ce47304 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 84.B9.08905.3EC3DE26; Sat, 6 Aug 2022 00:53:07 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220805155306epsmtip21e150df3b053853a14b764a604e4188b~Ifc36EXdx0383703837epsmtip2f; Fri, 5 Aug 2022 15:53:06 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, joshiiitr@gmail.com, gost.dev@samsung.com, Kanchan Joshi , Pankaj Raghav Subject: [PATCH 2/4] io_uring: add iopoll infrastructure for io_uring_cmd Date: Fri, 5 Aug 2022 21:12:24 +0530 Message-Id: <20220805154226.155008-3-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220805154226.155008-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJJsWRmVeSWpSXmKPExsWy7bCmuu4Tm7dJBotaOSxW3+1ns7h5YCeT xcrVR5ks3rWeY7E4+v8tm8X5t4eZLPbe0raYv+wpu8Whyc1MFp+XtrA7cHnsnHWX3ePy2VKP zUvqPXbfbGDzeL/vKptH35ZVjB6fN8kFsEdl22SkJqakFimk5iXnp2TmpdsqeQfHO8ebmhkY 6hpaWpgrKeQl5qbaKrn4BOi6ZeYAHaekUJaYUwoUCkgsLlbSt7Mpyi8tSVXIyC8usVVKLUjJ KTAp0CtOzC0uzUvXy0stsTI0MDAyBSpMyM44ecq84Jpcxd4LG1gaGP9IdDFyckgImEgseTyP sYuRi0NIYDejxKJjG9ghnE+MEus2bGCBcL4xSlzs+cUE07L3zRY2iMReRonfq8+yQjifGSW+ 7XkFVMXBwSagKXFhcilIg4iAvMSX22vBJjELvGKUmH25kx0kISzgKfHp02xGEJtFQFXi1OHX bCA2r4ClxPRVP1khtslLzLz0HayeU8BKovXwFmaIGkGJkzOfsIDYzEA1zVtnM4MskBDo5ZCY OnsCVLOLxKk9vxkhbGGJV8e3sEPYUhIv+9ug7GSJSzPPQb1WIvF4z0Eo216i9VQ/M8gzzEDP rN+lD7GLT6L39xOwHyUEeCU62oQgqhUl7k16CrVVXOLhjCVQtofEtt6n0FDsZZQ4f/gmywRG +VlIXpiF5IVZCNsWMDKvYpRMLSjOTU8tNi0wzEsth0dscn7uJkZwGtXy3MF498EHvUOMTByM hxglOJiVRHh/7nidJMSbklhZlVqUH19UmpNafIjRFBjGE5mlRJPzgYk8ryTe0MTSwMTMzMzE 0tjMUEmc1+vqpiQhgfTEktTs1NSC1CKYPiYOTqkGJkvzmNwDm24+Y3hsmrwry+Os5YRNf2Km PJzw+Q17wF2v5Xxd8xZxyl6282+ofNRzsu1NQ8fCpzvXfun4JzKhJkf/MjNbeNs5/rqJkxgF y5ruOKqcUb/wmXtGTInbZoFOnw1r45Z/+yF2+RnP7vjMn/t/X7+4Ml5wVpzGVLsJR777zGE5 UqW5Ofh/k5tG1w77Q3yTZKLKxPa3Gcqxvb17dvOlF5HHJEXneUm/V/hYnv2rfMGBzdW3P92t UqnKDj7mkxLCNv/pp6y8wOMXMySSHwk9mdK9+cmKrzwPfmT939m9skfmmOwk4SIHo5s92+7u YmbbwTBpcaWggNm9+w8F8y75m589FWO2XtPkOk9ZPYMSS3FGoqEWc1FxIgAiwpkILAQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrNLMWRmVeSWpSXmKPExsWy7bCSvO5jm7dJBls+61msvtvPZnHzwE4m i5WrjzJZvGs9x2Jx9P9bNovzbw8zWey9pW0xf9lTdotDk5uZLD4vbWF34PLYOesuu8fls6Ue m5fUe+y+2cDm8X7fVTaPvi2rGD0+b5ILYI/isklJzcksSy3St0vgyjh5yrzgmlzF3gsbWBoY /0h0MXJySAiYSOx9s4Wti5GLQ0hgN6PE5GNzmCAS4hLN136wQ9jCEiv/PWeHKPrIKHHoygWg Dg4ONgFNiQuTS0FqRAQUJTZ+bGIEqWEW+MQoce7SbbBmYQFPiU+fZjOC2CwCqhKnDr9mA7F5 BSwlpq/6yQqxQF5i5qXvYPWcAlYSrYe3MIPMFwKqmX+HC6JcUOLkzCcsIDYzUHnz1tnMExgF ZiFJzUKSWsDItIpRMrWgODc9t9iwwDAvtVyvODG3uDQvXS85P3cTIzj8tTR3MG5f9UHvECMT B+MhRgkOZiUR3p87XicJ8aYkVlalFuXHF5XmpBYfYpTmYFES573QdTJeSCA9sSQ1OzW1ILUI JsvEwSnVwHQq6U/4Ggs55Rkr7i9/9zAomNlpavdGqVvda//OaVNzlPZg2V/50HPta0tt9vn1 era7y9dOrBHu3MsvqOX98rld0vSbt9hWR+3+0XeMKeDQTYOUmXJlc9c9PTAxcsHPRy9OL7iy a2vHN6bFXy5HfDvz5xNrx0HO1FrGS2sfvnsl5f7+UeUau8n/GCdlVpSlGQmV8Nx91rVVvjnG aYEyZ+DTrhsba36csjW3Oy4o//oWa+tiM4sAPzbJLTndeWtkTHL797Q+2Os33SUh/oa7p7f7 Uosv3423LJNepaei2egsffpD9Db1jrpDW5uu7cu/cdzJdmX2htx9S155X87Rnqz6ZEZ4uK7J /ZvffumdfP5KiaU4I9FQi7moOBEAgw4j/+4CAAA= X-CMS-MailID: 20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c References: <20220805154226.155008-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Put this up in the same way as iopoll is done for regular read/write IO. Make place for storing a cookie into struct io_uring_cmd on its submission. Perform the completion using the ->uring_cmd_iopoll handler. Signed-off-by: Kanchan Joshi Signed-off-by: Pankaj Raghav --- include/linux/io_uring.h | 8 ++++++-- io_uring/io_uring.c | 6 ++++++ io_uring/opdef.c | 1 + io_uring/rw.c | 8 +++++++- io_uring/uring_cmd.c | 11 +++++++++-- 5 files changed, 29 insertions(+), 5 deletions(-) diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 4a2f6cc5a492..58676c0a398f 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -20,8 +20,12 @@ enum io_uring_cmd_flags { struct io_uring_cmd { struct file *file; const void *cmd; - /* callback to defer completions to task context */ - void (*task_work_cb)(struct io_uring_cmd *cmd); + union { + /* callback to defer completions to task context */ + void (*task_work_cb)(struct io_uring_cmd *cmd); + /* used for polled completion */ + void *cookie; + }; u32 cmd_op; u32 pad; u8 pdu[32]; /* available inline for free use */ diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index b54218da075c..48a430a86b50 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1296,6 +1296,12 @@ static int io_iopoll_check(struct io_ring_ctx *ctx, long min) wq_list_empty(&ctx->iopoll_list)) break; } + + if (task_work_pending(current)) { + mutex_unlock(&ctx->uring_lock); + io_run_task_work(); + mutex_lock(&ctx->uring_lock); + } ret = io_do_iopoll(ctx, !min); if (ret < 0) break; diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 72dd2b2d8a9d..9a0df19306fe 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -466,6 +466,7 @@ const struct io_op_def io_op_defs[] = { .needs_file = 1, .plug = 1, .name = "URING_CMD", + .iopoll = 1, .async_size = uring_cmd_pdu_size(1), .prep = io_uring_cmd_prep, .issue = io_uring_cmd, diff --git a/io_uring/rw.c b/io_uring/rw.c index 2b784795103c..1a4fb8a44b9a 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -1005,7 +1005,13 @@ int io_do_iopoll(struct io_ring_ctx *ctx, bool force_nonspin) if (READ_ONCE(req->iopoll_completed)) break; - ret = rw->kiocb.ki_filp->f_op->iopoll(&rw->kiocb, &iob, poll_flags); + if (req->opcode == IORING_OP_URING_CMD) { + struct io_uring_cmd *ioucmd = (struct io_uring_cmd *)rw; + + ret = req->file->f_op->uring_cmd_iopoll(ioucmd); + } else + ret = rw->kiocb.ki_filp->f_op->iopoll(&rw->kiocb, &iob, + poll_flags); if (unlikely(ret < 0)) return ret; else if (ret) diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 0a421ed51e7e..5cc339fba8b8 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -49,7 +49,11 @@ void io_uring_cmd_done(struct io_uring_cmd *ioucmd, ssize_t ret, ssize_t res2) io_req_set_res(req, 0, ret); if (req->ctx->flags & IORING_SETUP_CQE32) io_req_set_cqe32_extra(req, res2, 0); - __io_req_complete(req, 0); + if (req->ctx->flags & IORING_SETUP_IOPOLL) + /* order with io_iopoll_req_issued() checking ->iopoll_completed */ + smp_store_release(&req->iopoll_completed, 1); + else + __io_req_complete(req, 0); } EXPORT_SYMBOL_GPL(io_uring_cmd_done); @@ -89,8 +93,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int issue_flags) issue_flags |= IO_URING_F_SQE128; if (ctx->flags & IORING_SETUP_CQE32) issue_flags |= IO_URING_F_CQE32; - if (ctx->flags & IORING_SETUP_IOPOLL) + if (ctx->flags & IORING_SETUP_IOPOLL) { issue_flags |= IO_URING_F_IOPOLL; + req->iopoll_completed = 0; + WRITE_ONCE(ioucmd->cookie, NULL); + } if (req_has_async_data(req)) ioucmd->cmd = req->async_data; From patchwork Fri Aug 5 15:42:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12937492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7872C25B08 for ; Fri, 5 Aug 2022 15:55:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241145AbiHEPzX (ORCPT ); Fri, 5 Aug 2022 11:55:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241361AbiHEPzF (ORCPT ); Fri, 5 Aug 2022 11:55:05 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3C137969F for ; Fri, 5 Aug 2022 08:53:30 -0700 (PDT) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20220805155314epoutp04c855916f3e406f4574a0609a68fcf6da~Ifc-5yha70030300303epoutp04Y for ; Fri, 5 Aug 2022 15:53:14 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20220805155314epoutp04c855916f3e406f4574a0609a68fcf6da~Ifc-5yha70030300303epoutp04Y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1659714794; bh=5IRAjs6+m4OB4xTk3tclAQOLtoFiORzCHerBHOWaHZk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pi6VduDbQO3FvD0OGWcZ797fl67h4GdvYod+R9ieYWd2SdUF6qmyH1TUuDT8Sf86p 1vIb3uLhccOU3tUAnYDriP4aAr98h95dVzxkG4eoILKw2PJNC+x+rP0i1EX/uW062f Hj4/YDIC67HadgiiZHy0KmjbEygkG+KGfjKGonCY= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20220805155313epcas5p3b900662fb2a530dad95248adf6276cdc~Ifc_0k9j12910829108epcas5p35; Fri, 5 Aug 2022 15:53:13 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.183]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4Lzqr72TJXz4x9Pt; Fri, 5 Aug 2022 15:53:11 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 7B.15.09639.7EC3DE26; Sat, 6 Aug 2022 00:53:11 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0~Ifc8FvJhX2017620176epcas5p2U; Fri, 5 Aug 2022 15:53:10 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20220805155310epsmtrp2040be2f0d9c1a34a63a932dd1253ec62~Ifc8FEgkV2288122881epsmtrp20; Fri, 5 Aug 2022 15:53:10 +0000 (GMT) X-AuditID: b6c32a4b-e83ff700000025a7-49-62ed3ce7dee3 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 55.B9.08905.6EC3DE26; Sat, 6 Aug 2022 00:53:10 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220805155309epsmtip2230ca8e5ac0461a35d0f554ac625fb4a~Ifc6wGjCc0383603836epsmtip2T; Fri, 5 Aug 2022 15:53:09 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, joshiiitr@gmail.com, gost.dev@samsung.com, Kanchan Joshi Subject: [PATCH 3/4] block: export blk_rq_is_poll Date: Fri, 5 Aug 2022 21:12:25 +0530 Message-Id: <20220805154226.155008-4-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220805154226.155008-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCJsWRmVeSWpSXmKPExsWy7bCmuu5zm7dJBluO8lusvtvPZnHzwE4m i5WrjzJZvGs9x2Jx9P9bNovzbw8zWey9pW0xf9lTdotDk5uZHDg9ds66y+5x+Wypx+Yl9R67 bzawebzfd5XNo2/LKkaPz5vkAtijsm0yUhNTUosUUvOS81My89JtlbyD453jTc0MDHUNLS3M lRTyEnNTbZVcfAJ03TJzgO5SUihLzCkFCgUkFhcr6dvZFOWXlqQqZOQXl9gqpRak5BSYFOgV J+YWl+al6+WlllgZGhgYmQIVJmRnzH1zj7VgEnfFpp4jrA2MSzm7GDk5JARMJC58PM3axcjF ISSwm1Hiz6w3zBDOJ0aJo9/mMUI43xglln54zgzTsuzpE3aIxF5GiQlXT0H1f2aUOPe4AaiF g4NNQFPiwuRSkAYRAXmJL7fXsoDUMAscYpR4vuQV2CRhAUOJles3gNWzCKhKrLhQCRLmFbCU 2DynE2qZvMTMS9/ZQWxOASuJ1sNbmCFqBCVOznzCAmIzA9U0b50NdraEQCuHxJr1/6CaXSQu 7/nECGELS7w6voUdwpaS+PxuLxuEnSxxaeY5Jgi7ROLxnoNQtr1E66l+ZpDbmIF+Wb9LH2IX n0Tv7ydMIGEJAV6JjjYhiGpFiXuTnrJC2OISD2csgbI9JLbcuAYNq15GicM/21knMMrPQvLC LCQvzELYtoCReRWjZGpBcW56arFpgXFeajk8YpPzczcxgpOnlvcOxkcPPugdYmTiYAQGLQez kgjvzx2vk4R4UxIrq1KL8uOLSnNSiw8xmgKDeCKzlGhyPjB955XEG5pYGpiYmZmZWBqbGSqJ 83pd3ZQkJJCeWJKanZpakFoE08fEwSnVwMR/oi0imY9H3e/9yhucDDJnuHX3XrJ2nx1WaeXF 9XjddfZpnQoF7888PPJ107ztwZdm7rnhfX7zb+mL2yvFT6zoiU88slmE2Zf9qsbE7pXsjab+ bOWXdx7U3tckZNw2S0T1lsjNY+ynfWN3FP1pbFm9vLBnLZetRdeCGYaVV1QbPnEn37/Zzyd1 2iT9Stlb769Hdm4/s+DeoVuG/L33a7bJxu3alyevqldfMcd03ar7f9Lc3n01/nh/5+MvldKb 7vv6XJwkkLPj/UqDn9PEp622aoxm2KTTqHObv/XqHY69dkaNbvsMpN4c3KZ9/uVSfXOBn3vn 6f/NY9oT4znr87nOg+cuLexrYny0Wejg83N6SizFGYmGWsxFxYkAH8iXoicEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrGLMWRmVeSWpSXmKPExsWy7bCSvO4zm7dJBg93alqsvtvPZnHzwE4m i5WrjzJZvGs9x2Jx9P9bNovzbw8zWey9pW0xf9lTdotDk5uZHDg9ds66y+5x+Wypx+Yl9R67 bzawebzfd5XNo2/LKkaPz5vkAtijuGxSUnMyy1KL9O0SuDLmvrnHWjCJu2JTzxHWBsalnF2M nBwSAiYSy54+Ye9i5OIQEtjNKPG85wQjREJcovnaD3YIW1hi5b/nUEUfGSXe9e5m62Lk4GAT 0JS4MLkUpEZEQFFi48cmRpAaZoFTjBLvj75hAUkICxhKrFy/gRGknkVAVWLFhUqQMK+ApcTm OZ3MEPPlJWZe+g62i1PASqL18BZmkHIhoJr5d7ggygUlTs58AjaRGai8eets5gmMArOQpGYh SS1gZFrFKJlaUJybnltsWGCYl1quV5yYW1yal66XnJ+7iREc9FqaOxi3r/qgd4iRiYPxEKME B7OSCO/PHa+ThHhTEiurUovy44tKc1KLDzFKc7AoifNe6DoZLySQnliSmp2aWpBaBJNl4uCU amCqq34WZMh371PW47mqARMYDiy6F/0giC3wt8DH68Hi55/2BFhufHV04/KcOf9OGb2a1xZ8 d8Yf6fd37jxym51UUvtUN/AH978E+989ftsPZcnX3hCuO8IgIjFpDhOTvZJqmfMzrbkB71++ +Sp1bFfHyv2a4tG1paL/eywrbysvuCCbt/nANoUTO6ZOV/2hPYf56cQj/AUiOe7f1+f0f172 +E/okbUNpkxqXJ1XZns+EJgWtLZaY07o6Y7yLbrmad3s2wLvhDMEyU7zEAnv/vDmUeq7GxpH di99IJ95ITU5wT9U1Fpu96RPr8951Vq5PQ/KlKzLLj8mo7FwYcHqeREXPs6LSj4wSel1Qldg UsVNJZbijERDLeai4kQAp56ntekCAAA= X-CMS-MailID: 20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0 References: <20220805154226.155008-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org This is being done as preparation to support iopoll for nvme passthrough Signed-off-by: Kanchan Joshi --- block/blk-mq.c | 3 ++- include/linux/blk-mq.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 5ee62b95f3e5..de42f7237bad 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1233,7 +1233,7 @@ static void blk_end_sync_rq(struct request *rq, blk_status_t ret) complete(&wait->done); } -static bool blk_rq_is_poll(struct request *rq) +bool blk_rq_is_poll(struct request *rq) { if (!rq->mq_hctx) return false; @@ -1243,6 +1243,7 @@ static bool blk_rq_is_poll(struct request *rq) return false; return true; } +EXPORT_SYMBOL_GPL(blk_rq_is_poll); static void blk_rq_poll_completion(struct request *rq, struct completion *wait) { diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index effee1dc715a..8f841caaa4cb 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -981,6 +981,7 @@ int blk_rq_map_kern(struct request_queue *, struct request *, void *, int blk_rq_append_bio(struct request *rq, struct bio *bio); void blk_execute_rq_nowait(struct request *rq, bool at_head); blk_status_t blk_execute_rq(struct request *rq, bool at_head); +bool blk_rq_is_poll(struct request *rq); struct req_iterator { struct bvec_iter iter; From patchwork Fri Aug 5 15:42:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 12937493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CBC5C3F6B0 for ; Fri, 5 Aug 2022 15:55:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241509AbiHEPzZ (ORCPT ); Fri, 5 Aug 2022 11:55:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238648AbiHEPzI (ORCPT ); Fri, 5 Aug 2022 11:55:08 -0400 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FA337E007 for ; Fri, 5 Aug 2022 08:53:32 -0700 (PDT) Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20220805155316epoutp040d5c7d9283eb71bcefe5079a3a624f2a~IfdBPv8un0030600306epoutp04b for ; Fri, 5 Aug 2022 15:53:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20220805155316epoutp040d5c7d9283eb71bcefe5079a3a624f2a~IfdBPv8un0030600306epoutp04b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1659714796; bh=qxbl+D7T7pLG1KnB6mV1fU5QffkSLPLNszuu3vmJO/4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AbHdx7O/O7RjThXhHnEhtA/25fvttZguD96xWDGnowOQEMuF+iSjLgHXt2ong85t4 5AVnW5LnAmKw9PsYASJK/ssQ/T+5Q5yWAmOsAtuqygTHIBUjxao346LPpHznDzagUc qD56sMU2enadiYPg+QZEDr7svePwKQwo7Cqy4lD8= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20220805155315epcas5p17e44e94a063d8e28ffbf77ed7268f871~IfdAvvIm82221822218epcas5p1y; Fri, 5 Aug 2022 15:53:15 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.175]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4LzqrB1504z4x9Pt; Fri, 5 Aug 2022 15:53:14 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 05.4A.09662.9EC3DE26; Sat, 6 Aug 2022 00:53:14 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0~Ifc_2719A0917409174epcas5p24; Fri, 5 Aug 2022 15:53:13 +0000 (GMT) Received: from epsmgms1p2.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20220805155313epsmtrp1f2f0d47e45eedd08b64029b060919afb~Ifc_2JopT0820308203epsmtrp1i; Fri, 5 Aug 2022 15:53:13 +0000 (GMT) X-AuditID: b6c32a49-86fff700000025be-6a-62ed3ce93da0 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2.samsung.com (Symantec Messaging Gateway) with SMTP id 3D.FD.08802.9EC3DE26; Sat, 6 Aug 2022 00:53:13 +0900 (KST) Received: from localhost.localdomain (unknown [107.110.206.5]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20220805155312epsmtip267b29e932ee9955ee8efa409b2995a04~Ifc9Y8xbW1265212652epsmtip28; Fri, 5 Aug 2022 15:53:12 +0000 (GMT) From: Kanchan Joshi To: axboe@kernel.dk, hch@lst.de Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, ming.lei@redhat.com, joshiiitr@gmail.com, gost.dev@samsung.com, Kanchan Joshi , Anuj Gupta Subject: [PATCH 4/4] nvme: wire up async polling for io passthrough commands Date: Fri, 5 Aug 2022 21:12:26 +0530 Message-Id: <20220805154226.155008-5-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220805154226.155008-1-joshi.k@samsung.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprOJsWRmVeSWpSXmKPExsWy7bCmuu4rm7dJBtM3MVs0TfjLbLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLM6/PcxksfeWtsX8ZU/ZLQ5NbmZy4PLYOesuu8fls6Ue m5fUe+y+2cDm8X7fVTaPvi2rGD0+b5ILYI/KtslITUxJLVJIzUvOT8nMS7dV8g6Od443NTMw 1DW0tDBXUshLzE21VXLxCdB1y8wBOk5JoSwxpxQoFJBYXKykb2dTlF9akqqQkV9cYquUWpCS U2BSoFecmFtcmpeul5daYmVoYGBkClSYkJ1x+8Yc5oI+k4qtPQdZGhiPaHcxcnBICJhIXP0m 38XIxSEksJtR4uvmHywQzidGieP3DrF1MXICOZ8ZJd49igaxQRr2/74GFd/FKHHhgihEA1BN /+FmJpCpbAKaEhcml4LUiAjIS3y5vRZsKLPAc0aJpt/N7CAJYQEfiab7PUwgNouAqsTuawtY QGxeAUuJmUuPMkIsk5eYeek7WD2ngJVE6+EtzBA1ghInZz4Bq2cGqmneOpsZZIGEQCeHxOor i5kgml0k/n/th7KFJV4d38IOYUtJvOxvg7KTJS7NPAdVUyLxeM9BKNteovVUPzPIM8xAz6zf pQ+xi0+i9/cTJkjI8Up0tAlBVCtK3Jv0lBXCFpd4OGMJlO0hMXP/fyZI+PQySmxq+8EygVF+ FpIXZiF5YRbCtgWMzKsYJVMLinPTU4tNCwzzUsvh0Zqcn7uJEZxCtTx3MN598EHvECMTB+Mh RgkOZiUR3p87XicJ8aYkVlalFuXHF5XmpBYfYjQFhvFEZinR5HxgEs8riTc0sTQwMTMzM7E0 NjNUEuf1uropSUggPbEkNTs1tSC1CKaPiYNTqoGpc/Px4A+FRx+GJvOwn10fdauM+evVyD3K 6WbFT8/PzOa+l/G8IL1tlajkrevvnP03chms84nfsbj00y2RP2ZpxYf7w32NjAVS5usqO7Cv Z+9YdOTQ0b8LPY0z755pNbqVLyxy6LZvzPtJxzacqbi7y7f1oV4h4wyJhFqzouLzt0ymbfA/ saV0dbaaMNP8CTwsyctVvuWGsLmzb3x8pIPjjnl8uuHJopXey05tjr24Yw/HAXe2IN/9K0Jk SsuvdqWIz3W9JfTrkreiv+HdjV5vtgkKNR6LY/jNEq6X1jhr3rNDt2LX2B9d32KTcOSX8+wZ 0pF725o0dV5vbTy9oUni3tXpNoovzlbULF22OzpPiaU4I9FQi7moOBEAdtQlICoEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrDLMWRmVeSWpSXmKPExsWy7bCSvO5Lm7dJBpOnaVg0TfjLbLH6bj+b xc0DO5ksVq4+ymTxrvUci8XR/2/ZLM6/PcxksfeWtsX8ZU/ZLQ5NbmZy4PLYOesuu8fls6Ue m5fUe+y+2cDm8X7fVTaPvi2rGD0+b5ILYI/isklJzcksSy3St0vgyrh9Yw5zQZ9JxdaegywN jEe0uxg5OSQETCT2/77G1sXIxSEksINRouvAYSaIhLhE87Uf7BC2sMTKf8/ZIYo+Mkr82rkV yOHgYBPQlLgwuRSkRkRAUWLjxyZGkBpmgfeMEnfm/WQGSQgL+Eg03e8BG8oioCqx+9oCFhCb V8BSYubSo4wQC+QlZl76DraMU8BKovXwFmaQ+UJANfPvcEGUC0qcnPkErJUZqLx562zmCYwC s5CkZiFJLWBkWsUomVpQnJueW2xYYJSXWq5XnJhbXJqXrpecn7uJERwBWlo7GPes+qB3iJGJ g/EQowQHs5II788dr5OEeFMSK6tSi/Lji0pzUosPMUpzsCiJ817oOhkvJJCeWJKanZpakFoE k2Xi4JRqYBK6bdlz+/9Vt8/8UdseOoQUhJ2QPHLX5fmCLnPR8jiZrgbV3zcdLqb0N2+aorz1 6amgWpcNM9bMuyibbnm6ZIvTsulactN3rXnwafeqn8YzKgunmf7XVlS87fVmRv7fbfbulzfw vBO9pZfAEuEX+GuSTligl/jNFwFqIW/q/v7bzC2UPGeJWXO/P49csfDF9tg9k8wbtzlOfnGV 729N23vW82mT3z+o1Hb3Z17UdMiu3a/rSeuDpriFwopPTBounuL4rbuTY3maadStnqdndC5J MFXt8tQpYBLWD9FWdrL/kvXofHH1n57g+971QnxbF95UCtD58q9SpWqdnsLys49bJuoHJDJq 1PqvTva7osRSnJFoqMVcVJwIAHoMtjHvAgAA X-CMS-MailID: 20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0 References: <20220805154226.155008-1-joshi.k@samsung.com> Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Store a cookie during submission, and use that to implement completion-polling inside the ->uring_cmd_iopoll handler. This handler makes use of existing bio poll facility. Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- drivers/nvme/host/core.c | 1 + drivers/nvme/host/ioctl.c | 73 ++++++++++++++++++++++++++++++++--- drivers/nvme/host/multipath.c | 1 + drivers/nvme/host/nvme.h | 2 + 4 files changed, 72 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 2429b11eb9a8..77b6c2882afd 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3976,6 +3976,7 @@ static const struct file_operations nvme_ns_chr_fops = { .unlocked_ioctl = nvme_ns_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .uring_cmd = nvme_ns_chr_uring_cmd, + .uring_cmd_iopoll = nvme_ns_chr_uring_cmd_iopoll, }; static int nvme_add_ns_cdev(struct nvme_ns *ns) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 27614bee7380..136f0fd25710 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -391,11 +391,19 @@ static void nvme_uring_cmd_end_io(struct request *req, blk_status_t err) struct nvme_uring_cmd_pdu *pdu = nvme_uring_cmd_pdu(ioucmd); /* extract bio before reusing the same field for request */ struct bio *bio = pdu->bio; + void *cookie = READ_ONCE(ioucmd->cookie); pdu->req = req; req->bio = bio; - /* this takes care of moving rest of completion-work to task context */ - io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); + + /* + * For iopoll, complete it directly. + * Otherwise, move the completion to task work. + */ + if (cookie != NULL && blk_rq_is_poll(req)) + nvme_uring_task_cb(ioucmd); + else + io_uring_cmd_complete_in_task(ioucmd, nvme_uring_task_cb); } static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, @@ -445,7 +453,10 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, rq_flags = REQ_NOWAIT; blk_flags = BLK_MQ_REQ_NOWAIT; } + if (issue_flags & IO_URING_F_IOPOLL) + rq_flags |= REQ_POLLED; +retry: req = nvme_alloc_user_request(q, &c, nvme_to_user_ptr(d.addr), d.data_len, nvme_to_user_ptr(d.metadata), d.metadata_len, 0, &meta, d.timeout_ms ? @@ -456,6 +467,17 @@ static int nvme_uring_cmd_io(struct nvme_ctrl *ctrl, struct nvme_ns *ns, req->end_io = nvme_uring_cmd_end_io; req->end_io_data = ioucmd; + if (issue_flags & IO_URING_F_IOPOLL && rq_flags & REQ_POLLED) { + if (unlikely(!req->bio)) { + /* we can't poll this, so alloc regular req instead */ + blk_mq_free_request(req); + rq_flags &= ~REQ_POLLED; + goto retry; + } else { + WRITE_ONCE(ioucmd->cookie, req->bio); + req->bio->bi_opf |= REQ_POLLED; + } + } /* to free bio on completion, as req->bio will be null at that time */ pdu->bio = req->bio; pdu->meta = meta; @@ -559,9 +581,6 @@ long nvme_ns_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg) static int nvme_uring_cmd_checks(unsigned int issue_flags) { - /* IOPOLL not supported yet */ - if (issue_flags & IO_URING_F_IOPOLL) - return -EOPNOTSUPP; /* NVMe passthrough requires big SQE/CQE support */ if ((issue_flags & (IO_URING_F_SQE128|IO_URING_F_CQE32)) != @@ -604,6 +623,23 @@ int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) return nvme_ns_uring_cmd(ns, ioucmd, issue_flags); } +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +{ + struct bio *bio; + int ret; + struct nvme_ns *ns; + struct request_queue *q; + + rcu_read_lock(); + bio = READ_ONCE(ioucmd->cookie); + ns = container_of(file_inode(ioucmd->file)->i_cdev, + struct nvme_ns, cdev); + q = ns->queue; + if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio && bio->bi_bdev) + ret = bio_poll(bio, 0, 0); + rcu_read_unlock(); + return ret; +} #ifdef CONFIG_NVME_MULTIPATH static int nvme_ns_head_ctrl_ioctl(struct nvme_ns *ns, unsigned int cmd, void __user *argp, struct nvme_ns_head *head, int srcu_idx) @@ -685,6 +721,29 @@ int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd, srcu_read_unlock(&head->srcu, srcu_idx); return ret; } + +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd) +{ + struct cdev *cdev = file_inode(ioucmd->file)->i_cdev; + struct nvme_ns_head *head = container_of(cdev, struct nvme_ns_head, cdev); + int srcu_idx = srcu_read_lock(&head->srcu); + struct nvme_ns *ns = nvme_find_path(head); + struct bio *bio; + int ret = 0; + struct request_queue *q; + + if (ns) { + rcu_read_lock(); + bio = READ_ONCE(ioucmd->private); + q = ns->queue; + if (test_bit(QUEUE_FLAG_POLL, &q->queue_flags) && bio + && bio->bi_bdev) + ret = bio_poll(bio, 0, 0); + rcu_read_unlock(); + } + srcu_read_unlock(&head->srcu, srcu_idx); + return ret; +} #endif /* CONFIG_NVME_MULTIPATH */ int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) @@ -692,6 +751,10 @@ int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags) struct nvme_ctrl *ctrl = ioucmd->file->private_data; int ret; + /* IOPOLL not supported yet */ + if (issue_flags & IO_URING_F_IOPOLL) + return -EOPNOTSUPP; + ret = nvme_uring_cmd_checks(issue_flags); if (ret) return ret; diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 6ef497c75a16..00f2f81e20fa 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -439,6 +439,7 @@ static const struct file_operations nvme_ns_head_chr_fops = { .unlocked_ioctl = nvme_ns_head_chr_ioctl, .compat_ioctl = compat_ptr_ioctl, .uring_cmd = nvme_ns_head_chr_uring_cmd, + .uring_cmd_iopoll = nvme_ns_head_chr_uring_cmd_iopoll, }; static int nvme_add_ns_head_cdev(struct nvme_ns_head *head) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index bdc0ff7ed9ab..3f2d3dda6e6c 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -821,6 +821,8 @@ long nvme_ns_head_chr_ioctl(struct file *file, unsigned int cmd, unsigned long arg); long nvme_dev_ioctl(struct file *file, unsigned int cmd, unsigned long arg); +int nvme_ns_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); +int nvme_ns_head_chr_uring_cmd_iopoll(struct io_uring_cmd *ioucmd); int nvme_ns_chr_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_flags); int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *ioucmd,