From patchwork Fri Sep 30 15:55:59 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9358573 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DF4E16077B for ; Fri, 30 Sep 2016 15:56:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CE84B2A0CC for ; Fri, 30 Sep 2016 15:56:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C257D2A0CE; Fri, 30 Sep 2016 15:56:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, T_TVD_MIME_EPI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 135112A0CC for ; Fri, 30 Sep 2016 15:56:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933377AbcI3P4R (ORCPT ); Fri, 30 Sep 2016 11:56:17 -0400 Received: from mail-cys01nam02on0078.outbound.protection.outlook.com ([104.47.37.78]:6976 "EHLO NAM02-CY1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933082AbcI3P4O (ORCPT ); Fri, 30 Sep 2016 11:56:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandiskcorp.onmicrosoft.com; s=selector1-sandisk-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=6qJy0FLnhim4ldjByMwl70uVakRBYPK0Yl03CVYI9vs=; b=TfUUSSOzg7lnSaUZXeYUItbMxBcUPSUqMtb2Bz6bUwbSYgIY028wGrg4KcBRH1G9RczT2lRh/Wo2Ox7aTmmpa7l8oqGekugWg7c+nI8SQmR83LYp1PFBeuOTocT8c+UC/pi6yKqOpao0MDZuE/6hntRjSzuCmZHc0oqvicCBKe8= Received: from MWHPR02CA0002.namprd02.prod.outlook.com (10.168.209.140) by CY1PR02MB1625.namprd02.prod.outlook.com (10.162.160.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.649.16; Fri, 30 Sep 2016 15:56:12 +0000 Received: from BY2FFO11FD013.protection.gbl (2a01:111:f400:7c0c::194) by MWHPR02CA0002.outlook.office365.com (2603:10b6:300:4b::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.639.5 via Frontend Transport; Fri, 30 Sep 2016 15:56:12 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.21) smtp.mailfrom=sandisk.com; lst.de; dkim=none (message not signed) header.d=none;lst.de; dmarc=bestguesspass action=none header.from=sandisk.com; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.21 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.21; helo=milsmgep15.sandisk.com; Received: from milsmgep15.sandisk.com (63.163.107.21) by BY2FFO11FD013.mail.protection.outlook.com (10.1.14.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.5 via Frontend Transport; Fri, 30 Sep 2016 15:56:06 +0000 Received: from MILHUBIP03.sdcorp.global.sandisk.com (Unknown_Domain [10.201.67.162]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 3A.CE.26959.DB88EE75; Fri, 30 Sep 2016 08:46:06 -0700 (PDT) Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.301.0; Fri, 30 Sep 2016 08:56:05 -0700 X-AuditID: 0ac94369-babff7000001694f-31-57ee88bd8ccc Received: from [10.50.231.78] ( [10.177.8.100]) by (Symantec Messaging Gateway) with SMTP id D9.50.03606.11B8EE75; Fri, 30 Sep 2016 08:56:04 -0700 (PDT) Subject: Re: [PATCH v2 4/7] blk-mq: Introduce blk_quiesce_queue() and blk_resume_queue() To: Ming Lei References: CC: Jens Axboe , Christoph Hellwig , James Bottomley , "Martin K. Petersen" , Mike Snitzer , Doug Ledford , "Keith Busch" , "linux-block@vger.kernel.org" , "linux-scsi@vger.kernel.org" , "linux-rdma@vger.kernel.org" , "linux-nvme@lists.infradead.org" From: Bart Van Assche Message-ID: <9fb67745-954c-a70f-a89a-d184c99ae9aa@sandisk.com> Date: Fri, 30 Sep 2016 08:55:59 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrOIsWRmVeSWpSXmKPExsXCddJ5ke6+jnfhBlfe6Fv833OMzeLl+Q+s FitXH2WyWDUxz+Le0S9MFntvaVvMX/aU3eLZoV4Wi+7rO9gslh//x2TRtvEro8X7H9fZHXg8 Jja/Y/fYOesuu8fiPS+ZPB4c2szisXlJvcfumw1sHh+f3mLxeL/vKpvH501yAZxRXDYpqTmZ ZalF+nYJXBmbHmQWNHtWND+0bmC8aN3FyMEhIWAi8eMJXxcjJ4eQwFImicv3fLsYuYDsHYwS xxd3soIkQGq+7N/GApGYyyjR++MsG0hCWCBKYs2C54wgtoiAisSjns3sEJN2MUps+xUA0sAs 0M0i8WzOe7AEm4CRxLf3M1lAbF4BO4nLp56CbWARUJU4sHUDM4gtKhAhcethB1SNoMTJmU9Y QC7lFAiU6HmYAGIyCwRInPisDjJeQuAnq8Tms42MEHvVJU4umc80gVFoFpLuWQgtIGFmAQuJ mfPPM0LY8hLb385hhrBDJbbMfcCEKR4n0X7kAtMCRvZVjGK5mTnFuempBYamesWJeSmZxdl6 yfm5mxghUZy5g/HuE+9DjAIcjEo8vCei3oULsSaWFVfmHmJUARr0aMPqC4xSLHn5ealKIrwn W4HSvCmJlVWpRfnxRaU5qcWHGKU5WJTEed3ufQkTEkhPLEnNTk0tSC2CyTJxcEo1MG6ZxXXs zCbrhBPplw4fjr50hlNpX9LOrK+HYxzlhdZw33zw/KqtyNfLNw+5eRbmmvM1MEi8Pa008+q8 95H/Fi1W37119bWvym0Znj5Wt6VvrWKvqbzoHbnKpHu994NDezxe+xkuNoz4V//x3Vupntdh 02wvxFmaTZcOqxZ8drBYb7btnyO6XExKLMUZiYZazEXFiQANH3LO6gIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupiluLIzCtJLcpLzFFi42Lh2siRoivS/S7coGGFmcX/PcfYLF6e/8Bq sXL1USaLVRPzLO4d/cJksfeWtsX8ZU/ZLZ4d6mWx6L6+g81i+fF/TBZtG78yWrz/cZ3dgcdj YvM7do+ds+6yeyze85LJ48GhzSwem5fUe+y+2cDm8fHpLRaP9/uusnl83iQXwBnFZZOSmpNZ llqkb5fAlbF/eTNzwXrPiqN3djM3MH6x7mLk5JAQMJH4sn8bSxcjF4eQwGxGiWv35zCCJIQF oiTWLHgOZosIqEg86tnMDmILCexilNj2KwCkgVmgl0XiQVcrE0iCTcBI4tv7mSwgNq+AncTl U09ZQWwWAVWJA1s3MIPYogIRErcedkDVCEqcnPkEyObg4BQIlOh5mAASZhYIkHj3aybrBEbe WUiqZiFJLWBkWsUolpuZU5ybnllgaKRXnJiXklmcrZecn7uJERzanFE7GK9PND/EyMTBKdXA KJvXv3XDtT8trqdsLc0PfLzwxdUiidt1TfKty8uCDxSns+cvzv7ILfa684269TTVUouVFybZ ChqEiC1me/fI46hI3ZvdPxtudJ1ol759/aiDc6R5XVRJ3Yr5G1ezaG0RaxV+HJjuMHWzL2P6 7RoGnt07ptTfbWU9YxRfXOjuL3esoirY4aKIEktxRqKhFnNRcSIAHIVhMB0CAAA= X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:63.163.107.21; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(189002)(24454002)(377454003)(199003)(81166006)(270700001)(8676002)(4326007)(76176999)(54356999)(7846002)(11100500001)(65826007)(568964002)(31686004)(50986999)(8936002)(86362001)(626004)(106466001)(31696002)(356003)(4610100001)(5660300001)(7416002)(33646002)(84326002)(97736004)(87936001)(4001350100001)(189998001)(92566002)(110136003)(6666003)(81156014)(2476003)(83506001)(5890100001)(68736007)(65956001)(2950100002)(6916009)(77096005)(586003)(512874002)(69596002)(2906002)(64126003)(65806001)(36756003)(19580405001)(19580395003)(2270400002)(305945005); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR02MB1625; H:milsmgep15.sandisk.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11FD013; 1:GjuzeFQQsBLeUWP+RRcVwoIMpAlWFk4MIw8HXK7/ZtzRHubY/bMpo+d9vJY0Esi+A9G42SzR8COt79TkD9rbXvT0vpyLFsbI5V0WqXfBkOvGhmQOks1W9Zj4gTib7U5L7c8U9e9lWMiMzuRW2Hqv4g72liDAK4sIfSPtcigOksMcvITjalPjbK6Qfjmcffdx8HWtGbNL5LxJjuMQxogjTXQfEpvQeEeizBc389igF6A6GvZAhc21tjlqsD83601a12ynoJxVx0vcUKf1MhanfIWzjSdUaG7btHeASegULakVW2nwyhxd4UY8pNj20d+jZFFUSkOaDtNZfJ/6KH75btJBVB2v9CSG9oHkIN45e65wRlKArY5OToeJ+UTMP+99JfT+VMFnkulnw1c6QUgBtrzd6t1tcmi/Q7VyTR40iqjUwKYEe2iTTlxnuvC9w48o7mqh2a61OSr+iAQ7Ak2WFlucnBmRRDhdpnZLYKStFzWADJGZJF3jv9liyOQtGoS9GKR6ZnphdU/JvQePUrQO1k7fECFULbE//BCU6Ril3wV2vxvW+lsmvdGdwo1XtvBz X-MS-Office365-Filtering-Correlation-Id: 9210ea1e-b1d1-4b66-1874-08d3e94a4c9a X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 2:WLQSoQAsK06CN51sNNopShz0cMr0IxHgAsqkj8LfMKmhhuf7UPDgzXh1dEOKaIFDhWL7GhainTzgjGPjqsFDFbiLuQvzpDpG7TBDqsqg5aEOm7lTWK1f2riTkMxZ280gstKhjxvO/bYaKR0C2MLqvCkxlv/47zw0rkxB0QwEsicO+Xf2ivK12yQVApqqdwWWQzslc+QBKkpxWLKvDdyjhw==; 3:1zMjs+meAat+Hy1p0H/YGcmV4Ye5piZG7Kg8UH1fIPWOIzILklWCoQ5krR8y1SutUSicIb5Ezx6rEa5Rj7zpZu3hwrflRatDl8V+601WmLukPSLPtajKxaPvyV7U9nttTYsz9/oKG4gEtlu0ugObODgmv8fJAy9AMcOj7GuTvEZ8xnh3lHwi1PREJBZBN4OQIgdmovwNFRqEQL6s+1y2LGjux3aPc2l2lOVJTVbge6beKdkIKBEaozaL5+C0OuyggA1i/6rx7jQx5e3smGjts+G0+avchI4P7e4lCI7dG5w= X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:CY1PR02MB1625; X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 25:nkM8OGazcxMVp26DxOvkOwYHN7NY9GgIS8bKbAFByIEMFqFVfkd9UGRbbUyPiSGx9hD9Tmuo36W3T22YozSau5wS59VLc59f0I4AA3fKv4ygEkCcTfIpWoC773lcEzVoEsqN1VPK2S5ZTCFQk9dFro/T8wVx6sFmK1tcqB7JF9GPhTClMxMAa/QuFMAi3mw9LrSeI/GKCpDk88o+HN9ZrxzpxsjqbJTT1TBpyLJofmxHVxS9RzVaoCjkLS3zi5Qj4h1V6DsBRFGmz8swtxt2XILYCAqW1xnVwaCS7T9NhgYGNC588nG8BViaPiYgM4NQg1HC7nK9BItxoDwORC7UTQvgC0I+ogQkfIx1xJZC67FMNQqQPIbBJ+xfzkDo4cWcIsOk0y/rXwA4JSlJss3vfqhlxqsgqiniCeH53XUkKbF6H8xnF/ek1UPGYqh3AA8JqHSPw6SUQl37QbmS24jnrqcNL4UqnN173FgI5IJr9Tp6n1H98PST7e4gYOYMZYG6Xer5uFgtO/4ivHVGA0Gin7OLNJawnx1Y7/V4FpbOqqlr6KFUk4JF43AjS62xyhXcc5x8XAso6fOQkGu8ayp7AXzmoTUHnk7nnzTNe9hGUUHXsTih3u41Bt7lzqWiHmqSGh2ULgaCrZCAOmWgf6Z0kARc2t4aL2X3+0XxFqFmfqsacQm/Sj+8Axxg8sCFehPffIuElBHg7h8wIL4+QUIfjFVLbt5F7+Za+QH5dx6EAMituaVuoo559/35JuWEDnqInEPURVFmBs2mgCZhlLv9YWCdI5QSjLlBTg6rOGwp7fGx7bRPegOpuMDIUt06cSSdycuq4Wb7KV6GsvB1ByoUyE9SPi6L6Dig1TSBMHmop6yLBK3vT2Fsqzy5diWVAV/aqMcNBAQEE9fI7eI1UlgBrSt5l+iTbfr79kb8VrVMj xY= X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 31:1VQ03tDcotNPLrmZf7btIiIUYs++lamCLiihBCcin/r4ECr6qgcUYrgk9SkODdbRZmBzFdntLy2fL38yyYao6G5fj4vEW3hotRf+vcTu3blgSzALtxEDuyo9PKzeQoFDiQYjyk8ewOv6jusoOwF4MjEsvevor4+Rqi+MnDErzoZhRbanfw5ND7r3DAg2FFSLxbz3bNNlOLk8xtWtVuyOmv0LKz5SLfRrVR5rIKZKIsmCVeXJD8mdCewEoDJlCbbIPDT46uPrkH+/oo/4tPNOcw==; 20:FMVz5Btu8qe3echQWXbY4TYQnMWLif1SZoeWJkcfiEl+43X4StSX7LCnknCmokyxamocThqsZ962iSyTzhKrlP6EeikjJJhIROfOH/g5Ap/a6hCIdiK+Zo0fXQye+O3UsAsM7Njr17yJBlzjZS84YZcxwJXedzo4Ei+Je6Bo5wPdU+jrz6+/ltR64VrjVH21CLMSLekvagcQwCSILQvA/iW5GuvM2iE4aZCUkBbvB0ZkrKSH4yg/woWUGiyGHMaChsuUpfhDiWSixiw99UHFb40MmazZvpKifuwjIDDSL7o9QrAPzhxL2fboqdKquyFcbGBvQxwynIAmNG7yRSRKLBPEyZIPmj7LjHv4IBoIUfibvdl/PzjCaiveu8Gl1DA9zXFzbtpm3pz6bYE/EUc1TGyvtU3EQuc6gwaYfj6Oz++UcBv020bcUu+uJTnmwZg/ES9YxCcv0mlXWaIP6vnTeCo8buS96gc6zWRYd/NlDi55aLDy/Sz6lhhFfaacBx9a X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(42932892334569); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(102415321)(6040176)(601004)(2401047)(8121501046)(13018025)(13016025)(5005006)(3002001)(10201501046)(6055026); SRVR:CY1PR02MB1625; BCL:0; PCL:0; RULEID:; SRVR:CY1PR02MB1625; X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 4:/MM3vmb13FN0To/rZ2LRWkCmi9pmQAKRCuRSvj4XBjS3krghHZ+1zqS19UFQIBEb2J9l59OyGTWCwdZd9dLVlIVc2XRj5BNdUS9OPOtUoCRm9NgaUPitm0RM4EdIGS6TT6MaystWwX2TDRsOqYSpEOnMW62LNAOAMfVw1TwupznBaWUkNT3tUqOYZycC0rjSE4Bl2emRRJEvSZo29rrLem91Z+Og94Swva7OUvft8C+3AWN+fRio1xXFp/4RwLahN0AzzAGgTXs6sH1Y318z0UDFyuye5WThQ2UsPBhlseJg640Zqq6jzeuI2QTFPBkIJnRfccMTvnAEaU5sU3lgGiIeeYdC4mGaKEYynVhJihmcl5mmuIJMmGEB6rCLsLj5FCZdfLca005QjfIIdQ1Im1W/L8TCe3FTUhluKFqh0kucNt6SUixuX6j/X8WurIUSsBV33/GlrvYpO4S9Zxk+0P1mMXGwxAmlhWLejH8g2N2PKIUaqeozkdFwB36K26OPfpqvR1pK4r03+gk5ekVioA== X-Forefront-PRVS: 008184426E X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR02MB1625; 23:GFlnHEXvUFf1W/38OJLyjfv1fkgQZY/5FjKsDO3aH?= =?us-ascii?Q?TZruKDarHAWczhC27C7uJ3rGSOL96tpixMGZzX9r2IC1Zx/gdAFH/gCgQ8SK?= =?us-ascii?Q?6AleZgawM0qzME1aF7SIiFxodQ8cP8IvZ60DPSSZKYPgsatADb9MmbC+4V3w?= =?us-ascii?Q?fm6/FcUdsVjHOeHcMaKFJsLoTgCzlF6qJIHhL7jBd1ij2RRiKeEBQyNSKf5V?= =?us-ascii?Q?PUjKoX1ILUeJaa9lpdfi40UNfp+BmMWVQKTVLaMUB5z9yColX9VWl+BJaTPp?= =?us-ascii?Q?dU/XirHx0HNR9kfO5y95MtZIOUExg+BEY0w81CePQb80MTb5lvLZ12yF6BIV?= =?us-ascii?Q?CQmxg5x+iqtR2WjmWK8EEcjMuAsumhFbgb98YWX2t4KyUvLOgTXQdNSwJmnK?= =?us-ascii?Q?msGwXRgMke5iHICV3Un5eVjSnBc9OIjkVPCbn1+mMQNO0yVNPg1/hfL59kU8?= =?us-ascii?Q?y9YLogJvt/7NJAMcQLQUGEonxn/wTMuuSaRDft6MjiY/lV6H9jZK5k/Re9Ki?= =?us-ascii?Q?USv2zSbJhKoZMkaulv4UPQbMA+7l0ZUxnTJVV3XVj4672089svgho68jkslE?= =?us-ascii?Q?NYL7xIYqOWIBxvIOluOoqmg/PyqPM2mJJH5Z4tkRSIoXeW/5OgIeQuqxJ1t7?= =?us-ascii?Q?60C946hqb5AX/XsA5bq+wjZdbDSv2Nxnt8Xx0A5Nt7BfgH0aKkoGA/LW95YE?= =?us-ascii?Q?X7xLKQL8Q0zOxUaLVy8NYUh17da+v4xlvYTUycsiKMn4MCmc/+gZcVc/4rLS?= =?us-ascii?Q?jtBDXtUgt6XnT2f48Uo1B7pA2DoZ3F/o3x6+8FZ6F2e2Y4EYL/t7tTZPgusR?= =?us-ascii?Q?0PCUhIG6QP2Ojzk7sq2pwv5ZJ6q7av2dIEv5x7BVc3gR+N4o8FDHS5ek6t0J?= =?us-ascii?Q?ipBbXH8CJWdlNiUflpDauheRnU8IvweW5qH7ocsXTCz607ZUSwUs+QcDRj86?= =?us-ascii?Q?0u88xfF4PufK/aO283QkO4faNDDj8NHAUUtkFb276mddexa7WTrMvpqlfP8G?= =?us-ascii?Q?bZ6zB6NrN9PzFEHmR7QInBZhVXET6OyCXIiJsCuYj1nPVgVpbgKVnivIZvCp?= =?us-ascii?Q?PzOiPINTCY376kDFhLVmVj34y5lBRha8p/WBIzhf8O7MJNL/R/yckSD8x1Fr?= =?us-ascii?Q?KKuQRVwUlpv8kZ464Wt7Bvdyb8sRPYgcdL2fzxG5pHQy8BxxUi4iTocivKPD?= =?us-ascii?Q?X2lkzKz5gA4DfU4BnTagrufqwNgjSfts7QdN9imU+tmQQZELb8eVU9AQ0TFf?= =?us-ascii?Q?Wxf8jroTxfr5o4QVpRY4Y9iW1l6MEZt0PI1wYmKfQY3fZ2dXnEFKxHLkZHIt?= =?us-ascii?Q?nuYg/pzt3q3Rx1SIDmDi3A/PHpXgk9nIkY7IFL9O19foIi2dm4s0xo1dY5KJ?= =?us-ascii?Q?O+s6D4wH5weWyn14UZCjkTnTLRiidGt16kj0LpaA9IWRPoJ?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 6:JZaegz67v6BBzKVaCJuTjQgSwwwzbwaM6TM3ALYDpxnwgfljouJsqBR1EaUks0BrDE6cKMckmSskXinXQNze4au1AQamPHfJBd7BB/LXr0Yk2ZhwyVU81D9iyyVutcNbB5KNsfAUDAPPd14aMONwlPby9PSBl8qOJRIUN5AbAGuNWioKOH9q54jP0PMzXYi0oNmHNtv1hXUoxZ/GUjUj5EX3DDW9nqsux7bDJ+fJZ+LQ92erTf6tOfrAe7ZDWXkBmEwHP/C8cNTkKCZUlUWx8qaahtFZNRoKaq9aWFJ8nz/wQ1mhORNGpABYX/R3NFuC3R7qgb5lpkLGu+5ItObmRU7gvwyPEPIxIGVY6zQ9d/0=; 5:O+8ZFDPvaHH5W/SHkG0RxcVMRuLN0PuDK/2sIxbu1Mjp8LwDTnyx84mRyCdhm/o1LyXKANGIs5+f5jawaCx5aj49r66Wkg5nx8pd6XVgBCvsgS8N59ZRcNuJcq/D/yeGgQEwHprne5baR/ujp3Zi1KRHSiCDpU+KH/w9G76S+ow=; 24:WWcMacl/pQlVeS4XjpAeL5iKo+r+vWighaimOtPDlNzmjLaCrp37L7ah0BM/e0kwG7H/m1Jik/f2SFvun1LAe19kyv7jWQ+t+m2BwuBvgic= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR02MB1625; 7:zgpp40ffeyO146lYmtBDxHohzy2VF037ARNbLrtVpmYkXkiwQap0kjUiJsHWXOqtM1CVSUpsXC3/lfjLbr+UgK3Jm5D3ODBE7kFgrSOnWK9nRnFuiCLh4gRzuOQ3U9ZlshzZZY6YXaHaLr1KauhXBU21yBrT4DSaSyE4CzL1eO0GMzsSCLkaCBBP2u6DeBnbmBB3dbDp4D6cv3fG0tKKDtr1xjLr8Og1bD5keyOCB+Rd4uW8mr9q6DFOtsIc+YHNq9/XmZ8iQdWa5r+08ng3EMkqj+4a4N+Xo0XRUvr/fNYnsgjlUiiX6rkPYiM6DA8w81V/BkiQKecYr62swcDGzg==; 20:DGFy5twWZKBhva2igoXcoymHgRAVjewTVpj3nomP/VkI5ynUCnx1p6vC6/82i2smwJWvGCGo9C2LjbnJ2vT1oTlNY9VfBJUOhnyC+NWj1ES3feLLGkeIgSpEFduii8R1ofpa84FBe5sY7JGPKBPRp3A1C3m2YWkNIyLD3QVCA7FFZDjiSTMOghzltH8RfIuzwaLj4z8qe+p5jg/k7yZkCEnPV/5fYQb7AzQUVSPeTRffJHAz1pmbKd7vioIa0qQV X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2016 15:56:06.8334 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.21]; Helo=[milsmgep15.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR02MB1625 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 09/29/16 14:51, Ming Lei wrote: > On Thu, Sep 29, 2016 at 7:59 AM, Bart Van Assche > wrote: >> blk_quiesce_queue() prevents that new queue_rq() invocations > > blk_mq_quiesce_queue() Thanks, I will update the patch title and patch description. >> +void blk_mq_quiesce_queue(struct request_queue *q) >> +{ >> + struct blk_mq_hw_ctx *hctx; >> + unsigned int i; >> + bool res, rcu = false; >> + >> + spin_lock_irq(q->queue_lock); >> + WARN_ON_ONCE(blk_queue_quiescing(q)); >> + queue_flag_set(QUEUE_FLAG_QUIESCING, q); >> + spin_unlock_irq(q->queue_lock); >> + >> + res = __blk_mq_freeze_queue_start(q, false); > > Looks the implementation is a bit tricky and complicated, if the percpu > usage counter isn't killed, it isn't necessary to touch .mq_freeze_depth > since you use QUEUE_FLAG_QUIESCING to set/get this status of > the queue. > > Then using synchronize_rcu() and rcu_read_lock()/rcu_read_unlock() > with the flag of QUIESCING may be enough to wait for completing > of ongoing invocations of .queue_rq() and avoid to run new .queue_rq, > right? That's an interesting thought. Can you have a look at the attached patch in which blk_mq_quiesce_queue() no longer manipulates the mq_freeze_depth counter? >> + WARN_ON_ONCE(!res); >> + queue_for_each_hw_ctx(q, hctx, i) { >> + if (hctx->flags & BLK_MQ_F_BLOCKING) { >> + mutex_lock(&hctx->queue_rq_mutex); >> + mutex_unlock(&hctx->queue_rq_mutex); > > Could you explain a bit why all BLOCKING is treated so special? And > that flag just means the hw queue always need to schedule asynchronously, > and of course even though the flag isn't set, it still may be run > asynchronously too. So looks it should be enough to just use RCU. The mutex manipulations introduce atomic stores in the hot path. Hence the use of synchronize_rcu() and rcu_read_lock()/rcu_read_unlock() when possible. Since BLK_MQ_F_BLOCKING indicates that .queue_rq() may sleep and since sleeping is not allowed while holding the RCU read lock, queue_rq_mutex has been introduced for the BLK_MQ_F_BLOCKING case. Bart. From 3a549df53eba84fad94279484e54f9c43b2d6626 Mon Sep 17 00:00:00 2001 From: Bart Van Assche Date: Tue, 27 Sep 2016 10:52:36 -0700 Subject: [PATCH] blk-mq: Introduce blk_mq_quiesce_queue() blk_mq_quiesce_queue() waits until ongoing .queue_rq() invocations have finished. This function does *not* wait until all outstanding requests have finished (this means invocation of request.end_io()). Signed-off-by: Bart Van Assche Cc: Hannes Reinecke Cc: Johannes Thumshirn --- block/blk-mq.c | 69 +++++++++++++++++++++++++++++++++++++++++++++----- include/linux/blk-mq.h | 2 ++ include/linux/blkdev.h | 4 +++ 3 files changed, 69 insertions(+), 6 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index d8c45de..57701c5 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -115,6 +115,32 @@ void blk_mq_unfreeze_queue(struct request_queue *q) } EXPORT_SYMBOL_GPL(blk_mq_unfreeze_queue); +/** + * blk_mq_quiesce_queue() - wait until all ongoing queue_rq calls have finished + * + * Note: this function does not prevent that the struct request end_io() + * callback function is invoked. Additionally, it is not prevented that + * new queue_rq() calls occur unless the queue has been stopped first. + */ +void blk_mq_quiesce_queue(struct request_queue *q) +{ + struct blk_mq_hw_ctx *hctx; + unsigned int i; + bool res, rcu = false; + + queue_for_each_hw_ctx(q, hctx, i) { + if (hctx->flags & BLK_MQ_F_BLOCKING) { + mutex_lock(&hctx->queue_rq_mutex); + mutex_unlock(&hctx->queue_rq_mutex); + } else { + rcu = true; + } + } + if (rcu) + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(blk_mq_quiesce_queue); + void blk_mq_wake_waiters(struct request_queue *q) { struct blk_mq_hw_ctx *hctx; @@ -782,7 +808,7 @@ static inline unsigned int queued_to_index(unsigned int queued) * of IO. In particular, we'd like FIFO behaviour on handling existing * items on the hctx->dispatch list. Ignore that for now. */ -static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) +static void blk_mq_process_rq_list(struct blk_mq_hw_ctx *hctx) { struct request_queue *q = hctx->queue; struct request *rq; @@ -791,7 +817,7 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) struct list_head *dptr; int queued; - if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state))) + if (unlikely(blk_queue_quiescing(q))) return; WARN_ON(!cpumask_test_cpu(raw_smp_processor_id(), hctx->cpumask) && @@ -887,6 +913,22 @@ static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) } } +static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx) +{ + if (unlikely(test_bit(BLK_MQ_S_STOPPED, &hctx->state))) + return; + + if (hctx->flags & BLK_MQ_F_BLOCKING) { + mutex_lock(&hctx->queue_rq_mutex); + blk_mq_process_rq_list(hctx); + mutex_unlock(&hctx->queue_rq_mutex); + } else { + rcu_read_lock(); + blk_mq_process_rq_list(hctx); + rcu_read_unlock(); + } +} + /* * It'd be great if the workqueue API had a way to pass * in a mask and had some smarts for more clever placement. @@ -1341,7 +1383,7 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) blk_mq_bio_to_request(rq, bio); /* - * We do limited pluging. If the bio can be merged, do that. + * We do limited plugging. If the bio can be merged, do that. * Otherwise the existing request in the plug list will be * issued. So the plug list will have one request at most */ @@ -1361,9 +1403,23 @@ static blk_qc_t blk_mq_make_request(struct request_queue *q, struct bio *bio) blk_mq_put_ctx(data.ctx); if (!old_rq) goto done; - if (!blk_mq_direct_issue_request(old_rq, &cookie)) - goto done; - blk_mq_insert_request(old_rq, false, true, true); + + if (data.hctx->flags & BLK_MQ_F_BLOCKING) { + mutex_lock(&data.hctx->queue_rq_mutex); + if (blk_queue_quiescing(q) || + blk_mq_direct_issue_request(old_rq, &cookie) != 0) + blk_mq_insert_request(old_rq, false, true, + true); + mutex_unlock(&data.hctx->queue_rq_mutex); + } else { + rcu_read_lock(); + if (blk_queue_quiescing(q) || + blk_mq_direct_issue_request(old_rq, &cookie) != 0) + blk_mq_insert_request(old_rq, false, true, + true); + rcu_read_unlock(); + } + goto done; } @@ -1702,6 +1758,7 @@ static int blk_mq_init_hctx(struct request_queue *q, INIT_DELAYED_WORK(&hctx->delay_work, blk_mq_delay_work_fn); spin_lock_init(&hctx->lock); INIT_LIST_HEAD(&hctx->dispatch); + mutex_init(&hctx->queue_rq_mutex); hctx->queue = q; hctx->queue_num = hctx_idx; hctx->flags = set->flags & ~BLK_MQ_F_TAG_SHARED; diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 368c460d..4b970f1 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -41,6 +41,8 @@ struct blk_mq_hw_ctx { struct blk_mq_tags *tags; + struct mutex queue_rq_mutex; + unsigned long queued; unsigned long run; #define BLK_MQ_MAX_DISPATCH_ORDER 7 diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index c47c358..6c2d987 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -505,6 +505,7 @@ struct request_queue { #define QUEUE_FLAG_FUA 24 /* device supports FUA writes */ #define QUEUE_FLAG_FLUSH_NQ 25 /* flush not queueuable */ #define QUEUE_FLAG_DAX 26 /* device supports DAX */ +#define QUEUE_FLAG_QUIESCING 27 #define QUEUE_FLAG_DEFAULT ((1 << QUEUE_FLAG_IO_STAT) | \ (1 << QUEUE_FLAG_STACKABLE) | \ @@ -595,6 +596,8 @@ static inline void queue_flag_clear(unsigned int flag, struct request_queue *q) #define blk_queue_secure_erase(q) \ (test_bit(QUEUE_FLAG_SECERASE, &(q)->queue_flags)) #define blk_queue_dax(q) test_bit(QUEUE_FLAG_DAX, &(q)->queue_flags) +#define blk_queue_quiescing(q) test_bit(QUEUE_FLAG_QUIESCING, \ + &(q)->queue_flags) #define blk_noretry_request(rq) \ ((rq)->cmd_flags & (REQ_FAILFAST_DEV|REQ_FAILFAST_TRANSPORT| \ @@ -824,6 +827,7 @@ extern void __blk_run_queue(struct request_queue *q); extern void __blk_run_queue_uncond(struct request_queue *q); extern void blk_run_queue(struct request_queue *); extern void blk_run_queue_async(struct request_queue *q); +extern void blk_mq_quiesce_queue(struct request_queue *q); extern int blk_rq_map_user(struct request_queue *, struct request *, struct rq_map_data *, void __user *, unsigned long, gfp_t); -- 2.10.0