From patchwork Sun May 30 04:21:06 2083 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Shetty X-Patchwork-Id: 10207219 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E6BA4602D8 for ; Thu, 8 Feb 2018 14:37:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1CCF29505 for ; Thu, 8 Feb 2018 14:37:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5EA42951D; Thu, 8 Feb 2018 14:37:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,RCVD_IN_DNSWL_HI,T_DATE_IN_FUTURE_Q_PLUS autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6760F29519 for ; Thu, 8 Feb 2018 14:37:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751706AbeBHOhB (ORCPT ); Thu, 8 Feb 2018 09:37:01 -0500 Received: from mailout4.samsung.com ([203.254.224.34]:54294 "EHLO mailout4.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750961AbeBHOg7 (ORCPT ); Thu, 8 Feb 2018 09:36:59 -0500 Received: from epcas1p4.samsung.com (unknown [182.195.41.48]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20180208143657epoutp04425092afd888005cafece3d23525af67~RYHf0D2452521625216epoutp04A; Thu, 8 Feb 2018 14:36:57 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20180208143657epoutp04425092afd888005cafece3d23525af67~RYHf0D2452521625216epoutp04A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1518100617; bh=OOsZmnIVL9TG7TZZD5FaPtSRaqJKxOGnUVVJ/OjRODE=; h=From:To:Cc:Subject:Date:References:From; b=hqprCbjf6hox8hqUo6ZfCKTLTmM49Gij7kFiAX2Gczx1QuvC39rGrDVU8bm4xV7n9 tHTxuBvhtdjTLOwYNObjQtxc6F3RwwB4fs8vRT3Ox+oHMLubA6HKQjs3q3UWLFFi3r 1lrMeZlRlK2pIss5be+nrmGQYjsmiRSTWW1mzAyI= Received: from epsmges1p5.samsung.com (unknown [182.195.42.57]) by epcas1p2.samsung.com (KnoxPortal) with ESMTP id 20180208143657epcas1p24f578a3068c52d93d2b0cde7086df1d0~RYHfKOm9M1677016770epcas1p28; Thu, 8 Feb 2018 14:36:57 +0000 (GMT) Received: from epcas1p4.samsung.com ( [182.195.41.48]) by epsmges1p5.samsung.com (Symantec Messaging Gateway) with SMTP id D6.42.04037.8806C7A5; Thu, 8 Feb 2018 23:36:56 +0900 (KST) Received: from epsmgms2p1new.samsung.com (unknown [182.195.42.142]) by epcas1p4.samsung.com (KnoxPortal) with ESMTP id 20180208143655epcas1p444bf34705526b7839a5a135f82761aad~RYHeEZknX2552225522epcas1p4Q; Thu, 8 Feb 2018 14:36:55 +0000 (GMT) X-AuditID: b6c32a39-467ff70000000fc5-35-5a7c6088ad84 Received: from epmmp2 ( [203.254.227.17]) by epsmgms2p1new.samsung.com (Symantec Messaging Gateway) with SMTP id CA.06.03826.7806C7A5; Thu, 8 Feb 2018 23:36:55 +0900 (KST) Received: from test-PowerEdge-R720.sisodomain.com ([107.108.221.212]) by mmp2.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0P3U00DA65XD6500@mmp2.samsung.com>; Thu, 08 Feb 2018 23:36:55 +0900 (KST) From: Nitesh Shetty To: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, viro@zeniv.linux.org.uk Cc: linux-block@vger.kernel.org, axboe@fb.com, linux-nvme@lists.infradead.org, nj.shetty@samsung.com, joshi.k@samsung.com Subject: [PATCH] blk: optimization for classic polling Date: Sun, 30 May 2083 09:51:06 +0530 Message-id: <3578876466-3733-1-git-send-email-nj.shetty@samsung.com> X-Mailer: git-send-email 2.7.4 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrGIsWRmVeSWpSXmKPExsWy7bCmgW5HQk2UwbYllhb/9xxjszj6/y2b xd5b2hZ79p5ksbi8aw6bxfxlT9kttv2ez2xx/u9xVgcOj4nN79g9Ni+p9+jbsorR4/MmOY9N T94yBbBGcdmkpOZklqUW6dslcGW8nd7JWHBYomLRhn7WBsbPwl2MnBwSAiYSdyY0MYPYQgI7 GCU2v+brYuQCsr8zStx9uosRpujTuXYWiMQGRonGYwsZIZx2Jonf9zrZuxg5ONgEtCVO/+cA aRARiJLY1naHESTMLFAtsfCCKkhYWMBMYt+sLrBqFgFViY4rSSBhXgEXidkvnzJBrJKTuHmu kxlkuoTAV1aJZ+e+sUMkXCQenfoKdY+wxKvjW8DmSAhIS1w6agtR380ocaN1NVTNFEaJGXP1 IWx7iYt7/oItYBbgk3j3tYcVopdXoqNNCKLEQ2LRiQ5WCNtRYuH+Z0yQMImVeHZqLusERskF jAyrGMVSC4pz01OLDQtM9YoTc4tL89L1kvNzNzGCY0/LcgfjsXM+hxgFOBiVeHhfxNVECbEm lhVX5h5ilOBgVhLh3SMKFOJNSaysSi3Kjy8qzUktPsQozcGiJM4bEOASJSSQnliSmp2aWpBa BJNl4uCUamBcvu2pzmnjFOvzUvuE5vuebDt9aI9qUbbKzNMGXzcufnNSZ5Ln2aVpYSzijLdK 736O26gcIiKxU0ny3KSJ13ST1Y/6bXh0X/2y8upJpvNrFhQsrmkJEb1iU9zkHC+Zs/qd9+Qn T66xB+f/rg8OWjc3b7t9SefrpyfOFoeeqZPz31n4tGX/E8f7SizFGYmGWsxFxYkAlHHv47kC AAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrCJMWRmVeSWpSXmKPExsVy+t9jQd32hJoog9PrdS3+7znGZnH0/1s2 i723tC327D3JYnF51xw2i/nLnrJbbPs9n9ni/N/jrA4cHhOb37F7bF5S79G3ZRWjx+dNch6b nrxlCmCN4rJJSc3JLEst0rdL4Mp4O72TseCwRMWiDf2sDYyfhbsYOTkkBEwkPp1rZ+li5OIQ EljHKLHw2gY2CKeTSWJZ+1kgh4ODTUBb4vR/DpAGEYEoiYWfmllBwswC1RJn59SAhIUFzCT2 zepiBwmzCKhKdFxJAgnzCrhIzH75lAlilZzEzXOdzBMYuRYwMqxilEwtKM5Nzy02KjDMSy3X K07MLS7NS9dLzs/dxAgMi22Htfp2MN5fEn+IUYCDUYmHd0JMTZQQa2JZcWXuIUYJDmYlEd49 okAh3pTEyqrUovz4otKc1OJDjNIcLErivLfzjkUKCaQnlqRmp6YWpBbBZJk4OKUaGLeLiMfd 07zcU77s9PtXX/8XuuguKAzfJ/zvzwGxrvxl11PPpG7191oyY4lxx47mm12V5abnE7oP6a1X TbwU7bMgp+LO2c5Vau80p5iJ9zhvMsu5yrU6d2fctNqzkwOvGX3SOTG5ReSFYM6740oqd7Z0 veheO/N4ksShufkut/5f+//hzOoz2ZOUWIozEg21mIuKEwGv71ViBwIAAA== X-CMS-MailID: 20180208143655epcas1p444bf34705526b7839a5a135f82761aad X-Msg-Generator: CA CMS-TYPE: 101P X-CMS-RootMailID: 20180208143655epcas1p444bf34705526b7839a5a135f82761aad X-RootMTR: 20180208143655epcas1p444bf34705526b7839a5a135f82761aad References: Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This removes the dependency on interrupts to wake up task. Set task state as TASK_RUNNING, if need_resched() returns true, while polling for IO completion. Earlier, polling task used to sleep, relying on interrupt to wake it up. This made some IO take very long when interrupt-coalescing is enabled in NVMe. Reference: http://lists.infradead.org/pipermail/linux-nvme/2018-February/015435.html Signed-off-by: Nitesh Shetty --- fs/block_dev.c | 16 ++++++++++++---- fs/direct-io.c | 8 ++++++-- fs/iomap.c | 10 +++++++--- 3 files changed, 25 insertions(+), 9 deletions(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 4a181fc..a87d8b7 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -236,9 +236,13 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, set_current_state(TASK_UNINTERRUPTIBLE); if (!READ_ONCE(bio.bi_private)) break; - if (!(iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(bdev_get_queue(bdev), qc)) + if (!(iocb->ki_flags & IOCB_HIPRI)) io_schedule(); + else if (!blk_poll(bdev_get_queue(bdev), qc)) { + if (need_resched()) + set_current_state(TASK_RUNNING); + io_schedule(); + } } __set_current_state(TASK_RUNNING); @@ -401,9 +405,13 @@ __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, int nr_pages) if (!READ_ONCE(dio->waiter)) break; - if (!(iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(bdev_get_queue(bdev), qc)) + if (!(iocb->ki_flags & IOCB_HIPRI)) io_schedule(); + else if (!blk_poll(bdev_get_queue(bdev), qc)) { + if (need_resched()) + set_current_state(TASK_RUNNING); + io_schedule(); + } } __set_current_state(TASK_RUNNING); diff --git a/fs/direct-io.c b/fs/direct-io.c index a0ca9e4..c815ac9 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -518,9 +518,13 @@ static struct bio *dio_await_one(struct dio *dio) __set_current_state(TASK_UNINTERRUPTIBLE); dio->waiter = current; spin_unlock_irqrestore(&dio->bio_lock, flags); - if (!(dio->iocb->ki_flags & IOCB_HIPRI) || - !blk_poll(dio->bio_disk->queue, dio->bio_cookie)) + if (!(dio->iocb->ki_flags & IOCB_HIPRI)) io_schedule(); + else if (!blk_poll(dio->bio_disk->queue, dio->bio_cookie)) { + if (need_resched()) + __set_current_state(TASK_RUNNING); + io_schedule(); + } /* wake up sets us TASK_RUNNING */ spin_lock_irqsave(&dio->bio_lock, flags); dio->waiter = NULL; diff --git a/fs/iomap.c b/fs/iomap.c index afd1635..b51569d 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -1072,10 +1072,14 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, break; if (!(iocb->ki_flags & IOCB_HIPRI) || - !dio->submit.last_queue || - !blk_poll(dio->submit.last_queue, - dio->submit.cookie)) + !dio->submit.last_queue) io_schedule(); + else if (!blk_poll(dio->submit.last_queue, + dio->submit.cookie)) { + if (need_resched()) + set_current_state(TASK_RUNNING); + io_schedule(); + } } __set_current_state(TASK_RUNNING); }