From patchwork Mon Nov 14 17:28:39 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9428065 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CA0B160471 for ; Mon, 14 Nov 2016 17:29:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAAEB28AA3 for ; Mon, 14 Nov 2016 17:29:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AEEF128AA9; Mon, 14 Nov 2016 17:29:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4184D28AA3 for ; Mon, 14 Nov 2016 17:29:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753292AbcKNR3q (ORCPT ); Mon, 14 Nov 2016 12:29:46 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:53022 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934981AbcKNR3k (ORCPT ); Mon, 14 Nov 2016 12:29:40 -0500 Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.17/8.16.0.17) with SMTP id uAEHSKj9030725; Mon, 14 Nov 2016 09:28:45 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=hvGN0wntjpWdaUmOWZY1ABazcmQf0QpB526nYrFv1ro=; b=CBmb9IV54eIdmayEJqBmLtQx5Ptc/WQkp6oV6WudB6D1xaI1bgVg/ffRokuItFePGvtM cZnE2w3j55+G3nGA5PgEeKBoV/XshCwPGSPo80ls/UMnMrgDFVwdw8ByhNqXMddSa4DR joDVE2gtApHc0iHNYLacfWmdysy/Bb/V0oQ= Received: from mail.thefacebook.com ([199.201.64.23]) by m0089730.ppops.net with ESMTP id 26qesj90tq-2 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 14 Nov 2016 09:28:45 -0800 Received: from localhost.localdomain (192.168.54.13) by mail.thefacebook.com (192.168.16.24) with Microsoft SMTP Server (TLS) id 14.3.294.0; Mon, 14 Nov 2016 09:28:43 -0800 From: Jens Axboe To: , CC: , Jens Axboe Subject: [PATCH 2/2] block: add support for async simple direct-io for bdevs Date: Mon, 14 Nov 2016 10:28:39 -0700 Message-ID: <1479144519-15738-3-git-send-email-axboe@fb.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1479144519-15738-1-git-send-email-axboe@fb.com> References: <1479144519-15738-1-git-send-email-axboe@fb.com> MIME-Version: 1.0 X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-11-14_10:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Jens Axboe --- fs/block_dev.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 66 insertions(+), 10 deletions(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 2010997fd326..62ca4ce21222 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -176,9 +176,68 @@ static struct inode *bdev_file_inode(struct file *file) return file->f_mapping->host; } +static void blkdev_bio_end_io_async(struct bio *bio) +{ + struct kiocb *iocb = bio->bi_private; + + iocb->ki_complete(iocb, bio->bi_error, 0); + + if (bio_op(bio) == REQ_OP_READ) + bio_check_pages_dirty(bio); + else + bio_put(bio); +} + +static ssize_t +__blkdev_direct_IO_async(struct kiocb *iocb, struct iov_iter *iter, + int nr_pages) +{ + struct file *file = iocb->ki_filp; + struct block_device *bdev = I_BDEV(bdev_file_inode(file)); + unsigned blkbits = blksize_bits(bdev_logical_block_size(bdev)); + loff_t pos = iocb->ki_pos; + struct bio *bio; + ssize_t ret; + + if ((pos | iov_iter_alignment(iter)) & ((1 << blkbits) - 1)) + return -EINVAL; + + bio = bio_alloc(GFP_KERNEL, nr_pages); + if (!bio) + return -ENOMEM; + + bio->bi_bdev = bdev; + bio->bi_iter.bi_sector = pos >> blkbits; + bio->bi_private = iocb; + bio->bi_end_io = blkdev_bio_end_io_async; + + ret = bio_iov_iter_get_pages(bio, iter); + if (unlikely(ret)) + return ret; + + /* + * Overload bio size in error. If it gets set, we lose the + * size, but we don't need the size for that case. IO is limited + * to BIO_MAX_PAGES, so we can't overflow. + */ + ret = bio->bi_error = bio->bi_iter.bi_size; + + if (iov_iter_rw(iter) == READ) { + bio_set_op_attrs(bio, REQ_OP_READ, 0); + bio_set_pages_dirty(bio); + } else { + bio_set_op_attrs(bio, REQ_OP_WRITE, REQ_SYNC | REQ_IDLE); + task_io_account_write(ret); + } + + submit_bio(bio); + iocb->ki_pos += ret; + return -EIOCBQUEUED; +} + #define DIO_INLINE_BIO_VECS 4 -static void blkdev_bio_end_io_simple(struct bio *bio) +static void blkdev_bio_end_io_sync(struct bio *bio) { struct task_struct *waiter = bio->bi_private; @@ -187,8 +246,7 @@ static void blkdev_bio_end_io_simple(struct bio *bio) } static ssize_t -__blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, - int nr_pages) +__blkdev_direct_IO_sync(struct kiocb *iocb, struct iov_iter *iter, int nr_pages) { struct file *file = iocb->ki_filp; struct block_device *bdev = I_BDEV(bdev_file_inode(file)); @@ -218,7 +276,7 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, bio.bi_bdev = bdev; bio.bi_iter.bi_sector = pos >> blkbits; bio.bi_private = current; - bio.bi_end_io = blkdev_bio_end_io_simple; + bio.bi_end_io = blkdev_bio_end_io_sync; ret = bio_iov_iter_get_pages(&bio, iter); if (unlikely(ret)) @@ -263,18 +321,16 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter, static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter) { - struct file *file = iocb->ki_filp; - struct inode *inode = bdev_file_inode(file); int nr_pages; nr_pages = iov_iter_npages(iter, BIO_MAX_PAGES); if (!nr_pages) return 0; + if (is_sync_kiocb(iocb)) - return __blkdev_direct_IO_simple(iocb, iter, nr_pages); - return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter, - blkdev_get_block, NULL, NULL, - DIO_SKIP_DIO_COUNT); + return __blkdev_direct_IO_sync(iocb, iter, nr_pages); + + return __blkdev_direct_IO_async(iocb, iter, nr_pages); } int __sync_blockdev(struct block_device *bdev, int wait)