From patchwork Wed Nov 21 03:23:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10691763 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 78B5013BB for ; Wed, 21 Nov 2018 03:28:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 664D82B412 for ; Wed, 21 Nov 2018 03:28:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 599FE2B415; Wed, 21 Nov 2018 03:28:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEEB72B412 for ; Wed, 21 Nov 2018 03:28:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728294AbeKUOAt (ORCPT ); Wed, 21 Nov 2018 09:00:49 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37826 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726705AbeKUOAs (ORCPT ); Wed, 21 Nov 2018 09:00:48 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CD8E4309706B; Wed, 21 Nov 2018 03:28:15 +0000 (UTC) Received: from localhost (ovpn-8-21.pek2.redhat.com [10.72.8.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id B383B17188; Wed, 21 Nov 2018 03:27:57 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split Date: Wed, 21 Nov 2018 11:23:22 +0800 Message-Id: <20181121032327.8434-15-ming.lei@redhat.com> In-Reply-To: <20181121032327.8434-1-ming.lei@redhat.com> References: <20181121032327.8434-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Wed, 21 Nov 2018 03:28:16 +0000 (UTC) Sender: linux-xfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We will enable multi-page bvec soon, but non-cluster queue can't handle the multi-page bvec at all. This patch borrows bounce's idea to clone new single-page bio for non-cluster queue, and moves its handling out of blk_bio_segment_split(). Signed-off-by: Ming Lei Signed-off-by: Christoph Hellwig --- block/Makefile | 3 ++- block/blk-merge.c | 6 ++++- block/blk.h | 2 ++ block/non-cluster.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 79 insertions(+), 2 deletions(-) create mode 100644 block/non-cluster.c diff --git a/block/Makefile b/block/Makefile index eee1b4ceecf9..e07d59438c4b 100644 --- a/block/Makefile +++ b/block/Makefile @@ -9,7 +9,8 @@ obj-$(CONFIG_BLOCK) := bio.o elevator.o blk-core.o blk-sysfs.o \ blk-lib.o blk-mq.o blk-mq-tag.o blk-stat.o \ blk-mq-sysfs.o blk-mq-cpumap.o blk-mq-sched.o ioctl.o \ genhd.o partition-generic.o ioprio.o \ - badblocks.o partitions/ blk-rq-qos.o + badblocks.o partitions/ blk-rq-qos.o \ + non-cluster.o obj-$(CONFIG_BOUNCE) += bounce.o obj-$(CONFIG_BLK_SCSI_REQUEST) += scsi_ioctl.o diff --git a/block/blk-merge.c b/block/blk-merge.c index 8829c51b4e75..7c44216c1b58 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -247,7 +247,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, goto split; } - if (bvprvp && blk_queue_cluster(q)) { + if (bvprvp) { if (seg_size + bv.bv_len > queue_max_segment_size(q)) goto new_segment; if (!biovec_phys_mergeable(q, bvprvp, &bv)) @@ -307,6 +307,10 @@ void blk_queue_split(struct request_queue *q, struct bio **bio) split = blk_bio_write_same_split(q, *bio, &q->bio_split, &nsegs); break; default: + if (!blk_queue_cluster(q)) { + blk_queue_non_cluster_bio(q, bio); + return; + } split = blk_bio_segment_split(q, *bio, &q->bio_split, &nsegs); break; } diff --git a/block/blk.h b/block/blk.h index 31c0e45aba3a..6fc5821ced55 100644 --- a/block/blk.h +++ b/block/blk.h @@ -338,6 +338,8 @@ struct bio *blk_next_bio(struct bio *bio, unsigned int nr_pages, gfp_t gfp); struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask, struct bio_set *bs); +void blk_queue_non_cluster_bio(struct request_queue *q, struct bio **bio_orig); + #ifdef CONFIG_BLK_DEV_ZONED void blk_queue_free_zone_bitmaps(struct request_queue *q); #else diff --git a/block/non-cluster.c b/block/non-cluster.c new file mode 100644 index 000000000000..9c2910be9404 --- /dev/null +++ b/block/non-cluster.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0 +/* non-cluster handling for block devices */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "blk.h" + +static struct bio_set non_cluster_bio_set, non_cluster_bio_split; + +static __init int init_non_cluster_bioset(void) +{ + WARN_ON(bioset_init(&non_cluster_bio_set, BIO_POOL_SIZE, 0, + BIOSET_NEED_BVECS)); + WARN_ON(bioset_integrity_create(&non_cluster_bio_set, BIO_POOL_SIZE)); + WARN_ON(bioset_init(&non_cluster_bio_split, BIO_POOL_SIZE, 0, 0)); + + return 0; +} +__initcall(init_non_cluster_bioset); + +static void non_cluster_end_io(struct bio *bio) +{ + struct bio *bio_orig = bio->bi_private; + + bio_orig->bi_status = bio->bi_status; + bio_endio(bio_orig); + bio_put(bio); +} + +void blk_queue_non_cluster_bio(struct request_queue *q, struct bio **bio_orig) +{ + struct bio *bio; + struct bvec_iter iter; + struct bio_vec from; + unsigned i = 0; + unsigned sectors = 0; + unsigned short max_segs = min_t(unsigned short, BIO_MAX_PAGES, + queue_max_segments(q)); + + bio_for_each_segment(from, *bio_orig, iter) { + if (i++ < max_segs) + sectors += from.bv_len >> 9; + else + break; + } + + if (sectors < bio_sectors(*bio_orig)) { + bio = bio_split(*bio_orig, sectors, GFP_NOIO, + &non_cluster_bio_split); + bio_chain(bio, *bio_orig); + generic_make_request(*bio_orig); + *bio_orig = bio; + } + bio = bio_clone_bioset(*bio_orig, GFP_NOIO, &non_cluster_bio_set); + + bio->bi_phys_segments = bio_segments(bio); + bio_set_flag(bio, BIO_SEG_VALID); + bio->bi_end_io = non_cluster_end_io; + + bio->bi_private = *bio_orig; + *bio_orig = bio; +}