From patchwork Wed Nov 21 03:23:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10691761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A33A13BB for ; Wed, 21 Nov 2018 03:28:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 382E92B412 for ; Wed, 21 Nov 2018 03:28:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2B3192B415; Wed, 21 Nov 2018 03:28:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A52DF2B412 for ; Wed, 21 Nov 2018 03:28:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B36E06B23B0; Tue, 20 Nov 2018 22:28:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AE4E76B23B1; Tue, 20 Nov 2018 22:28:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D7F96B23B2; Tue, 20 Nov 2018 22:28:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 6CA1A6B23B0 for ; Tue, 20 Nov 2018 22:28:18 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id q3so2203683qtq.15 for ; Tue, 20 Nov 2018 19:28:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=u/GwWcK6UN3Z/q7XxvOIQRRKJq08yyNNXRcBbF1NJ+s=; b=Yzs7hYT736vvk+wowv+gJ14qk1HwFMWBjb+UfIdbEgnNVG9W5oVKkjCjQxRU4NhDTO fH1hwa2BVjlcf+y4n5JkKdg92W4yqZ2IlVVEdCa6LAWOd+LMW7xn7ELR73RpZ4ZD0YE/ 4UQ2TUBKaH2qh0OkR2+DyamfAMFLt2f95ThduxA+ESEqqd8+frkJgiZlTPEI343tAH5x dBlDKbv5rH8Ilf2gqvaCM5BAYqfXDNuHveAq5aPP9Y/+vEL2CKOivcfiVdYZyHJNl6Eb RF53SUNuC7cA8neUZOUobiH7SJ/gkMNoHWpCsufKChOzIatOKzSovZDPi1AXjKdMa1WN ZBGw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWaMNzwP+jiFueUQyby7iw3Q/7IMBv+61lHuvrqWbwf2ROk1MTkO zUangdFh2FyZxCCNySDP5PGVPOyVOedEleihBPADuysX6Go54gS7aWN3AcMdzTUn+GaStRq0PBf sOHfBgNfqry33bPLRsNHEnng15tZbXJDpfQAokNlwWwUfkfWTe61OYsBJha4giX5GOA== X-Received: by 2002:aed:3802:: with SMTP id j2mr4468653qte.146.1542770898209; Tue, 20 Nov 2018 19:28:18 -0800 (PST) X-Google-Smtp-Source: AJdET5eX3SoqgO6KmiiWjso5hy2ojcYWW7vF/pLcWsceMGOZIFUQ75jHGyo3FksmM49UW71Szwkc X-Received: by 2002:aed:3802:: with SMTP id j2mr4468607qte.146.1542770897005; Tue, 20 Nov 2018 19:28:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542770896; cv=none; d=google.com; s=arc-20160816; b=aU9PgzM5JsFoFw3aVcgv1AGlOQ91kq6Yp+RBQN60+9A4Tb1fo8RzdfRH1sOk+Jn97I G5eDPWzd/GRklUhHA5eWnJNk2nCEfevp36FXgB/L3dJGX/TcB0shjD8iDfYY8QIR9vwE mW4ld1fivIrX3wfFrUv0rPJ6W5dvxhMk8UPydOe3wsE/+EjKqU0NrNkZjK514RAr2vKm uip0E4nrn6Q5AmZwlBszKRCfic8n6euPDPjJe0t9F2p8R3qijraQqnsmvW5KcZk7MRny 1FviFzA5nrLFh6CO6ci7v7LTZRwZXxjbHX8zMAhzroB4b8aO35cu/LfejWPv6ChyOojE uPeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=u/GwWcK6UN3Z/q7XxvOIQRRKJq08yyNNXRcBbF1NJ+s=; b=PhjKdoiz8bRLX2SJGLVkGG6dLHU8NSqfaEVf8AxerysbPCsWRJGwtIV+BiUp5METBb 8Hc993fhMadXbQidPnXfSkfYaraqo/fgX70A22LB0mHaYNI0osWlIW2zPJ+uLYSxzSQ+ OzudZmBMfPiRWbNYcvf/cU7FPtKV7MGEf3F6sTJgbEJfhu/+EK07fcZjldDv0AiqQDur fiMzxCD06uqoOkPimbBSyLSz8Cv4hUewn1lya7Al2TtrdMbvI9JthI6S5sQHYS84jm4g RBIjwI3gbrQPrGf4UiHFuxu1Rv+LS0iU6Y6IhrMTW8TgdI7M3+EbZnxfY3TnUo8bBizb qvUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id d42si5034877qve.68.2018.11.20.19.28.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 19:28:16 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CD8E4309706B; Wed, 21 Nov 2018 03:28:15 +0000 (UTC) Received: from localhost (ovpn-8-21.pek2.redhat.com [10.72.8.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id B383B17188; Wed, 21 Nov 2018 03:27:57 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V11 14/19] block: handle non-cluster bio out of blk_bio_segment_split Date: Wed, 21 Nov 2018 11:23:22 +0800 Message-Id: <20181121032327.8434-15-ming.lei@redhat.com> In-Reply-To: <20181121032327.8434-1-ming.lei@redhat.com> References: <20181121032327.8434-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Wed, 21 Nov 2018 03:28:16 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP We will enable multi-page bvec soon, but non-cluster queue can't handle the multi-page bvec at all. This patch borrows bounce's idea to clone new single-page bio for non-cluster queue, and moves its handling out of blk_bio_segment_split(). Signed-off-by: Ming Lei Signed-off-by: Christoph Hellwig --- block/Makefile | 3 ++- block/blk-merge.c | 6 ++++- block/blk.h | 2 ++ block/non-cluster.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 79 insertions(+), 2 deletions(-) create mode 100644 block/non-cluster.c diff --git a/block/Makefile b/block/Makefile index eee1b4ceecf9..e07d59438c4b 100644 --- a/block/Makefile +++ b/block/Makefile @@ -9,7 +9,8 @@ obj-$(CONFIG_BLOCK) := bio.o elevator.o blk-core.o blk-sysfs.o \ blk-lib.o blk-mq.o blk-mq-tag.o blk-stat.o \ blk-mq-sysfs.o blk-mq-cpumap.o blk-mq-sched.o ioctl.o \ genhd.o partition-generic.o ioprio.o \ - badblocks.o partitions/ blk-rq-qos.o + badblocks.o partitions/ blk-rq-qos.o \ + non-cluster.o obj-$(CONFIG_BOUNCE) += bounce.o obj-$(CONFIG_BLK_SCSI_REQUEST) += scsi_ioctl.o diff --git a/block/blk-merge.c b/block/blk-merge.c index 8829c51b4e75..7c44216c1b58 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -247,7 +247,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, goto split; } - if (bvprvp && blk_queue_cluster(q)) { + if (bvprvp) { if (seg_size + bv.bv_len > queue_max_segment_size(q)) goto new_segment; if (!biovec_phys_mergeable(q, bvprvp, &bv)) @@ -307,6 +307,10 @@ void blk_queue_split(struct request_queue *q, struct bio **bio) split = blk_bio_write_same_split(q, *bio, &q->bio_split, &nsegs); break; default: + if (!blk_queue_cluster(q)) { + blk_queue_non_cluster_bio(q, bio); + return; + } split = blk_bio_segment_split(q, *bio, &q->bio_split, &nsegs); break; } diff --git a/block/blk.h b/block/blk.h index 31c0e45aba3a..6fc5821ced55 100644 --- a/block/blk.h +++ b/block/blk.h @@ -338,6 +338,8 @@ struct bio *blk_next_bio(struct bio *bio, unsigned int nr_pages, gfp_t gfp); struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask, struct bio_set *bs); +void blk_queue_non_cluster_bio(struct request_queue *q, struct bio **bio_orig); + #ifdef CONFIG_BLK_DEV_ZONED void blk_queue_free_zone_bitmaps(struct request_queue *q); #else diff --git a/block/non-cluster.c b/block/non-cluster.c new file mode 100644 index 000000000000..9c2910be9404 --- /dev/null +++ b/block/non-cluster.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0 +/* non-cluster handling for block devices */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "blk.h" + +static struct bio_set non_cluster_bio_set, non_cluster_bio_split; + +static __init int init_non_cluster_bioset(void) +{ + WARN_ON(bioset_init(&non_cluster_bio_set, BIO_POOL_SIZE, 0, + BIOSET_NEED_BVECS)); + WARN_ON(bioset_integrity_create(&non_cluster_bio_set, BIO_POOL_SIZE)); + WARN_ON(bioset_init(&non_cluster_bio_split, BIO_POOL_SIZE, 0, 0)); + + return 0; +} +__initcall(init_non_cluster_bioset); + +static void non_cluster_end_io(struct bio *bio) +{ + struct bio *bio_orig = bio->bi_private; + + bio_orig->bi_status = bio->bi_status; + bio_endio(bio_orig); + bio_put(bio); +} + +void blk_queue_non_cluster_bio(struct request_queue *q, struct bio **bio_orig) +{ + struct bio *bio; + struct bvec_iter iter; + struct bio_vec from; + unsigned i = 0; + unsigned sectors = 0; + unsigned short max_segs = min_t(unsigned short, BIO_MAX_PAGES, + queue_max_segments(q)); + + bio_for_each_segment(from, *bio_orig, iter) { + if (i++ < max_segs) + sectors += from.bv_len >> 9; + else + break; + } + + if (sectors < bio_sectors(*bio_orig)) { + bio = bio_split(*bio_orig, sectors, GFP_NOIO, + &non_cluster_bio_split); + bio_chain(bio, *bio_orig); + generic_make_request(*bio_orig); + *bio_orig = bio; + } + bio = bio_clone_bioset(*bio_orig, GFP_NOIO, &non_cluster_bio_set); + + bio->bi_phys_segments = bio_segments(bio); + bio_set_flag(bio, BIO_SEG_VALID); + bio->bi_end_io = non_cluster_end_io; + + bio->bi_private = *bio_orig; + *bio_orig = bio; +}