From patchwork Mon Nov 26 02:17:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10697361 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66A0A1750 for ; Mon, 26 Nov 2018 02:19:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5469B29A30 for ; Mon, 26 Nov 2018 02:19:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4661D29A32; Mon, 26 Nov 2018 02:19:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B483429A30 for ; Mon, 26 Nov 2018 02:19:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF1456B3F76; Sun, 25 Nov 2018 21:19:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CC7C46B3F78; Sun, 25 Nov 2018 21:19:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B91F46B3F79; Sun, 25 Nov 2018 21:19:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f200.google.com (mail-qt1-f200.google.com [209.85.160.200]) by kanga.kvack.org (Postfix) with ESMTP id 88CA06B3F76 for ; Sun, 25 Nov 2018 21:19:32 -0500 (EST) Received: by mail-qt1-f200.google.com with SMTP id p24so15405031qtl.2 for ; Sun, 25 Nov 2018 18:19:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=LvI5Ds5bnvZaqKCMcY3iOwKbE8ds7I7FAm6kP7ysdT8=; b=Lx4g3GbkkLHY6AnbrCn7SygPii8NmTu5HVin87BpGTUZ0ljMGbwJ9qMyxRzOC6G3AI WymUOjwE6nQ6/rA+2STO3VgadmQdtRTEu9Csrke8xAOtT+hjGCfpxYLGeqndiktaKJAO lXOhoJOYnMntyobiJ/YhzBRTTEwpu3pOwOAswALZqniKNYBMAvNbqHTEy3iq7n7P9s0l Wew37iCDZvtUaXF+ZQynif9t/9+I9D0K8VRkPLQsXR4v4I8BshUrgIYrclMGV2F60euN diW19us8hmuNY38MtpW5y+8sojPfrJa47RzjJmIudL8WEDvQ43F6DKmbzrczIMhASeCx +fLg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AGRZ1gJDwtQKPfT/MDJBjH+K6fYYsS/b/rxA4Dspaqtld/DJG5OWTr4l +qaM02HmG75p4DH6iPH6Huttlsv/CQIkAVoYQTSa7J7Zr0CSStnJ+207dWkWzWju6HmthYZqSEn AyoAJM8WOAXuVwsXtrWKcYwbo/wro0OpHfDYud0/fFV6f/AYGqPuR/iDgd9jE06vcjA== X-Received: by 2002:ac8:3fc9:: with SMTP id v9mr23614888qtk.108.1543198772313; Sun, 25 Nov 2018 18:19:32 -0800 (PST) X-Google-Smtp-Source: AFSGD/VP1LDmmDtRHLBCtJHw0KyJVcjpfbd5x6LJ6T2aNWcoHd0kWUWqSMwWEQ6FHeNlctoTl1yu X-Received: by 2002:ac8:3fc9:: with SMTP id v9mr23614869qtk.108.1543198771512; Sun, 25 Nov 2018 18:19:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543198771; cv=none; d=google.com; s=arc-20160816; b=NtBc7EQdJ76KpSNZ+AWCQ2stmssl24coEZlnDaPUb0lWyi6dQd6nsTENi3u5Nz1igT VnmeT8kVmJmLa/27lWonUEI/DOt1y9K924k/cBwMT1yjjvgTAE4am2UZRbLkLtISbMez Z8KXZ2x4nprOwixMB7feIm9HIpBSzuVWUuQsZxUoUZBwrRCyQcpOONaAaa3Xxk7wLKFt uxfWX6Uc9dCEfGFpK+5MsujuDAK8Ez7cguU15MNsoAD7BCtsh/g5d6cXXPcFcqByHjr6 QVDHZqxcMyGwPsKzomGVRESaj3rp3wsMbNFYdosnIAxQ1Xg8cIonj1BI0AVVOntR+kve xM9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=LvI5Ds5bnvZaqKCMcY3iOwKbE8ds7I7FAm6kP7ysdT8=; b=KEDIUWjCFx3V1F62CHxKmqHZa2AG0yFoWqb3FVdj6bcsz9tn69L1Moid4NdG6WA3a6 q+rDsVetCep1Kqjx/Mkb90laqxVkYJsnDq3sx2HqlU2Zry6LjWKWICMIzClNH8hWfsob YwW0hrAwU5hst/vDLLb55ivqUvqM0B/e2QOUyAkaG8OkE4Ym7MSF93QTZfIkelboguz5 clei8UVnI/wel7t78gbhUyLRggU6o8mMpkUIeLF7Cd6eY3z4HbM2eYAkRCOl1wDbeess JGAN6vnOuyBu8sRhzjO/4/YzC/k+rl58WwkoUcZQO/91td5wl6Ip6E7MmA8YzI+Y52my Pkow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id h12si5457532qti.361.2018.11.25.18.19.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Nov 2018 18:19:31 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 648E899C4B; Mon, 26 Nov 2018 02:19:30 +0000 (UTC) Received: from localhost (ovpn-8-19.pek2.redhat.com [10.72.8.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0FBBD194A7; Mon, 26 Nov 2018 02:19:20 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V12 09/20] block: use bio_for_each_bvec() to compute multi-page bvec count Date: Mon, 26 Nov 2018 10:17:09 +0800 Message-Id: <20181126021720.19471-10-ming.lei@redhat.com> In-Reply-To: <20181126021720.19471-1-ming.lei@redhat.com> References: <20181126021720.19471-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 26 Nov 2018 02:19:30 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than max segment size, so we have to split the big bvec into several segments. Thirdly when splitting multi-page bvec into segments, the max segment limit may be reached, so the bio split need to be considered under this situation too. Signed-off-by: Ming Lei Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval --- block/blk-merge.c | 100 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 80 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 51ec6ca56a0a..2d8f388d43de 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -161,6 +161,70 @@ static inline unsigned get_max_io_size(struct request_queue *q, return sectors; } +static unsigned get_max_segment_size(struct request_queue *q, + unsigned offset) +{ + unsigned long mask = queue_segment_boundary(q); + + return min_t(unsigned long, mask - (mask & offset) + 1, + queue_max_segment_size(q)); +} + +/* + * Split the bvec @bv into segments, and update all kinds of + * variables. + */ +static bool bvec_split_segs(struct request_queue *q, struct bio_vec *bv, + unsigned *nsegs, unsigned *last_seg_size, + unsigned *front_seg_size, unsigned *sectors) +{ + unsigned len = bv->bv_len; + unsigned total_len = 0; + unsigned new_nsegs = 0, seg_size = 0; + + /* + * Multipage bvec may be too big to hold in one segment, + * so the current bvec has to be splitted as multiple + * segments. + */ + while (len && new_nsegs + *nsegs < queue_max_segments(q)) { + seg_size = get_max_segment_size(q, bv->bv_offset + total_len); + seg_size = min(seg_size, len); + + new_nsegs++; + total_len += seg_size; + len -= seg_size; + + if ((bv->bv_offset + total_len) & queue_virt_boundary(q)) + break; + } + + if (!new_nsegs) + return !!len; + + /* update front segment size */ + if (!*nsegs) { + unsigned first_seg_size; + + if (new_nsegs == 1) + first_seg_size = get_max_segment_size(q, bv->bv_offset); + else + first_seg_size = queue_max_segment_size(q); + + if (*front_seg_size < first_seg_size) + *front_seg_size = first_seg_size; + } + + /* update other varibles */ + *last_seg_size = seg_size; + *nsegs += new_nsegs; + if (sectors) + *sectors += total_len >> 9; + + /* split in the middle of the bvec if len != 0 */ + return !!len; +} + static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *bio, struct bio_set *bs, @@ -174,7 +238,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *new = NULL; const unsigned max_sectors = get_max_io_size(q, bio); - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If the queue doesn't support SG gaps and adding this * offset would create a gap, disallow it. @@ -189,8 +253,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, */ if (nsegs < queue_max_segments(q) && sectors < max_sectors) { - nsegs++; - sectors = max_sectors; + /* split in the middle of bvec */ + bv.bv_len = (max_sectors - sectors) << 9; + bvec_split_segs(q, &bv, &nsegs, + &seg_size, + &front_seg_size, + §ors); } goto split; } @@ -212,14 +280,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, if (nsegs == queue_max_segments(q)) goto split; - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; - - nsegs++; bvprv = bv; bvprvp = &bvprv; - seg_size = bv.bv_len; - sectors += bv.bv_len >> 9; + + if (bvec_split_segs(q, &bv, &nsegs, &seg_size, + &front_seg_size, §ors)) + goto split; } @@ -233,8 +299,6 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, bio = new; } - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; bio->bi_seg_front_size = front_seg_size; if (seg_size > bio->bi_seg_back_size) bio->bi_seg_back_size = seg_size; @@ -296,6 +360,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, { struct bio_vec bv, bvprv = { NULL }; unsigned int seg_size, nr_phys_segs; + unsigned front_seg_size = bio->bi_seg_front_size; struct bio *fbio, *bbio; struct bvec_iter iter; bool prev = false; @@ -316,7 +381,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, seg_size = 0; nr_phys_segs = 0; for_each_bio(bio) { - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If SG merging is disabled, each bio vector is * a segment @@ -336,20 +401,15 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, continue; } new_segment: - if (nr_phys_segs == 1 && seg_size > - fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; - - nr_phys_segs++; bvprv = bv; prev = true; - seg_size = bv.bv_len; + bvec_split_segs(q, &bv, &nr_phys_segs, &seg_size, + &front_seg_size, NULL); } bbio = bio; } - if (nr_phys_segs == 1 && seg_size > fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; + fbio->bi_seg_front_size = front_seg_size; if (seg_size > bbio->bi_seg_back_size) bbio->bi_seg_back_size = seg_size;