From patchwork Wed Nov 21 03:23:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10691775 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 562FE13BB for ; Wed, 21 Nov 2018 03:28:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41EFD2AB6F for ; Wed, 21 Nov 2018 03:28:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 34FAD2B440; Wed, 21 Nov 2018 03:28:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4EA12B3F4 for ; Wed, 21 Nov 2018 03:28:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D7EF36B23B1; Tue, 20 Nov 2018 22:28:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D2FBC6B23B3; Tue, 20 Nov 2018 22:28:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C21566B23B4; Tue, 20 Nov 2018 22:28:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id 906416B23B1 for ; Tue, 20 Nov 2018 22:28:33 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id n68so5398387qkn.8 for ; Tue, 20 Nov 2018 19:28:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=9NZ8XI0wSMWu+PvCtOk6AcrbND/96y9gK36RFJAfcOQ=; b=g9lz21REOQhSrq4EvUzLJVWh8l4TXpe1oASbbtMGFM3Haw+zDA7Fjk0JfElX4KAYSf Ggukz1/hnQnXatQuJqekpVW9OvA6z2rsVW8EpHmL9h9Yp9jLtwYqgxZD+f8IzSwJRtaw VdFy5iGMh2Rivkx2DJgUH98on+Z97D8mIavEzvNa81yQ6Vc/Y9unEXpmwFGhnIwhCs/l 6BIF0yraSN/qUQk1xkHY1rR8TN7L4udvATIxrarQ0PhsDJDYVQWvN+CevMRnvZFDT4nn hIFGBsiKCr0IJS/Iy2Xv/YAaJ0qmWjM9ngRqnnhgOVT1WSCumEKSe1Ac10btN5PN2IXT Uu8Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWbnUeH3lhGtFIBugWJr+BDszXktEDmxT1H5j8pzZanLvGEWpkQp HLIkQPkMk3bOCyyBBvsxOE8dNb/HoiYkedAZK4OUf55A/K1CV5oZ4uzUERHn/vyXJgXzwMHDRvU SESvtE1eXoPkmJy8fwGJ8SobCg01hQpEVlaZFhVzNplZ9axu6sA6byOlg/UKjDStrxQ== X-Received: by 2002:a37:b1c7:: with SMTP id a190mr3983525qkf.94.1542770913344; Tue, 20 Nov 2018 19:28:33 -0800 (PST) X-Google-Smtp-Source: AFSGD/UV2YjV+8Kq1wZnkcsF6OMjciwvZhKhrqZRWwHykVBW4Mn0Vhd4Sc2FipBml7mfG/c0+bzR X-Received: by 2002:a37:b1c7:: with SMTP id a190mr3983497qkf.94.1542770912092; Tue, 20 Nov 2018 19:28:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542770912; cv=none; d=google.com; s=arc-20160816; b=Zxnt1XH7p2UEY1gE034CYfJmFUQAuo+8VLvx9Mem5DpCK28HoCP6/mtOoAia8U9DNr eVgZHTTYv837YfMpFI87A0QT8DdORrAbdUm7Im1NFP1V0bvsZQ40rZ8U9RHNpIdhwOoP 1cFRKj+8tdH3FbInejkEeztiHFXX2dfQHC5XUL4ajPo4U85rVGVoPpfjX6jo6BfQ4ed0 hc+Qw/RUeL6yJq0Xi4MCCO9kuOC1/wQS8S6yRyyw+SvJ6mNdkdne8mQ8fL507999Y+Us KerzZ6gaX/2fdN68bPpCfrW0c8GqjtZQpjSdR9YODPZTtL6o97UC5yxwVHHdaFaFUtXN CWkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=9NZ8XI0wSMWu+PvCtOk6AcrbND/96y9gK36RFJAfcOQ=; b=wrvhmY0XvFm6RFXaGxhAgFgrik17yj6o7ObbVD//P+DlUm4gSrJLSR8IFcbcXasXBU R0tBb4KRts8tmQZoFhOjJ3i0a5R3LIS5fZY3bMP5KLj0K/5vVKXMFkQyksLjtuIudp9j aO9eQ4TsR3MUEOI/N06BWuO2S1I79NI0hBlYHiW5Qq3+Wp8UqrpgIsg1oK8m1KwCq9Mk 4vTpyvyVz/CRZMvBCEq29Bz1eVuZnh/M6/ucvlYlgpMPDgu6RQ6XIkm4jiCahyqEdPCR MsmFZkJ0GayQb7OeZRiTUwC+N73Vh0uGOUl37ZmIlDLOimHXSuUjWwYl7c52OWdDtXGL 9mow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id a22si153514qtc.162.2018.11.20.19.28.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 19:28:32 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9E45D3083390; Wed, 21 Nov 2018 03:28:30 +0000 (UTC) Received: from localhost (ovpn-8-21.pek2.redhat.com [10.72.8.21]) by smtp.corp.redhat.com (Postfix) with ESMTP id 072B117A6A; Wed, 21 Nov 2018 03:28:17 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V11 15/19] block: enable multipage bvecs Date: Wed, 21 Nov 2018 11:23:23 +0800 Message-Id: <20181121032327.8434-16-ming.lei@redhat.com> In-Reply-To: <20181121032327.8434-1-ming.lei@redhat.com> References: <20181121032327.8434-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Wed, 21 Nov 2018 03:28:31 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch pulls the trigger for multi-page bvecs. Signed-off-by: Ming Lei --- block/bio.c | 32 +++++++++++++++++++++++++++----- fs/iomap.c | 2 +- fs/xfs/xfs_aops.c | 2 +- 3 files changed, 29 insertions(+), 7 deletions(-) diff --git a/block/bio.c b/block/bio.c index 0f1635b9ec50..854676edc438 100644 --- a/block/bio.c +++ b/block/bio.c @@ -823,7 +823,7 @@ EXPORT_SYMBOL(bio_add_pc_page); * @len: length of the data to add * @off: offset of the data in @page * - * Try to add the data at @page + @off to the last bvec of @bio. This is a + * Try to add the data at @page + @off to the last page of @bio. This is a * a useful optimisation for file systems with a block size smaller than the * page size. * @@ -836,10 +836,13 @@ bool __bio_try_merge_page(struct bio *bio, struct page *page, return false; if (bio->bi_vcnt > 0) { - struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; + struct bio_vec bv; + struct bio_vec *seg = &bio->bi_io_vec[bio->bi_vcnt - 1]; - if (page == bv->bv_page && off == bv->bv_offset + bv->bv_len) { - bv->bv_len += len; + bvec_last_segment(seg, &bv); + + if (page == bv.bv_page && off == bv.bv_offset + bv.bv_len) { + seg->bv_len += len; bio->bi_iter.bi_size += len; return true; } @@ -848,6 +851,25 @@ bool __bio_try_merge_page(struct bio *bio, struct page *page, } EXPORT_SYMBOL_GPL(__bio_try_merge_page); +static bool bio_try_merge_segment(struct bio *bio, struct page *page, + unsigned int len, unsigned int off) +{ + if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) + return false; + + if (bio->bi_vcnt > 0) { + struct bio_vec *seg = &bio->bi_io_vec[bio->bi_vcnt - 1]; + + if (page_to_phys(seg->bv_page) + seg->bv_offset + seg->bv_len == + page_to_phys(page) + off) { + seg->bv_len += len; + bio->bi_iter.bi_size += len; + return true; + } + } + return false; +} + /** * __bio_add_page - add page to a bio in a new segment * @bio: destination bio @@ -888,7 +910,7 @@ EXPORT_SYMBOL_GPL(__bio_add_page); int bio_add_page(struct bio *bio, struct page *page, unsigned int len, unsigned int offset) { - if (!__bio_try_merge_page(bio, page, len, offset)) { + if (!bio_try_merge_segment(bio, page, len, offset)) { if (bio_full(bio)) return 0; __bio_add_page(bio, page, len, offset); diff --git a/fs/iomap.c b/fs/iomap.c index f5fb8bf75cc8..ccc2ba115f4d 100644 --- a/fs/iomap.c +++ b/fs/iomap.c @@ -344,7 +344,7 @@ iomap_readpage_actor(struct inode *inode, loff_t pos, loff_t length, void *data, ctx->bio->bi_end_io = iomap_read_end_io; } - __bio_add_page(ctx->bio, page, plen, poff); + bio_add_page(ctx->bio, page, plen, poff); done: /* * Move the caller beyond our range so that it keeps making progress. diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 1f1829e506e8..5c2190216614 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -621,7 +621,7 @@ xfs_add_to_ioend( atomic_inc(&iop->write_count); if (bio_full(wpc->ioend->io_bio)) xfs_chain_bio(wpc->ioend, wbc, bdev, sector); - __bio_add_page(wpc->ioend->io_bio, page, len, poff); + bio_add_page(wpc->ioend->io_bio, page, len, poff); } wpc->ioend->io_size += len;