From patchwork Fri Jan 22 16:01:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D9E9C433DB for ; Fri, 22 Jan 2021 16:03:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A099823159 for ; Fri, 22 Jan 2021 16:03:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A099823159 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 01CBD6B000D; Fri, 22 Jan 2021 11:03:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F0F006B000E; Fri, 22 Jan 2021 11:03:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFD8D6B0010; Fri, 22 Jan 2021 11:03:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0113.hostedemail.com [216.40.44.113]) by kanga.kvack.org (Postfix) with ESMTP id C9AAE6B000D for ; Fri, 22 Jan 2021 11:03:26 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 95337181AEF3E for ; Fri, 22 Jan 2021 16:03:26 +0000 (UTC) X-FDA: 77733880812.27.key93_3410e4e2756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 2CD563D66D for ; Fri, 22 Jan 2021 16:03:24 +0000 (UTC) X-HE-Tag: key93_3410e4e2756d X-Filterd-Recvd-Size: 6316 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:03:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Nbi5ZwqvpfNoyWg2clLlXNPhBFB3kJ+6Hvh4lIGwZ/E=; b=iRAzJ6JycFyFh3zYptXJb8DWFa fXTvgDs25OJfwxkakbLiNU+B64KYHyJkDMuvor2E36MnBIhTElJYZDwj2G0VS+Cno1LIkBEx937XS ESDbNnW19ZqAuhRUHTaof/xEJDooyADFWMiabiNX+wFYKIEzz0Pijv/q7gjpWgSFBUaVNWQxMbsVk NXYxEgVL/tQs15xAYQW6VnOhH6ip6vo6TYJ6k3WBespUcvSO5e0LLFR1SGy013kBxmXjhm80OPUEK i8W4DtNb0hmSOYYzKn/Hkr0n1D1SmG2ivnytn8eVyCsH9c3sno6kru34FrkaHTgo4hr3XlXMZkPUB aMyw2yUg==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2ysu-000w6U-BR; Fri, 22 Jan 2021 16:02:07 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig , Miaohe Lin Subject: [PATCH v5 01/18] mm/filemap: Rename generic_file_buffered_read subfunctions Date: Fri, 22 Jan 2021 16:01:23 +0000 Message-Id: <20210122160140.223228-2-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The recent split of generic_file_buffered_read() created some very long function names which are hard to distinguish from each other. Rename as follows: generic_file_buffered_read_readpage -> filemap_read_page generic_file_buffered_read_pagenotuptodate -> filemap_update_page generic_file_buffered_read_no_cached_page -> filemap_create_page generic_file_buffered_read_get_pages -> filemap_get_pages Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Reviewed-by: Miaohe Lin --- mm/filemap.c | 44 +++++++++++++++----------------------------- 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index bb28dd6d9e22a..afc0f674f2242 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2191,11 +2191,8 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } -static struct page * -generic_file_buffered_read_readpage(struct kiocb *iocb, - struct file *filp, - struct address_space *mapping, - struct page *page) +static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, + struct address_space *mapping, struct page *page) { struct file_ra_state *ra = &filp->f_ra; int error; @@ -2246,12 +2243,9 @@ generic_file_buffered_read_readpage(struct kiocb *iocb, return page; } -static struct page * -generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, - struct file *filp, - struct iov_iter *iter, - struct page *page, - loff_t pos, loff_t count) +static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, + struct iov_iter *iter, struct page *page, loff_t pos, + loff_t count) { struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; @@ -2314,12 +2308,11 @@ generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, return page; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static struct page * -generic_file_buffered_read_no_cached_page(struct kiocb *iocb, - struct iov_iter *iter) +static struct page *filemap_create_page(struct kiocb *iocb, + struct iov_iter *iter) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2330,10 +2323,6 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, if (iocb->ki_flags & IOCB_NOIO) return ERR_PTR(-EAGAIN); - /* - * Ok, it wasn't cached, so we need to create a new - * page.. - */ page = page_cache_alloc(mapping); if (!page) return ERR_PTR(-ENOMEM); @@ -2345,13 +2334,11 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, return error != -EEXIST ? ERR_PTR(error) : NULL; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static int generic_file_buffered_read_get_pages(struct kiocb *iocb, - struct iov_iter *iter, - struct page **pages, - unsigned int nr) +static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, + struct page **pages, unsigned int nr) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2378,7 +2365,7 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, if (nr_got) goto got_pages; - pages[0] = generic_file_buffered_read_no_cached_page(iocb, iter); + pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pages[0]); if (!IS_ERR_OR_NULL(pages[0])) nr_got = 1; @@ -2412,8 +2399,8 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, break; } - page = generic_file_buffered_read_pagenotuptodate(iocb, - filp, iter, page, pg_pos, pg_count); + page = filemap_update_page(iocb, filp, iter, page, + pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { for (j = i + 1; j < nr_got; j++) put_page(pages[j]); @@ -2492,8 +2479,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, iocb->ki_flags |= IOCB_NOWAIT; i = 0; - pg_nr = generic_file_buffered_read_get_pages(iocb, iter, - pages, nr_pages); + pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); if (pg_nr < 0) { error = pg_nr; break; From patchwork Fri Jan 22 16:01:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB145C433E0 for ; Fri, 22 Jan 2021 16:04:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4109C23159 for ; Fri, 22 Jan 2021 16:04:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4109C23159 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7E2B16B0007; Fri, 22 Jan 2021 11:04:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 792DF6B000A; Fri, 22 Jan 2021 11:04:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CF426B000C; Fri, 22 Jan 2021 11:04:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id 599A46B0007 for ; Fri, 22 Jan 2021 11:04:42 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 21676824999B for ; Fri, 22 Jan 2021 16:04:42 +0000 (UTC) X-FDA: 77733884004.19.land12_17088352756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 95D5414D15 for ; Fri, 22 Jan 2021 16:04:12 +0000 (UTC) X-HE-Tag: land12_17088352756d X-Filterd-Recvd-Size: 3494 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:04:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bJrHcc3TRuo3zacTOm7nFFwlB8n6a0vSf1BsFy5yvWQ=; b=UKPS+BewMN+ObHxdDV7pNY8in4 d9ph4PhrCy1FYoao5Lt9YvjBfIYfrHSEGPWJj5E9L9MqjqtRwEjM/ExkPiDCNm2Iz3kEYRU4oMpTF Bh206af6zhRXC8+wx7nYXN6SP3Ve3EiviJFg1ykq+SVM+jtVamePjfj5weYJDZYOgpXTa2fRcI+dB oU0g91XHwKHqpQ2mDQztKYZ1B3Jjh29uPgGR8Id45+TdAm/32EKm9715xAh/u3/DjrrzaAwFmJsbE +kIe6kkJZ6JqiFpGYQJlh0s4tFyqSWDzqI+mPBiplMywYnHsIIDzUZuJSfpSdVSMFhZ8RxrPbcUr9 9ZJlmsRg==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2ytq-000wAw-An; Fri, 22 Jan 2021 16:03:09 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Miaohe Lin Subject: [PATCH v5 02/18] mm/filemap: Remove dynamically allocated array from filemap_read Date: Fri, 22 Jan 2021 16:01:24 +0000 Message-Id: <20210122160140.223228-3-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Increasing the batch size runs into diminishing returns. It's probably better to make, eg, three calls to filemap_get_pages() than it is to call into kmalloc(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Reviewed-by: Miaohe Lin --- mm/filemap.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index afc0f674f2242..a5c4f7ddfc40c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2444,8 +2444,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages_onstack[PAGEVEC_SIZE], **pages = NULL; - unsigned int nr_pages = min_t(unsigned int, 512, + struct page *pages[PAGEVEC_SIZE]; + unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - (iocb->ki_pos >> PAGE_SHIFT)); int i, pg_nr, error = 0; @@ -2459,14 +2459,6 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, iov_iter_truncate(iter, inode->i_sb->s_maxbytes); - if (nr_pages > ARRAY_SIZE(pages_onstack)) - pages = kmalloc_array(nr_pages, sizeof(void *), GFP_KERNEL); - - if (!pages) { - pages = pages_onstack; - nr_pages = min_t(unsigned int, nr_pages, ARRAY_SIZE(pages_onstack)); - } - do { cond_resched(); @@ -2551,9 +2543,6 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, file_accessed(filp); - if (pages != pages_onstack) - kfree(pages); - return written ? written : error; } EXPORT_SYMBOL_GPL(generic_file_buffered_read); From patchwork Fri Jan 22 16:01:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E23BC433E0 for ; Fri, 22 Jan 2021 16:05:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 287EC22EBE for ; Fri, 22 Jan 2021 16:05:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 287EC22EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AEEEC6B000A; Fri, 22 Jan 2021 11:05:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A9F146B000C; Fri, 22 Jan 2021 11:05:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DD486B000D; Fri, 22 Jan 2021 11:05:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id 891C96B000A for ; Fri, 22 Jan 2021 11:05:08 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 3C39318149422 for ; Fri, 22 Jan 2021 16:05:08 +0000 (UTC) X-FDA: 77733885096.01.laugh20_360b22d2756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 564521004BA99 for ; Fri, 22 Jan 2021 16:04:49 +0000 (UTC) X-HE-Tag: laugh20_360b22d2756d X-Filterd-Recvd-Size: 8315 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:04:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Xjym9lcL0CxKzUararhEBi3kSrlAKm+bXyK6l1ORLjE=; b=FGYXtlxIJvPCIiOqQR0M1ZHLC8 5AzpbpS9EeuHIzh+NMA6ca8Q6lvkmyxYD76jR9JhhKm5iYY0T1e5RbDKFl0UqvrXsWOMQUgVB7S3Z BRFoqAtBnRs6K75+ZBBJgEGoPX6eGjXpmF0i8/ZvbMKyvn99zAFOIRmVilDrxZbU8EAl2K1wP8gTJ 3gzvo8h/XWw5gjt7+CFvumE5tFhnm3qapaF3CnqoKwx/HBe/XL1oDtqDHx0Po1ye6pnQZdbTbGMrK jvilDFdzdxnVcMaiHS+1poRQ+fD7xjtEcUgX88MOUXO2Qr46Eh3YY0HI37gxNxGipmSlad74Z0AOT zpwbDhQw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yu8-000wCI-H1; Fri, 22 Jan 2021 16:03:26 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig Subject: [PATCH v5 03/18] mm/filemap: Convert filemap_get_pages to take a pagevec Date: Fri, 22 Jan 2021 16:01:25 +0000 Message-Id: <20210122160140.223228-4-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Using a pagevec lets us keep the pages and the number of pages together which simplifies a lot of the calling conventions. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/filemap.c | 82 ++++++++++++++++++++++++---------------------------- 1 file changed, 38 insertions(+), 44 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index a5c4f7ddfc40c..ca4141dd63a4b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2338,22 +2338,22 @@ static struct page *filemap_create_page(struct kiocb *iocb, } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, - struct page **pages, unsigned int nr) + struct pagevec *pvec) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - int i, j, nr_got, err = 0; + unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); + int i, j, err = 0; - nr = min_t(unsigned long, last_index - index, nr); find_page: if (fatal_signal_pending(current)) return -EINTR; - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; if (iocb->ki_flags & IOCB_NOIO) @@ -2361,17 +2361,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; - pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pages[0]); - if (!IS_ERR_OR_NULL(pages[0])) - nr_got = 1; + pvec->pages[0] = filemap_create_page(iocb, iter); + err = PTR_ERR_OR_ZERO(pvec->pages[0]); + if (!IS_ERR_OR_NULL(pvec->pages[0])) + pvec->nr = 1; got_pages: - for (i = 0; i < nr_got; i++) { - struct page *page = pages[i]; + for (i = 0; i < pvec->nr; i++) { + struct page *page = pvec->pages[i]; pgoff_t pg_index = index + i; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); @@ -2379,9 +2379,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2392,9 +2392,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2402,17 +2402,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i + 1; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = PTR_ERR_OR_ZERO(page); break; } } } - if (likely(nr_got)) - return nr_got; + if (likely(pvec->nr)) + return 0; if (err) return err; /* @@ -2444,11 +2444,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages[PAGEVEC_SIZE]; - unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, - ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - - (iocb->ki_pos >> PAGE_SHIFT)); - int i, pg_nr, error = 0; + struct pagevec pvec; + int i, error = 0; bool writably_mapped; loff_t isize, end_offset; @@ -2470,12 +2467,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, if ((iocb->ki_flags & IOCB_WAITQ) && written) iocb->ki_flags |= IOCB_NOWAIT; - i = 0; - pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); - if (pg_nr < 0) { - error = pg_nr; + error = filemap_get_pages(iocb, iter, &pvec); + if (error < 0) break; - } /* * i_size must be checked after we know the pages are Uptodate. @@ -2491,9 +2485,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pg_nr > + while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pages[--pg_nr]); + put_page(pvec.pages[--pvec.nr]); /* * Once we start copying data, we don't want to be touching any @@ -2507,11 +2501,11 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, */ if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) - mark_page_accessed(pages[0]); - for (i = 1; i < pg_nr; i++) - mark_page_accessed(pages[i]); + mark_page_accessed(pvec.pages[0]); + for (i = 1; i < pagevec_count(&pvec); i++) + mark_page_accessed(pvec.pages[i]); - for (i = 0; i < pg_nr; i++) { + for (i = 0; i < pagevec_count(&pvec); i++) { unsigned int offset = iocb->ki_pos & ~PAGE_MASK; unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, PAGE_SIZE - offset); @@ -2523,9 +2517,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, * before reading the page on the kernel side. */ if (writably_mapped) - flush_dcache_page(pages[i]); + flush_dcache_page(pvec.pages[i]); - copied = copy_page_to_iter(pages[i], offset, bytes, iter); + copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); written += copied; iocb->ki_pos += copied; @@ -2537,8 +2531,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, } } put_pages: - for (i = 0; i < pg_nr; i++) - put_page(pages[i]); + for (i = 0; i < pagevec_count(&pvec); i++) + put_page(pvec.pages[i]); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); file_accessed(filp); From patchwork Fri Jan 22 16:01:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF78BC433DB for ; Fri, 22 Jan 2021 16:05:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 24E96233FC for ; Fri, 22 Jan 2021 16:05:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 24E96233FC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A05656B000C; Fri, 22 Jan 2021 11:05:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B4AA6B000D; Fri, 22 Jan 2021 11:05:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F18B6B000E; Fri, 22 Jan 2021 11:05:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0003.hostedemail.com [216.40.44.3]) by kanga.kvack.org (Postfix) with ESMTP id 708946B000C for ; Fri, 22 Jan 2021 11:05:39 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 695734DC5 for ; Fri, 22 Jan 2021 16:05:36 +0000 (UTC) X-FDA: 77733886272.10.sail99_59095482756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 8F9EA16A4D4 for ; Fri, 22 Jan 2021 16:05:35 +0000 (UTC) X-HE-Tag: sail99_59095482756d X-Filterd-Recvd-Size: 9481 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:05:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Ck3hyfvZ+ikY0vayhTOqigRy8fhYspMMGICObptFY94=; b=mjOWFDHg1xYhyKbqz7930dlX88 Nxdis8nDTT1ZfOtDsKl0ovK74UvkcAB7Etb9b36E0U06MDPL0zv6q4I68Ac8OFomlxKNiPbm7tpFm AGkEi7xEPhXM5nQjBWopEtY1Ns0skHWlAGuVbPjGlexAlyXTHP6GBRALqW/vWuaQkTvG94ammfM9h +BG93Secn1//P+9GZzHsiCcB6gEv7s87EqEzxl5XQnDmmFKEggqLIwwqm42RAPFytQROtJH/NbYz5 I3Enz6wekHYa1Sb9iH9Dg9fxm8dyoQPxIH9+Lbu8UmnmTqbMHF1EOX27iGr8fZZoQoWo3YfjnJxuY SUI0xAJQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yuw-000wEo-I6; Fri, 22 Jan 2021 16:04:21 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 04/18] mm/filemap: Use head pages in generic_file_buffered_read Date: Fri, 22 Jan 2021 16:01:26 +0000 Message-Id: <20210122160140.223228-5-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add filemap_get_read_batch() which returns the head pages which represent a contiguous array of bytes in the file. It also stops when encountering a page marked as Readahead or !Uptodate (but does return that page) so it can be handled appropriately by filemap_get_pages(). That lets us remove the loop in filemap_get_pages() and check only the last page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 122 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 85 insertions(+), 37 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ca4141dd63a4b..c5d8b6baf656a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2191,6 +2191,51 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } +/* + * filemap_get_read_batch - Get a batch of pages for read + * + * Get a batch of pages which represent a contiguous range of bytes + * in the file. No tail pages will be returned. If @index is in the + * middle of a THP, the entire THP will be returned. The last page in + * the batch may have Readahead set or be not Uptodate so that the + * caller can take the appropriate action. + */ +static void filemap_get_read_batch(struct address_space *mapping, + pgoff_t index, pgoff_t max, struct pagevec *pvec) +{ + XA_STATE(xas, &mapping->i_pages, index); + struct page *head; + + rcu_read_lock(); + for (head = xas_load(&xas); head; head = xas_next(&xas)) { + if (xas_retry(&xas, head)) + continue; + if (xas.xa_index > max || xa_is_value(head)) + break; + if (!page_cache_get_speculative(head)) + goto retry; + + /* Has the page moved or been split? */ + if (unlikely(head != xas_reload(&xas))) + goto put_page; + + if (!pagevec_add(pvec, head)) + break; + if (!PageUptodate(head)) + break; + if (PageReadahead(head)) + break; + xas.xa_index = head->index + thp_nr_pages(head) - 1; + xas.xa_offset = (xas.xa_index >> xas.xa_shift) & XA_CHUNK_MASK; + continue; +put_page: + put_page(head); +retry: + xas_reset(&xas); + } + rcu_read_unlock(); +} + static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct address_space *mapping, struct page *page) { @@ -2344,15 +2389,15 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; - pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); - int i, j, err = 0; + pgoff_t last_index; + int err = 0; + last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); find_page: if (fatal_signal_pending(current)) return -EINTR; - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; @@ -2361,29 +2406,30 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; pvec->pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (!IS_ERR_OR_NULL(pvec->pages[0])) - pvec->nr = 1; + if (IS_ERR_OR_NULL(pvec->pages[0])) + goto err; + pvec->nr = 1; + return 0; got_pages: - for (i = 0; i < pvec->nr; i++) { - struct page *page = pvec->pages[i]; - pgoff_t pg_index = index + i; + { + struct page *page = pvec->pages[pvec->nr - 1]; + pgoff_t pg_index = page->index; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page_cache_async_readahead(mapping, ra, filp, page, pg_index, last_index - pg_index); @@ -2391,26 +2437,23 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + pvec->nr--; err = PTR_ERR_OR_ZERO(page); - break; } } } +err: if (likely(pvec->nr)) return 0; if (err) @@ -2455,6 +2498,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, return 0; iov_iter_truncate(iter, inode->i_sb->s_maxbytes); + pagevec_init(&pvec); do { cond_resched(); @@ -2482,13 +2526,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, isize = i_size_read(inode); if (unlikely(iocb->ki_pos >= isize)) goto put_pages; - end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > - (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pvec.pages[--pvec.nr]); - /* * Once we start copying data, we don't want to be touching any * cachelines that might be contended: @@ -2502,24 +2541,32 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) mark_page_accessed(pvec.pages[0]); - for (i = 1; i < pagevec_count(&pvec); i++) - mark_page_accessed(pvec.pages[i]); for (i = 0; i < pagevec_count(&pvec); i++) { - unsigned int offset = iocb->ki_pos & ~PAGE_MASK; - unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, - PAGE_SIZE - offset); - unsigned int copied; + struct page *page = pvec.pages[i]; + size_t page_size = thp_size(page); + size_t offset = iocb->ki_pos & (page_size - 1); + size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos, + page_size - offset); + size_t copied; + if (end_offset < page_offset(page)) + break; + if (i > 0) + mark_page_accessed(page); /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. */ - if (writably_mapped) - flush_dcache_page(pvec.pages[i]); + if (writably_mapped) { + int j; + + for (j = 0; j < thp_nr_pages(page); j++) + flush_dcache_page(page + j); + } - copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); + copied = copy_page_to_iter(page, offset, bytes, iter); written += copied; iocb->ki_pos += copied; @@ -2533,6 +2580,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, put_pages: for (i = 0; i < pagevec_count(&pvec); i++) put_page(pvec.pages[i]); + pagevec_reinit(&pvec); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); file_accessed(filp); From patchwork Fri Jan 22 16:01:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E546C433DB for ; Fri, 22 Jan 2021 16:06:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2BED522EBE for ; Fri, 22 Jan 2021 16:06:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2BED522EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C258C6B0007; Fri, 22 Jan 2021 11:06:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD6626B000D; Fri, 22 Jan 2021 11:06:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B140D6B000E; Fri, 22 Jan 2021 11:06:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id 9CA156B0007 for ; Fri, 22 Jan 2021 11:06:15 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 650298249980 for ; Fri, 22 Jan 2021 16:06:15 +0000 (UTC) X-FDA: 77733887910.20.love23_07151412756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 3AD3F180C063E for ; Fri, 22 Jan 2021 16:06:15 +0000 (UTC) X-HE-Tag: love23_07151412756d X-Filterd-Recvd-Size: 5529 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:06:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=wMfE7bzqqAfe5q1gI/PP7mCc/7GwzPnfL/VmkkXAWNc=; b=ieDp0oWMjBBrQwYQdQPsYPdY3h l6bL3UT6bUuM2NoRcZ72f+sR1I4rK9w5l8yscKf1Zha2oxogAxAO8k1IR+K7tkoQs8ouiWXlfuqc5 DtI2NxSv6AytZtsUFIwjryWNGTYpitcEY7MDGZbMtN5RUtAeZAluk6LMpa9ru+NIupOUbgapLGfNe YZtMKhkE19qfM8cA15xg2KATgoNTHd2pE50SzVIBbArBuHiwuOtHce6C38ekamcnDakHxyJGF013F SjKHJIWEQEZoQjkKpuX7eGKHjmmNKXGjSrYOfwZDVulEl15autUmchJuswsu/KfArhCVN8ys1rwks OQXeyA6A==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yvU-000wGo-GI; Fri, 22 Jan 2021 16:05:01 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 05/18] mm/filemap: Pass a sleep state to put_and_wait_on_page_locked Date: Fri, 22 Jan 2021 16:01:27 +0000 Message-Id: <20210122160140.223228-6-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is prep work for the next patch, but I think at least one of the current callers would prefer a killable sleep to an uninterruptible one. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 7 +++++-- mm/huge_memory.c | 4 ++-- mm/migrate.c | 4 ++-- 4 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 4317f34866c75..20225b067583a 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -681,8 +681,7 @@ static inline int wait_on_page_locked_killable(struct page *page) return wait_on_page_bit_killable(compound_head(page), PG_locked); } -extern void put_and_wait_on_page_locked(struct page *page); - +int put_and_wait_on_page_locked(struct page *page, int state); void wait_on_page_writeback(struct page *page); extern void end_page_writeback(struct page *page); void wait_for_stable_page(struct page *page); diff --git a/mm/filemap.c b/mm/filemap.c index c5d8b6baf656a..c71cd95e5372b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1379,20 +1379,23 @@ static int wait_on_page_locked_async(struct page *page, /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. + * @state: The sleep state (TASK_KILLABLE, TASK_UNINTERRUPTIBLE, etc). * * The caller should hold a reference on @page. They expect the page to * become unlocked relatively soon, but do not wish to hold up migration * (for example) by holding the reference while waiting for the page to * come unlocked. After this function returns, the caller should not * dereference @page. + * + * Return: 0 if the page was unlocked or -EINTR if interrupted by a signal. */ -void put_and_wait_on_page_locked(struct page *page) +int put_and_wait_on_page_locked(struct page *page, int state) { wait_queue_head_t *q; page = compound_head(page); q = page_waitqueue(page); - wait_on_page_bit_common(q, page, PG_locked, TASK_UNINTERRUPTIBLE, DROP); + return wait_on_page_bit_common(q, page, PG_locked, state, DROP); } /** diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5aa045e3b5dc1..1764ecd5beba5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1441,7 +1441,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } @@ -1477,7 +1477,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } diff --git a/mm/migrate.c b/mm/migrate.c index a3e1acc72ad78..62b81d5257aaa 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -331,7 +331,7 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, if (!get_page_unless_zero(page)) goto out; pte_unmap_unlock(ptep, ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; out: pte_unmap_unlock(ptep, ptl); @@ -365,7 +365,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd) if (!get_page_unless_zero(page)) goto unlock; spin_unlock(ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; unlock: spin_unlock(ptl); From patchwork Fri Jan 22 16:01:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA129C433E0 for ; Fri, 22 Jan 2021 16:07:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5235922EBE for ; Fri, 22 Jan 2021 16:07:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5235922EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E94736B000D; Fri, 22 Jan 2021 11:07:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E44966B000E; Fri, 22 Jan 2021 11:07:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D35096B0010; Fri, 22 Jan 2021 11:07:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id BDC906B000D for ; Fri, 22 Jan 2021 11:07:28 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 79F541801A238 for ; Fri, 22 Jan 2021 16:07:28 +0000 (UTC) X-FDA: 77733890976.23.crook91_150a0732756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 43F8637608 for ; Fri, 22 Jan 2021 16:07:28 +0000 (UTC) X-HE-Tag: crook91_150a0732756d X-Filterd-Recvd-Size: 5866 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:07:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=trvRejywQoT6u/VJzqGTxTieNBzKtlrLfHekLYCLg+E=; b=pkm7pd/SU24bpEMYVzjjiAch67 hGpQwHm3+s2B0OXEh/ZsWr5xDtyU/afV/WNdlSgV2T958XK//4rhmUZ+MmM24CV4Y2R/hpl7kj8wC 0BEzSdgRt90z1i3bGFNcPrD9ku5meM5JDg521s/Y0zmiBUNPcjo5rrL4SnviNsaSK0p0hUPA25fqp J4l5DhH+HwWRaqodqRako7iQPM83VuMzOaKkjEWBFJqGujoLBVQbURhpSPS9lzC+E+ztyKWJQYMun 0oeGs4T6XGgVb4zc1aGiFwy+dpvlddy8hM2f591kiM6PNty44uhrUd94LhPW5SE5+ULTdyOHZyZSe EAJl04nQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yw9-000wM6-PT; Fri, 22 Jan 2021 16:05:37 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 06/18] mm/filemap: Support readpage splitting a page Date: Fri, 22 Jan 2021 16:01:28 +0000 Message-Id: <20210122160140.223228-7-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For page splitting to succeed, the thread asking to split the page has to be the only one with a reference to the page. Calling wait_on_page_locked() while holding a reference to the page will effectively prevent this from happening with sufficient threads waiting on the same page. Use put_and_wait_on_page_locked() to sleep without holding a reference to the page, then retry the page lookup after the page is unlocked. Since we now get the page lock a little earlier in filemap_update_page(), we can eliminate a number of duplicate checks. The original intent (commit ebded02788b5 ("avoid unnecessary calls to lock_page when waiting for IO to complete during a read")) behind getting the page lock later was to avoid re-locking the page after it has been brought uptodate by another thread. We still avoid that because we go through the normal lookup path again after the winning thread has brought the page uptodate. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 76 ++++++++++++++++------------------------------------ 1 file changed, 23 insertions(+), 53 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index c71cd95e5372b..88201506ad3ef 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1368,14 +1368,6 @@ static int __wait_on_page_locked_async(struct page *page, return ret; } -static int wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait) -{ - if (!PageLocked(page)) - return 0; - return __wait_on_page_locked_async(compound_head(page), wait, false); -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -2299,64 +2291,42 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, struct inode *inode = mapping->host; int error; - /* - * See comment in do_read_cache_page on why - * wait_on_page_locked is used to avoid unnecessarily - * serialisations and why it's safe. - */ if (iocb->ki_flags & IOCB_WAITQ) { - error = wait_on_page_locked_async(page, - iocb->ki_waitq); + error = lock_page_async(page, iocb->ki_waitq); + if (error) { + put_page(page); + return ERR_PTR(error); + } } else { - error = wait_on_page_locked_killable(page); - } - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); + if (!trylock_page(page)) { + put_and_wait_on_page_locked(page, TASK_KILLABLE); + return NULL; + } } - if (PageUptodate(page)) - return page; + if (!page->mapping) + goto truncated; + if (PageUptodate(page)) + goto uptodate; if (inode->i_blkbits == PAGE_SHIFT || !mapping->a_ops->is_partially_uptodate) - goto page_not_up_to_date; + goto readpage; /* pipes can't handle partially uptodate pages */ if (unlikely(iov_iter_is_pipe(iter))) - goto page_not_up_to_date; - if (!trylock_page(page)) - goto page_not_up_to_date; - /* Did it get truncated before we got the lock? */ - if (!page->mapping) - goto page_not_up_to_date_locked; + goto readpage; if (!mapping->a_ops->is_partially_uptodate(page, - pos & ~PAGE_MASK, count)) - goto page_not_up_to_date_locked; + pos & (thp_size(page) - 1), count)) + goto readpage; +uptodate: unlock_page(page); return page; -page_not_up_to_date: - /* Get exclusive access to the page ... */ - error = lock_page_for_iocb(iocb, page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - -page_not_up_to_date_locked: - /* Did it get truncated before we got the lock? */ - if (!page->mapping) { - unlock_page(page); - put_page(page); - return NULL; - } - - /* Did somebody else fill it already? */ - if (PageUptodate(page)) { - unlock_page(page); - return page; - } - +readpage: return filemap_read_page(iocb, filp, mapping, page); +truncated: + unlock_page(page); + put_page(page); + return NULL; } static struct page *filemap_create_page(struct kiocb *iocb, From patchwork Fri Jan 22 16:01:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6823C433E0 for ; Fri, 22 Jan 2021 16:08:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 98525233FC for ; Fri, 22 Jan 2021 16:08:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98525233FC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2D3476B000E; Fri, 22 Jan 2021 11:08:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 25D3B6B0010; Fri, 22 Jan 2021 11:08:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 125936B0012; Fri, 22 Jan 2021 11:08:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id ED5B86B000E for ; Fri, 22 Jan 2021 11:08:14 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id AFC5F1DFC for ; Fri, 22 Jan 2021 16:08:14 +0000 (UTC) X-FDA: 77733892908.12.feast12_0106ae72756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 6A8751802B9D0 for ; Fri, 22 Jan 2021 16:08:14 +0000 (UTC) X-HE-Tag: feast12_0106ae72756d X-Filterd-Recvd-Size: 4168 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:08:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=2l7KhQmeVEgzNv6gIsz+bp1NSbblHe5jWS/lA7k9oQo=; b=e2kb8wiG1N46lGY957HUHmq5pv g19AKKENfHTGsmbGFoLuVQem/Ih/91LxZDDPYd5lrPYSLMImVj83zZAJOu8uxoU/do0tj+4xrftFr E17ne5wQgOr2gBMqgkM5bLHjfUDJ38DwdwzPf5sGYQPsIKrbLgwvMoubHMP4kayuE/bPK81MT6Hi2 fQ33Uuw7C7qOVBRLqeUKzzc1ApsWufEvfbzmP+EfC3ODeEbMteQp3OSXwBCzKaNraqiuSd5cpMKMb WqhNNQSCZrLgKCOsO7+J3ssw0VyYaD/ZxbBRBHyiambwyxH/BMztffIH18CdoCrTorLjbRbuHqG8s yhTl3CZQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2ywo-000wNk-Ks; Fri, 22 Jan 2021 16:06:29 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 07/18] mm/filemap: Inline __wait_on_page_locked_async into caller Date: Fri, 22 Jan 2021 16:01:29 +0000 Message-Id: <20210122160140.223228-8-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The previous patch removed wait_on_page_locked_async(), so inline __wait_on_page_locked_async into __lock_page_async(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 53 ++++++++++++++++++++++------------------------------ 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 88201506ad3ef..abd596931f699 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1338,36 +1338,6 @@ int wait_on_page_bit_killable(struct page *page, int bit_nr) } EXPORT_SYMBOL(wait_on_page_bit_killable); -static int __wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait, bool set) -{ - struct wait_queue_head *q = page_waitqueue(page); - int ret = 0; - - wait->page = page; - wait->bit_nr = PG_locked; - - spin_lock_irq(&q->lock); - __add_wait_queue_entry_tail(q, &wait->wait); - SetPageWaiters(page); - if (set) - ret = !trylock_page(page); - else - ret = PageLocked(page); - /* - * If we were successful now, we know we're still on the - * waitqueue as we're still under the lock. This means it's - * safe to remove and return success, we know the callback - * isn't going to trigger. - */ - if (!ret) - __remove_wait_queue(q, &wait->wait); - else - ret = -EIOCBQUEUED; - spin_unlock_irq(&q->lock); - return ret; -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -1543,7 +1513,28 @@ EXPORT_SYMBOL_GPL(__lock_page_killable); int __lock_page_async(struct page *page, struct wait_page_queue *wait) { - return __wait_on_page_locked_async(page, wait, true); + struct wait_queue_head *q = page_waitqueue(page); + int ret = 0; + + wait->page = page; + wait->bit_nr = PG_locked; + + spin_lock_irq(&q->lock); + __add_wait_queue_entry_tail(q, &wait->wait); + SetPageWaiters(page); + ret = !trylock_page(page); + /* + * If we were successful now, we know we're still on the + * waitqueue as we're still under the lock. This means it's + * safe to remove and return success, we know the callback + * isn't going to trigger. + */ + if (!ret) + __remove_wait_queue(q, &wait->wait); + else + ret = -EIOCBQUEUED; + spin_unlock_irq(&q->lock); + return ret; } /* From patchwork Fri Jan 22 16:01:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 770CBC433DB for ; Fri, 22 Jan 2021 16:08:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2204C22EBE for ; Fri, 22 Jan 2021 16:08:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2204C22EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AEA576B0010; Fri, 22 Jan 2021 11:08:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A72D36B0012; Fri, 22 Jan 2021 11:08:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 913946B0022; Fri, 22 Jan 2021 11:08:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 7736F6B0010 for ; Fri, 22 Jan 2021 11:08:58 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4039318014943 for ; Fri, 22 Jan 2021 16:08:58 +0000 (UTC) X-FDA: 77733894756.09.cord54_0515fd22756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 2554A18012CC1 for ; Fri, 22 Jan 2021 16:08:58 +0000 (UTC) X-HE-Tag: cord54_0515fd22756d X-Filterd-Recvd-Size: 3446 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:08:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bs18mHWbgxEoTtD6bBUn7VUlixvEct0iYWwE1wWQVR8=; b=dDBSQGUGaQ2h865+PRYt2as8Mw 6ElxeUl+AyKNn6ie8vU58KGmB8gWPB4cLdstDwKByFwt9jwR+JSb/djeiEP40PB/jJx14CIS4zPzI NPoY8fcrorVopBy8aQqWHaEy5CR7o+LnDcWXiD7EOHfm/xVuu35qTvN/rrrPqZDWbcOW6xElYutWs ZhLwll2HEQMwWMJ9EMTT4t9xpP3faycE/uIeek0PTv2rp7mLeYiIVQVyjzxr2yt6m2dhAElqatjo/ lqQ3ZOaZvTiFCXxyq2NX9UYDqY962ChR8iJlvU2uBQ4jFJldX0LxGpdQ+mTAP4o4Usgq4o2aWYBB7 sNjpgKgw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yxp-000wPN-4a; Fri, 22 Jan 2021 16:07:41 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 08/18] mm/filemap: Don't call ->readpage if IOCB_WAITQ is set Date: Fri, 22 Jan 2021 16:01:30 +0000 Message-Id: <20210122160140.223228-9-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The readpage operation can block in many (most?) filesystems, so we should punt to a work queue instead of calling it. This was the last caller of lock_page_for_iocb(), so remove it. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index abd596931f699..ee1255a88106f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2167,16 +2167,6 @@ static void shrink_readahead_size_eio(struct file_ra_state *ra) ra->ra_pages /= 4; } -static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) -{ - if (iocb->ki_flags & IOCB_WAITQ) - return lock_page_async(page, iocb->ki_waitq); - else if (iocb->ki_flags & IOCB_NOWAIT) - return trylock_page(page) ? 0 : -EAGAIN; - else - return lock_page_killable(page); -} - /* * filemap_get_read_batch - Get a batch of pages for read * @@ -2228,7 +2218,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT)) { + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); put_page(page); return ERR_PTR(-EAGAIN); @@ -2249,7 +2239,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, } if (!PageUptodate(page)) { - error = lock_page_for_iocb(iocb, page); + error = lock_page_killable(page); if (unlikely(error)) { put_page(page); return ERR_PTR(error); From patchwork Fri Jan 22 16:01:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E3C0C433E0 for ; Fri, 22 Jan 2021 16:09:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0979123A9A for ; Fri, 22 Jan 2021 16:09:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0979123A9A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8ED0E6B000E; Fri, 22 Jan 2021 11:09:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 89C486B0012; Fri, 22 Jan 2021 11:09:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78BF86B0022; Fri, 22 Jan 2021 11:09:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0219.hostedemail.com [216.40.44.219]) by kanga.kvack.org (Postfix) with ESMTP id 632A46B000E for ; Fri, 22 Jan 2021 11:09:55 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2FCB28249980 for ; Fri, 22 Jan 2021 16:09:55 +0000 (UTC) X-FDA: 77733897150.16.spy16_3d014b92756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 056A2100E6903 for ; Fri, 22 Jan 2021 16:09:54 +0000 (UTC) X-HE-Tag: spy16_3d014b92756d X-Filterd-Recvd-Size: 6077 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:09:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=NgCVYx229NtKZXdmCxsIRHe+oOGX7Fnf1zniwjiydYo=; b=kNbOa25ba1q9ABDoUTkLwUX+5t XQLG827nMTfDlbb2ms7uhY1ZxYpWqq82kbmbUfRIrUSkzIpCmvMtFoGMNORJrnnFkylyyQZYao42k 15aSYaedAUi0OXTkn690ljBpM/tfLKxwOQJvGFYL/SuaMwoxzFB0Tw5g7s/Kc7ga2scTxfEDLX9Wk g2FU0eaPWIESTxkm1aRSZ18UMv/dLyi04pzfTq85fPn5gzw/aGiQV8adr4/zSrrJCcmnOPTUQgEX5 LK/JS0i0RuaZr94+wc8CrNGUNrKoQ2528YmPtOeYCYyDnCVllvarUFcuOyTeb+mbAt0ZCbEQnNG7h kygnAODQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yyY-000wQm-2n; Fri, 22 Jan 2021 16:08:04 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 09/18] mm/filemap: Change filemap_read_page calling conventions Date: Fri, 22 Jan 2021 16:01:31 +0000 Message-Id: <20210122160140.223228-10-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make this function more generic by passing the file instead of the iocb. Check in the callers whether we should call readpage or not. Also make it return an errno / 0 / AOP_TRUNCATED_PAGE, and make calling put_page() the caller's responsibility. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 89 +++++++++++++++++++++++++--------------------------- 1 file changed, 42 insertions(+), 47 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ee1255a88106f..ceb2931f5725f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2212,56 +2212,38 @@ static void filemap_get_read_batch(struct address_space *mapping, rcu_read_unlock(); } -static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, - struct address_space *mapping, struct page *page) +static int filemap_read_page(struct file *file, struct address_space *mapping, + struct page *page) { - struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); - } - /* - * A previous I/O error may have been due to temporary - * failures, eg. multipath errors. - * PG_error will be set again if readpage fails. + * A previous I/O error may have been due to temporary failures, + * eg. multipath errors. PG_error will be set again if readpage + * fails. */ ClearPageError(page); /* Start the actual read. The read will unlock the page. */ - error = mapping->a_ops->readpage(filp, page); - - if (unlikely(error)) { - put_page(page); - return error != AOP_TRUNCATED_PAGE ? ERR_PTR(error) : NULL; - } + error = mapping->a_ops->readpage(file, page); + if (error) + return error; + if (PageUptodate(page)) + return 0; + error = lock_page_killable(page); + if (error) + return error; if (!PageUptodate(page)) { - error = lock_page_killable(page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* - * invalidate_mapping_pages got it - */ - unlock_page(page); - put_page(page); - return NULL; - } - unlock_page(page); - shrink_readahead_size_eio(ra); - put_page(page); - return ERR_PTR(-EIO); + if (page->mapping == NULL) { + /* page truncated */ + error = AOP_TRUNCATED_PAGE; + } else { + shrink_readahead_size_eio(&file->f_ra); + error = -EIO; } - unlock_page(page); } - - return page; + unlock_page(page); + return error; } static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, @@ -2303,7 +2285,18 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return page; readpage: - return filemap_read_page(iocb, filp, mapping, page); + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { + unlock_page(page); + put_page(page); + return ERR_PTR(-EAGAIN); + } + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); truncated: unlock_page(page); put_page(page); @@ -2319,7 +2312,7 @@ static struct page *filemap_create_page(struct kiocb *iocb, struct page *page; int error; - if (iocb->ki_flags & IOCB_NOIO) + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) return ERR_PTR(-EAGAIN); page = page_cache_alloc(mapping); @@ -2328,12 +2321,14 @@ static struct page *filemap_create_page(struct kiocb *iocb, error = add_to_page_cache_lru(page, mapping, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (error) { - put_page(page); - return error != -EEXIST ? ERR_PTR(error) : NULL; - } - - return filemap_read_page(iocb, filp, mapping, page); + if (!error) + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, From patchwork Fri Jan 22 16:01:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E5A1C433DB for ; Fri, 22 Jan 2021 16:10:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A052222EBE for ; Fri, 22 Jan 2021 16:10:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A052222EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 458086B0022; Fri, 22 Jan 2021 11:10:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 42FC26B0023; Fri, 22 Jan 2021 11:10:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 345CF6B0024; Fri, 22 Jan 2021 11:10:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 1B9C86B0022 for ; Fri, 22 Jan 2021 11:10:16 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 62D11362A for ; Fri, 22 Jan 2021 16:10:15 +0000 (UTC) X-FDA: 77733897990.10.love67_170bd482756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 06E171698E3 for ; Fri, 22 Jan 2021 16:10:13 +0000 (UTC) X-HE-Tag: love67_170bd482756d X-Filterd-Recvd-Size: 4628 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:10:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=w4OKuHj4/BCRiAsSC83gFZEiMSBcxYv4BKBdF2yd9kM=; b=tfVELe3M1fgchmKA8VzTnNLlYK j7POjKW2ESI11ic4ImrLLmletJWtNDP9NupJu6VesxDPpVo30vi0x2AcfCHK++3zbcdlg3hqXfh4h PMJxpdIAdnL3eZRkjPsc70ITU6WKqQvQyG+mtDY4kyKeUCzLH8ZtMg4SnwVyjT+3bidCh1aAeaNcZ 3c4IZvtkxw0O7T1jPdclEXYXAYvY//vhrUYvcHksjEhWTrgJYgMal4Qbpjv05kKakD3iPKn/NR3XT tPJB7i5XEcr4OPP97QVJU0dv0g5WNNrpbPZbe5WVLOm1/EyoC9ks6ZvgcoijBolyyf1+FKyGXB3Ow 9szD27hw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2yzC-000wS5-Mx; Fri, 22 Jan 2021 16:08:54 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 10/18] mm/filemap: Change filemap_create_page calling conventions Date: Fri, 22 Jan 2021 16:01:32 +0000 Message-Id: <20210122160140.223228-11-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: By moving the iocb flag checks to the caller, we can pass the file and the page index instead of the iocb. It never needed the iter. By passing the pagevec, we can return an errno (or AOP_TRUNCATED_PAGE) instead of an ERR_PTR. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 53 ++++++++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ceb2931f5725f..3e0790626addf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2303,32 +2303,33 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return NULL; } -static struct page *filemap_create_page(struct kiocb *iocb, - struct iov_iter *iter) +static int filemap_create_page(struct file *file, + struct address_space *mapping, pgoff_t index, + struct pagevec *pvec) { - struct file *filp = iocb->ki_filp; - struct address_space *mapping = filp->f_mapping; - pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; struct page *page; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) - return ERR_PTR(-EAGAIN); - page = page_cache_alloc(mapping); if (!page) - return ERR_PTR(-ENOMEM); + return -ENOMEM; error = add_to_page_cache_lru(page, mapping, index, - mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (!error) - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; + mapping_gfp_constraint(mapping, GFP_KERNEL)); + if (error == -EEXIST) + error = AOP_TRUNCATED_PAGE; + if (error) + goto error; + + error = filemap_read_page(file, mapping, page); + if (error) + goto error; + + pagevec_add(pvec, page); + return 0; +error: put_page(page); - if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + return error; } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, @@ -2356,15 +2357,15 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - pvec->pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (IS_ERR_OR_NULL(pvec->pages[0])) - goto err; - pvec->nr = 1; - return 0; + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + err = filemap_create_page(filp, mapping, + iocb->ki_pos >> PAGE_SHIFT, pvec); + if (err == AOP_TRUNCATED_PAGE) + goto find_page; + return err; + } got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; From patchwork Fri Jan 22 16:01:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE7B7C433E9 for ; Fri, 22 Jan 2021 16:11:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 99AF522EBE for ; Fri, 22 Jan 2021 16:11:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99AF522EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3BA016B000E; Fri, 22 Jan 2021 11:11:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36A296B0022; Fri, 22 Jan 2021 11:11:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A6FA6B0023; Fri, 22 Jan 2021 11:11:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0166.hostedemail.com [216.40.44.166]) by kanga.kvack.org (Postfix) with ESMTP id 150946B000E for ; Fri, 22 Jan 2021 11:11:19 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D42161801D3D3 for ; Fri, 22 Jan 2021 16:11:18 +0000 (UTC) X-FDA: 77733900636.24.knot63_24033952756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id B6E381A4A0 for ; Fri, 22 Jan 2021 16:11:18 +0000 (UTC) X-HE-Tag: knot63_24033952756d X-Filterd-Recvd-Size: 4714 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:11:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=jfugzsM0alWjT34BzPisiavewfjggn230xNmr/mWe+s=; b=WUq91r2/3iaaw23zTGy4jLmeuu VS98pc5hFHoZ6VgsdGQo8IJ/ghUkzxAnnxQXKev3+G1vzMFqmIHggoyKjAIL3sMM4HF8/XEDWo3us B53ow4VEmTKPmCpA4T6SM3A0rMuIKkNRTiweLDFTfOX1LfzK07jdINkLn6GxSbCxOZBcP3/E4ryDu YfxSPytawQrw9A1Iv7wi1NzH+d1hqBo3rh2sCCLCVBcH8pL5cE2JMt/c9+bhdpDRHqH1UlN5rIoXl 1OsKpwIfzJkAvC7tZzwiLSCvOaVDhpCkSLS2WvgfO3TlXtriTR/n+hB74rQyro5MIOvDzN+OdxcI5 VOhxf0kA==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z03-000wTi-Pe; Fri, 22 Jan 2021 16:09:47 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 11/18] mm/filemap: Convert filemap_update_page to return an errno Date: Fri, 22 Jan 2021 16:01:33 +0000 Message-Id: <20210122160140.223228-12-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use AOP_TRUNCATED_PAGE to indicate that no error occurred, but the page we looked up is no longer valid. In this case, the reference to the page will have been removed; if we hit any other error, the caller will release the reference. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 38 +++++++++++++++++--------------------- 1 file changed, 17 insertions(+), 21 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3e0790626addf..469fc97659cb5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2246,24 +2246,21 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } -static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, - struct iov_iter *iter, struct page *page, loff_t pos, - loff_t count) +static int filemap_update_page(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page, loff_t pos, loff_t count) { - struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; int error; if (iocb->ki_flags & IOCB_WAITQ) { error = lock_page_async(page, iocb->ki_waitq); - if (error) { - put_page(page); - return ERR_PTR(error); - } + if (error) + return error; } else { if (!trylock_page(page)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); - return NULL; + return AOP_TRUNCATED_PAGE; } } @@ -2282,25 +2279,21 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, goto readpage; uptodate: unlock_page(page); - return page; + return 0; readpage: if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); + return -EAGAIN; } error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; - put_page(page); if (error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + put_page(page); + return error; truncated: unlock_page(page); put_page(page); - return NULL; + return AOP_TRUNCATED_PAGE; } static int filemap_create_page(struct file *file, @@ -2394,11 +2387,12 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, goto err; } - page = filemap_update_page(iocb, filp, iter, page, + err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); - if (IS_ERR_OR_NULL(page)) { + if (err) { + if (err < 0) + put_page(page); pvec->nr--; - err = PTR_ERR_OR_ZERO(page); } } } @@ -2406,6 +2400,8 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, err: if (likely(pvec->nr)) return 0; + if (err == AOP_TRUNCATED_PAGE) + goto find_page; if (err) return err; /* From patchwork Fri Jan 22 16:01:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5270DC433E0 for ; Fri, 22 Jan 2021 16:11:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EABDD206F6 for ; Fri, 22 Jan 2021 16:11:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EABDD206F6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8C38C6B0022; Fri, 22 Jan 2021 11:11:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 872F56B0023; Fri, 22 Jan 2021 11:11:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7AF356B0024; Fri, 22 Jan 2021 11:11:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0032.hostedemail.com [216.40.44.32]) by kanga.kvack.org (Postfix) with ESMTP id 6546D6B0022 for ; Fri, 22 Jan 2021 11:11:38 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 29FC0181AF5C4 for ; Fri, 22 Jan 2021 16:11:38 +0000 (UTC) X-FDA: 77733901476.21.pipe80_540580a2756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 8A939180220C9 for ; Fri, 22 Jan 2021 16:11:37 +0000 (UTC) X-HE-Tag: pipe80_540580a2756d X-Filterd-Recvd-Size: 3479 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:11:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=e1dqgx7QZzF3L8VaYSOlRMVWVBZcwhnmiE8oxge+DEQ=; b=t5K7WMmRr+FAuouatu6/DI/DdL iOgTBwpbWzuKJYr5nYFcCK0orqCF84qm96OLA2hhXMbJj2nAdWP1fkxIsKEWC6psFhcU8Pnqb+ihA HYWH8j9vaGAnuhMrGM6+YGtfs4ntdsemtIy5fiH1tjoXEuWYnFLFqoPhwA6s2UdV0s0cZYI/3TDQF Ut73yAgcU57TYv2tg67aS1ASBCmUlXXMOdYZ5v+gpCO42OCk7xEWrkbwuglriuBY2Atq/AiZcs86Q lYQhNbywBzeLvQvTjBqMMRjkyvN+16EsDBrHT6i+tszyv8SWiRyrDwqXzp2vEY4mpUM/KbD4UTAx4 pemeKiPQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z0m-000waS-3W; Fri, 22 Jan 2021 16:10:22 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 12/18] mm/filemap: Move the iocb checks into filemap_update_page Date: Fri, 22 Jan 2021 16:01:34 +0000 Message-Id: <20210122160140.223228-13-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We don't need to give up when a non-blocking request sees a !Uptodate page. We may be able to satisfy the read from a partially-uptodate page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 469fc97659cb5..9fbfb27483147 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2253,15 +2253,16 @@ static int filemap_update_page(struct kiocb *iocb, struct inode *inode = mapping->host; int error; - if (iocb->ki_flags & IOCB_WAITQ) { - error = lock_page_async(page, iocb->ki_waitq); - if (error) - return error; - } else { - if (!trylock_page(page)) { + if (!trylock_page(page)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_NOIO)) + return -EAGAIN; + if (!(iocb->ki_flags & IOCB_WAITQ)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); return AOP_TRUNCATED_PAGE; } + error = __lock_page_async(page, iocb->ki_waitq); + if (error) + return error; } if (!page->mapping) @@ -2379,14 +2380,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { - put_page(page); - pvec->nr--; - err = -EAGAIN; - goto err; - } - + if ((iocb->ki_flags & IOCB_WAITQ) && + pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); if (err) { From patchwork Fri Jan 22 16:01:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F17E3C433E0 for ; Fri, 22 Jan 2021 16:12:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7DED722EBE for ; Fri, 22 Jan 2021 16:12:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7DED722EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E4E26B000E; Fri, 22 Jan 2021 11:12:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 194186B0023; Fri, 22 Jan 2021 11:12:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D1036B0024; Fri, 22 Jan 2021 11:12:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id EA8486B000E for ; Fri, 22 Jan 2021 11:12:21 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B82E71803F91C for ; Fri, 22 Jan 2021 16:12:21 +0000 (UTC) X-FDA: 77733903282.24.pear50_17125ae2756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 98E421A4A0 for ; Fri, 22 Jan 2021 16:12:21 +0000 (UTC) X-HE-Tag: pear50_17125ae2756d X-Filterd-Recvd-Size: 5381 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:12:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=25Fu4UHrJmMz1eHRSy0HQHdJ9c5Kkv8Xt9zJLZGKkEs=; b=NY6Q/3yhi01TJK3WZKUdF13bp3 YMnXRSKesbilGTBI+ZatfWHukw1mK7Mkj5gZlvQkHuLMyf5X9eUHnRukxvXeKMGmXkYDvODrFwjaG BMW1OgXFi8wmhIOCJo9hmWdqxRJk+1fgY4gfO3lmDTyUtIHRRvevwhP4sqTLp3ghtH8yOhDoNXpZq OW4NQjX8ZqcOIX8lDPjberoeszWDQHOYV7ieX/p6RDzGKbbm7NS36yD/BBqUAhsQApy8rYZapSAO0 d0bazHvHkMu81GSA75oB6mL7P80GApWthu2MSl+rDFTJ/2QJiMUzUFzuoYk9jeXLPbDiq4dMx9EBO 6pGYU9jQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z1T-000wca-Mh; Fri, 22 Jan 2021 16:11:07 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 13/18] mm/filemap: Add filemap_range_uptodate Date: Fri, 22 Jan 2021 16:01:35 +0000 Message-Id: <20210122160140.223228-14-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move the complicated condition and the calculations out of filemap_update_page() into its own function. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 68 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 40 insertions(+), 28 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 9fbfb27483147..a3ebc6082022e 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2246,11 +2246,36 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } +static bool filemap_range_uptodate(struct address_space *mapping, + loff_t pos, struct iov_iter *iter, struct page *page) +{ + int count; + + if (PageUptodate(page)) + return true; + /* pipes can't handle partially uptodate pages */ + if (iov_iter_is_pipe(iter)) + return false; + if (!mapping->a_ops->is_partially_uptodate) + return false; + if (mapping->host->i_blkbits >= (PAGE_SHIFT + thp_order(page))) + return false; + + count = iter->count; + if (page_offset(page) > pos) { + count -= page_offset(page) - pos; + pos = 0; + } else { + pos -= page_offset(page); + } + + return mapping->a_ops->is_partially_uptodate(page, pos, count); +} + static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, - struct page *page, loff_t pos, loff_t count) + struct page *page) { - struct inode *inode = mapping->host; int error; if (!trylock_page(page)) { @@ -2267,34 +2292,25 @@ static int filemap_update_page(struct kiocb *iocb, if (!page->mapping) goto truncated; - if (PageUptodate(page)) - goto uptodate; - if (inode->i_blkbits == PAGE_SHIFT || - !mapping->a_ops->is_partially_uptodate) - goto readpage; - /* pipes can't handle partially uptodate pages */ - if (unlikely(iov_iter_is_pipe(iter))) - goto readpage; - if (!mapping->a_ops->is_partially_uptodate(page, - pos & (thp_size(page) - 1), count)) - goto readpage; -uptodate: - unlock_page(page); - return 0; -readpage: - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - return -EAGAIN; - } + error = 0; + if (filemap_range_uptodate(mapping, iocb->ki_pos, iter, page)) + goto unlock; + + error = -EAGAIN; + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) + goto unlock; + error = filemap_read_page(iocb->ki_filp, mapping, page); if (error == AOP_TRUNCATED_PAGE) put_page(page); return error; truncated: - unlock_page(page); + error = AOP_TRUNCATED_PAGE; put_page(page); - return AOP_TRUNCATED_PAGE; +unlock: + unlock_page(page); + return error; } static int filemap_create_page(struct file *file, @@ -2364,9 +2380,6 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, { struct page *page = pvec->pages[pvec->nr - 1]; pgoff_t pg_index = page->index; - loff_t pg_pos = max(iocb->ki_pos, - (loff_t) pg_index << PAGE_SHIFT); - loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { @@ -2383,8 +2396,7 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page, - pg_pos, pg_count); + err = filemap_update_page(iocb, mapping, iter, page); if (err) { if (err < 0) put_page(page); From patchwork Fri Jan 22 16:01:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADBADC433E0 for ; Fri, 22 Jan 2021 16:12:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5DCE7206F6 for ; Fri, 22 Jan 2021 16:12:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5DCE7206F6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F028B6B0022; Fri, 22 Jan 2021 11:12:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB2136B0024; Fri, 22 Jan 2021 11:12:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA1D36B0025; Fri, 22 Jan 2021 11:12:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id C363E6B0022 for ; Fri, 22 Jan 2021 11:12:43 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9182C180238C6 for ; Fri, 22 Jan 2021 16:12:43 +0000 (UTC) X-FDA: 77733904206.16.title33_390d5602756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 6AB2E100E6912 for ; Fri, 22 Jan 2021 16:12:43 +0000 (UTC) X-HE-Tag: title33_390d5602756d X-Filterd-Recvd-Size: 3301 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:12:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=H2OcqOIMq0mcv5yQgY/W8VUwwxU4vHAWx5dRYzYPb2U=; b=I0D4E/EtTL95duVjzt01mHGoVU foiuIH3AEhr3uDWLJmuJHac2e2oZAohCznE89JQCJXdmISMqKDTKQsNRvC0kMldRfoXOX+CKdcYwj kX1ofuUXbyQyIEkuIXg2DqP5Ll1ctUNv/w1wcqT5wiYIRc7NLlmHVLn7oIEXsEVOybqe7H1l1f7Jw 6vJEyFckzSICuwnvXwCTtx4pzsFG1Kvxl+yeiW3NOQhGB+uqYPvdHYOa+TSItOHy7MbYrALr156BM hmZ5/npfFDHzzqq8wepllJFFO3ThmrtsNUwmmeLww0BHorc4E9JH0d+RqtJhuIbAp080PficLddlt cSIJSvtw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z20-000weN-6H; Fri, 22 Jan 2021 16:11:43 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 14/18] mm/filemap: Split filemap_readahead out of filemap_get_pages Date: Fri, 22 Jan 2021 16:01:36 +0000 Message-Id: <20210122160140.223228-15-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This simplifies the error handling. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index a3ebc6082022e..ba20baef9056c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2342,6 +2342,17 @@ static int filemap_create_page(struct file *file, return error; } +static int filemap_readahead(struct kiocb *iocb, struct file *file, + struct address_space *mapping, struct page *page, + pgoff_t last_index) +{ + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_async_readahead(mapping, &file->f_ra, file, page, + page->index, last_index - page->index); + return 0; +} + static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct pagevec *pvec) { @@ -2379,17 +2390,15 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; - pgoff_t pg_index = page->index; if (PageReadahead(page)) { - if (iocb->ki_flags & IOCB_NOIO) { + err = filemap_readahead(iocb, filp, mapping, page, + last_index); + if (err) { put_page(page); pvec->nr--; - err = -EAGAIN; goto err; } - page_cache_async_readahead(mapping, ra, filp, page, - pg_index, last_index - pg_index); } if (!PageUptodate(page)) { From patchwork Fri Jan 22 16:01:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039921 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD4ABC433E0 for ; Fri, 22 Jan 2021 16:13:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8941C22EBE for ; Fri, 22 Jan 2021 16:13:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8941C22EBE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2A5896B000E; Fri, 22 Jan 2021 11:13:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 27C5B6B0024; Fri, 22 Jan 2021 11:13:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B95E6B0025; Fri, 22 Jan 2021 11:13:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0145.hostedemail.com [216.40.44.145]) by kanga.kvack.org (Postfix) with ESMTP id 05BF46B000E for ; Fri, 22 Jan 2021 11:13:36 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C2F3A612C for ; Fri, 22 Jan 2021 16:13:35 +0000 (UTC) X-FDA: 77733906390.15.offer36_6306f4d2756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 808901814B0D4 for ; Fri, 22 Jan 2021 16:13:35 +0000 (UTC) X-HE-Tag: offer36_6306f4d2756d X-Filterd-Recvd-Size: 4783 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:13:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=2Z2U/bAknO5pacSZOQIOc8zERYqgUMTGzojmzvHTzDA=; b=Hc4W6u/NLFRNaNTdc15sxIiLXf gitT3ZYXCqjwDrOE6gxzxJjPmnGrQ48PaVc6JDYaDjxLLFblVHIJWHFZU6OQQ4kPfJaBeXfyVTcBp /j2YWid4mDmgpcW7Sp6NqJMcU8isKSdheSc/LM/bopD25Aa6GgCG6/yubDfMbwpi6WPxXevW3EWw+ +Eb6Qhwp01sbk09vKRESUz6qkNo/S/iS11XcOPyrZ1UuWYVp6hxywmdztTZVWrYay0fGf9Uzs54dk 16cbKL3RMQ7SAsUSREujd8A6OHdtynP3lhHaXbcZuGSy/o5JEcxcxjqJHl5Hz1Y0rtMFR5kEeZnSE uG5Gp5Og==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z2k-000wg7-Ng; Fri, 22 Jan 2021 16:12:18 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 15/18] mm/filemap: Restructure filemap_get_pages Date: Fri, 22 Jan 2021 16:01:37 +0000 Message-Id: <20210122160140.223228-16-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Remove the got_pages label, remove indentation, rename find_page to retry, simplify error handling. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 71 +++++++++++++++++++++------------------------------- 1 file changed, 28 insertions(+), 43 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ba20baef9056c..1fba96dab4b8e 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2361,70 +2361,55 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; + struct page *page; int err = 0; last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); -find_page: +retry: if (fatal_signal_pending(current)) return -EINTR; - filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - if (iocb->ki_flags & IOCB_NOIO) - return -EAGAIN; - - page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - filemap_get_read_batch(mapping, index, last_index, pvec); + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_sync_readahead(mapping, ra, filp, index, + last_index - index); + filemap_get_read_batch(mapping, index, last_index, pvec); + } if (!pagevec_count(pvec)) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; err = filemap_create_page(filp, mapping, iocb->ki_pos >> PAGE_SHIFT, pvec); if (err == AOP_TRUNCATED_PAGE) - goto find_page; + goto retry; return err; } -got_pages: - { - struct page *page = pvec->pages[pvec->nr - 1]; - - if (PageReadahead(page)) { - err = filemap_readahead(iocb, filp, mapping, page, - last_index); - if (err) { - put_page(page); - pvec->nr--; - goto err; - } - } - if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_WAITQ) && - pagevec_count(pvec) > 1) - iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page); - if (err) { - if (err < 0) - put_page(page); - pvec->nr--; - } - } + page = pvec->pages[pagevec_count(pvec) - 1]; + if (PageReadahead(page)) { + err = filemap_readahead(iocb, filp, mapping, page, last_index); + if (err) + goto err; + } + if (!PageUptodate(page)) { + if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; + err = filemap_update_page(iocb, mapping, iter, page); + if (err) + goto err; } + return 0; err: - if (likely(pvec->nr)) + if (err < 0) + put_page(page); + if (likely(--pvec->nr)) return 0; if (err == AOP_TRUNCATED_PAGE) - goto find_page; - if (err) - return err; - /* - * No pages and no error means we raced and should retry: - */ - goto find_page; + goto retry; + return err; } /** From patchwork Fri Jan 22 16:01:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A2B5C433E0 for ; Fri, 22 Jan 2021 16:14:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0CC68206F6 for ; Fri, 22 Jan 2021 16:14:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CC68206F6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9F7386B0024; Fri, 22 Jan 2021 11:14:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A72F6B0025; Fri, 22 Jan 2021 11:14:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E3EB6B0026; Fri, 22 Jan 2021 11:14:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id 7CD6D6B0024 for ; Fri, 22 Jan 2021 11:14:34 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 481B1181AF5C3 for ; Fri, 22 Jan 2021 16:14:34 +0000 (UTC) X-FDA: 77733908868.22.floor92_0f0edc72756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 107641801F0EB for ; Fri, 22 Jan 2021 16:14:34 +0000 (UTC) X-HE-Tag: floor92_0f0edc72756d X-Filterd-Recvd-Size: 3144 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:14:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vPBpsG8Y9bd3KuPs8xt0zB/EQXJ9nDnfU8YkfkH81Kg=; b=H2FzzfYxjwuwUPc3YQp7VL8e5T pyC8W5VRSTd72GjAqFTEjjUdHAdFi00S52ZyuRwO7qfUcAe7WtrRtquIavHruu+fLiA2+6YFk5jWv xBCa7jzD4a9BIfbpAWnWPmqBRZc9rn5KaupGbI1n6rchCv/B+9vRly/QYXIzlQXwbWJEdzH+pBDHA 0F2lgVnsZN+Dbj5dmpZDeqEcLvb7gRvqxadaZ3V4FWs62RxbsCm7Fdvyow0UJkzj9398ZuwgrNwo7 FNehBlWbeApF5LOHDSnYW0gatvcqi0GF4tCcJr1nzAcHho3Jq0jyq7LyDQsfwKYExDNcZWM0uvcPh tLNnbf+w==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z35-000whC-LO; Fri, 22 Jan 2021 16:12:54 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Christoph Hellwig Subject: [PATCH v5 16/18] mm/filemap: Don't relock the page after calling readpage Date: Fri, 22 Jan 2021 16:01:38 +0000 Message-Id: <20210122160140.223228-17-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We don't need to get the page lock again; we just need to wait for the I/O to finish, so use wait_on_page_locked_killable() like the other callers of ->readpage. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 1fba96dab4b8e..542d9c93732c2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2227,23 +2227,16 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, error = mapping->a_ops->readpage(file, page); if (error) return error; - if (PageUptodate(page)) - return 0; - error = lock_page_killable(page); + error = wait_on_page_locked_killable(page); if (error) return error; - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* page truncated */ - error = AOP_TRUNCATED_PAGE; - } else { - shrink_readahead_size_eio(&file->f_ra); - error = -EIO; - } - } - unlock_page(page); - return error; + if (PageUptodate(page)) + return 0; + if (!page->mapping) /* page truncated */ + return AOP_TRUNCATED_PAGE; + shrink_readahead_size_eio(&file->f_ra); + return -EIO; } static bool filemap_range_uptodate(struct address_space *mapping, From patchwork Fri Jan 22 16:01:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B78D3C433E0 for ; Fri, 22 Jan 2021 16:14:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5B424233FC for ; Fri, 22 Jan 2021 16:14:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B424233FC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E3C776B000E; Fri, 22 Jan 2021 11:14:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DED6D6B0025; Fri, 22 Jan 2021 11:14:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D032E6B0026; Fri, 22 Jan 2021 11:14:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id BA9896B000E for ; Fri, 22 Jan 2021 11:14:50 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7C6DD2C1E for ; Fri, 22 Jan 2021 16:14:50 +0000 (UTC) X-FDA: 77733909540.26.use99_58065182756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 8AA55180BB967 for ; Fri, 22 Jan 2021 16:14:49 +0000 (UTC) X-HE-Tag: use99_58065182756d X-Filterd-Recvd-Size: 6404 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:14:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=A1+HIPy+gc0pcVpTnVIkfesangqzonHz4LbQrFN5pFU=; b=T1Xkr38TmFFJMUcKHRZ0+IrU/+ EtookqUKM9mM59iDg6TlakiUy42NTTUhmddrYBUKn5gln/VxGjMVg+lott3N5Ff3+e9dCbaypvvJk rHlsuRRB0iRh4HGcE2KNLAdwJfN5wACrTQ5wZzhwwqnbef6Nqg69N38nMz5nEX7jaxXds1eO3P/Gu mbfQQGl0U9sNdMZmGgZbUMpzBAvYhVymUafFAc2sO6uvK8sVTrDoynPmAW6RI3jC/YyGUJ1MtXghN aLorOXSi3pcJKQSkkfax4obpy0f2leS5Jh+kVkp68YCrQbv45kZnybcFCdwbB0FQe9UOz6iqveQzu WbZpZekw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z41-000wkQ-RY; Fri, 22 Jan 2021 16:13:37 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: Christoph Hellwig , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Kent Overstreet Subject: [PATCH v5 17/18] mm/filemap: Rename generic_file_buffered_read to filemap_read Date: Fri, 22 Jan 2021 16:01:39 +0000 Message-Id: <20210122160140.223228-18-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Rename generic_file_buffered_read to match the naming of filemap_fault, also update the written parameter to a more descriptive name and improve the kerneldoc comment. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- fs/btrfs/file.c | 2 +- include/linux/fs.h | 4 ++-- mm/filemap.c | 35 ++++++++++++++++------------------- 3 files changed, 19 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index d81ae1f518f23..b0ff2e62a65ca 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -3636,7 +3636,7 @@ static ssize_t btrfs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } - return generic_file_buffered_read(iocb, to, ret); + return filemap_read(iocb, to, ret); } const struct file_operations btrfs_file_operations = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 43ba79ddbd680..11a399967ad80 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2985,8 +2985,8 @@ extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *); extern int generic_write_check_limits(struct file *file, loff_t pos, loff_t *count); extern int generic_file_rw_checks(struct file *file_in, struct file *file_out); -extern ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *to, ssize_t already_read); +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *to, + ssize_t already_read); extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *); extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *); extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); diff --git a/mm/filemap.c b/mm/filemap.c index 542d9c93732c2..ef910eca9e1a2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2406,23 +2406,20 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } /** - * generic_file_buffered_read - generic file read routine - * @iocb: the iocb to read - * @iter: data destination - * @written: already copied + * filemap_read - Read data from the page cache. + * @iocb: The iocb to read. + * @iter: Destination for the data. + * @already_read: Number of bytes already read by the caller. * - * This is a generic file read routine, and uses the - * mapping->a_ops->readpage() function for the actual low-level stuff. + * Copies data from the page cache. If the data is not currently present, + * uses the readahead and readpage address_space operations to fetch it. * - * This is really ugly. But the goto's actually try to clarify some - * of the logic when it comes to error handling etc. - * - * Return: - * * total number of bytes copied, including those the were already @written - * * negative error code if nothing was copied + * Return: Total number of bytes copied, including those already read by + * the caller. If an error happens before any bytes are copied, returns + * a negative error number. */ -ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *iter, ssize_t written) +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, + ssize_t already_read) { struct file *filp = iocb->ki_filp; struct file_ra_state *ra = &filp->f_ra; @@ -2449,7 +2446,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, * can no longer safely return -EIOCBQUEUED. Hence mark * an async read NOWAIT at that point. */ - if ((iocb->ki_flags & IOCB_WAITQ) && written) + if ((iocb->ki_flags & IOCB_WAITQ) && already_read) iocb->ki_flags |= IOCB_NOWAIT; error = filemap_get_pages(iocb, iter, &pvec); @@ -2509,7 +2506,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, copied = copy_page_to_iter(page, offset, bytes, iter); - written += copied; + already_read += copied; iocb->ki_pos += copied; ra->prev_pos = iocb->ki_pos; @@ -2526,9 +2523,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, file_accessed(filp); - return written ? written : error; + return already_read ? already_read : error; } -EXPORT_SYMBOL_GPL(generic_file_buffered_read); +EXPORT_SYMBOL_GPL(filemap_read); /** * generic_file_read_iter - generic filesystem read routine @@ -2603,7 +2600,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) goto out; } - retval = generic_file_buffered_read(iocb, iter, retval); + retval = filemap_read(iocb, iter, retval); out: return retval; } From patchwork Fri Jan 22 16:01:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 12039927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC6F0C433DB for ; Fri, 22 Jan 2021 16:16:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5001B206F6 for ; Fri, 22 Jan 2021 16:16:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5001B206F6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E144C6B000E; Fri, 22 Jan 2021 11:16:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9D676B0025; Fri, 22 Jan 2021 11:16:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8D0B6B0026; Fri, 22 Jan 2021 11:16:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id AF1A96B000E for ; Fri, 22 Jan 2021 11:16:03 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6EEF01DF1 for ; Fri, 22 Jan 2021 16:16:03 +0000 (UTC) X-FDA: 77733912606.24.jewel44_34136602756d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 49D311A4A0 for ; Fri, 22 Jan 2021 16:16:03 +0000 (UTC) X-HE-Tag: jewel44_34136602756d X-Filterd-Recvd-Size: 3100 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 16:16:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=mw1PgASVvzvnZ/mdEiLUwI2zgqaIpbnSB3KmajCDg3c=; b=i3zZ+JVUpCCuZmhey68sNtPI4U JEkV2uJnFz2GWqvhJO/N+xXQKgtnQjcDjfyHj+ydGEek+ErXFpCB+dcV5/DvSAxf+mniFU3ZooQbR tn4iJ/9a+28/PuPUUC+mCXtkC7D6mMLJWIioeWEt5ZVaFROroOFlR1IEYyEWZ7sBgQLKTQzTyJ0No oA5UtG8QVbg0KXAnxImKNaS+Tc00OfgNGxW0RQFlb8EQNoU1HnZjCt1P2Gq/r1nruklb7n4H+f20e dpEX6nhT3FyZF5mMLsOoL7dV1S0YVYoJrr/5c95A0cjlfhdfhWCNWg+ZY26rgMPac1Xd6peFb6185 wD1LWSOw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1l2z4F-000wlr-Ri; Fri, 22 Jan 2021 16:14:04 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: Christoph Hellwig , linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Kent Overstreet Subject: [PATCH v5 18/18] mm/filemap: Simplify generic_file_read_iter Date: Fri, 22 Jan 2021 16:01:40 +0000 Message-Id: <20210122160140.223228-19-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210122160140.223228-1-willy@infradead.org> References: <20210122160140.223228-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Avoid the pointless goto out just for returning retval. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ef910eca9e1a2..f5903494cf173 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2555,7 +2555,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) ssize_t retval = 0; if (!count) - goto out; /* skip atime */ + return 0; /* skip atime */ if (iocb->ki_flags & IOCB_DIRECT) { struct file *file = iocb->ki_filp; @@ -2573,7 +2573,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) iocb->ki_pos, iocb->ki_pos + count - 1); if (retval < 0) - goto out; + return retval; } file_accessed(file); @@ -2597,12 +2597,10 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) */ if (retval < 0 || !count || iocb->ki_pos >= size || IS_DAX(inode)) - goto out; + return retval; } - retval = filemap_read(iocb, iter, retval); -out: - return retval; + return filemap_read(iocb, iter, retval); } EXPORT_SYMBOL(generic_file_read_iter);