From patchwork Wed Nov 4 20:42:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2836174A for ; Wed, 4 Nov 2020 20:42:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C746820637 for ; Wed, 4 Nov 2020 20:42:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="rV6cEyU1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732304AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731475AbgKDUmY (ORCPT ); Wed, 4 Nov 2020 15:42:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29BC8C0613D3 for ; Wed, 4 Nov 2020 12:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=obfMMHdQIANHMdZvpP1HFoeuMZsOOwUrMgjktC9GoWA=; b=rV6cEyU1IipIm8SmZ8AQFCa4CF YekjCXMkmr8Fj7OTnsqpLkcypg4aet2mad1bpJsOc/qAGJZ7Nj/oiGwYN1eA3G85PdJo7JRHBrIO/ AiUa7wtg+dCm2DuW8J4TbmRJe1ry/B1JYTptBnhEPUih45kbkjDvNJeAfbvpwnUdhdZGfmddzzFDD XJYFKulSRoJYi6he8KRFevO1/9UTsgCUPo5sUsZnFk0oVLRHCTm0Q0mlqFyuFrei97Y8xFFyPuaUe dRAfTkD/hc2fKAPC2WUd6+IcQEVQWlxyEI80NcuvR2NN9GDETRZ1TZFsahwcAnNa+uQzLm4zy7qDn Di6Grv6A==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbq-0006Ct-1f; Wed, 04 Nov 2020 20:42:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 01/18] mm/filemap: Rename generic_file_buffered_read subfunctions Date: Wed, 4 Nov 2020 20:42:02 +0000 Message-Id: <20201104204219.23810-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The recent split of generic_file_buffered_read() created some very long function names which are hard to distinguish from each other. Rename as follows: generic_file_buffered_read_readpage -> filemap_read_page generic_file_buffered_read_pagenotuptodate -> filemap_update_page generic_file_buffered_read_no_cached_page -> filemap_create_page generic_file_buffered_read_get_pages -> filemap_get_pages Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 44 +++++++++++++++----------------------------- 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index a68516ddeddc..23e3781b3aef 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2176,11 +2176,8 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } -static struct page * -generic_file_buffered_read_readpage(struct kiocb *iocb, - struct file *filp, - struct address_space *mapping, - struct page *page) +static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, + struct address_space *mapping, struct page *page) { struct file_ra_state *ra = &filp->f_ra; int error; @@ -2231,12 +2228,9 @@ generic_file_buffered_read_readpage(struct kiocb *iocb, return page; } -static struct page * -generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, - struct file *filp, - struct iov_iter *iter, - struct page *page, - loff_t pos, loff_t count) +static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, + struct iov_iter *iter, struct page *page, loff_t pos, + loff_t count) { struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; @@ -2299,12 +2293,11 @@ generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, return page; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static struct page * -generic_file_buffered_read_no_cached_page(struct kiocb *iocb, - struct iov_iter *iter) +static struct page *filemap_create_page(struct kiocb *iocb, + struct iov_iter *iter) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2315,10 +2308,6 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, if (iocb->ki_flags & IOCB_NOIO) return ERR_PTR(-EAGAIN); - /* - * Ok, it wasn't cached, so we need to create a new - * page.. - */ page = page_cache_alloc(mapping); if (!page) return ERR_PTR(-ENOMEM); @@ -2330,13 +2319,11 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, return error != -EEXIST ? ERR_PTR(error) : NULL; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static int generic_file_buffered_read_get_pages(struct kiocb *iocb, - struct iov_iter *iter, - struct page **pages, - unsigned int nr) +static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, + struct page **pages, unsigned int nr) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2363,7 +2350,7 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, if (nr_got) goto got_pages; - pages[0] = generic_file_buffered_read_no_cached_page(iocb, iter); + pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pages[0]); if (!IS_ERR_OR_NULL(pages[0])) nr_got = 1; @@ -2397,8 +2384,8 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, break; } - page = generic_file_buffered_read_pagenotuptodate(iocb, - filp, iter, page, pg_pos, pg_count); + page = filemap_update_page(iocb, filp, iter, page, + pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { for (j = i + 1; j < nr_got; j++) put_page(pages[j]); @@ -2474,8 +2461,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, iocb->ki_flags |= IOCB_NOWAIT; i = 0; - pg_nr = generic_file_buffered_read_get_pages(iocb, iter, - pages, nr_pages); + pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); if (pg_nr < 0) { error = pg_nr; break; From patchwork Wed Nov 4 20:42:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882111 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 113BF921 for ; Wed, 4 Nov 2020 20:42:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B79EC20825 for ; Wed, 4 Nov 2020 20:42:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="sUt0T6Yo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732297AbgKDUmY (ORCPT ); Wed, 4 Nov 2020 15:42:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731732AbgKDUmY (ORCPT ); Wed, 4 Nov 2020 15:42:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2E76AC0613D4 for ; Wed, 4 Nov 2020 12:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=IEtkIPQsQeLEcKSFUu4N+/Cun43SD1V6eMts9/zmGx4=; b=sUt0T6Yor6TcXUXshkMis8mEHW 9rbbLkYXwAwhL+MAP3FUiLSYOI0WuKpTbft1uqz2k9GHrSbZLkFdLtmvxzzYioLGKNSKNStUFlTBx p96csI/BsUrVuNZXKU2ANFlb4OHt4y4sDz2Dci7vuOz3CujqpPr0j6TyfKgVoWMhj9bH8iRE2Ihqm yVt7ydXJIjjg0EpJzKUayXCg4HBFNMzD3U80+DFLgmB2uIiPf3+9g6ztkClCesJkVF9ejDo2BMBKf W0NC7t8WlKU9oTJWhMIOz40bZJi+wo9mDeQP8f6jbG5BStWJ3qf200wK4El41FDhTl9YvBXID26bi gaWNT+2A==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbq-0006Cx-8x; Wed, 04 Nov 2020 20:42:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 02/18] mm/filemap: Remove dynamically allocated array from filemap_read Date: Wed, 4 Nov 2020 20:42:03 +0000 Message-Id: <20201104204219.23810-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Increasing the batch size runs into diminishing returns. It's probably better to make, eg, three calls to filemap_get_pages() than it is to call into kmalloc(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/filemap.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 23e3781b3aef..bb1c42d0223c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2429,8 +2429,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages_onstack[PAGEVEC_SIZE], **pages = NULL; - unsigned int nr_pages = min_t(unsigned int, 512, + struct page *pages[PAGEVEC_SIZE]; + unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - (iocb->ki_pos >> PAGE_SHIFT)); int i, pg_nr, error = 0; @@ -2441,14 +2441,6 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, return 0; iov_iter_truncate(iter, inode->i_sb->s_maxbytes); - if (nr_pages > ARRAY_SIZE(pages_onstack)) - pages = kmalloc_array(nr_pages, sizeof(void *), GFP_KERNEL); - - if (!pages) { - pages = pages_onstack; - nr_pages = min_t(unsigned int, nr_pages, ARRAY_SIZE(pages_onstack)); - } - do { cond_resched(); @@ -2533,9 +2525,6 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, file_accessed(filp); - if (pages != pages_onstack) - kfree(pages); - return written ? written : error; } EXPORT_SYMBOL_GPL(generic_file_buffered_read); From patchwork Wed Nov 4 20:42:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CD5F1130 for ; Wed, 4 Nov 2020 20:42:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1885420795 for ; Wed, 4 Nov 2020 20:42:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mXhnGsJF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732321AbgKDUmj (ORCPT ); Wed, 4 Nov 2020 15:42:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732142AbgKDUmg (ORCPT ); Wed, 4 Nov 2020 15:42:36 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23B91C0613D3 for ; Wed, 4 Nov 2020 12:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=0WgTAjyCsNlpTBvR7hpK1MJfnGK7KKLu7MHBBLK3GK8=; b=mXhnGsJFyMdLwH+85xVqtmM5Yv zIPGEjt4IsHPwDKLfNNoMxl+/CyINAXNsJdEbEU+M+QBf5tkiTWQoqPYVdW1nP6foE3Q1PPGDRpbV AVf3MEtze21dl863KHZsRp0Hh4Bh9zzPB7GvtDpwbJYxP4HzqJuPOXmnvqGvpQGHeDjtqgYpXMsIE KyTUhCOoC4QjZmBTbSGE3Su+yFAEupLKVQlKRFHwQnYPm2h7ndJT98vCHKuNaEOgpycYDMQFlFKgd UosMjzeq+vLDu9yrnOcQshPqfXl2gE36SrlX1bWkryg98zCoQSiEI19y7UFgJ0d9kjJ+r7s+se5HH 8i9lG/TA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbq-0006D1-FI; Wed, 04 Nov 2020 20:42:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 03/18] mm/filemap: Convert filemap_get_pages to take a pagevec Date: Wed, 4 Nov 2020 20:42:04 +0000 Message-Id: <20201104204219.23810-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Using a pagevec lets us keep the pages and the number of pages together which simplifies a lot of the calling conventions. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/filemap.c | 82 ++++++++++++++++++++++++---------------------------- 1 file changed, 38 insertions(+), 44 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index bb1c42d0223c..bd02820601f8 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2323,22 +2323,22 @@ static struct page *filemap_create_page(struct kiocb *iocb, } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, - struct page **pages, unsigned int nr) + struct pagevec *pvec) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - int i, j, nr_got, err = 0; + unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); + int i, j, err = 0; - nr = min_t(unsigned long, last_index - index, nr); find_page: if (fatal_signal_pending(current)) return -EINTR; - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; if (iocb->ki_flags & IOCB_NOIO) @@ -2346,17 +2346,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; - pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pages[0]); - if (!IS_ERR_OR_NULL(pages[0])) - nr_got = 1; + pvec->pages[0] = filemap_create_page(iocb, iter); + err = PTR_ERR_OR_ZERO(pvec->pages[0]); + if (!IS_ERR_OR_NULL(pvec->pages[0])) + pvec->nr = 1; got_pages: - for (i = 0; i < nr_got; i++) { - struct page *page = pages[i]; + for (i = 0; i < pvec->nr; i++) { + struct page *page = pvec->pages[i]; pgoff_t pg_index = index + i; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); @@ -2364,9 +2364,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2377,9 +2377,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2387,17 +2387,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i + 1; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = PTR_ERR_OR_ZERO(page); break; } } } - if (likely(nr_got)) - return nr_got; + if (likely(pvec->nr)) + return 0; if (err) return err; /* @@ -2429,11 +2429,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages[PAGEVEC_SIZE]; - unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, - ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - - (iocb->ki_pos >> PAGE_SHIFT)); - int i, pg_nr, error = 0; + struct pagevec pvec; + int i, error = 0; bool writably_mapped; loff_t isize, end_offset; @@ -2452,12 +2449,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, if ((iocb->ki_flags & IOCB_WAITQ) && written) iocb->ki_flags |= IOCB_NOWAIT; - i = 0; - pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); - if (pg_nr < 0) { - error = pg_nr; + error = filemap_get_pages(iocb, iter, &pvec); + if (error < 0) break; - } /* * i_size must be checked after we know the pages are Uptodate. @@ -2473,9 +2467,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pg_nr > + while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pages[--pg_nr]); + put_page(pvec.pages[--pvec.nr]); /* * Once we start copying data, we don't want to be touching any @@ -2489,11 +2483,11 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, */ if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) - mark_page_accessed(pages[0]); - for (i = 1; i < pg_nr; i++) - mark_page_accessed(pages[i]); + mark_page_accessed(pvec.pages[0]); + for (i = 1; i < pagevec_count(&pvec); i++) + mark_page_accessed(pvec.pages[i]); - for (i = 0; i < pg_nr; i++) { + for (i = 0; i < pagevec_count(&pvec); i++) { unsigned int offset = iocb->ki_pos & ~PAGE_MASK; unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, PAGE_SIZE - offset); @@ -2505,9 +2499,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, * before reading the page on the kernel side. */ if (writably_mapped) - flush_dcache_page(pages[i]); + flush_dcache_page(pvec.pages[i]); - copied = copy_page_to_iter(pages[i], offset, bytes, iter); + copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); written += copied; iocb->ki_pos += copied; @@ -2519,8 +2513,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, } } put_pages: - for (i = 0; i < pg_nr; i++) - put_page(pages[i]); + for (i = 0; i < pagevec_count(&pvec); i++) + put_page(pvec.pages[i]); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); file_accessed(filp); From patchwork Wed Nov 4 20:42:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A60F1130 for ; Wed, 4 Nov 2020 20:42:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2DF4A20782 for ; Wed, 4 Nov 2020 20:42:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hMFrg5OS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730980AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732284AbgKDUmY (ORCPT ); Wed, 4 Nov 2020 15:42:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56672C061A4A for ; Wed, 4 Nov 2020 12:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=gLAcQdMdGC7/GAWc07qo/xfEb/oaxKwpC8eBA17ucnI=; b=hMFrg5OSH5zMkgYpMib8gN2PKv nAYuOjGbuwzgGKmVAVPuL19UMKsiFvN85NSVtD5uV7fb+kUNUq92XilRqrxEDqP15dx5AQ9mZ1S0c A53M0e93UYV+aW2nRvEql4KI5y9pGPSoxNMTRl/jhEmCXBe9DW5vZ4jnID3lzRj54qZqbtqjquItB u7wf56lntXZZ0qf5Zg5CcV+Atz4NfDMJKN0bv4kV/Il/nXioT6Ss734RNWoxSesqdZbzDYeaRUPs4 pQpuqV34TCWi8HmO/7GZ5nQzXF7NhkhzSIVdSs8rYKKZMphjSlPRi400An3L6Tgc+YO28Uk6VacxD VPPwXPgA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbq-0006D6-Lm; Wed, 04 Nov 2020 20:42:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 04/18] mm/filemap: Use THPs in generic_file_buffered_read Date: Wed, 4 Nov 2020 20:42:05 +0000 Message-Id: <20201104204219.23810-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add filemap_get_read_batch() which returns the THPs which represent a contiguous array of bytes in the file. It also stops when encountering a page marked as Readahead or !Uptodate (but does return that page) so it can be handled appropriately by filemap_get_pages(). That lets us remove the loop in filemap_get_pages() and check only the last page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 121 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 84 insertions(+), 37 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index bd02820601f8..def9c5513975 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2176,6 +2176,51 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } +/* + * filemap_get_read_batch - Get a batch of pages for read + * + * Get a batch of pages which represent a contiguous range of bytes + * in the file. No tail pages will be returned. If @index is in the + * middle of a THP, the entire THP will be returned. The last page in + * the batch may have Readahead set or be not Uptodate so that the + * caller can take the appropriate action. + */ +static void filemap_get_read_batch(struct address_space *mapping, + pgoff_t index, pgoff_t max, struct pagevec *pvec) +{ + XA_STATE(xas, &mapping->i_pages, index); + struct page *head; + + rcu_read_lock(); + for (head = xas_load(&xas); head; head = xas_next(&xas)) { + if (xas_retry(&xas, head)) + continue; + if (xas.xa_index > max || xa_is_value(head)) + break; + if (!page_cache_get_speculative(head)) + goto retry; + + /* Has the page moved or been split? */ + if (unlikely(head != xas_reload(&xas))) + goto put_page; + + if (!pagevec_add(pvec, head)) + break; + if (!PageUptodate(head)) + break; + if (PageReadahead(head)) + break; + xas.xa_index = head->index + thp_nr_pages(head) - 1; + xas.xa_offset = (xas.xa_index >> xas.xa_shift) & XA_CHUNK_MASK; + continue; +put_page: + put_page(head); +retry: + xas_reset(&xas); + } + rcu_read_unlock(); +} + static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct address_space *mapping, struct page *page) { @@ -2329,15 +2374,16 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; - pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); - int i, j, err = 0; + pgoff_t last_index; + int err = 0; + last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); find_page: if (fatal_signal_pending(current)) return -EINTR; - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + pagevec_init(pvec); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; @@ -2346,29 +2392,30 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; pvec->pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (!IS_ERR_OR_NULL(pvec->pages[0])) - pvec->nr = 1; + if (IS_ERR_OR_NULL(pvec->pages[0])) + goto err; + pvec->nr = 1; + return 0; got_pages: - for (i = 0; i < pvec->nr; i++) { - struct page *page = pvec->pages[i]; - pgoff_t pg_index = index + i; + { + struct page *page = pvec->pages[pvec->nr - 1]; + pgoff_t pg_index = page->index; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page_cache_async_readahead(mapping, ra, filp, page, pg_index, last_index - pg_index); @@ -2376,26 +2423,23 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + pvec->nr--; err = PTR_ERR_OR_ZERO(page); - break; } } } +err: if (likely(pvec->nr)) return 0; if (err) @@ -2464,13 +2508,8 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, isize = i_size_read(inode); if (unlikely(iocb->ki_pos >= isize)) goto put_pages; - end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > - (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pvec.pages[--pvec.nr]); - /* * Once we start copying data, we don't want to be touching any * cachelines that might be contended: @@ -2484,24 +2523,32 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) mark_page_accessed(pvec.pages[0]); - for (i = 1; i < pagevec_count(&pvec); i++) - mark_page_accessed(pvec.pages[i]); for (i = 0; i < pagevec_count(&pvec); i++) { - unsigned int offset = iocb->ki_pos & ~PAGE_MASK; - unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, - PAGE_SIZE - offset); - unsigned int copied; + struct page *page = pvec.pages[i]; + size_t page_size = thp_size(page); + size_t offset = iocb->ki_pos & (page_size - 1); + size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos, + page_size - offset); + size_t copied; + if (end_offset < page_offset(page)) + break; + if (i > 0) + mark_page_accessed(page); /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. */ - if (writably_mapped) - flush_dcache_page(pvec.pages[i]); + if (writably_mapped) { + int j; + + for (j = 0; j < thp_nr_pages(page); j++) + flush_dcache_page(page + j); + } - copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); + copied = copy_page_to_iter(page, offset, bytes, iter); written += copied; iocb->ki_pos += copied; From patchwork Wed Nov 4 20:42:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20941174A for ; Wed, 4 Nov 2020 20:42:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB63520782 for ; Wed, 4 Nov 2020 20:42:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="cQI0Mrmm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732308AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732289AbgKDUmY (ORCPT ); Wed, 4 Nov 2020 15:42:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CB60C061A4C for ; Wed, 4 Nov 2020 12:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9QjKBpl/45Y4xLBzGyfR89N0yrDj4sRG1yoSBSSZloc=; b=cQI0Mrmm3ipLVo/Hv/z2qF4pCf w0cOyHze1v5iEtGP+zoYdm5GCSSh7jm4I9oMve6f/brsFHaARBGjI9kBQ8cV/FPVWY6MwZB3v5ZuE j6sm+Zi9el6cnnc4Y0VQJXFIW1QFmgwEiSOR53k5Usa3MqR8Uqai09HdH6oyGe71mJGoNiYZupwU8 cpr4zKiFhGNavMhi86ZQmb8jOLF7TAIFb3t3CfXzaqy91ZIFaNLoK7iIC2XQTfPzYGOP0sYmTa3K8 4X7n3RplWRd0t88gAXdeoBmHppc4Q5zdBRloTS9TVOGodpZmWul/IZA97LRQviyhYG6bqJHh06ScH zV6fEFTg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbq-0006DE-UQ; Wed, 04 Nov 2020 20:42:23 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 05/18] mm/filemap: Pass a sleep state to put_and_wait_on_page_locked Date: Wed, 4 Nov 2020 20:42:06 +0000 Message-Id: <20201104204219.23810-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This is prep work for the next patch, but I think at least one of the current callers would prefer a killable sleep to an uninterruptible one. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 7 +++++-- mm/huge_memory.c | 4 ++-- mm/migrate.c | 4 ++-- 4 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 00288ed24698..71b36b275e4d 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -681,8 +681,7 @@ static inline int wait_on_page_locked_killable(struct page *page) return wait_on_page_bit_killable(compound_head(page), PG_locked); } -extern void put_and_wait_on_page_locked(struct page *page); - +int put_and_wait_on_page_locked(struct page *page, int state); void wait_on_page_writeback(struct page *page); extern void end_page_writeback(struct page *page); void wait_for_stable_page(struct page *page); diff --git a/mm/filemap.c b/mm/filemap.c index def9c5513975..bc35de079dd0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1358,20 +1358,23 @@ static int wait_on_page_locked_async(struct page *page, /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. + * @state: The sleep state (TASK_KILLABLE, TASK_UNINTERRUPTIBLE, etc). * * The caller should hold a reference on @page. They expect the page to * become unlocked relatively soon, but do not wish to hold up migration * (for example) by holding the reference while waiting for the page to * come unlocked. After this function returns, the caller should not * dereference @page. + * + * Return: 0 if the page was unlocked or -EINTR if interrupted by a signal. */ -void put_and_wait_on_page_locked(struct page *page) +int put_and_wait_on_page_locked(struct page *page, int state) { wait_queue_head_t *q; page = compound_head(page); q = page_waitqueue(page); - wait_on_page_bit_common(q, page, PG_locked, TASK_UNINTERRUPTIBLE, DROP); + return wait_on_page_bit_common(q, page, PG_locked, state, DROP); } /** diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 616102ba3682..ac114d265950 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1432,7 +1432,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } @@ -1468,7 +1468,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } diff --git a/mm/migrate.c b/mm/migrate.c index 39663dfbc273..a50bbb0e029b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -335,7 +335,7 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, if (!get_page_unless_zero(page)) goto out; pte_unmap_unlock(ptep, ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; out: pte_unmap_unlock(ptep, ptl); @@ -369,7 +369,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd) if (!get_page_unless_zero(page)) goto unlock; spin_unlock(ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; unlock: spin_unlock(ptl); From patchwork Wed Nov 4 20:42:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C14DE921 for ; Wed, 4 Nov 2020 20:42:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9538D20825 for ; Wed, 4 Nov 2020 20:42:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="C3wWQqjy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732289AbgKDUm0 (ORCPT ); Wed, 4 Nov 2020 15:42:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730685AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CB187C0613D3 for ; Wed, 4 Nov 2020 12:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=k3NeLpURfpzDiKzHovJBso4aC4OX+4B5YCiKOkDBH9A=; b=C3wWQqjyQ7aa8qWwQdBcT9uWdf zHXtboHTnyYwqPy1xnXxeUIdge2ABUH7o9IyaAduwp6/ZZlBFnvTrdd8BdAmd3fIEm/ow2J/QqORI rOk4XPzh5w6RVDAqWsNyxljTbTsm/SYOq2Z/hzPZvW4rD8YPb6YC2Pw/r/HgZJ5RXHzDTvKcKm359 ny1WKUlQgr0k+LY9Q1LavrgjAMa+9vcg8HZ0SQlQnwjPn0SX61sx2y/V+YztWh5mVERDrvQmInAYb 3Ry0A4E5+mGqjmr0PqvSwsqZPM6P9DGYYsU8uGRbtn59c2e3MciJqCe97YLlV5hRoCAY4CM+aa8OJ cRnBv/qw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbr-0006DP-7G; Wed, 04 Nov 2020 20:42:23 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 06/18] mm/filemap: Support readpage splitting a page Date: Wed, 4 Nov 2020 20:42:07 +0000 Message-Id: <20201104204219.23810-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org For page splitting to succeed, the thread asking to split the page has to be the only one with a reference to the page. Calling wait_on_page_locked() while holding a reference to the page will effectively prevent this from happening with sufficient threads waiting on the same page. Use put_and_wait_on_page_locked() to sleep without holding a reference to the page, then retry the page lookup after the page is unlocked. Since we now get the page lock a little earlier in filemap_update_page(), we can eliminate a number of duplicate checks. The original intent (commit ebded02788b5 ("avoid unnecessary calls to lock_page when waiting for IO to complete during a read")) behind getting the page lock later was to avoid re-locking the page after it has been brought uptodate by another thread. We still avoid that because we go through the normal lookup path again after the winning thread has brought the page uptodate. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 76 ++++++++++++++++------------------------------------ 1 file changed, 23 insertions(+), 53 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index bc35de079dd0..02fe068c434b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1347,14 +1347,6 @@ static int __wait_on_page_locked_async(struct page *page, return ret; } -static int wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait) -{ - if (!PageLocked(page)) - return 0; - return __wait_on_page_locked_async(compound_head(page), wait, false); -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -2284,64 +2276,42 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, struct inode *inode = mapping->host; int error; - /* - * See comment in do_read_cache_page on why - * wait_on_page_locked is used to avoid unnecessarily - * serialisations and why it's safe. - */ if (iocb->ki_flags & IOCB_WAITQ) { - error = wait_on_page_locked_async(page, - iocb->ki_waitq); + error = lock_page_async(page, iocb->ki_waitq); + if (error) { + put_page(page); + return ERR_PTR(error); + } } else { - error = wait_on_page_locked_killable(page); - } - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); + if (!trylock_page(page)) { + put_and_wait_on_page_locked(page, TASK_KILLABLE); + return NULL; + } } - if (PageUptodate(page)) - return page; + if (!page->mapping) + goto truncated; + if (PageUptodate(page)) + goto uptodate; if (inode->i_blkbits == PAGE_SHIFT || !mapping->a_ops->is_partially_uptodate) - goto page_not_up_to_date; + goto readpage; /* pipes can't handle partially uptodate pages */ if (unlikely(iov_iter_is_pipe(iter))) - goto page_not_up_to_date; - if (!trylock_page(page)) - goto page_not_up_to_date; - /* Did it get truncated before we got the lock? */ - if (!page->mapping) - goto page_not_up_to_date_locked; + goto readpage; if (!mapping->a_ops->is_partially_uptodate(page, - pos & ~PAGE_MASK, count)) - goto page_not_up_to_date_locked; + pos & (thp_size(page) - 1), count)) + goto readpage; +uptodate: unlock_page(page); return page; -page_not_up_to_date: - /* Get exclusive access to the page ... */ - error = lock_page_for_iocb(iocb, page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - -page_not_up_to_date_locked: - /* Did it get truncated before we got the lock? */ - if (!page->mapping) { - unlock_page(page); - put_page(page); - return NULL; - } - - /* Did somebody else fill it already? */ - if (PageUptodate(page)) { - unlock_page(page); - return page; - } - +readpage: return filemap_read_page(iocb, filp, mapping, page); +truncated: + unlock_page(page); + put_page(page); + return NULL; } static struct page *filemap_create_page(struct kiocb *iocb, From patchwork Wed Nov 4 20:42:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD69B1130 for ; Wed, 4 Nov 2020 20:42:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BECB20795 for ; Wed, 4 Nov 2020 20:42:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YX8YVnGf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732088AbgKDUm0 (ORCPT ); Wed, 4 Nov 2020 15:42:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731475AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1615DC0613D4 for ; Wed, 4 Nov 2020 12:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=BPzTREsrmdHWYJMRCCp6qEMsAlXPYJa2AwEZf+keIHs=; b=YX8YVnGfiivk2X6kdP4wD4vGkf LC4Nvd3cI5suEUZSlfTU6E5iiGAoiAQFBztAjtEFO1e7Q5U+qera9mFgche78+nEqtVmwg5Hx7SBC sd8RoFv86AlicuI6MxztKvfHPwKXvnEY/RKHH7q2uEU/IolivG9OqUkz9XGRMhNSUVRM1xMCWgMYK AGh/oua0wILOsIa6lkkvwSKrPuPbnPp+9g+1Oj9ycB83F2b8AOclNln15weeOmE0HC+gQNxZAt21+ BKE9vA7aA7PrmLD7PzcJtD6lK2QmsZISA6s/Bm9Yh/jeRSEX5VyPd1VPSFIlWEn4ur5f44IsJJCzb RJMty7lA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbr-0006DX-Gl; Wed, 04 Nov 2020 20:42:23 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 07/18] mm/filemap: Inline __wait_on_page_locked_async into caller Date: Wed, 4 Nov 2020 20:42:08 +0000 Message-Id: <20201104204219.23810-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The previous patch removed wait_on_page_locked_async(), so inline __wait_on_page_locked_async into __lock_page_async(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 53 ++++++++++++++++++++++------------------------------ 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 02fe068c434b..6ad067fe5d47 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1317,36 +1317,6 @@ int wait_on_page_bit_killable(struct page *page, int bit_nr) } EXPORT_SYMBOL(wait_on_page_bit_killable); -static int __wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait, bool set) -{ - struct wait_queue_head *q = page_waitqueue(page); - int ret = 0; - - wait->page = page; - wait->bit_nr = PG_locked; - - spin_lock_irq(&q->lock); - __add_wait_queue_entry_tail(q, &wait->wait); - SetPageWaiters(page); - if (set) - ret = !trylock_page(page); - else - ret = PageLocked(page); - /* - * If we were succesful now, we know we're still on the - * waitqueue as we're still under the lock. This means it's - * safe to remove and return success, we know the callback - * isn't going to trigger. - */ - if (!ret) - __remove_wait_queue(q, &wait->wait); - else - ret = -EIOCBQUEUED; - spin_unlock_irq(&q->lock); - return ret; -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -1514,7 +1484,28 @@ EXPORT_SYMBOL_GPL(__lock_page_killable); int __lock_page_async(struct page *page, struct wait_page_queue *wait) { - return __wait_on_page_locked_async(page, wait, true); + struct wait_queue_head *q = page_waitqueue(page); + int ret = 0; + + wait->page = page; + wait->bit_nr = PG_locked; + + spin_lock_irq(&q->lock); + __add_wait_queue_entry_tail(q, &wait->wait); + SetPageWaiters(page); + ret = !trylock_page(page); + /* + * If we were succesful now, we know we're still on the + * waitqueue as we're still under the lock. This means it's + * safe to remove and return success, we know the callback + * isn't going to trigger. + */ + if (!ret) + __remove_wait_queue(q, &wait->wait); + else + ret = -EIOCBQUEUED; + spin_unlock_irq(&q->lock); + return ret; } /* From patchwork Wed Nov 4 20:42:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4323C921 for ; Wed, 4 Nov 2020 20:42:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 18AA920637 for ; Wed, 4 Nov 2020 20:42:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="IwcwNlmf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732114AbgKDUm2 (ORCPT ); Wed, 4 Nov 2020 15:42:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47840 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732284AbgKDUmZ (ORCPT ); Wed, 4 Nov 2020 15:42:25 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F55EC0613D3 for ; Wed, 4 Nov 2020 12:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=YNNeXBhVHv699q0LtJ4ucb2dmsW+OxEQo3+eO1Ahvb4=; b=IwcwNlmfy7T+QEBcoL3iK+lN0F 4R1xJ6qpLTU1QWkf0ZKNCW6u+iOIUwggGmWCSIzXwVEpsKPzO7vXrJDVpWOVyAwWv+YYCcd7sBjve rdWMdR0lIZBQ+INl7MHQsI09C0XhTYUmQrJ5L84j3RnwyEa0GPW6G7nqN1wT5VNCp0kd2ONtXO7GR nsYEsmw2P/Hs37bX3EgTh6Fy4+WAjrESS0yeHHij4VWW2Bzipe+pdVxwi9TN+vAeomIxFl67xlfOm u4C4AaatKA7ROAYpID67JY/Avv+USumnGAMEfCUIxuv+pvvbjWssTjAsDOXKJdhhyPXW22ZvZZWjH 4JVWtkBw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbr-0006Dd-Nd; Wed, 04 Nov 2020 20:42:23 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 08/18] mm/filemap: Don't call ->readpage if IOCB_WAITQ is set Date: Wed, 4 Nov 2020 20:42:09 +0000 Message-Id: <20201104204219.23810-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The readpage operation can block in many (most?) filesystems, so we should punt to a work queue instead of calling it. This was the last caller of lock_page_async(), so remove it. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 6ad067fe5d47..6699581bd4f5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2152,16 +2152,6 @@ static void shrink_readahead_size_eio(struct file_ra_state *ra) ra->ra_pages /= 4; } -static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) -{ - if (iocb->ki_flags & IOCB_WAITQ) - return lock_page_async(page, iocb->ki_waitq); - else if (iocb->ki_flags & IOCB_NOWAIT) - return trylock_page(page) ? 0 : -EAGAIN; - else - return lock_page_killable(page); -} - /* * filemap_get_read_batch - Get a batch of pages for read * @@ -2213,7 +2203,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT)) { + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); put_page(page); return ERR_PTR(-EAGAIN); @@ -2234,7 +2224,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, } if (!PageUptodate(page)) { - error = lock_page_for_iocb(iocb, page); + error = lock_page_killable(page); if (unlikely(error)) { put_page(page); return ERR_PTR(error); From patchwork Wed Nov 4 20:42:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882143 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1197921 for ; Wed, 4 Nov 2020 20:42:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C3CB220795 for ; Wed, 4 Nov 2020 20:42:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="TOuwzlzV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732322AbgKDUmk (ORCPT ); Wed, 4 Nov 2020 15:42:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732309AbgKDUm0 (ORCPT ); Wed, 4 Nov 2020 15:42:26 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D44BDC0613D3 for ; Wed, 4 Nov 2020 12:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=E3PFYu4ADvGDg7UWuMRVEPqs5KTe07zEAN8nBfGPndg=; b=TOuwzlzV2qY3A9WI/LvjxAet6O uhY3g6rlHc2YAFlti6I9+QF+Q9N2wAojqUuekt1933ekXKKJOFrqVY2o2G6b0QSOOcxi6qSeL5hgt PfO3+7/A60WokNdluYiJ48Zb7xaz/38OsbB+ZtIZCl4uK28y+oXlN+09Rl341hRAurkOy7jbz0DUE RKbZj2VzKS/DJ7+ujdjpCP0cASLSp5GvNX3f+V8GNC1WNM/IxTthIsxHaEAOcFsgWRxO8chOe60Zd 0VOqSQez62lfq8XPVBRiAKultYEKwGJDslgpn2RfjgVEqjT2pgCCbAuv5qL4AndjRmeqBqg1XPr9N bWgQk45Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbs-0006Dl-0G; Wed, 04 Nov 2020 20:42:24 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 09/18] mm/filemap: Change filemap_read_page calling conventions Date: Wed, 4 Nov 2020 20:42:10 +0000 Message-Id: <20201104204219.23810-10-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Make this function more generic by passing the file instead of the iocb. Check in the callers whether we should call readpage or not. Also make it return an errno / 0 / AOP_TRUNCATED_PAGE, and make calling put_page() the caller's responsibility. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 89 +++++++++++++++++++++++++--------------------------- 1 file changed, 42 insertions(+), 47 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 6699581bd4f5..8c6444a6296c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2197,56 +2197,38 @@ static void filemap_get_read_batch(struct address_space *mapping, rcu_read_unlock(); } -static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, - struct address_space *mapping, struct page *page) +static int filemap_read_page(struct file *file, struct address_space *mapping, + struct page *page) { - struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); - } - /* - * A previous I/O error may have been due to temporary - * failures, eg. multipath errors. - * PG_error will be set again if readpage fails. + * A previous I/O error may have been due to temporary failures, + * eg. multipath errors. PG_error will be set again if readpage + * fails. */ ClearPageError(page); /* Start the actual read. The read will unlock the page. */ - error = mapping->a_ops->readpage(filp, page); - - if (unlikely(error)) { - put_page(page); - return error != AOP_TRUNCATED_PAGE ? ERR_PTR(error) : NULL; - } + error = mapping->a_ops->readpage(file, page); + if (error) + return error; + if (PageUptodate(page)) + return 0; + error = lock_page_killable(page); + if (error) + return error; if (!PageUptodate(page)) { - error = lock_page_killable(page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* - * invalidate_mapping_pages got it - */ - unlock_page(page); - put_page(page); - return NULL; - } - unlock_page(page); - shrink_readahead_size_eio(ra); - put_page(page); - return ERR_PTR(-EIO); + if (page->mapping == NULL) { + /* page truncated */ + error = AOP_TRUNCATED_PAGE; + } else { + shrink_readahead_size_eio(&file->f_ra); + error = -EIO; } - unlock_page(page); } - - return page; + unlock_page(page); + return error; } static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, @@ -2288,7 +2270,18 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return page; readpage: - return filemap_read_page(iocb, filp, mapping, page); + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { + unlock_page(page); + put_page(page); + return ERR_PTR(-EAGAIN); + } + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); truncated: unlock_page(page); put_page(page); @@ -2304,7 +2297,7 @@ static struct page *filemap_create_page(struct kiocb *iocb, struct page *page; int error; - if (iocb->ki_flags & IOCB_NOIO) + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) return ERR_PTR(-EAGAIN); page = page_cache_alloc(mapping); @@ -2313,12 +2306,14 @@ static struct page *filemap_create_page(struct kiocb *iocb, error = add_to_page_cache_lru(page, mapping, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (error) { - put_page(page); - return error != -EEXIST ? ERR_PTR(error) : NULL; - } - - return filemap_read_page(iocb, filp, mapping, page); + if (!error) + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, From patchwork Wed Nov 4 20:42:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882147 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE431921 for ; Wed, 4 Nov 2020 20:42:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B67F0207BB for ; Wed, 4 Nov 2020 20:42:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pZG00wgm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732325AbgKDUmm (ORCPT ); Wed, 4 Nov 2020 15:42:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730685AbgKDUm0 (ORCPT ); Wed, 4 Nov 2020 15:42:26 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E294C0613D4 for ; Wed, 4 Nov 2020 12:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=DEk+f7g2Jq8Q5PYfmhOauv7ViHdI47tt4KnQl1DEUZ8=; b=pZG00wgmzom43LkAcdoZD+b7BN RVwRrB3sEJpjm6E/jPhxxrQdmn6u+Lbzt55EAPJ/2VOq22dd4s3nQqukWHUonZ+QInXx/hb2xAyGP 5+Z+jgl8hkoVlvnD/rRkPXlt0a9x/XdJw2EVOr3tqO+Oydzv/tsErHQEUMmz61RWaXI+nGJYoXHcn ge9se/MBditIFOWJ3UPRcqkxPdqo4yltxB23+7et5eSuqm6xNNYOorL0dCyHWBVqlvLWyw+xDvxFe y8ldNCNO8P0vjfGm6Ci1QqL9sst58ArRVzOKAyHTm4baW0Y3xZ7jm4PZHQULLRpIU0wSSBsjXJafj NBTQ3epw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbs-0006Dr-Bi; Wed, 04 Nov 2020 20:42:24 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 10/18] mm/filemap: Change filemap_create_page calling conventions Date: Wed, 4 Nov 2020 20:42:11 +0000 Message-Id: <20201104204219.23810-11-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org By moving the iocb flag checks to the caller, we can pass the file and the page index instead of the iocb. It never needed the iter. By passing the pagevec, we can return an errno (or AOP_TRUNCATED_PAGE) instead of an ERR_PTR. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 53 ++++++++++++++++++++++++++-------------------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 8c6444a6296c..25945fefdd39 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2288,32 +2288,33 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return NULL; } -static struct page *filemap_create_page(struct kiocb *iocb, - struct iov_iter *iter) +static int filemap_create_page(struct file *file, + struct address_space *mapping, pgoff_t index, + struct pagevec *pvec) { - struct file *filp = iocb->ki_filp; - struct address_space *mapping = filp->f_mapping; - pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; struct page *page; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) - return ERR_PTR(-EAGAIN); - page = page_cache_alloc(mapping); if (!page) - return ERR_PTR(-ENOMEM); + return -ENOMEM; error = add_to_page_cache_lru(page, mapping, index, - mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (!error) - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; + mapping_gfp_constraint(mapping, GFP_KERNEL)); + if (error == -EEXIST) + error = AOP_TRUNCATED_PAGE; + if (error) + goto error; + + error = filemap_read_page(file, mapping, page); + if (error) + goto error; + + pagevec_add(pvec, page); + return 0; +error: put_page(page); - if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + return error; } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, @@ -2342,15 +2343,15 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - pvec->pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (IS_ERR_OR_NULL(pvec->pages[0])) - goto err; - pvec->nr = 1; - return 0; + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + err = filemap_create_page(filp, mapping, + iocb->ki_pos >> PAGE_SHIFT, pvec); + if (err == AOP_TRUNCATED_PAGE) + goto find_page; + return err; + } got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; From patchwork Wed Nov 4 20:42:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882145 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F5B5174A for ; Wed, 4 Nov 2020 20:42:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69A7920795 for ; Wed, 4 Nov 2020 20:42:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="vyKezbBR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732323AbgKDUml (ORCPT ); Wed, 4 Nov 2020 15:42:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732073AbgKDUm0 (ORCPT ); Wed, 4 Nov 2020 15:42:26 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EABFC061A4A for ; Wed, 4 Nov 2020 12:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=IO+k8uenUtokvjHXHB5nYnweRPf35rV98FYUgve2yF8=; b=vyKezbBR3DfMqOj6dGV6N3I8L5 SfF90eLmdvYDOKniftPV/yi9JOGBHxrKccqoebzlwaByDv4Lh7cDxkM9UXhgjifoKh9uHQjkJn5tZ g58d8qNCbnINd554bIfgqi9C45hC4Snjj3JyjEznukna/RhXrgSzXggB+A4gY1pgv4lvRnWJ15+I3 i4pYdTg+J8tOlYuPMsaMyXKFGWi3jCvgBW37pqHNImSx57RqkEW7GMPhOcZgCk0vb2SANn5t7e/dv JIfyCz5gjlJ8BAWYXcyBdHIkIYW+udIHTfKM3cXfWl3nM9qBNTWwZclPOd9rkUYDZwGhQhHqtwRNJ lkG/zx4g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbs-0006Dz-Ms; Wed, 04 Nov 2020 20:42:24 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 11/18] mm/filemap: Convert filemap_update_page to return an errno Date: Wed, 4 Nov 2020 20:42:12 +0000 Message-Id: <20201104204219.23810-12-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Use AOP_TRUNCATED_PAGE to indicate that no error occurred, but the page we looked up is no longer valid. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 42 +++++++++++++++++++----------------------- 1 file changed, 19 insertions(+), 23 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 25945fefdd39..93c054f51677 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2231,24 +2231,21 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } -static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, - struct iov_iter *iter, struct page *page, loff_t pos, - loff_t count) +static int filemap_update_page(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page, loff_t pos, loff_t count) { - struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; int error; if (iocb->ki_flags & IOCB_WAITQ) { error = lock_page_async(page, iocb->ki_waitq); - if (error) { - put_page(page); - return ERR_PTR(error); - } + if (error) + goto error; } else { if (!trylock_page(page)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); - return NULL; + return AOP_TRUNCATED_PAGE; } } @@ -2267,25 +2264,24 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, goto readpage; uptodate: unlock_page(page); - return page; + return 0; readpage: if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); + error = -EAGAIN; + } else { + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return 0; } - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; +error: put_page(page); - if (error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + return error; truncated: unlock_page(page); put_page(page); - return NULL; + return AOP_TRUNCATED_PAGE; } static int filemap_create_page(struct file *file, @@ -2380,18 +2376,18 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, goto err; } - page = filemap_update_page(iocb, filp, iter, page, + err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); - if (IS_ERR_OR_NULL(page)) { + if (err) pvec->nr--; - err = PTR_ERR_OR_ZERO(page); - } } } err: if (likely(pvec->nr)) return 0; + if (err == AOP_TRUNCATED_PAGE) + goto find_page; if (err) return err; /* From patchwork Wed Nov 4 20:42:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882141 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D387E174A for ; Wed, 4 Nov 2020 20:42:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A917320795 for ; Wed, 4 Nov 2020 20:42:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="C0Z8ozkg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732235AbgKDUmh (ORCPT ); Wed, 4 Nov 2020 15:42:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731475AbgKDUm1 (ORCPT ); Wed, 4 Nov 2020 15:42:27 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5DC8C061A4C for ; Wed, 4 Nov 2020 12:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9CL+S7Fi5xbteEEqYAzcQmFqJbUHwSVAW/7+GqlnElo=; b=C0Z8ozkgSg6mNgnD3anPbyeVoW bkLT+SwwEaO2KHeph6uQAn/So2NFZ5XUN/nG4kF+0dn1AfmbjSxl8ZO5QZggf/YeLNHij3viO1HHk wMs2vARWeHFz81/RRFtot6jGpwPhHD/FIOnYL0MQwu1uCqK7d4AbtM+4e/azKSVBusyr3BaTB6dPB mCYrf/x2xdZB3RG4E73xM8ZkkjMwxwQjtASJZaUc8gfiO7HyOgUSLbWcYa/Yau7KCIxvk4ESGfnsn ZCw0zSqtODXjVX4isvGSsszB+5hFAI1pVwkQx2JAS78SJupnswH2v9Db64Iu4JLrUYOIQoR7iUV3y WpPvvmEA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbt-0006E6-0o; Wed, 04 Nov 2020 20:42:25 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 12/18] mm/filemap: Move the iocb checks into filemap_update_page Date: Wed, 4 Nov 2020 20:42:13 +0000 Message-Id: <20201104204219.23810-13-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We don't need to give up when a non-blocking request sees a !Uptodate page. We may be able to satisfy the read from a partially-uptodate page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 93c054f51677..9db94876122c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2236,17 +2236,18 @@ static int filemap_update_page(struct kiocb *iocb, struct page *page, loff_t pos, loff_t count) { struct inode *inode = mapping->host; - int error; + int error = -EAGAIN; - if (iocb->ki_flags & IOCB_WAITQ) { - error = lock_page_async(page, iocb->ki_waitq); - if (error) + if (!trylock_page(page)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_NOIO)) goto error; - } else { - if (!trylock_page(page)) { + if (!(iocb->ki_flags & IOCB_WAITQ)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); return AOP_TRUNCATED_PAGE; } + error = __lock_page_async(page, iocb->ki_waitq); + if (error) + goto error; } if (!page->mapping) @@ -2368,14 +2369,9 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { - put_page(page); - pvec->nr--; - err = -EAGAIN; - goto err; - } - + if ((iocb->ki_flags & IOCB_WAITQ) && + pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); if (err) From patchwork Wed Nov 4 20:42:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 680C8174A for ; Wed, 4 Nov 2020 20:42:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 40E7A20795 for ; Wed, 4 Nov 2020 20:42:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lvIt+pYO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732319AbgKDUmi (ORCPT ); Wed, 4 Nov 2020 15:42:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732083AbgKDUm1 (ORCPT ); Wed, 4 Nov 2020 15:42:27 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02D48C061A4D for ; Wed, 4 Nov 2020 12:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=wZuYwxOpDaNwN62+yWqx9L6GKStO72TcNJvurdU40aM=; b=lvIt+pYOq0An9fz4WqPx05EZn3 uN0PUS0ZM1/GBVFrIaSNvNS4IiiK1jgRfMCct+spcMVgr5c16HTp3ERZzI0qq4R1wSCnL18iYtDCt QNkV6J71ailMlGUQWmUANfnK+rW/fpnr4KvmrKzNn5EZbet5gXv02pCE87ZsVS+A84E+KSTWcRCVq iwyf1JKp6lZYtyET8OfXPbVehLSToD0m2BI7L+vH9by3GCfD+rkn+CS0YuuGJtm0cl+XkOGP4RvRZ F1ANuMdYcsFx1zxsfcrPIz0cIrE7k6KOazU75izmhuVcbfbvQ/mDLQFDmFN50HOKvkp+/iEQsGnHa nP0vH5bg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbt-0006EI-B2; Wed, 04 Nov 2020 20:42:25 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 13/18] mm/filemap: Add filemap_range_uptodate Date: Wed, 4 Nov 2020 20:42:14 +0000 Message-Id: <20201104204219.23810-14-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Move the complicated condition and the calculations out of filemap_update_page() into its own function. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 80 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 46 insertions(+), 34 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 9db94876122c..281538a05dc1 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2231,11 +2231,39 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } +static bool filemap_range_uptodate(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page) +{ + loff_t pos; + int count; + + if (PageUptodate(page)) + return true; + /* pipes can't handle partially uptodate pages */ + if (iov_iter_is_pipe(iter)) + return false; + if (!mapping->a_ops->is_partially_uptodate) + return false; + if (mapping->host->i_blkbits >= (PAGE_SHIFT + thp_order(page))) + return false; + + pos = (loff_t) page->index << PAGE_SHIFT; + if (pos > iocb->ki_pos) { + count = iocb->ki_pos + iter->count - pos; + pos = 0; + } else { + count = iter->count; + pos = iocb->ki_pos & (thp_size(page) - 1); + } + + return mapping->a_ops->is_partially_uptodate(page, pos, count); +} + static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, - struct page *page, loff_t pos, loff_t count) + struct page *page) { - struct inode *inode = mapping->host; int error = -EAGAIN; if (!trylock_page(page)) { @@ -2250,39 +2278,27 @@ static int filemap_update_page(struct kiocb *iocb, goto error; } + error = AOP_TRUNCATED_PAGE; if (!page->mapping) - goto truncated; - if (PageUptodate(page)) - goto uptodate; - if (inode->i_blkbits == PAGE_SHIFT || - !mapping->a_ops->is_partially_uptodate) - goto readpage; - /* pipes can't handle partially uptodate pages */ - if (unlikely(iov_iter_is_pipe(iter))) - goto readpage; - if (!mapping->a_ops->is_partially_uptodate(page, - pos & (thp_size(page) - 1), count)) - goto readpage; -uptodate: - unlock_page(page); - return 0; - -readpage: - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { + goto unlock; + if (filemap_range_uptodate(iocb, mapping, iter, page)) { unlock_page(page); - error = -EAGAIN; - } else { - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return 0; + return 0; } + + error = -EAGAIN; + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) + goto unlock; + + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (error) + goto error; + return 0; +unlock: + unlock_page(page); error: put_page(page); return error; -truncated: - unlock_page(page); - put_page(page); - return AOP_TRUNCATED_PAGE; } static int filemap_create_page(struct file *file, @@ -2353,9 +2369,6 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, { struct page *page = pvec->pages[pvec->nr - 1]; pgoff_t pg_index = page->index; - loff_t pg_pos = max(iocb->ki_pos, - (loff_t) pg_index << PAGE_SHIFT); - loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { @@ -2372,8 +2385,7 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page, - pg_pos, pg_count); + err = filemap_update_page(iocb, mapping, iter, page); if (err) pvec->nr--; } From patchwork Wed Nov 4 20:42:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21744921 for ; Wed, 4 Nov 2020 20:42:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EAE6920795 for ; Wed, 4 Nov 2020 20:42:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UgsEfTsq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732316AbgKDUmf (ORCPT ); Wed, 4 Nov 2020 15:42:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732142AbgKDUmd (ORCPT ); Wed, 4 Nov 2020 15:42:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 386B3C040201 for ; Wed, 4 Nov 2020 12:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=EzLyDg0XjuxcQhMXYgM8pHY3Yys4o9DoqSKTg0VJ+k4=; b=UgsEfTsqUzBi7FXX/y+OqivOBu XopUIp7L0yTQvF5sqU8LUUg+x6oxrfidiQPe7xk+KVUXWYiO0mw7tsxDieodZ360IJKCG3LIYPM6G YU7fKtmu085x4d8fCCw7WpLf+3zBPhhmofZJ2lu6rAk9+BDkr5OeT+Vr6CvNOXNa9mB7fHnZqvRaP CKhfzvMHzmD/Ob0Z+fUjnYTY6jUB1amkiBci91rwCjz6Zii4NQnxdMhUBQ0LV5ItpZTAfTso9oFJl wdF+kVH5UL4GPKc1BlC49HbK3eovy8MRLEEtFXV/GC5aT34CibiQuksie9gJI7dELYeOkKyPRxe3v lz5l8fqg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbt-0006EP-LX; Wed, 04 Nov 2020 20:42:25 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 14/18] mm/filemap: Split filemap_readahead out of filemap_get_pages Date: Wed, 4 Nov 2020 20:42:15 +0000 Message-Id: <20201104204219.23810-15-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This simplifies the error handling. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 281538a05dc1..7af0e656c5f6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2330,6 +2330,17 @@ static int filemap_create_page(struct file *file, return error; } +static int filemap_readahead(struct kiocb *iocb, struct file *file, + struct address_space *mapping, struct page *page, + pgoff_t last_index) +{ + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_async_readahead(mapping, &file->f_ra, file, page, + page->index, last_index - page->index); + return 0; +} + static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct pagevec *pvec) { @@ -2368,17 +2379,14 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; - pgoff_t pg_index = page->index; if (PageReadahead(page)) { - if (iocb->ki_flags & IOCB_NOIO) { - put_page(page); + err = filemap_readahead(iocb, filp, mapping, page, + last_index); + if (err) { pvec->nr--; - err = -EAGAIN; goto err; } - page_cache_async_readahead(mapping, ra, filp, page, - pg_index, last_index - pg_index); } if (!PageUptodate(page)) { From patchwork Wed Nov 4 20:42:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D839174A for ; Wed, 4 Nov 2020 20:42:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E7EF20795 for ; Wed, 4 Nov 2020 20:42:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="A+juBz0+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732174AbgKDUme (ORCPT ); Wed, 4 Nov 2020 15:42:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732218AbgKDUmd (ORCPT ); Wed, 4 Nov 2020 15:42:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9C84C040203 for ; Wed, 4 Nov 2020 12:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=p52FzgOz2FkBPUpnXRWkpt0LYlXomk3WlbSKftpD8c0=; b=A+juBz0+LHy7JTvLeBVBKewTTR X/CvHXGs8uyBLGTy6KRUItl4+ruaI/d23fCFh5VUyqDx+lMTHT8pr/ITZklMTsxNzOTSK+RVAoZ2i rdWiVTpFByEcLysgEswUZF9uNeznw62defgCcvDRh09E1zQNLOBewikFTImphLoLQFIhFGdWTr24c xfj2O1yUOAovnWbjcTj6ZbRw7fR1nqAOEiTIA7jJrO33KzdrG2l7LJ+y4AI8TKwTF9P+tAER5tbOD oy7jn8eGQQpOLql+b9/0JWR6k2MF1ni6QMqwBOlNKY0bwMJQ9ZmtznHvnbO270vL7NOGb8JxqGu8n MGii6dnA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbu-0006EW-2z; Wed, 04 Nov 2020 20:42:26 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 15/18] mm/filemap: Restructure filemap_get_pages Date: Wed, 4 Nov 2020 20:42:16 +0000 Message-Id: <20201104204219.23810-16-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Remove the got_pages label, remove indentation, rename find_page to retry, simplify error handling. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 64 +++++++++++++++++++++------------------------------- 1 file changed, 26 insertions(+), 38 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 7af0e656c5f6..22716f4bc977 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2349,67 +2349,55 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; + struct page *page; int err = 0; last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); -find_page: +retry: if (fatal_signal_pending(current)) return -EINTR; pagevec_init(pvec); filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - if (iocb->ki_flags & IOCB_NOIO) - return -EAGAIN; - - page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - - filemap_get_read_batch(mapping, index, last_index, pvec); + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_sync_readahead(mapping, ra, filp, index, + last_index - index); + filemap_get_read_batch(mapping, index, last_index, pvec); + } if (!pagevec_count(pvec)) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; err = filemap_create_page(filp, mapping, iocb->ki_pos >> PAGE_SHIFT, pvec); if (err == AOP_TRUNCATED_PAGE) - goto find_page; + goto retry; return err; } -got_pages: - { - struct page *page = pvec->pages[pvec->nr - 1]; - - if (PageReadahead(page)) { - err = filemap_readahead(iocb, filp, mapping, page, - last_index); - if (err) { - pvec->nr--; - goto err; - } - } - if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_WAITQ) && - pagevec_count(pvec) > 1) - iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page); - if (err) - pvec->nr--; - } + page = pvec->pages[pagevec_count(pvec) - 1]; + if (PageReadahead(page)) { + err = filemap_readahead(iocb, filp, mapping, page, last_index); + if (err) + goto err; + } + if (!PageUptodate(page)) { + if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; + err = filemap_update_page(iocb, mapping, iter, page); + if (err) + goto err; } + return 0; err: + pvec->nr--; if (likely(pvec->nr)) return 0; if (err == AOP_TRUNCATED_PAGE) - goto find_page; - if (err) - return err; - /* - * No pages and no error means we raced and should retry: - */ - goto find_page; + goto retry; + return err; } /** From patchwork Wed Nov 4 20:42:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A39E6174A for ; Wed, 4 Nov 2020 20:42:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 70E9920795 for ; Wed, 4 Nov 2020 20:42:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="QBhGioKn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732318AbgKDUmg (ORCPT ); Wed, 4 Nov 2020 15:42:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732235AbgKDUmd (ORCPT ); Wed, 4 Nov 2020 15:42:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00D58C0401C1 for ; Wed, 4 Nov 2020 12:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=FZ0ilWfNk+LpFZcrrKvvNb3YXA8+YdnxL9l6nFjVvBM=; b=QBhGioKn7YTkyVA5GiUrUBp9Z2 ROw64aIQhY2vCO1ljIts11NuNBX9xp/yYJN6Wn9wtGFyuMepl+bRZQprB52tYKyrl4zj9SrrK9pMv aWn8pZ0dEv7Sli9o9maFjiRPe8nqorXEkOHJujnEfbKRevmvV8vgGGlWEtcrWnHtgmRXCCTLA6RlS L2PIvLSuNZrTyzMLQd3lJKH1LJi1oO1PZTVorBaVnH/CWTgXg4QKD/Asvu4x0DLyqgdrNdl2RdHsp WDGJXFR5dYrWVODdxD5N0Pa21oAuoWkNSFbKa6dLANqxnIRfMAW+jMhmf4zWYUWaE7Z4KL1f01+sM sDUxLsOw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbu-0006Em-Bn; Wed, 04 Nov 2020 20:42:26 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH v2 16/18] mm/filemap: Don't relock the page after calling readpage Date: Wed, 4 Nov 2020 20:42:17 +0000 Message-Id: <20201104204219.23810-17-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We don't need to get the page lock again; we just need to wait for the I/O to finish, so use wait_on_page_locked_killable() like the other callers of ->readpage. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 22716f4bc977..721dcb580657 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2212,23 +2212,16 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, error = mapping->a_ops->readpage(file, page); if (error) return error; - if (PageUptodate(page)) - return 0; - error = lock_page_killable(page); + error = wait_on_page_locked_killable(page); if (error) return error; - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* page truncated */ - error = AOP_TRUNCATED_PAGE; - } else { - shrink_readahead_size_eio(&file->f_ra); - error = -EIO; - } - } - unlock_page(page); - return error; + if (PageUptodate(page)) + return 0; + if (!page->mapping) /* page truncated */ + return AOP_TRUNCATED_PAGE; + shrink_readahead_size_eio(&file->f_ra); + return -EIO; } static bool filemap_range_uptodate(struct kiocb *iocb, From patchwork Wed Nov 4 20:42:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E20391130 for ; Wed, 4 Nov 2020 20:42:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8CF120782 for ; Wed, 4 Nov 2020 20:42:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DhQFY2zq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732284AbgKDUme (ORCPT ); Wed, 4 Nov 2020 15:42:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732174AbgKDUmd (ORCPT ); Wed, 4 Nov 2020 15:42:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 752E2C0401C2 for ; Wed, 4 Nov 2020 12:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=MraftUqfwZApKy2Od3kiQnSYjUqSFSHBSGn0SK6jrOE=; b=DhQFY2zq3uO0goOPKmIFZZKk8E iIAyQXQAT25do/SdoqZn2c8hzIKpwIskkQJP07+dcZy+XTuADO2rrFND982cOO9rcd4MmV9DTIORe 8eC/nBvc/F6Q5BUldKZ5tIz0SCs6hpSXZOOBnTn6BgWz2CPqdT5VvCTwWwLCgmwnzDgaI21tpzH8B Yt0N+w1slhVyExEiqJzGfL9Mrh9nSyIBUoDlDcwU1mutkxqjl4ljCmkPyTaI0DVPIx4ckl0/Shx5S LWo9u7r/ou7GPGjp6use6JVA1vSAflKaFkz05wRC1z9Tv04poUY0+uP6DMJtk+380kd6wu8CCGTLM hjDQXXsw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbu-0006Eu-OB; Wed, 04 Nov 2020 20:42:26 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , kent.overstreet@gmail.com, Matthew Wilcox Subject: [PATCH v2 17/18] mm/filemap: Rename generic_file_buffered_read to filemap_read Date: Wed, 4 Nov 2020 20:42:18 +0000 Message-Id: <20201104204219.23810-18-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Christoph Hellwig Rename generic_file_buffered_read to match the naming of filemap_fault, also update the written parameter to a more descriptive name and improve the kerneldoc comment. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- fs/btrfs/file.c | 2 +- include/linux/fs.h | 4 ++-- mm/filemap.c | 35 ++++++++++++++++------------------- 3 files changed, 19 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 87355a38a654..1a4913e1fd12 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -3633,7 +3633,7 @@ static ssize_t btrfs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } - return generic_file_buffered_read(iocb, to, ret); + return filemap_read(iocb, to, ret); } const struct file_operations btrfs_file_operations = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 4ccc879ae845..413e327fa1c6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2946,8 +2946,8 @@ extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *); extern int generic_write_check_limits(struct file *file, loff_t pos, loff_t *count); extern int generic_file_rw_checks(struct file *file_in, struct file *file_out); -extern ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *to, ssize_t already_read); +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *to, + ssize_t already_read); extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *); extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *); extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); diff --git a/mm/filemap.c b/mm/filemap.c index 721dcb580657..d24c25345bae 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2394,23 +2394,20 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } /** - * generic_file_buffered_read - generic file read routine - * @iocb: the iocb to read - * @iter: data destination - * @written: already copied + * filemap_read - Read data from the page cache. + * @iocb: The iocb to read. + * @iter: Destination for the data. + * @already_read: Number of bytes already read by the caller. * - * This is a generic file read routine, and uses the - * mapping->a_ops->readpage() function for the actual low-level stuff. + * Copies data from the page cache. If the data is not currently present, + * uses the readahead and readpage address_space operations to fetch it. * - * This is really ugly. But the goto's actually try to clarify some - * of the logic when it comes to error handling etc. - * - * Return: - * * total number of bytes copied, including those the were already @written - * * negative error code if nothing was copied + * Return: Total number of bytes copied, including those already read by + * the caller. If an error happens before any bytes are copied, returns + * a negative error number. */ -ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *iter, ssize_t written) +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, + ssize_t already_read) { struct file *filp = iocb->ki_filp; struct file_ra_state *ra = &filp->f_ra; @@ -2433,7 +2430,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, * can no longer safely return -EIOCBQUEUED. Hence mark * an async read NOWAIT at that point. */ - if ((iocb->ki_flags & IOCB_WAITQ) && written) + if ((iocb->ki_flags & IOCB_WAITQ) && already_read) iocb->ki_flags |= IOCB_NOWAIT; error = filemap_get_pages(iocb, iter, &pvec); @@ -2493,7 +2490,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, copied = copy_page_to_iter(page, offset, bytes, iter); - written += copied; + already_read += copied; iocb->ki_pos += copied; ra->prev_pos = iocb->ki_pos; @@ -2509,9 +2506,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, file_accessed(filp); - return written ? written : error; + return already_read ? already_read : error; } -EXPORT_SYMBOL_GPL(generic_file_buffered_read); +EXPORT_SYMBOL_GPL(filemap_read); /** * generic_file_read_iter - generic filesystem read routine @@ -2585,7 +2582,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) goto out; } - retval = generic_file_buffered_read(iocb, iter, retval); + retval = filemap_read(iocb, iter, retval); out: return retval; } From patchwork Wed Nov 4 20:42:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11882135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA267921 for ; Wed, 4 Nov 2020 20:42:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A376D20795 for ; Wed, 4 Nov 2020 20:42:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lQwBbmSo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732314AbgKDUmf (ORCPT ); Wed, 4 Nov 2020 15:42:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732207AbgKDUmd (ORCPT ); Wed, 4 Nov 2020 15:42:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95C4EC0401C3 for ; Wed, 4 Nov 2020 12:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=M4qshmF8EuAc2b6PT9kIoX64mldjJyTot7vJt+nRP0k=; b=lQwBbmSoM8Sg5wMDf1lIkmXriF hepWxVPDUt0VKg+vZmva5WB/5fDx3mNXEV/dvm+Z7bh8ySaCc4UpHqDWQAqSRfGTO7A6iQOnYTW5x Zv+p25yTVikF4h3r/Mz5jpoaSE1hc4sGCiD+83spCeUz4jYLPf33kIlOua6g3ATkMHZ0wIuyZ4odR FyJwu61SvDqBdVnYAptVYSe2zzPSfSO1Nwa11DqLKxfdEodue3BMl4o4Y0TRHM7hTL5b114+3so4U mgwxRqK5jLOjIxwekOs1D4V1yrDNLbv2ZVyF/Aup0iMixR0u/b4Hw0suIZ9VVJPVK/TvGw0K4TJBe N/y0aPEQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kaPbv-0006F3-2t; Wed, 04 Nov 2020 20:42:27 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , kent.overstreet@gmail.com, Matthew Wilcox Subject: [PATCH v2 18/18] mm/filemap: Simplify generic_file_read_iter Date: Wed, 4 Nov 2020 20:42:19 +0000 Message-Id: <20201104204219.23810-19-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201104204219.23810-1-willy@infradead.org> References: <20201104204219.23810-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Christoph Hellwig Avoid the pointless goto out just for returning retval. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d24c25345bae..e84845ec7cd4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2538,7 +2538,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) ssize_t retval = 0; if (!count) - goto out; /* skip atime */ + return 0; /* skip atime */ if (iocb->ki_flags & IOCB_DIRECT) { struct file *file = iocb->ki_filp; @@ -2556,7 +2556,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) iocb->ki_pos, iocb->ki_pos + count - 1); if (retval < 0) - goto out; + return retval; } file_accessed(file); @@ -2579,12 +2579,10 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) */ if (retval < 0 || !count || iocb->ki_pos >= size || IS_DAX(inode)) - goto out; + return retval; } - retval = filemap_read(iocb, iter, retval); -out: - return retval; + return filemap_read(iocb, iter, retval); } EXPORT_SYMBOL(generic_file_read_iter);