From patchwork Mon Nov 2 18:42:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874903 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1C1192C for ; Mon, 2 Nov 2020 18:43:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7FF97222BA for ; Mon, 2 Nov 2020 18:43:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="HyxiyGwg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726369AbgKBSnW (ORCPT ); Mon, 2 Nov 2020 13:43:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725809AbgKBSnV (ORCPT ); Mon, 2 Nov 2020 13:43:21 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43006C061A04 for ; Mon, 2 Nov 2020 10:43:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bkzaNHacm07eY9PD61blsMkIGUR+U2cxbSYg2JzcjUQ=; b=HyxiyGwgx6A0ltc2kjP0vnfOhA BIjxrfBZysvV1VC0TjRYLcusCWvqQhd0SoqvKtD0kxedanTfSO61Q/m9jZVegvIfMT/iT9MOZUuRq letxeCZ7494gGCSFRWmqwOV3YfLls/cIBzrZgmSVZ8Nt9PBVUcGnfNN9QdQUrN+qU9EtX8mrIMDjv DaHJa8BPeKMaJykDH9EZK9euWyGEvk2QY3jDTvlimTJDzowJnWgbw3Ge/CUJOq/EITsGnU1zXAo9t e0JPXZLC8POzJWjsbFp2vyZ65IeUQX86y+gIQYEuA0VBZwgP4XcHL58SRHflMPPVA0G+l+0x+Y6u1 Fm7TYqrQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenT-0006ln-Hj; Mon, 02 Nov 2020 18:43:15 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 01/17] mm/filemap: Rename generic_file_buffered_read subfunctions Date: Mon, 2 Nov 2020 18:42:56 +0000 Message-Id: <20201102184312.25926-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The recent split of generic_file_buffered_read() created some very long function names which are hard to distinguish from each other. Rename as follows: generic_file_buffered_read_readpage -> filemap_read_page generic_file_buffered_read_pagenotuptodate -> filemap_update_page generic_file_buffered_read_no_cached_page -> filemap_create_page generic_file_buffered_read_get_pages -> filemap_get_pages Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 44 +++++++++++++++----------------------------- 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index a68516ddeddc..23e3781b3aef 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2176,11 +2176,8 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } -static struct page * -generic_file_buffered_read_readpage(struct kiocb *iocb, - struct file *filp, - struct address_space *mapping, - struct page *page) +static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, + struct address_space *mapping, struct page *page) { struct file_ra_state *ra = &filp->f_ra; int error; @@ -2231,12 +2228,9 @@ generic_file_buffered_read_readpage(struct kiocb *iocb, return page; } -static struct page * -generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, - struct file *filp, - struct iov_iter *iter, - struct page *page, - loff_t pos, loff_t count) +static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, + struct iov_iter *iter, struct page *page, loff_t pos, + loff_t count) { struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; @@ -2299,12 +2293,11 @@ generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, return page; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static struct page * -generic_file_buffered_read_no_cached_page(struct kiocb *iocb, - struct iov_iter *iter) +static struct page *filemap_create_page(struct kiocb *iocb, + struct iov_iter *iter) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2315,10 +2308,6 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, if (iocb->ki_flags & IOCB_NOIO) return ERR_PTR(-EAGAIN); - /* - * Ok, it wasn't cached, so we need to create a new - * page.. - */ page = page_cache_alloc(mapping); if (!page) return ERR_PTR(-ENOMEM); @@ -2330,13 +2319,11 @@ generic_file_buffered_read_no_cached_page(struct kiocb *iocb, return error != -EEXIST ? ERR_PTR(error) : NULL; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static int generic_file_buffered_read_get_pages(struct kiocb *iocb, - struct iov_iter *iter, - struct page **pages, - unsigned int nr) +static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, + struct page **pages, unsigned int nr) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2363,7 +2350,7 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, if (nr_got) goto got_pages; - pages[0] = generic_file_buffered_read_no_cached_page(iocb, iter); + pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pages[0]); if (!IS_ERR_OR_NULL(pages[0])) nr_got = 1; @@ -2397,8 +2384,8 @@ static int generic_file_buffered_read_get_pages(struct kiocb *iocb, break; } - page = generic_file_buffered_read_pagenotuptodate(iocb, - filp, iter, page, pg_pos, pg_count); + page = filemap_update_page(iocb, filp, iter, page, + pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { for (j = i + 1; j < nr_got; j++) put_page(pages[j]); @@ -2474,8 +2461,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, iocb->ki_flags |= IOCB_NOWAIT; i = 0; - pg_nr = generic_file_buffered_read_get_pages(iocb, iter, - pages, nr_pages); + pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); if (pg_nr < 0) { error = pg_nr; break; From patchwork Mon Nov 2 18:42:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0E641130 for ; Mon, 2 Nov 2020 18:43:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7808822245 for ; Mon, 2 Nov 2020 18:43:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="feTu5T5f" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726366AbgKBSnV (ORCPT ); Mon, 2 Nov 2020 13:43:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726066AbgKBSnU (ORCPT ); Mon, 2 Nov 2020 13:43:20 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0283C0617A6 for ; Mon, 2 Nov 2020 10:43:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5UKc/Mp3kM/zU/rfSseS3aHyFa9ZSPW/UoKsd9ADFZo=; b=feTu5T5feJ6zWA/qLVIF/Oky/w Vw73MyXMDSRqiFT0dZ8OsEhC6QPAguPa6ENwq9UwGL6ktPpxs97kxG+TXQQvrtRK/tt2bZ0A+U4dD QtklTvGihf5gEzdukR6R4zA7AAz8jAna/VizB74sImnXFbYD8Il92xNX+O7ETFy7L2pLDTd7MTaoF bPITaSSP++gdoSio8+4hir/DP8hql42Eg2wrVEiaYHobVP9r8Cs0yjUdVYixIap95y4eHiplBCS8d BDlrNX6nh7b6EfTnWPWeQSGXOSXLWI0+pHjKfkbyKh6BR3oxdpP/LXLXk30HoEcVd2YOuTeQsIxLq iKutK6HA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenU-0006ls-2G; Mon, 02 Nov 2020 18:43:17 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 02/17] mm/filemap: Use THPs in generic_file_buffered_read Date: Mon, 2 Nov 2020 18:42:57 +0000 Message-Id: <20201102184312.25926-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add mapping_get_read_thps() which returns the THPs which represent a contiguous array of bytes in the file. It also stops when encountering a page marked as Readahead or !Uptodate (but does return that page) so it can be handled appropriately by filemap_get_pages(). That lets us remove the loop in filemap_get_pages() and check only the last page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 96 ++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 70 insertions(+), 26 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 23e3781b3aef..d9636ccf87ff 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2176,6 +2176,48 @@ static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) return lock_page_killable(page); } +static unsigned mapping_get_read_thps(struct address_space *mapping, + pgoff_t index, unsigned int nr_pages, struct page **pages) +{ + XA_STATE(xas, &mapping->i_pages, index); + struct page *head; + unsigned int ret = 0; + + if (unlikely(!nr_pages)) + return 0; + + rcu_read_lock(); + for (head = xas_load(&xas); head; head = xas_next(&xas)) { + if (xas_retry(&xas, head)) + continue; + if (xa_is_value(head)) + break; + if (!page_cache_get_speculative(head)) + goto retry; + + /* Has the page moved or been split? */ + if (unlikely(head != xas_reload(&xas))) + goto put_page; + + pages[ret++] = head; + if (ret == nr_pages) + break; + if (!PageUptodate(head)) + break; + if (PageReadahead(head)) + break; + xas.xa_index = head->index + thp_nr_pages(head) - 1; + xas.xa_offset = (xas.xa_index >> xas.xa_shift) & XA_CHUNK_MASK; + continue; +put_page: + put_page(head); +retry: + xas_reset(&xas); + } + rcu_read_unlock(); + return ret; +} + static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct address_space *mapping, struct page *page) { @@ -2330,14 +2372,14 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - int i, j, nr_got, err = 0; + int nr_got, err = 0; nr = min_t(unsigned long, last_index - index, nr); find_page: if (fatal_signal_pending(current)) return -EINTR; - nr_got = find_get_pages_contig(mapping, index, nr, pages); + nr_got = mapping_get_read_thps(mapping, index, nr, pages); if (nr_got) goto got_pages; @@ -2346,7 +2388,7 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - nr_got = find_get_pages_contig(mapping, index, nr, pages); + nr_got = mapping_get_read_thps(mapping, index, nr, pages); if (nr_got) goto got_pages; @@ -2355,20 +2397,19 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!IS_ERR_OR_NULL(pages[0])) nr_got = 1; got_pages: - for (i = 0; i < nr_got; i++) { - struct page *page = pages[i]; - pgoff_t pg_index = index + i; + if (nr_got > 0) { + struct page *page = pages[nr_got - 1]; + pgoff_t pg_index = page->index; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + put_page(page); + nr_got--; err = -EAGAIN; - break; + goto err; } page_cache_async_readahead(mapping, ra, filp, page, pg_index, last_index - pg_index); @@ -2376,26 +2417,23 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + ((iocb->ki_flags & IOCB_WAITQ) && nr_got > 1)) { + put_page(page); + nr_got--; err = -EAGAIN; - break; + goto err; } page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + nr_got--; err = PTR_ERR_OR_ZERO(page); - break; } } } +err: if (likely(nr_got)) return nr_got; if (err) @@ -2502,20 +2540,26 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, mark_page_accessed(pages[i]); for (i = 0; i < pg_nr; i++) { - unsigned int offset = iocb->ki_pos & ~PAGE_MASK; - unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, - PAGE_SIZE - offset); - unsigned int copied; + struct page *page = pages[i]; + size_t page_size = thp_size(page); + size_t offset = iocb->ki_pos & (page_size - 1); + size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos, + page_size - offset); + size_t copied; /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. */ - if (writably_mapped) - flush_dcache_page(pages[i]); + if (writably_mapped) { + int j; + + for (j = 0; j < thp_nr_pages(page); j++) + flush_dcache_page(page + j); + } - copied = copy_page_to_iter(pages[i], offset, bytes, iter); + copied = copy_page_to_iter(page, offset, bytes, iter); written += copied; iocb->ki_pos += copied; From patchwork Mon Nov 2 18:42:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874909 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA59A92C for ; Mon, 2 Nov 2020 18:43:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3ED822243 for ; Mon, 2 Nov 2020 18:43:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="kuwYmLU/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726392AbgKBSnY (ORCPT ); Mon, 2 Nov 2020 13:43:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725927AbgKBSnY (ORCPT ); Mon, 2 Nov 2020 13:43:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC80DC061A48 for ; Mon, 2 Nov 2020 10:43:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=nz9pMjX52oOyDOQK/h+mNIPY0q/AM3o8K/VbuCzMZaY=; b=kuwYmLU/mZ/d9vl0TNDOVl0kr5 KldrIXp47MXZcbKp9y2YXpRv7h6ilE0IIMDw70F/0OJvuFqannPt2ctFwJ8VP90/kFgwlqe+HYFOZ T+W8SMx6f4aIO9amkV63M3X9Luhg6wbK9vzGzC2Rd8N8yXtrRxhX8O1Gh0WJa/bY3bdxpkDbDxu9Q 1hIgFFvHy8Pz80WbAbaR3ipLd9+CYxu7w3wi3T9Pjd+//axIiCaGeol/0bGKEDsnH/hH/g/05ef0O tx5BPFzERozE8HmytwgtYOUvb+9t7/iWE+6fZAnVWbuy7C+RqE+4Pw6tZ9U7qJxCkOvfSqbgmYP6x aqLsgDhA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenW-0006m2-Bq; Mon, 02 Nov 2020 18:43:18 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 03/17] mm/filemap: Pass a sleep state to put_and_wait_on_page_locked Date: Mon, 2 Nov 2020 18:42:58 +0000 Message-Id: <20201102184312.25926-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This is prep work for the next patch, but I think at least one of the current callers would prefer a killable sleep to an uninterruptible one. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 7 +++++-- mm/huge_memory.c | 4 ++-- mm/migrate.c | 4 ++-- 4 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 00288ed24698..71b36b275e4d 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -681,8 +681,7 @@ static inline int wait_on_page_locked_killable(struct page *page) return wait_on_page_bit_killable(compound_head(page), PG_locked); } -extern void put_and_wait_on_page_locked(struct page *page); - +int put_and_wait_on_page_locked(struct page *page, int state); void wait_on_page_writeback(struct page *page); extern void end_page_writeback(struct page *page); void wait_for_stable_page(struct page *page); diff --git a/mm/filemap.c b/mm/filemap.c index d9636ccf87ff..709774a60379 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1358,20 +1358,23 @@ static int wait_on_page_locked_async(struct page *page, /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. + * @state: The sleep state (TASK_KILLABLE, TASK_UNINTERRUPTIBLE, etc). * * The caller should hold a reference on @page. They expect the page to * become unlocked relatively soon, but do not wish to hold up migration * (for example) by holding the reference while waiting for the page to * come unlocked. After this function returns, the caller should not * dereference @page. + * + * Return: 0 if the page was unlocked or -EINTR if interrupted by a signal. */ -void put_and_wait_on_page_locked(struct page *page) +int put_and_wait_on_page_locked(struct page *page, int state) { wait_queue_head_t *q; page = compound_head(page); q = page_waitqueue(page); - wait_on_page_bit_common(q, page, PG_locked, TASK_UNINTERRUPTIBLE, DROP); + return wait_on_page_bit_common(q, page, PG_locked, state, DROP); } /** diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 616102ba3682..ac114d265950 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1432,7 +1432,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } @@ -1468,7 +1468,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } diff --git a/mm/migrate.c b/mm/migrate.c index 39663dfbc273..a50bbb0e029b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -335,7 +335,7 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, if (!get_page_unless_zero(page)) goto out; pte_unmap_unlock(ptep, ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; out: pte_unmap_unlock(ptep, ptl); @@ -369,7 +369,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd) if (!get_page_unless_zero(page)) goto unlock; spin_unlock(ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; unlock: spin_unlock(ptl); From patchwork Mon Nov 2 18:42:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874921 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36B8392C for ; Mon, 2 Nov 2020 18:43:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E53B22275 for ; Mon, 2 Nov 2020 18:43:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="GoDl4tFn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726449AbgKBSn2 (ORCPT ); Mon, 2 Nov 2020 13:43:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725789AbgKBSnY (ORCPT ); Mon, 2 Nov 2020 13:43:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DA23C0617A6 for ; Mon, 2 Nov 2020 10:43:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=kmWLmtiAbwte+d4YCV97fdpcfgVqkIryTOcgsMsHcjA=; b=GoDl4tFnNwEk47mYiaP6cFFmeZ BhMQXb7ti+z7lcWzFR2ilm4YipyVQrqh1HdA/vq1WUKyy39tL+HvpacXBxN7zStFRikgTtOsGPhxs /pzLjcQvgwAQNqF/ia4oi4K/xFcEHd9mET7Q8B11DYz8YTBrfQI1P2YNv0coaL7UWa0WjKyoy9lgL 5pCG8tMpCGSxus4VwUzZ4FiyZCx2MDlVI4sfQHJ/gY++Bsr4tmOdc2bUky2x8hNgLcpIFRPchxED3 yaq8iz4pMw9vyFmQpg9mvSZ0N79nN7yZ1PTQ7BgR04mbV85DA3NLeRgRNkHUCOIkkYhFXtsu3UhEL T6EPQe5g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenX-0006mH-4J; Mon, 02 Nov 2020 18:43:19 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 04/17] mm/filemap: Support readpage splitting a page Date: Mon, 2 Nov 2020 18:42:59 +0000 Message-Id: <20201102184312.25926-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org For page splitting to succeed, the thread asking to split the page has to be the only one with a reference to the page. Calling wait_on_page_locked() while holding a reference to the page will effectively prevent this from happening with sufficient threads waiting on the same page. Use put_and_wait_on_page_locked() to sleep without holding a reference to the page, then retry the page lookup after the page is unlocked. Since we now get the page lock a little earlier in filemap_update_page(), we can eliminate a number of duplicate checks. The original intent (commit ebded02788b5 ("avoid unnecessary calls to lock_page when waiting for IO to complete during a read")) behind getting the page lock later was to avoid re-locking the page after it has been brought uptodate by another thread. We still avoid that because we go through the normal lookup path again after the winning thread has brought the page uptodate. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 76 ++++++++++++++++------------------------------------ 1 file changed, 23 insertions(+), 53 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 709774a60379..550e023abb52 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1347,14 +1347,6 @@ static int __wait_on_page_locked_async(struct page *page, return ret; } -static int wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait) -{ - if (!PageLocked(page)) - return 0; - return __wait_on_page_locked_async(compound_head(page), wait, false); -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -2281,64 +2273,42 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, struct inode *inode = mapping->host; int error; - /* - * See comment in do_read_cache_page on why - * wait_on_page_locked is used to avoid unnecessarily - * serialisations and why it's safe. - */ if (iocb->ki_flags & IOCB_WAITQ) { - error = wait_on_page_locked_async(page, - iocb->ki_waitq); + error = lock_page_async(page, iocb->ki_waitq); + if (error) { + put_page(page); + return ERR_PTR(error); + } } else { - error = wait_on_page_locked_killable(page); - } - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); + if (!trylock_page(page)) { + put_and_wait_on_page_locked(page, TASK_KILLABLE); + return NULL; + } } - if (PageUptodate(page)) - return page; + if (!page->mapping) + goto truncated; + if (PageUptodate(page)) + goto uptodate; if (inode->i_blkbits == PAGE_SHIFT || !mapping->a_ops->is_partially_uptodate) - goto page_not_up_to_date; + goto readpage; /* pipes can't handle partially uptodate pages */ if (unlikely(iov_iter_is_pipe(iter))) - goto page_not_up_to_date; - if (!trylock_page(page)) - goto page_not_up_to_date; - /* Did it get truncated before we got the lock? */ - if (!page->mapping) - goto page_not_up_to_date_locked; + goto readpage; if (!mapping->a_ops->is_partially_uptodate(page, - pos & ~PAGE_MASK, count)) - goto page_not_up_to_date_locked; + pos & (thp_size(page) - 1), count)) + goto readpage; +uptodate: unlock_page(page); return page; -page_not_up_to_date: - /* Get exclusive access to the page ... */ - error = lock_page_for_iocb(iocb, page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - -page_not_up_to_date_locked: - /* Did it get truncated before we got the lock? */ - if (!page->mapping) { - unlock_page(page); - put_page(page); - return NULL; - } - - /* Did somebody else fill it already? */ - if (PageUptodate(page)) { - unlock_page(page); - return page; - } - +readpage: return filemap_read_page(iocb, filp, mapping, page); +truncated: + unlock_page(page); + put_page(page); + return NULL; } static struct page *filemap_create_page(struct kiocb *iocb, From patchwork Mon Nov 2 18:43:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874907 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43B1214B4 for ; Mon, 2 Nov 2020 18:43:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1E5F922268 for ; Mon, 2 Nov 2020 18:43:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="epiHgR/H" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726236AbgKBSnY (ORCPT ); Mon, 2 Nov 2020 13:43:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725789AbgKBSnW (ORCPT ); Mon, 2 Nov 2020 13:43:22 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3165AC0617A6 for ; Mon, 2 Nov 2020 10:43:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5XmDfsPihqOhYwqzIA37D9dhvkyh36adUczq+IZmr9Y=; b=epiHgR/H4u8NML86wxA83kdrTT rpPliY6SDYRT/j7chybGWvl0zPcb4MM+5TGVgt/10yYyPQ2glczZgMRzSMtkw9wfAhn3jPf9z+aUf KDtyN0aH6Y4nWmEVFxWGUV8en0H4LPiIL7acEfbK8d9HRXXQeik4MayNR4U0VFipL2vtDFhvpVA3r geBZgQ/GuLIvdHOQgWM8V9byLGLMgQX2gdZJe7ZdqfiLxuo29KgFu4+rGhdQTcbvBVP9hG6mGGddu iRua/eagnO9lQM4GWOf83TT5NB/phWcU5LvtiiEtH/G03+5mWzrYF1gReqQTFfl9WAQJFy/W/17Ql Py6cAhJA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenX-0006mY-Qi; Mon, 02 Nov 2020 18:43:19 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 05/17] mm/filemap: Inline __wait_on_page_locked_async into caller Date: Mon, 2 Nov 2020 18:43:00 +0000 Message-Id: <20201102184312.25926-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The previous patch removed wait_on_page_locked_async(), so inline __wait_on_page_locked_async into __lock_page_async(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 53 ++++++++++++++++++++++------------------------------ 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 550e023abb52..7a4101ceb106 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1317,36 +1317,6 @@ int wait_on_page_bit_killable(struct page *page, int bit_nr) } EXPORT_SYMBOL(wait_on_page_bit_killable); -static int __wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait, bool set) -{ - struct wait_queue_head *q = page_waitqueue(page); - int ret = 0; - - wait->page = page; - wait->bit_nr = PG_locked; - - spin_lock_irq(&q->lock); - __add_wait_queue_entry_tail(q, &wait->wait); - SetPageWaiters(page); - if (set) - ret = !trylock_page(page); - else - ret = PageLocked(page); - /* - * If we were succesful now, we know we're still on the - * waitqueue as we're still under the lock. This means it's - * safe to remove and return success, we know the callback - * isn't going to trigger. - */ - if (!ret) - __remove_wait_queue(q, &wait->wait); - else - ret = -EIOCBQUEUED; - spin_unlock_irq(&q->lock); - return ret; -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -1514,7 +1484,28 @@ EXPORT_SYMBOL_GPL(__lock_page_killable); int __lock_page_async(struct page *page, struct wait_page_queue *wait) { - return __wait_on_page_locked_async(page, wait, true); + struct wait_queue_head *q = page_waitqueue(page); + int ret = 0; + + wait->page = page; + wait->bit_nr = PG_locked; + + spin_lock_irq(&q->lock); + __add_wait_queue_entry_tail(q, &wait->wait); + SetPageWaiters(page); + ret = !trylock_page(page); + /* + * If we were succesful now, we know we're still on the + * waitqueue as we're still under the lock. This means it's + * safe to remove and return success, we know the callback + * isn't going to trigger. + */ + if (!ret) + __remove_wait_queue(q, &wait->wait); + else + ret = -EIOCBQUEUED; + spin_unlock_irq(&q->lock); + return ret; } /* From patchwork Mon Nov 2 18:43:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874915 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E33814B4 for ; Mon, 2 Nov 2020 18:43:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2752C222BA for ; Mon, 2 Nov 2020 18:43:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="V+0iGqDS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726433AbgKBSn0 (ORCPT ); Mon, 2 Nov 2020 13:43:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725809AbgKBSnW (ORCPT ); Mon, 2 Nov 2020 13:43:22 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E1B9C061A04 for ; Mon, 2 Nov 2020 10:43:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=XAUjZnLVwPJBDOtFZ9diYzCnxqB06ylLLNQKCZj7LGU=; b=V+0iGqDSPAjmEncASUjeFL3L2/ /4ljNA5OjXuJ2R1aYmjW2s3hJkF6vbvH4iPF+fGsceHR8t1TTJBDZZyRyPU44NWMaYbfK3c9+YAh6 P4PlJn6rC954as35y2dck6LIAB0HVglupPMVGjKBvA3KEeA7x+UJNsnCwPBrk4k2JFqBCBTjmttUn W0eWP8QQAen5qkF0Kr9IVjEhr+BV0+u06pIJw8jmdHlmA+kd/NULcS1GbzYg17KiwWy48xilRd12u yhi3W6jAmXEVtt/TB51/i0KdX/bx/u7nn6hgzlnf+dI4ThjeSR+Dt6hJlvSxr4uwVxRuGDcwKkyec 4OGnKsqg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenY-0006mk-LI; Mon, 02 Nov 2020 18:43:20 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 06/17] mm/filemap: Don't call ->readpage if IOCB_WAITQ is set Date: Mon, 2 Nov 2020 18:43:01 +0000 Message-Id: <20201102184312.25926-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The readpage operation can block in many (most?) filesystems, so we should punt to a work queue instead of calling it. This was the last caller of lock_page_async(), so remove it. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 7a4101ceb106..5bafd2dc830c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2152,16 +2152,6 @@ static void shrink_readahead_size_eio(struct file_ra_state *ra) ra->ra_pages /= 4; } -static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) -{ - if (iocb->ki_flags & IOCB_WAITQ) - return lock_page_async(page, iocb->ki_waitq); - else if (iocb->ki_flags & IOCB_NOWAIT) - return trylock_page(page) ? 0 : -EAGAIN; - else - return lock_page_killable(page); -} - static unsigned mapping_get_read_thps(struct address_space *mapping, pgoff_t index, unsigned int nr_pages, struct page **pages) { @@ -2210,7 +2200,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT)) { + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); put_page(page); return ERR_PTR(-EAGAIN); @@ -2231,7 +2221,7 @@ static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, } if (!PageUptodate(page)) { - error = lock_page_for_iocb(iocb, page); + error = lock_page_killable(page); if (unlikely(error)) { put_page(page); return ERR_PTR(error); From patchwork Mon Nov 2 18:43:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874917 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AFB3892C for ; Mon, 2 Nov 2020 18:43:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B7C2223AC for ; Mon, 2 Nov 2020 18:43:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="uOzVdl1m" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726429AbgKBSnZ (ORCPT ); Mon, 2 Nov 2020 13:43:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725824AbgKBSnX (ORCPT ); Mon, 2 Nov 2020 13:43:23 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55186C061A47 for ; Mon, 2 Nov 2020 10:43:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ZB3YT97pDvW/WHAiwrWuiAsryKmJBMD/HsAlKXX0sB0=; b=uOzVdl1m8+3mVmg/FNORt6+tfC wVbuVmGB+QLRKIZKCZ0Zs/1U+o8HhrYVopNoRix5pJF8GLAGv6a7pIvHuQK0veIV5QQDXe8KWibXM JiUCKx/q9N3vntkl8pkJ9UVF/xeFCo/usSnfm92Dleu8IJGL63rOvQFyY2ntibufaQ1rBBoW8kF+z l2qD5Ldh1fmU1UU9kXb4qK7VGAI8F0vmhIjPxMtEH/E3oqNryKO5CkMvLCHDV4ZqYarDki5HoKxTw lI5qXgfAs0AcNbEVYx7CJxNJugSz0l4Sof7Ez/5SOB8p9mwGe0dtO5dqgHUerq3XYHZ0jfcd1g7/j LMWR3Upw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenZ-0006mx-6u; Mon, 02 Nov 2020 18:43:21 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 07/17] mm/filemap: Change filemap_read_page calling conventions Date: Mon, 2 Nov 2020 18:43:02 +0000 Message-Id: <20201102184312.25926-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Make this function more generic by passing the file instead of the iocb. Check in the callers whether we should call readpage or not. Also make it return an errno / 0 / AOP_TRUNCATED_PAGE, and make calling put_page() the caller's responsibility. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 89 +++++++++++++++++++++++++--------------------------- 1 file changed, 42 insertions(+), 47 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 5bafd2dc830c..171da5c592fa 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2194,56 +2194,38 @@ static unsigned mapping_get_read_thps(struct address_space *mapping, return ret; } -static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, - struct address_space *mapping, struct page *page) +static int filemap_read_page(struct file *file, struct address_space *mapping, + struct page *page) { - struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); - } - /* - * A previous I/O error may have been due to temporary - * failures, eg. multipath errors. - * PG_error will be set again if readpage fails. + * A previous I/O error may have been due to temporary failures, + * eg. multipath errors. PG_error will be set again if readpage + * fails. */ ClearPageError(page); /* Start the actual read. The read will unlock the page. */ - error = mapping->a_ops->readpage(filp, page); - - if (unlikely(error)) { - put_page(page); - return error != AOP_TRUNCATED_PAGE ? ERR_PTR(error) : NULL; - } + error = mapping->a_ops->readpage(file, page); + if (error) + return error; + if (PageUptodate(page)) + return 0; + error = lock_page_killable(page); + if (error) + return error; if (!PageUptodate(page)) { - error = lock_page_killable(page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* - * invalidate_mapping_pages got it - */ - unlock_page(page); - put_page(page); - return NULL; - } - unlock_page(page); - shrink_readahead_size_eio(ra); - put_page(page); - return ERR_PTR(-EIO); + if (page->mapping == NULL) { + /* page truncated */ + error = AOP_TRUNCATED_PAGE; + } else { + shrink_readahead_size_eio(&file->f_ra); + error = -EIO; } - unlock_page(page); } - - return page; + unlock_page(page); + return error; } static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, @@ -2285,7 +2267,18 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return page; readpage: - return filemap_read_page(iocb, filp, mapping, page); + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { + unlock_page(page); + put_page(page); + return ERR_PTR(-EAGAIN); + } + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); truncated: unlock_page(page); put_page(page); @@ -2301,7 +2294,7 @@ static struct page *filemap_create_page(struct kiocb *iocb, struct page *page; int error; - if (iocb->ki_flags & IOCB_NOIO) + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) return ERR_PTR(-EAGAIN); page = page_cache_alloc(mapping); @@ -2310,12 +2303,14 @@ static struct page *filemap_create_page(struct kiocb *iocb, error = add_to_page_cache_lru(page, mapping, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (error) { - put_page(page); - return error != -EEXIST ? ERR_PTR(error) : NULL; - } - - return filemap_read_page(iocb, filp, mapping, page); + if (!error) + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, From patchwork Mon Nov 2 18:43:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874911 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 510F814B4 for ; Mon, 2 Nov 2020 18:43:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C0AD2225E for ; Mon, 2 Nov 2020 18:43:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DCaCsr2S" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726406AbgKBSnZ (ORCPT ); Mon, 2 Nov 2020 13:43:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726212AbgKBSnY (ORCPT ); Mon, 2 Nov 2020 13:43:24 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33781C061A49 for ; Mon, 2 Nov 2020 10:43:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=73msYa311R11WAkHAc3cSVAXeaQOWUerwPhWbLhOqVE=; b=DCaCsr2SioJGpGQUS75coYTkcw Y/6SR4iAb+jkb4MUbGIwu34RdotOOrFJF1qe4BfRZFdx0PAAgsoq8kUR6zEoI6DA2PlBhm4DI5z1B yM+cFtXZJsWF6VC0rmq0yOCewwamZMJZJI38bKB30+evYQWty+SVd5Lhq8r9rJMIDkg80rdRxj6hx w3NUzDOrWRJ3RptaiI3JFB7tk/ReLEsEHSV5qBwQJT/bTbw4G76eRiYfvLNigrh2Q0CthVui19zEx 2KgL/d82nyEnDIOKApXYd5YJCkIy9ZWVHg6w1vwX/BsjDNNq1Y2+HFTkpbESF6+2K59vYPcTyAjsL 7BfvRwpw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenZ-0006n8-S2; Mon, 02 Nov 2020 18:43:21 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 08/17] mm/filemap: Change filemap_create_page arguments Date: Mon, 2 Nov 2020 18:43:03 +0000 Message-Id: <20201102184312.25926-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org filemap_create_page() doesn't need the iocb or the iter. It just needs the file and the index. Move the iocb flag checks to the caller. We can skip checking GFP_NOIO as that's checked a few lines earlier. There's no need to fall through to filemap_update_page() -- if filemap_create_page couldn't update it, filemap_update_page will not be able to. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 171da5c592fa..5527b239771c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2285,26 +2285,20 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, return NULL; } -static struct page *filemap_create_page(struct kiocb *iocb, - struct iov_iter *iter) +static struct page *filemap_create_page(struct file *file, + struct address_space *mapping, pgoff_t index) { - struct file *filp = iocb->ki_filp; - struct address_space *mapping = filp->f_mapping; - pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; struct page *page; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) - return ERR_PTR(-EAGAIN); - page = page_cache_alloc(mapping); if (!page) return ERR_PTR(-ENOMEM); error = add_to_page_cache_lru(page, mapping, index, - mapping_gfp_constraint(mapping, GFP_KERNEL)); + mapping_gfp_constraint(mapping, GFP_KERNEL)); if (!error) - error = filemap_read_page(iocb->ki_filp, mapping, page); + error = filemap_read_page(file, mapping, page); if (!error) return page; put_page(page); @@ -2338,13 +2332,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); nr_got = mapping_get_read_thps(mapping, index, nr, pages); - if (nr_got) - goto got_pages; - - pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pages[0]); - if (!IS_ERR_OR_NULL(pages[0])) - nr_got = 1; + if (!nr_got) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + pages[0] = filemap_create_page(filp, mapping, + iocb->ki_pos >> PAGE_SHIFT); + if (!pages[0]) + goto find_page; + if (IS_ERR(pages[0])) + return PTR_ERR(pages[0]); + return 1; + } got_pages: if (nr_got > 0) { struct page *page = pages[nr_got - 1]; From patchwork Mon Nov 2 18:43:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874919 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C28792C for ; Mon, 2 Nov 2020 18:43:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E3B822275 for ; Mon, 2 Nov 2020 18:43:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="fYPMVz4b" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726157AbgKBSn1 (ORCPT ); Mon, 2 Nov 2020 13:43:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725927AbgKBSnZ (ORCPT ); Mon, 2 Nov 2020 13:43:25 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D65FCC061A04 for ; Mon, 2 Nov 2020 10:43:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=u754MWx1ZhWf2ZlBtQmrFs0jUJq0Q4zppwEcbORl0e8=; b=fYPMVz4bLScctBBqKzp5R+MwN6 LIwEN1f9+csABwucbT8VvHLzLEUgLUzcXgUc4Nb99cqhW0F6Dc9vSwIPm3cBYC5f2df0nfUzMYl1+ ld4y2ZaHpI5akIVAV3bLZPinXg06isSM65tCNndjynPwdy/lUOh/M4pN4xxa4bJeh0RPzsw2z4Hek TKB9FjCq/eA3Pnor+q1Jk9tsqAODWJZUbFtC1hc5RLMBpr4z37/CECDl9OszRPVwG03EGy+I+asHv pwTQ3X+rsBySeH9/TB+PhKtQ7JYiaqwweqJx8Ddzw0I2agT1Sn9WV15aSTC3qakjYfxt7i83TTriL bxbx6ofg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZena-0006nI-Az; Mon, 02 Nov 2020 18:43:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 09/17] mm/filemap: Convert filemap_update_page to return an errno Date: Mon, 2 Nov 2020 18:43:04 +0000 Message-Id: <20201102184312.25926-10-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Use AOP_TRUNCATED_PAGE to indicate that no error occurred, but the page we looked up is no longer valid. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 43 +++++++++++++++++++------------------------ 1 file changed, 19 insertions(+), 24 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 5527b239771c..ebb14fdec0cc 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2228,24 +2228,21 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } -static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, - struct iov_iter *iter, struct page *page, loff_t pos, - loff_t count) +static int filemap_update_page(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page, loff_t pos, loff_t count) { - struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; int error; if (iocb->ki_flags & IOCB_WAITQ) { error = lock_page_async(page, iocb->ki_waitq); - if (error) { - put_page(page); - return ERR_PTR(error); - } + if (error) + goto error; } else { if (!trylock_page(page)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); - return NULL; + return AOP_TRUNCATED_PAGE; } } @@ -2264,25 +2261,24 @@ static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, goto readpage; uptodate: unlock_page(page); - return page; + return 0; readpage: if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); + error = -EAGAIN; + } else { + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return 0; } - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; +error: put_page(page); - if (error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + return error; truncated: unlock_page(page); put_page(page); - return NULL; + return AOP_TRUNCATED_PAGE; } static struct page *filemap_create_page(struct file *file, @@ -2371,20 +2367,19 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, goto err; } - page = filemap_update_page(iocb, filp, iter, page, + err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); - if (IS_ERR_OR_NULL(page)) { + if (err) nr_got--; - err = PTR_ERR_OR_ZERO(page); - } } } err: if (likely(nr_got)) return nr_got; - if (err) + if (err < 0) return err; + err = 0; /* * No pages and no error means we raced and should retry: */ From patchwork Mon Nov 2 18:43:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874923 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBDFA1130 for ; Mon, 2 Nov 2020 18:43:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9C6ED2231B for ; Mon, 2 Nov 2020 18:43:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="fVkezwnW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726470AbgKBSn3 (ORCPT ); Mon, 2 Nov 2020 13:43:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725927AbgKBSn2 (ORCPT ); Mon, 2 Nov 2020 13:43:28 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36FE3C0617A6 for ; Mon, 2 Nov 2020 10:43:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=C03/K9nEYgbV/HZHtBn1uLOW4rAEhplcjX1jRdnEKuo=; b=fVkezwnWaBuKXNmMRZUtxfEoWl +WfNm+ZKGxwivLIvpQWZqUqnSZb6avA0hoXOXnr5z2pMgjIQc/GBedx7PCh0UsE1OdA562v7iIfEs KmcLfDniuSqLqOKBH+xd1OZtCQdAGNq+GYQtSVQR9V0UCViL+lfRkglaFlhrTdBmykHkZDFgrBcFQ F0n5UTwsgzkpC9Wk7LGaR4VxjYyQTi0oRUevScTO5vP9eJVZ0L6Snk0Tp9C/FNrFXT4KO+o03Orkg Gb+sCY4N9xDs5ruJ05pgozFhtTQ2yXLXiy60R59hZGnLGhm2cRZllw/fVFq53EfIDJE0WJ+D3/tyt YohTeBhQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenb-0006nZ-2W; Mon, 02 Nov 2020 18:43:23 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 10/17] mm/filemap: Move the iocb checks into filemap_update_page Date: Mon, 2 Nov 2020 18:43:05 +0000 Message-Id: <20201102184312.25926-11-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We don't need to give up when a special request sees a !Uptodate page. We may be able to satisfy the read from a partially-uptodate page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig --- mm/filemap.c | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ebb14fdec0cc..41b90243f4ee 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2230,17 +2230,21 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, - struct page *page, loff_t pos, loff_t count) + struct page *page, loff_t pos, loff_t count, bool first) { struct inode *inode = mapping->host; - int error; + int error = -EAGAIN; - if (iocb->ki_flags & IOCB_WAITQ) { - error = lock_page_async(page, iocb->ki_waitq); - if (error) + if (!trylock_page(page)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_NOIO)) goto error; - } else { - if (!trylock_page(page)) { + if (iocb->ki_flags & IOCB_WAITQ) { + if (!first) + goto error; + error = __lock_page_async(page, iocb->ki_waitq); + if (error) + goto error; + } else { put_and_wait_on_page_locked(page, TASK_KILLABLE); return AOP_TRUNCATED_PAGE; } @@ -2359,16 +2363,8 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && nr_got > 1)) { - put_page(page); - nr_got--; - err = -EAGAIN; - goto err; - } - err = filemap_update_page(iocb, mapping, iter, page, - pg_pos, pg_count); + pg_pos, pg_count, nr_got == 1); if (err) nr_got--; } From patchwork Mon Nov 2 18:43:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874925 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E06E31130 for ; Mon, 2 Nov 2020 18:43:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC7B52231B for ; Mon, 2 Nov 2020 18:43:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hmQCHLP7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726493AbgKBSna (ORCPT ); Mon, 2 Nov 2020 13:43:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726481AbgKBSn3 (ORCPT ); Mon, 2 Nov 2020 13:43:29 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94068C0617A6 for ; Mon, 2 Nov 2020 10:43:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=QtHfrKT5xabd5uqtY4b+yuJs0ExznL5pKGwHYW5XBsc=; b=hmQCHLP7INmBJOug4JStsn+6kQ qxiTWM4ps8wndza/iHprPu2ivWO6Ls5OPS5QECPXXC47rAuETRmAD+B2pK+L/jBy9BI6q6g08yEMY nestnIFVn3wQ++JSHDD+Ur5/Plqdp9s92ekaXawKR1HaRkdPwnOlFTySrHxdckqq51jlhTmoCgJKj bKJscoW4lwPWfg8wIcNVWD6mjQYEl0wj9z/mRv0RR6bDrTNOBqUvjAUHGyHzy+WYFKOy2W5gFISpD tt96oWq6uUZ8VSDw0m/taZah8hm9IePKQWhq6IwvbPuxn8o48K3qCHzXM1IUvtTJU+tRS0KDnTWgf K07Agg8Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenc-0006o3-5L; Mon, 02 Nov 2020 18:43:24 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 11/17] mm/filemap: Add filemap_range_uptodate Date: Mon, 2 Nov 2020 18:43:06 +0000 Message-Id: <20201102184312.25926-12-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Move the complicated condition and the calculations out of filemap_update_page() into its own function. Reviewed-by: Kent Overstreet Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 49 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 41b90243f4ee..81b569d818a3 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2228,11 +2228,39 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, return error; } +static bool filemap_range_uptodate(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page) +{ + loff_t pos; + int count; + + if (PageUptodate(page)) + return true; + /* pipes can't handle partially uptodate pages */ + if (iov_iter_is_pipe(iter)) + return false; + if (!mapping->a_ops->is_partially_uptodate) + return false; + if (mapping->host->i_blkbits >= (PAGE_SHIFT + thp_order(page))) + return false; + + pos = (loff_t) page->index << PAGE_SHIFT; + if (pos > iocb->ki_pos) { + count = iocb->ki_pos + iter->count - pos; + pos = 0; + } else { + count = iter->count; + pos = iocb->ki_pos & (thp_size(page) - 1); + } + + return mapping->a_ops->is_partially_uptodate(page, pos, count); +} + static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, struct page *page, loff_t pos, loff_t count, bool first) { - struct inode *inode = mapping->host; int error = -EAGAIN; if (!trylock_page(page)) { @@ -2252,22 +2280,11 @@ static int filemap_update_page(struct kiocb *iocb, if (!page->mapping) goto truncated; - if (PageUptodate(page)) - goto uptodate; - if (inode->i_blkbits == PAGE_SHIFT || - !mapping->a_ops->is_partially_uptodate) - goto readpage; - /* pipes can't handle partially uptodate pages */ - if (unlikely(iov_iter_is_pipe(iter))) - goto readpage; - if (!mapping->a_ops->is_partially_uptodate(page, - pos & (thp_size(page) - 1), count)) - goto readpage; -uptodate: - unlock_page(page); - return 0; + if (filemap_range_uptodate(iocb, mapping, iter, page)) { + unlock_page(page); + return 0; + } -readpage: if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); error = -EAGAIN; From patchwork Mon Nov 2 18:43:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874927 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F96414B4 for ; Mon, 2 Nov 2020 18:43:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 73657223AC for ; Mon, 2 Nov 2020 18:43:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="D+L+8if9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726501AbgKBSnb (ORCPT ); Mon, 2 Nov 2020 13:43:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725927AbgKBSna (ORCPT ); Mon, 2 Nov 2020 13:43:30 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F975C0617A6 for ; Mon, 2 Nov 2020 10:43:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TAKe5ShkzeUGiDLF7zD5jcjV0+ZkrFCKna03nopA5xQ=; b=D+L+8if9YPagHFPeQxIHYE/NWT zhtdCfDJJ9uLTM3OqOOla4ifoGgx/xBbtf9DvT9Gb+POXHrqv7PNIrfeN3pEO8AGVe+VWLwbXCBIK Hi90/+evC8MbwyrriKtp3VX/EBvB9NU8ke2aUlx3u8cAO9NX+8ZjmsNs9jhaATDcfrjHwKW2B5oSm m2wWqoyX0dWSPfgnK8+tRtAjvXB5AMYiZYMjPdI5llUohxz9M29f2TZ9aH27GM469q2jjWTR2ys5j Kxf4KJotfSfWOIYynYbQbLd/w9BZdjjIqOssHpb9oKEad6MkB680Sqv8YlV2ZMs5+EvTdNj28QMZu yaOJxSuw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenf-0006oz-Ml; Mon, 02 Nov 2020 18:43:27 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 12/17] mm/filemap: Split filemap_readahead out of filemap_get_pages Date: Mon, 2 Nov 2020 18:43:07 +0000 Message-Id: <20201102184312.25926-13-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This simplifies the error handling. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 81b569d818a3..7c6380a3a871 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2324,6 +2324,17 @@ static struct page *filemap_create_page(struct file *file, return ERR_PTR(error); } +static int filemap_readahead(struct kiocb *iocb, struct file *file, + struct address_space *mapping, struct page *page, + pgoff_t last_index) +{ + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_async_readahead(mapping, ra, filp, page, + pg_index, last_index - pg_index); + return 0; +} + static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct page **pages, unsigned int nr) { @@ -2368,23 +2379,14 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, (loff_t) pg_index << PAGE_SHIFT); loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; - if (PageReadahead(page)) { - if (iocb->ki_flags & IOCB_NOIO) { - put_page(page); - nr_got--; - err = -EAGAIN; - goto err; - } - page_cache_async_readahead(mapping, ra, filp, page, - pg_index, last_index - pg_index); - } - - if (!PageUptodate(page)) { + if (PageReadahead(page)) + err = filemap_readahead(iocb, filp, mapping, page, + last_index); + if (!err && !PageUptodate(page)) err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count, nr_got == 1); - if (err) - nr_got--; - } + if (err) + nr_got--; } err: From patchwork Mon Nov 2 18:43:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874929 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2A1492C for ; Mon, 2 Nov 2020 18:43:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 785CB22280 for ; Mon, 2 Nov 2020 18:43:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lKrZrlFO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726503AbgKBSnc (ORCPT ); Mon, 2 Nov 2020 13:43:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726481AbgKBSnb (ORCPT ); Mon, 2 Nov 2020 13:43:31 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33E27C0617A6 for ; Mon, 2 Nov 2020 10:43:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=MhaGM2ZGeDwxTv2rpzeXcoU4mMvsEnyn3vSYd9SK9Ns=; b=lKrZrlFOyo5i2BYg2CkDhPoyeo oVOeraG38fSUovw9klaZ8NccZMgZobBZMIzUdOSTnVK14ujz5UB/ZjFD+SBVz/G8DAx0h08TWEORI reHs/gyimctMbFymljtC0GX6MwlkC86AqENyTJADE4WDl3IAGLyBjwp+vfwDZRaKKnEsDGkOZtFB3 A3tJrlOctRFfoc+y4cBWMkvFi5pkL9iM896cTUNzy3gXIr56V04JFJ9YzhC+pOvVxiHGE3rCwZNOC lT2M3WyVJrkaoLa1lU9EEeLFfWWpF9Xf/M4C2HgTnIGqqrSRdMrb86M8zmdz9MzaDtc+bmFyeAwA9 9eaJaLSw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZeng-0006pb-Ji; Mon, 02 Nov 2020 18:43:28 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 13/17] mm/filemap: Remove parameters from filemap_update_page() Date: Mon, 2 Nov 2020 18:43:08 +0000 Message-Id: <20201102184312.25926-14-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The 'pos' and 'count' params are no longer used in filemap_update_page() Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 7c6380a3a871..0ae8305ccb97 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2259,7 +2259,7 @@ static bool filemap_range_uptodate(struct kiocb *iocb, static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, - struct page *page, loff_t pos, loff_t count, bool first) + struct page *page, bool first) { int error = -EAGAIN; @@ -2330,8 +2330,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, { if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; - page_cache_async_readahead(mapping, ra, filp, page, - pg_index, last_index - pg_index); + page_cache_async_readahead(mapping, &file->f_ra, file, page, + page->index, last_index - page->index); return 0; } @@ -2374,22 +2374,17 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, got_pages: if (nr_got > 0) { struct page *page = pages[nr_got - 1]; - pgoff_t pg_index = page->index; - loff_t pg_pos = max(iocb->ki_pos, - (loff_t) pg_index << PAGE_SHIFT); - loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) err = filemap_readahead(iocb, filp, mapping, page, last_index); if (!err && !PageUptodate(page)) err = filemap_update_page(iocb, mapping, iter, page, - pg_pos, pg_count, nr_got == 1); + nr_got == 1); if (err) nr_got--; } -err: if (likely(nr_got)) return nr_got; if (err < 0) From patchwork Mon Nov 2 18:43:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874937 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A62191130 for ; Mon, 2 Nov 2020 18:43:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E666223BF for ; Mon, 2 Nov 2020 18:43:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="azWrD4yj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726536AbgKBSni (ORCPT ); Mon, 2 Nov 2020 13:43:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726512AbgKBSni (ORCPT ); Mon, 2 Nov 2020 13:43:38 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77087C0617A6 for ; Mon, 2 Nov 2020 10:43:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=/CeGi8DzEW3bR+G9BPlfR5yNaaVvUzZl8ckDh3b7Jv8=; b=azWrD4yjPzI3HrnmonDw0uzTIV oQKZEKnN7T8C5HXUrNxupDQm4PkJ8qPZmmcICSQhSuKKJmjM9A/i7hJKf59FzaCnI6FhpDCXFldw4 J1XqH3F7NYUl7J10TZMRSDfwMaD+lXel7jfyYnHG8v3kcB1iyqPhrxBaBfN/yLWIyupuLp55XlJeF YTYQiKu5YZLNL9BzANbrv2ZIILnwArOlU7ZtYIFIaKbjNdoBgmG2C68yohQ1EgGfDgKK0opeYgCIV HF+AQsvW24tnWccvMGPt6YsPp5MY6H3NQuboZCvFA8jprtusI9DoLrCBvee+2ue3HHI8xZj6fN0El MCJRRH0A==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenh-0006q0-Fd; Mon, 02 Nov 2020 18:43:29 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 14/17] mm/filemap: Restructure filemap_get_pages Date: Mon, 2 Nov 2020 18:43:09 +0000 Message-Id: <20201102184312.25926-15-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Avoid a goto, and by the time we get to calling filemap_update_page(), we definitely have at least one page. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 0ae8305ccb97..f16b1eb03bca 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2343,6 +2343,7 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; + struct page *page; int nr_got, err = 0; nr = min_t(unsigned long, last_index - index, nr); @@ -2351,15 +2352,13 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, return -EINTR; nr_got = mapping_get_read_thps(mapping, index, nr, pages); - if (nr_got) - goto got_pages; - - if (iocb->ki_flags & IOCB_NOIO) - return -EAGAIN; - - page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - - nr_got = mapping_get_read_thps(mapping, index, nr, pages); + if (!nr_got) { + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_sync_readahead(mapping, ra, filp, index, + last_index - index); + nr_got = mapping_get_read_thps(mapping, index, nr, pages); + } if (!nr_got) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; @@ -2371,20 +2370,16 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, return PTR_ERR(pages[0]); return 1; } -got_pages: - if (nr_got > 0) { - struct page *page = pages[nr_got - 1]; - if (PageReadahead(page)) - err = filemap_readahead(iocb, filp, mapping, page, - last_index); - if (!err && !PageUptodate(page)) - err = filemap_update_page(iocb, mapping, iter, page, - nr_got == 1); - if (err) - nr_got--; - } + page = pages[nr_got - 1]; + if (PageReadahead(page)) + err = filemap_readahead(iocb, filp, mapping, page, last_index); + if (!err && !PageUptodate(page)) + err = filemap_update_page(iocb, mapping, iter, page, + nr_got == 1); + if (err) + nr_got--; if (likely(nr_got)) return nr_got; if (err < 0) From patchwork Mon Nov 2 18:43:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874931 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE9B91130 for ; Mon, 2 Nov 2020 18:43:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B41D22275 for ; Mon, 2 Nov 2020 18:43:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="M4UFc6+R" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726509AbgKBSnd (ORCPT ); Mon, 2 Nov 2020 13:43:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726481AbgKBSnd (ORCPT ); Mon, 2 Nov 2020 13:43:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3856C0617A6 for ; Mon, 2 Nov 2020 10:43:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=n+REdW1xoOea4wiyJWdr2S+YGLxyipm2rcZL5gYfVC8=; b=M4UFc6+RWWZ4YxauVIwZf4ObKR enRZBjteDCpQ4K18wTrH8TqUhXHXIshDmOAhQQKHybNLpQ79LULFq8sK3uh1UqZvKVisJYO8r4B4r RtBh0KahVKVU9QHVx9ZljhoQVh+xvSmiJ9/Z6+zlY6yafvqIX0bkWCSxqjIEzseUvis/ZXzuWYsUb NcIIVnVENX+/XrN+rAKSghdqIqSb0Ey0s17L3I46HApZsv/eWGkd7CJnYByUfvjbuGNBSvd3xycbT gBPY5ua8luCMoHo5DCBrJMdmPHg34P1guWSctRyNaZTbQYxXpazwFtkff1QGNYKL7wjIgy0Bo7uHt NgZJHZZw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZeni-0006qS-Gy; Mon, 02 Nov 2020 18:43:30 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , hch@lst.de, kent.overstreet@gmail.com Subject: [PATCH 15/17] mm/filemap: Don't relock the page after calling readpage Date: Mon, 2 Nov 2020 18:43:10 +0000 Message-Id: <20201102184312.25926-16-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We don't need to get the page lock again; we just need to wait for the I/O to finish, so use wait_on_page_locked_killable() like the other callers of ->readpage. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f16b1eb03bca..f2de97d51441 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2209,23 +2209,16 @@ static int filemap_read_page(struct file *file, struct address_space *mapping, error = mapping->a_ops->readpage(file, page); if (error) return error; - if (PageUptodate(page)) - return 0; - error = lock_page_killable(page); + error = wait_on_page_locked_killable(page); if (error) return error; - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* page truncated */ - error = AOP_TRUNCATED_PAGE; - } else { - shrink_readahead_size_eio(&file->f_ra); - error = -EIO; - } - } - unlock_page(page); - return error; + if (PageUptodate(page)) + return 0; + if (!page->mapping) /* page truncated */ + return AOP_TRUNCATED_PAGE; + shrink_readahead_size_eio(&file->f_ra); + return -EIO; } static bool filemap_range_uptodate(struct kiocb *iocb, From patchwork Mon Nov 2 18:43:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874933 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D93714B4 for ; Mon, 2 Nov 2020 18:43:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED548223BD for ; Mon, 2 Nov 2020 18:43:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="S/X44xGW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726513AbgKBSne (ORCPT ); Mon, 2 Nov 2020 13:43:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726512AbgKBSnd (ORCPT ); Mon, 2 Nov 2020 13:43:33 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B48A7C0617A6 for ; Mon, 2 Nov 2020 10:43:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=XQwtmp2e0naOceRdgszG0xu3cKjAYZmucqQPn2T+foA=; b=S/X44xGWBV+tRIrQZPUPhzyWKq mAJTKiQz3liZD8VTDlBC8mZagubiLSmU7fogl+7xh7V2Igvb4pPK2DsoAj0XqXvglWgJlL9PKYm+J YBXNOu+HXgHGdy/Z0yUJ7LDMQZcEKYqPkOrgKTXzpl/4AnEld//ocOnBd0+1ngCmG7xKRJOn15k09 CtGbbYpEoy2hD8iUfkVAElrKgdake421f0+iM9oetDVi3Hp0sjw95oU7CMoAqJTeBur6Q76eiRURD qF4/3o36DqwYhSwSeeRNX43l/Qcv8RfpXTEbTGw8NRdUTRX6kewW3EBz5VTYsrNYgkAH6Wr2NnWc2 lERoQlKw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenj-0006qt-B6; Mon, 02 Nov 2020 18:43:31 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , kent.overstreet@gmail.com, Matthew Wilcox Subject: [PATCH 16/17] mm/filemap: rename generic_file_buffered_read to filemap_read Date: Mon, 2 Nov 2020 18:43:11 +0000 Message-Id: <20201102184312.25926-17-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Christoph Hellwig Rename generic_file_buffered_read to match the naming of filemap_fault, also update the written parameter to a more descriptive name and improve the kerneldoc comment. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- fs/btrfs/file.c | 2 +- include/linux/fs.h | 4 ++-- mm/filemap.c | 35 ++++++++++++++++------------------- 3 files changed, 19 insertions(+), 22 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 87355a38a654..1a4913e1fd12 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -3633,7 +3633,7 @@ static ssize_t btrfs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } - return generic_file_buffered_read(iocb, to, ret); + return filemap_read(iocb, to, ret); } const struct file_operations btrfs_file_operations = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 4ccc879ae845..413e327fa1c6 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2946,8 +2946,8 @@ extern ssize_t generic_write_checks(struct kiocb *, struct iov_iter *); extern int generic_write_check_limits(struct file *file, loff_t pos, loff_t *count); extern int generic_file_rw_checks(struct file *file_in, struct file *file_out); -extern ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *to, ssize_t already_read); +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *to, + ssize_t already_read); extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *); extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *); extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); diff --git a/mm/filemap.c b/mm/filemap.c index f2de97d51441..92bb308029c3 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2385,23 +2385,20 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, } /** - * generic_file_buffered_read - generic file read routine - * @iocb: the iocb to read - * @iter: data destination - * @written: already copied + * filemap_read - Read data from the page cache. + * @iocb: The iocb to read. + * @iter: Destination for the data. + * @already_read: Number of bytes already read by the caller. * - * This is a generic file read routine, and uses the - * mapping->a_ops->readpage() function for the actual low-level stuff. + * Copies data from the page cache. If the data is not currently present, + * uses the readahead and readpage address_space operations to fetch it. * - * This is really ugly. But the goto's actually try to clarify some - * of the logic when it comes to error handling etc. - * - * Return: - * * total number of bytes copied, including those the were already @written - * * negative error code if nothing was copied + * Return: Total number of bytes copied, including those already read by + * the caller. If an error happens before any bytes are copied, returns + * a negative error number. */ -ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *iter, ssize_t written) +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, + ssize_t already_read) { struct file *filp = iocb->ki_filp; struct file_ra_state *ra = &filp->f_ra; @@ -2435,7 +2432,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, * can no longer safely return -EIOCBQUEUED. Hence mark * an async read NOWAIT at that point. */ - if ((iocb->ki_flags & IOCB_WAITQ) && written) + if ((iocb->ki_flags & IOCB_WAITQ) && already_read) iocb->ki_flags |= IOCB_NOWAIT; i = 0; @@ -2501,7 +2498,7 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, copied = copy_page_to_iter(page, offset, bytes, iter); - written += copied; + already_read += copied; iocb->ki_pos += copied; ra->prev_pos = iocb->ki_pos; @@ -2520,9 +2517,9 @@ ssize_t generic_file_buffered_read(struct kiocb *iocb, if (pages != pages_onstack) kfree(pages); - return written ? written : error; + return already_read ? already_read : error; } -EXPORT_SYMBOL_GPL(generic_file_buffered_read); +EXPORT_SYMBOL_GPL(filemap_read); /** * generic_file_read_iter - generic filesystem read routine @@ -2596,7 +2593,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) goto out; } - retval = generic_file_buffered_read(iocb, iter, retval); + retval = filemap_read(iocb, iter, retval); out: return retval; } From patchwork Mon Nov 2 18:43:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11874935 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2B3D92C for ; Mon, 2 Nov 2020 18:43:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BCE84223C7 for ; Mon, 2 Nov 2020 18:43:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hzZgkNvB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726525AbgKBSng (ORCPT ); Mon, 2 Nov 2020 13:43:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726512AbgKBSne (ORCPT ); Mon, 2 Nov 2020 13:43:34 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5DA5FC0617A6 for ; Mon, 2 Nov 2020 10:43:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=yGyAJgqIZQghEE7oZN0iVtA30eQFmzuryM2GY/sRLrw=; b=hzZgkNvB2NLMB2hjkr63NI85rq 6Wyh3yajoMs5mWxCOEx2ZXWUsmWlGpWSwJIWzJ7pBIXp3YLddBe7c4NGDdcCluTQ1hQtMycoW33Lo mmUx1XfJnruzkutworpYU8T6FTnevOCz7vWvOM4JU97YYLScdlsWWwHqayMS/hhfkEYhGq8y1uX8J aS4CA+6E1YiKbZDCD29bHo0ohcbqXyx5Lh3yf9FVgzUrOE1dsFkEC/ki2ll9N1pk+BEUUGP35crQl piv4IY6qlf/lYa7Rnr8rBBsRkFgu6LIR2jMoSP0jnAx4TMQEyZK4Y50726gKhtNB1eXvQckl0Zj+S sAt0LXvw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kZenj-0006rN-Vk; Mon, 02 Nov 2020 18:43:32 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: Christoph Hellwig , kent.overstreet@gmail.com, Matthew Wilcox Subject: [PATCH 17/17] mm: simplify generic_file_read_iter Date: Mon, 2 Nov 2020 18:43:12 +0000 Message-Id: <20201102184312.25926-18-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201102184312.25926-1-willy@infradead.org> References: <20201102184312.25926-1-willy@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org From: Christoph Hellwig Avoid the pointless goto out just for returning retval. Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet --- mm/filemap.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 92bb308029c3..4676ce15a9a5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2549,7 +2549,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) ssize_t retval = 0; if (!count) - goto out; /* skip atime */ + return 0; /* skip atime */ if (iocb->ki_flags & IOCB_DIRECT) { struct file *file = iocb->ki_filp; @@ -2567,7 +2567,7 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) iocb->ki_pos, iocb->ki_pos + count - 1); if (retval < 0) - goto out; + return retval; } file_accessed(file); @@ -2590,12 +2590,10 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) */ if (retval < 0 || !count || iocb->ki_pos >= size || IS_DAX(inode)) - goto out; + return retval; } - retval = filemap_read(iocb, iter, retval); -out: - return retval; + return filemap_read(iocb, iter, retval); } EXPORT_SYMBOL(generic_file_read_iter);