From patchwork Sun Jul 11 15:09:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12369209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1874FC07E96 for ; Sun, 11 Jul 2021 15:10:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C1B7B60FF4 for ; Sun, 11 Jul 2021 15:10:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C1B7B60FF4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DAD196B0085; Sun, 11 Jul 2021 11:10:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D83116B0087; Sun, 11 Jul 2021 11:10:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C256A6B0089; Sun, 11 Jul 2021 11:10:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0001.hostedemail.com [216.40.44.1]) by kanga.kvack.org (Postfix) with ESMTP id 92A7E6B0085 for ; Sun, 11 Jul 2021 11:10:03 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BC8C62197C for ; Sun, 11 Jul 2021 15:10:02 +0000 (UTC) X-FDA: 78350642244.28.CC4574E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id D5396700177D; Sun, 11 Jul 2021 15:10:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=8GduWunndRkt/zvmZxhBRXnTi7c25FVkVOoI9I1j1WI=; b=Xubiupq9tpL0QEZKdWBAtzT8HN AGDBM4xgt21nYmNICM1Nqm1oifADesE2s7aWm2aaHXv6JJ8YBDaK/J65rz9yZUplhcMabKl68AjjT ESAyLIohfKYP66suy7Lnbbd/rLbwiVRLgXuzO5AIjcOnN6ETDOX74DD2w4Tr6oSQt3Fqut7t9EPFy xyDAfC7+Ck/FH/WXEmn8X2tfSxbrglOlyE4tUcr79adLmnXJXGG3d12Tn8RyDB4DuzoBOZhDeg2HJ rkzysDuE+nom2VwDk0cPWyyrDsX0b/L+NzhQc6egsqyFMyvR2a/Df0IJapU6g+XF6yCP9a7CPK/bI p6zyYPRA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1m2b5P-00GMAF-Rm; Sun, 11 Jul 2021 15:09:42 +0000 From: "Matthew Wilcox (Oracle)" To: Jens Axboe Cc: "Matthew Wilcox (Oracle)" , io-uring@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-aio@kvack.org Subject: [PATCH 1/2] mm/readahead: Add gfp_flags to ractl Date: Sun, 11 Jul 2021 16:09:26 +0100 Message-Id: <20210711150927.3898403-2-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210711150927.3898403-1-willy@infradead.org> References: <20210711150927.3898403-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Xubiupq9; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D5396700177D X-Stat-Signature: yjqjqf8ecf5kqmyw4b6jcsmdptaggkdh X-HE-Tag: 1626016201-630666 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is currently possible for an I/O request that specifies IOCB_NOWAIT to sleep waiting for I/O to complete in order to allocate pages for readahead. In order to fix that, we need the caller to be able to specify the GFP flags to use for memory allocation in the rest of the readahead path. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 3 +++ mm/readahead.c | 16 ++++++++-------- 2 files changed, 11 insertions(+), 8 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index ed02aa522263..00abb2b2f158 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -833,11 +833,13 @@ static inline int add_to_page_cache(struct page *page, * May be NULL if invoked internally by the filesystem. * @mapping: Readahead this filesystem object. * @ra: File readahead state. May be NULL. + * @gfp_flags: Memory allocation flags to use. */ struct readahead_control { struct file *file; struct address_space *mapping; struct file_ra_state *ra; + gfp_t gfp_flags; /* private: use the readahead_* accessors instead */ pgoff_t _index; unsigned int _nr_pages; @@ -849,6 +851,7 @@ struct readahead_control { .file = f, \ .mapping = m, \ .ra = r, \ + .gfp_flags = readahead_gfp_mask(m), \ ._index = i, \ } diff --git a/mm/readahead.c b/mm/readahead.c index d589f147f4c2..58937d1fe3f7 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -177,7 +177,6 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, struct address_space *mapping = ractl->mapping; unsigned long index = readahead_index(ractl); LIST_HEAD(page_pool); - gfp_t gfp_mask = readahead_gfp_mask(mapping); unsigned long i; /* @@ -212,14 +211,14 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - page = __page_cache_alloc(gfp_mask); + page = __page_cache_alloc(ractl->gfp_flags); if (!page) break; if (mapping->a_ops->readpages) { page->index = index + i; list_add(&page->lru, &page_pool); } else if (add_to_page_cache_lru(page, mapping, index + i, - gfp_mask) < 0) { + ractl->gfp_flags) < 0) { put_page(page); read_pages(ractl, &page_pool, true); i = ractl->_index + ractl->_nr_pages - index - 1; @@ -663,7 +662,6 @@ void readahead_expand(struct readahead_control *ractl, struct address_space *mapping = ractl->mapping; struct file_ra_state *ra = ractl->ra; pgoff_t new_index, new_nr_pages; - gfp_t gfp_mask = readahead_gfp_mask(mapping); new_index = new_start / PAGE_SIZE; @@ -675,10 +673,11 @@ void readahead_expand(struct readahead_control *ractl, if (page && !xa_is_value(page)) return; /* Page apparently present */ - page = __page_cache_alloc(gfp_mask); + page = __page_cache_alloc(ractl->gfp_flags); if (!page) return; - if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) { + if (add_to_page_cache_lru(page, mapping, index, + ractl->gfp_flags) < 0) { put_page(page); return; } @@ -698,10 +697,11 @@ void readahead_expand(struct readahead_control *ractl, if (page && !xa_is_value(page)) return; /* Page apparently present */ - page = __page_cache_alloc(gfp_mask); + page = __page_cache_alloc(ractl->gfp_flags); if (!page) return; - if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) { + if (add_to_page_cache_lru(page, mapping, index, + ractl->gfp_flags) < 0) { put_page(page); return; } From patchwork Sun Jul 11 15:09:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12369211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29B3EC07E96 for ; Sun, 11 Jul 2021 15:10:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A42536115A for ; Sun, 11 Jul 2021 15:10:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A42536115A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B65196B008A; Sun, 11 Jul 2021 11:10:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B11986B0093; Sun, 11 Jul 2021 11:10:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98AD06B008C; Sun, 11 Jul 2021 11:10:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 6CD646B008A for ; Sun, 11 Jul 2021 11:10:24 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 90D9A204A1 for ; Sun, 11 Jul 2021 15:10:23 +0000 (UTC) X-FDA: 78350643126.30.0C94EE5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP id 0B9874002085; Sun, 11 Jul 2021 15:10:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=855AUMnreo1HN0f6x88VXU6uE3rYZIQf/z+jfpkStV8=; b=oVIZqa/cHJZKqXZURbCu4vT/My 3BjLnO4TkSqsT9gs6iXSZs8ludP41KEhRV5E8IsEhnliSztIFL3IpFtnoF5KLgWKKfArUBzubosmE rQN6FVFqnPpMJ0Gugb4rkaj3RbStL+V1/TJ0mQApGauvjMrAquxmHATfaodvJgGXrWNZvJ9WiAsdn q2N2nAdWb5ocKFZWL3YUhMKvAdc9wJCjCmKdubSAdWA1/JuiyBzqKMroaxPxefQW/vsiKTLWHbjKB otXyUSBk3qGPfV5NMqMT5cszCvX9w4DzRRsJW7zTEOdB7GtuzQhCXOwbyw9cnCcs6WUzArF6RaBjt ZU1FuerA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1m2b5b-00GMAM-0h; Sun, 11 Jul 2021 15:09:54 +0000 From: "Matthew Wilcox (Oracle)" To: Jens Axboe Cc: "Matthew Wilcox (Oracle)" , io-uring@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-aio@kvack.org Subject: [PATCH 2/2] mm/filemap: Prevent waiting for memory for NOWAIT reads Date: Sun, 11 Jul 2021 16:09:27 +0100 Message-Id: <20210711150927.3898403-3-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210711150927.3898403-1-willy@infradead.org> References: <20210711150927.3898403-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0B9874002085 X-Stat-Signature: yjfn5na4ooo3y9fgmo8zjuue4dufuhw6 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="oVIZqa/c"; dmarc=none; spf=none (imf18.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-HE-Tag: 1626016222-731311 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Readahead memory allocations won't block for much today, as they're already marked as NOFS and NORETRY, but they can still sleep, and they shouldn't if the read is marked as IOCB_NOWAIT. Clearing the DIRECT_RECLAIM flag will prevent sleeping. Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d1458ecf2f51..2be27b686518 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2435,21 +2435,27 @@ static int filemap_create_page(struct file *file, static int filemap_readahead(struct kiocb *iocb, struct file *file, struct address_space *mapping, struct page *page, - pgoff_t last_index) + pgoff_t index, pgoff_t last_index) { + DEFINE_READAHEAD(ractl, file, &file->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; - page_cache_async_readahead(mapping, &file->f_ra, file, page, - page->index, last_index - page->index); + if (iocb->ki_flags & IOCB_NOWAIT) + ractl.gfp_flags &= ~__GFP_DIRECT_RECLAIM; + + if (page) + page_cache_async_ra(&ractl, page, last_index - index); + else + page_cache_sync_ra(&ractl, last_index - index); return 0; } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct pagevec *pvec) { - struct file *filp = iocb->ki_filp; - struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; + struct file *file = iocb->ki_filp; + struct address_space *mapping = file->f_mapping; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct page *page; @@ -2462,16 +2468,16 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, filemap_get_read_batch(mapping, index, last_index, pvec); if (!pagevec_count(pvec)) { - if (iocb->ki_flags & IOCB_NOIO) - return -EAGAIN; - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + err = filemap_readahead(iocb, file, mapping, NULL, index, + last_index); + if (err) + return err; filemap_get_read_batch(mapping, index, last_index, pvec); } if (!pagevec_count(pvec)) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; - err = filemap_create_page(filp, mapping, + err = filemap_create_page(file, mapping, iocb->ki_pos >> PAGE_SHIFT, pvec); if (err == AOP_TRUNCATED_PAGE) goto retry; @@ -2480,7 +2486,8 @@ static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, page = pvec->pages[pagevec_count(pvec) - 1]; if (PageReadahead(page)) { - err = filemap_readahead(iocb, filp, mapping, page, last_index); + err = filemap_readahead(iocb, file, mapping, page, page->index, + last_index); if (err) goto err; }