From patchwork Thu Nov 12 21:26:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901641 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58B101391 for ; Thu, 12 Nov 2020 21:26:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F01C222241 for ; Thu, 12 Nov 2020 21:26:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="NXmVbUS7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F01C222241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DA37E6B005C; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CF2256B006C; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A38CE6B005C; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 6D9E66B005D for ; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0CF1E180AD801 for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) X-FDA: 77477051106.04.hate34_5013d0027309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id C89248008873 for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-Spam-Summary: 1,0,0,81ea7c5f9d3244c9,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2553:2559:2562:2895:2898:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:5007:6119:6261:6653:7875:10004:11026:11658:11914:12043:12050:12296:12297:12438:12555:12895:13894:14181:14394:14721:21080:21324:21451:21627:21990:30034:30054:30055:30064:30070:30090:30091,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yrioqckm78jhtp8poas6bigmgycocrmybhfpswr9hids499hgwja9abhaphg3.cujy6ujfu4jaxxi6tknyb5rq9t4gu4u8s7wygmwicp489tp3f1t7rhiijzd7k33.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:780,LUA_SUMMARY:none X-HE-Tag: hate34_5013d0027309 X-Filterd-Recvd-Size: 3888 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=hHTBK4BYPzK5sxPr2WzuBNxW83FIC+F2fBOyFQf+0ws=; b=NXmVbUS7kRm/236rRSQxvtyc2U ZZHRvCkKVgQjP/gVzR/xk/YlN5IHiVxBG0u+WMV5uEX2gjlgQ73lS/hgOm1Ha4Rs/6O+uRjHC+CLq fmShrL0C1mKrjpFXW77suorHT+7Kkl9bUcF9BCkdjqUr/ycSP8+qRzJzauuWslNUTX8GZy5VLNPTz piHbKC5WBneycVJN40+v62WUOGcRQpiMw1Hm4QK/mFyUHiIer9DD6yFQ4bhzkYe7eopd7TtwLC/Xl bzf26tY7NklsxMfzn85KbmdBIIqRMLBdnc7hgs3MojXF5HgJuESEIXUEA/hBj/XGmXroZuuIB8EFO Joy1oLtQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK79-0007G4-NF; Thu, 12 Nov 2020 21:26:43 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 01/16] mm: Make pagecache tagged lookups return only head pages Date: Thu, 12 Nov 2020 21:26:26 +0000 Message-Id: <20201112212641.27837-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pagecache tags are used for dirty page writeback. Since dirtiness is tracked on a per-THP basis, we only want to return the head page rather than each subpage of a tagged page. All the filesystems which use huge pages today are in-memory, so there are no tagged huge pages today. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- mm/filemap.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 249cf489f5df..bb6f2ae5a68c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2066,7 +2066,7 @@ unsigned find_get_pages_contig(struct address_space *mapping, pgoff_t index, EXPORT_SYMBOL(find_get_pages_contig); /** - * find_get_pages_range_tag - find and return pages in given range matching @tag + * find_get_pages_range_tag - Find and return head pages matching @tag. * @mapping: the address_space to search * @index: the starting page index * @end: The final page index (inclusive) @@ -2074,8 +2074,9 @@ EXPORT_SYMBOL(find_get_pages_contig); * @nr_pages: the maximum number of pages * @pages: where the resulting pages are placed * - * Like find_get_pages, except we only return pages which are tagged with - * @tag. We update @index to index the next page for the traversal. + * Like find_get_pages(), except we only return head pages which are tagged + * with @tag. @index is updated to the index immediately after the last + * page we return, ready for the next iteration. * * Return: the number of pages which were found. */ @@ -2109,9 +2110,9 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, if (unlikely(page != xas_reload(&xas))) goto put_page; - pages[ret] = find_subpage(page, xas.xa_index); + pages[ret] = page; if (++ret == nr_pages) { - *index = xas.xa_index + 1; + *index = page->index + thp_nr_pages(page); goto out; } continue; From patchwork Thu Nov 12 21:26:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901665 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51AB3697 for ; Thu, 12 Nov 2020 21:27:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F27292224B for ; Thu, 12 Nov 2020 21:27:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qUppFtXX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F27292224B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 531FF6B0092; Thu, 12 Nov 2020 16:27:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 470906B0093; Thu, 12 Nov 2020 16:27:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 337BE6B0095; Thu, 12 Nov 2020 16:27:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id F2AFA6B0092 for ; Thu, 12 Nov 2020 16:27:16 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8137733CD for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) X-FDA: 77477052072.14.cord50_2d1657727309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 624A918229818 for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) X-Spam-Summary: 1,0,0,8df47bd66ca45ced,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2553:2559:2562:2691:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3871:3872:3874:5007:6261:6653:7996:10004:11026:11658:11914:12043:12114:12294:12296:12297:12438:12555:12895:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21324:21451:21627:21987:21990:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04y8ncjsqyghzryfqygsbqetmq771opfzs3974d5tn51qmtagjx5i5djc6ctebc.riy5iib3qquofj4pcsnrjku4axrf66g6za896bgknqtenw6zxpzxwsfdh9rdqk6.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:69,LUA_SUMMARY:none X-HE-Tag: cord50_2d1657727309 X-Filterd-Recvd-Size: 3428 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:27:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=8GEXd5Ze60X4OYh6xIrRSvS9O7QAElyk7+6nVS10XRE=; b=qUppFtXXjLIsyxqhhhTIba7+5w Wz92hzrfGGRzwRAeLh3ftv3GoXSbkbKixG9PlAzzo8RHr+48BjLLPV2cqKTpcGHjDRHJWI9KPAxOR SEdNM7FB/9MtDEW9c+VO65VkfocuiZ0iSWIUJ/32eluFYoBwX8SFu3eQsHx0chO8aLITlbt6aWmzY x661/+vIxCVUckb2/uLhoVv9etRyhuz9g0UzgSd7V6kUcGvBsK8gkYhjVYc4JRzyPFxC2uTOWWUxl iu4NbxDXobKwDbrv50FSUPdAKCNR3RT9A3iONUCYwDmMkloUYojo+/iT09hLQfH01+oANSz2eHsZn oWIQ82UA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7A-0007G7-1Z; Thu, 12 Nov 2020 21:26:44 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 02/16] mm/shmem: Use pagevec_lookup in shmem_unlock_mapping Date: Thu, 12 Nov 2020 21:26:27 +0000 Message-Id: <20201112212641.27837-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The comment shows that the reason for using find_get_entries() is now stale; find_get_pages() will not return 0 if it hits a consecutive run of swap entries, and I don't believe it has since 2011. pagevec_lookup() is a simpler function to use than find_get_pages(), so use it instead. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- mm/shmem.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 028f4596fc16..8076c171731c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -842,7 +842,6 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma) void shmem_unlock_mapping(struct address_space *mapping) { struct pagevec pvec; - pgoff_t indices[PAGEVEC_SIZE]; pgoff_t index = 0; pagevec_init(&pvec); @@ -850,16 +849,8 @@ void shmem_unlock_mapping(struct address_space *mapping) * Minor point, but we might as well stop if someone else SHM_LOCKs it. */ while (!mapping_unevictable(mapping)) { - /* - * Avoid pagevec_lookup(): find_get_pages() returns 0 as if it - * has finished, if it hits a row of PAGEVEC_SIZE swap entries. - */ - pvec.nr = find_get_entries(mapping, index, - PAGEVEC_SIZE, pvec.pages, indices); - if (!pvec.nr) + if (!pagevec_lookup(&pvec, mapping, &index)) break; - index = indices[pvec.nr - 1] + 1; - pagevec_remove_exceptionals(&pvec); check_move_unevictable_pages(&pvec); pagevec_release(&pvec); cond_resched(); From patchwork Thu Nov 12 21:26:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901683 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 001C41391 for ; Thu, 12 Nov 2020 21:27:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8EEFA22252 for ; Thu, 12 Nov 2020 21:27:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="a9Dldi9i" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8EEFA22252 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C10DF6B007D; Thu, 12 Nov 2020 16:26:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B9B436B007E; Thu, 12 Nov 2020 16:26:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A679E6B0080; Thu, 12 Nov 2020 16:26:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 65C336B007D for ; Thu, 12 Nov 2020 16:26:58 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 17229180AD804 for ; Thu, 12 Nov 2020 21:26:58 +0000 (UTC) X-FDA: 77477051316.07.heat73_4a0062427309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id F129E1803F9A1 for ; Thu, 12 Nov 2020 21:26:57 +0000 (UTC) X-Spam-Summary: 1,0,0,5b9d079becc359d9,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1539:1711:1714:1730:1747:1777:1792:2282:2393:2559:2562:3138:3139:3140:3141:3142:3350:3865:3867:3868:3872:4321:5007:6261:6653:10004:11026:11658:11914:12043:12114:12297:12438:12555:12895:12986:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21324:21451:21627:21990:30054,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yf5eysxej95nu59ktaqzotkmbz7ocqoqi6abkencoxwrheooybxssguaz3xtp.xbncjo3nj4i3gyootdy4me4ntcxfjt7xryr1ca6ebg767zyypjqxadsckpynpze.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:68,LUA_SUMMARY:none X-HE-Tag: heat73_4a0062427309 X-Filterd-Recvd-Size: 2454 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=w06ZP8J1L1sNnrL6d7bu/K2RwiChw8umB6NkZXCjOJM=; b=a9Dldi9iEA7FsD03s2jU9jo1jw DCu12xlb5XFWmxWtgbqZO4LCcIUcnPn2cAeWN/YBHpH4yuWcCl3xgOxosVLAev/3+F+Oj2gZBcC2J +yWjYaqWuopWkvwOP7IPzWAznGyc1hWUmCXD1cIyIQ0plBCNNy3Mk1Rcj6F9GF0yQH+wiNOZTaFKE N2oR/gMEzTOCXt80wJZ1HPQyWkHSO1BrpcvUXkSGwHSQvsSUwXulC72JphaN3rocSWx3TGiT8UZlm 3gIQITyu4v+ifFvLr06G9cgYZ6Nrui1twyBnSOEFKIFiBHLqiELYE06q6lzb3KIJVPzCybJYqrWo3 Wv0E/QQA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7A-0007GF-Qj; Thu, 12 Nov 2020 21:26:44 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 03/16] mm/swap: Optimise get_shadow_from_swap_cache Date: Thu, 12 Nov 2020 21:26:28 +0000 Message-Id: <20201112212641.27837-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's no need to get a reference to the page, just load the entry and see if it's a shadow entry. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/swap_state.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index cf7b322a9abc..d2161154d873 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -113,11 +113,9 @@ void *get_shadow_from_swap_cache(swp_entry_t entry) pgoff_t idx = swp_offset(entry); struct page *page; - page = find_get_entry(address_space, idx); + page = xa_load(&address_space->i_pages, idx); if (xa_is_value(page)) return page; - if (page) - put_page(page); return NULL; } From patchwork Thu Nov 12 21:26:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901659 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 610A7697 for ; Thu, 12 Nov 2020 21:27:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D74E22258 for ; Thu, 12 Nov 2020 21:27:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="U30C+OFT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D74E22258 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 66D9B6B0081; Thu, 12 Nov 2020 16:27:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5D0956B0083; Thu, 12 Nov 2020 16:27:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 420A86B0085; Thu, 12 Nov 2020 16:27:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 089EA6B0081 for ; Thu, 12 Nov 2020 16:27:03 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id AB7E71EF1 for ; Thu, 12 Nov 2020 21:27:03 +0000 (UTC) X-FDA: 77477051526.11.peace65_2c0ea1a27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 8B9D7180F8B86 for ; Thu, 12 Nov 2020 21:27:03 +0000 (UTC) X-Spam-Summary: 1,0,0,47e08f9b1e2617d1,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1606:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4118:4250:4321:4605:5007:6261:6653:7875:7903:9010:9592:10004:10946:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12683:12895:12986:13146:13161:13229:13230:13894:14096:14110:14394:21063:21080:21324:21433:21451:21627:21990:30054:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yruste5pumux387a4aupz5pfo7hopfd6jew6zp6susi86uxami9mbezo4twga.xckz39qnggqzbgg3faut5m6t51dad1hxmdyewa15m39azszqrqyrcid9h68o3cq.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:68,LUA_SUMMARY:none X-HE-Tag: peace65_2c0ea1a27309 X-Filterd-Recvd-Size: 7193 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:27:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=HzBDjinxaIupRQuRgbw6uvZwV6m7KZ5Scf5MNsXtHDY=; b=U30C+OFTvsiV3WiiA64yPKrS2h 0L/nxwCXdOgcaaXy0QOtr0vExD0rIZdj13SubElhEIVymcC/X0I1lqHpr7YqsDTbQSfNJq12fQ+kb 5Im5l689oE7qlKyeNCr4+TDKh4lHN+49GU3Oc8X5WOZxgogY5eBwglqtR5AZRLNa0u1bzsfVG+IOe XLEwAvv2v58YK5Bb4sAhumPNmKarS1N3L0EfyVgFGOQJ0Epc73visw9eyLxFUhZ9AN8uNkG3EnlSd zcxBsEvIiXEZeenFlsM1SiiATMzH4v1o9iEhXIUxocoxgCIRKF1OoemThpn379JjDqQ/F9a7/qNgM 1166Y06g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7B-0007GN-90; Thu, 12 Nov 2020 21:26:45 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 04/16] mm: Add FGP_ENTRY Date: Thu, 12 Nov 2020 21:26:29 +0000 Message-Id: <20201112212641.27837-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The functionality of find_lock_entry() and find_get_entry() can be provided by pagecache_get_pages(), which lets us delete find_lock_entry() and make find_get_entry() static. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 1 + mm/filemap.c | 44 ++++++++--------------------------------- mm/internal.h | 3 --- mm/shmem.c | 3 ++- mm/swap_state.c | 3 ++- 5 files changed, 13 insertions(+), 41 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index edb990f3b930..55e1bff1c4b9 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -313,6 +313,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, #define FGP_NOWAIT 0x00000020 #define FGP_FOR_MMAP 0x00000040 #define FGP_HEAD 0x00000080 +#define FGP_ENTRY 0x00000100 struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset, int fgp_flags, gfp_t cache_gfp_mask); diff --git a/mm/filemap.c b/mm/filemap.c index bb6f2ae5a68c..01603f021740 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1662,7 +1662,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, } EXPORT_SYMBOL(page_cache_prev_miss); -/** +/* * find_get_entry - find and get a page cache entry * @mapping: the address_space to search * @index: The page cache index. @@ -1675,7 +1675,7 @@ EXPORT_SYMBOL(page_cache_prev_miss); * * Return: The head page or shadow entry, %NULL if nothing is found. */ -struct page *find_get_entry(struct address_space *mapping, pgoff_t index) +static struct page *find_get_entry(struct address_space *mapping, pgoff_t index) { XA_STATE(xas, &mapping->i_pages, index); struct page *page; @@ -1711,39 +1711,6 @@ struct page *find_get_entry(struct address_space *mapping, pgoff_t index) return page; } -/** - * find_lock_entry - Locate and lock a page cache entry. - * @mapping: The address_space to search. - * @index: The page cache index. - * - * Looks up the page at @mapping & @index. If there is a page in the - * cache, the head page is returned locked and with an increased refcount. - * - * If the slot holds a shadow entry of a previously evicted page, or a - * swap entry from shmem/tmpfs, it is returned. - * - * Context: May sleep. - * Return: The head page or shadow entry, %NULL if nothing is found. - */ -struct page *find_lock_entry(struct address_space *mapping, pgoff_t index) -{ - struct page *page; - -repeat: - page = find_get_entry(mapping, index); - if (page && !xa_is_value(page)) { - lock_page(page); - /* Has the page been truncated? */ - if (unlikely(page->mapping != mapping)) { - unlock_page(page); - put_page(page); - goto repeat; - } - VM_BUG_ON_PAGE(!thp_contains(page, index), page); - } - return page; -} - /** * pagecache_get_page - Find and get a reference to a page. * @mapping: The address_space to search. @@ -1759,6 +1726,8 @@ struct page *find_lock_entry(struct address_space *mapping, pgoff_t index) * * %FGP_LOCK - The page is returned locked. * * %FGP_HEAD - If the page is present and a THP, return the head page * rather than the exact page specified by the index. + * * %FGP_ENTRY - If there is a shadow / swap / DAX entry, return it + * instead of allocating a new page to replace it. * * %FGP_CREAT - If no page is present then a new page is allocated using * @gfp_mask and added to the page cache and the VM's LRU list. * The page is returned locked and with an increased refcount. @@ -1783,8 +1752,11 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, repeat: page = find_get_entry(mapping, index); - if (xa_is_value(page)) + if (xa_is_value(page)) { + if (fgp_flags & FGP_ENTRY) + return page; page = NULL; + } if (!page) goto no_page; diff --git a/mm/internal.h b/mm/internal.h index c43ccdddb0f6..93880d460e12 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -60,9 +60,6 @@ static inline void force_page_cache_readahead(struct address_space *mapping, force_page_cache_ra(&ractl, &file->f_ra, nr_to_read); } -struct page *find_get_entry(struct address_space *mapping, pgoff_t index); -struct page *find_lock_entry(struct address_space *mapping, pgoff_t index); - /** * page_evictable - test whether a page is evictable * @page: the page to test diff --git a/mm/shmem.c b/mm/shmem.c index 8076c171731c..c4feb05425f2 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1831,7 +1831,8 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, sbinfo = SHMEM_SB(inode->i_sb); charge_mm = vma ? vma->vm_mm : current->mm; - page = find_lock_entry(mapping, index); + page = pagecache_get_page(mapping, index, + FGP_ENTRY | FGP_HEAD | FGP_LOCK, 0); if (xa_is_value(page)) { error = shmem_swapin_page(inode, index, &page, sgp, gfp, vma, fault_type); diff --git a/mm/swap_state.c b/mm/swap_state.c index d2161154d873..7cb4433edaa2 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -427,7 +427,8 @@ struct page *find_get_incore_page(struct address_space *mapping, pgoff_t index) { swp_entry_t swp; struct swap_info_struct *si; - struct page *page = find_get_entry(mapping, index); + struct page *page = pagecache_get_page(mapping, index, + FGP_ENTRY | FGP_HEAD, 0); if (!page) return page; From patchwork Thu Nov 12 21:26:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901657 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CCA51391 for ; Thu, 12 Nov 2020 21:27:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E259822250 for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="nVlL3wYe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E259822250 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E393B6B0078; Thu, 12 Nov 2020 16:26:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DBBDB6B007D; Thu, 12 Nov 2020 16:26:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3A086B007E; Thu, 12 Nov 2020 16:26:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 8DA966B0078 for ; Thu, 12 Nov 2020 16:26:57 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 34D09180AD802 for ; Thu, 12 Nov 2020 21:26:57 +0000 (UTC) X-FDA: 77477051274.23.patch64_010e2c827309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 1636537606 for ; Thu, 12 Nov 2020 21:26:57 +0000 (UTC) X-Spam-Summary: 1,0,0,4c5ee9a2d881c814,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3867:3868:3871:3872:4321:5007:6261:6653:7875:10004:11026:11658:11914:12043:12296:12297:12438:12555:12895:12986:13069:13311:13357:13894:14096:14181:14384:14394:14721:21080:21324:21451:21627:21990:30054,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yroy8wqbhkh7dmqt4zriifcjadqoc3foekbihi3miem17e1x7qtkie11wx6sr.gxujz67rxowa54jb4yyx8qe5zypmjx8thq7g8t3531qww6gmzgpsxi47hrudkr4.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:68,LUA_SUMMARY:none X-HE-Tag: patch64_010e2c827309 X-Filterd-Recvd-Size: 3110 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ah+jL08Aq18bIod4sv8CxA+v//JQ2UBcaOuSobFGpKI=; b=nVlL3wYePUt8Dxl8hv5Jy858mR KfnUJXQAbqmuNRSOc2Wn1AL/zijiJKxUK4dm9ehGYXWUcetiIjm4gTqS2HEzgKnCl/GKm4IgpH9fZ +qth5eo3uHiuKLv478z3B786bFIlbAuYz3Ti3LW9VSMVQVjfTLXNto0EoGyK44+Z6RMsSVfN6WKVA WR+2spWkhua55IIm/WPaNh9lRiepVEyh7G2MogS0TpDH0ZBIAz6Jgd17cnQ48PffXW5zan7NwFwxV IEn0Aen2RuIRXxq91fFNu15ExLsXws8sRBG+lqhZKSyF68oa3djk02RWqPmNDMXurkLWSMDWvHQYY i3tm/QKA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7B-0007GR-IN; Thu, 12 Nov 2020 21:26:45 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 05/16] mm/filemap: Rename find_get_entry to mapping_get_entry Date: Thu, 12 Nov 2020 21:26:30 +0000 Message-Id: <20201112212641.27837-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: find_get_entry doesn't "find" anything. It returns the entry at a particular index. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/filemap.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 01603f021740..53073281f027 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1663,7 +1663,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, EXPORT_SYMBOL(page_cache_prev_miss); /* - * find_get_entry - find and get a page cache entry + * mapping_get_entry - Get a page cache entry. * @mapping: the address_space to search * @index: The page cache index. * @@ -1675,7 +1675,8 @@ EXPORT_SYMBOL(page_cache_prev_miss); * * Return: The head page or shadow entry, %NULL if nothing is found. */ -static struct page *find_get_entry(struct address_space *mapping, pgoff_t index) +static struct page *mapping_get_entry(struct address_space *mapping, + pgoff_t index) { XA_STATE(xas, &mapping->i_pages, index); struct page *page; @@ -1751,7 +1752,7 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, struct page *page; repeat: - page = find_get_entry(mapping, index); + page = mapping_get_entry(mapping, index); if (xa_is_value(page)) { if (fgp_flags & FGP_ENTRY) return page; From patchwork Thu Nov 12 21:26:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901671 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45A6615E6 for ; Thu, 12 Nov 2020 21:27:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E500322259 for ; Thu, 12 Nov 2020 21:27:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YKIYxP9A" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E500322259 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B9E5A6B0093; Thu, 12 Nov 2020 16:27:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD7B76B0095; Thu, 12 Nov 2020 16:27:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92B866B0096; Thu, 12 Nov 2020 16:27:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0087.hostedemail.com [216.40.44.87]) by kanga.kvack.org (Postfix) with ESMTP id 4EEA36B0095 for ; Thu, 12 Nov 2020 16:27:17 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id ED48C180AD801 for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) X-FDA: 77477052072.21.son06_340201927309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id CB265180442C0 for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) X-Spam-Summary: 1,0,0,39e11e636f5ecfb9,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4117:4250:4321:5007:6261:6653:7875:8957:9592:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12683:12895:12986:13161:13229:13894:14110:14181:14394:14721:21080:21324:21451:21627:21990:30012:30054:30055:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfj7siqtasdcm3agb57kx8bbny4oc1c4kxfpx6yf6w5sb6ahu66bun34quf3e.xpai679ftxzsoz4jw84rkq837ukyz34j8nnzu181mkks4rscbntizftqbem8s4d.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:70,LUA_SUMMARY:none X-HE-Tag: son06_340201927309 X-Filterd-Recvd-Size: 6478 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:27:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6XcAZ7OySglgZ9lx2gp5OHTvPB4mDq/mtcuSzOM0Egk=; b=YKIYxP9A/8LSWex39mvvG1NzQ0 Dca/uL4ewu2LDJxHpxWAl1Vc2lFbqodOz3eugS6t1VbAYlDfvsd6l17xAHTm0e4iY7ncvGG5v2saB 5PVjINN5fQIrk+1K7EYTisH7+y3CF9xr9jYNSj2WgS7xVW1/Sqrh2h2IyC80ujwCxNn23Bjk0dZGe 4m6+bSd7MC17OG1oZMtgTO2hKWvgCsLZtzC3RrWNllPPWjDlG2VmUcT+hlNNyeHuzB8HDY1Qt6cyQ 8FaswTkekpge4YtPyw0L0SflKpX9o9sItkpHQaYPvrpOrd3w7kSgfaSzJ+rWARP7NEEx2Iy6/+Y46 kUR9sNWA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7B-0007GV-Rg; Thu, 12 Nov 2020 21:26:45 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 06/16] mm/filemap: Add helper for finding pages Date: Thu, 12 Nov 2020 21:26:31 +0000 Message-Id: <20201112212641.27837-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a lot of common code in find_get_entries(), find_get_pages_range() and find_get_pages_range_tag(). Factor out find_get_entry() which simplifies all three functions. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- mm/filemap.c | 98 +++++++++++++++++++++++----------------------------- 1 file changed, 43 insertions(+), 55 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 53073281f027..e0d9cfa2ae90 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1829,6 +1829,43 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, } EXPORT_SYMBOL(pagecache_get_page); +static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, + xa_mark_t mark) +{ + struct page *page; + +retry: + if (mark == XA_PRESENT) + page = xas_find(xas, max); + else + page = xas_find_marked(xas, max, mark); + + if (xas_retry(xas, page)) + goto retry; + /* + * A shadow entry of a recently evicted page, a swap + * entry from shmem/tmpfs or a DAX entry. Return it + * without attempting to raise page count. + */ + if (!page || xa_is_value(page)) + return page; + + if (!page_cache_get_speculative(page)) + goto reset; + + /* Has the page moved or been split? */ + if (unlikely(page != xas_reload(xas))) { + put_page(page); + goto reset; + } + VM_BUG_ON_PAGE(!thp_contains(page, xas->xa_index), page); + + return page; +reset: + xas_reset(xas); + goto retry; +} + /** * find_get_entries - gang pagecache lookup * @mapping: The address_space to search @@ -1868,42 +1905,21 @@ unsigned find_get_entries(struct address_space *mapping, return 0; rcu_read_lock(); - xas_for_each(&xas, page, ULONG_MAX) { - if (xas_retry(&xas, page)) - continue; - /* - * A shadow entry of a recently evicted page, a swap - * entry from shmem/tmpfs or a DAX entry. Return it - * without attempting to raise page count. - */ - if (xa_is_value(page)) - goto export; - - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - + while ((page = find_get_entry(&xas, ULONG_MAX, XA_PRESENT))) { /* * Terminate early on finding a THP, to allow the caller to * handle it all at once; but continue if this is hugetlbfs. */ - if (PageTransHuge(page) && !PageHuge(page)) { + if (!xa_is_value(page) && PageTransHuge(page) && + !PageHuge(page)) { page = find_subpage(page, xas.xa_index); nr_entries = ret + 1; } -export: + indices[ret] = xas.xa_index; entries[ret] = page; if (++ret == nr_entries) break; - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } rcu_read_unlock(); return ret; @@ -1942,30 +1958,16 @@ unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, return 0; rcu_read_lock(); - xas_for_each(&xas, page, end) { - if (xas_retry(&xas, page)) - continue; + while ((page = find_get_entry(&xas, end, XA_PRESENT))) { /* Skip over shadow, swap and DAX entries */ if (xa_is_value(page)) continue; - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - pages[ret] = find_subpage(page, xas.xa_index); if (++ret == nr_pages) { *start = xas.xa_index + 1; goto out; } - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } /* @@ -2065,9 +2067,7 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, return 0; rcu_read_lock(); - xas_for_each_marked(&xas, page, end, tag) { - if (xas_retry(&xas, page)) - continue; + while ((page = find_get_entry(&xas, end, tag))) { /* * Shadow entries should never be tagged, but this iteration * is lockless so there is a window for page reclaim to evict @@ -2076,23 +2076,11 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, if (xa_is_value(page)) continue; - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - pages[ret] = page; if (++ret == nr_pages) { *index = page->index + thp_nr_pages(page); goto out; } - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } /* From patchwork Thu Nov 12 21:26:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901667 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 320DE697 for ; Thu, 12 Nov 2020 21:27:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CFDFE22258 for ; Thu, 12 Nov 2020 21:27:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="DjSUCPuP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CFDFE22258 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6DF8B6B0087; Thu, 12 Nov 2020 16:27:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 669956B0088; Thu, 12 Nov 2020 16:27:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BEBB6B0089; Thu, 12 Nov 2020 16:27:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 0DD2C6B0087 for ; Thu, 12 Nov 2020 16:27:10 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A1280181AEF10 for ; Thu, 12 Nov 2020 21:27:09 +0000 (UTC) X-FDA: 77477051778.28.group90_1e057ef27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 741336D62 for ; Thu, 12 Nov 2020 21:27:09 +0000 (UTC) X-Spam-Summary: 1,0,0,4f720bb89cc892e7,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4049:4118:4321:4605:5007:6261:6653:7875:8957:9036:9592:10004:11026:11473:11658:11914:12043:12114:12291:12296:12297:12438:12555:12683:12895:13894:14096:14110:14394:21080:21324:21451:21627:21987:21990:30003:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfthqiyj4x99etoextwz1qnduskoc1n8m6n1i7whjbsantp6kddrcxjc6fk47.w7zpc9j161edho45zjguzgs1ujextx6pzrtnfxpcgpme5ihdho373bhmmaoeakq.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:70,LUA_SUMMARY:none X-HE-Tag: group90_1e057ef27309 X-Filterd-Recvd-Size: 7874 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:27:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=adf3HaAGq3GYSYScBgfrGImWJpmiJMbnVTcYKwTbD/g=; b=DjSUCPuPtO63YmeC25BzRezanj p34lGa1kjErYiI7HZAZ+rMUDmRDGm4SY3OitbXv6N4ePjRWsjMLnBjXGozzU2qt3mLuquHsJCV/2c kH+/3QvTENN9IXRCnYS4/8ht0+ySf/RAC2eNAzdq8qFbTmu4r8UktfVtT0yBlDu2UUS+iDe3bnHMY NrRxt1Fv2ujNUNMXtTFdwWhrR+sIXpixOk9fMF8y4DdrD7OFh9Hry12iofIXK4D/2nqJuA/pU3Lyw kkaP0Gj8WVq+/Bo5eEjv/i2cx0XHJl766fqreFvaEIAAx+URtpabb2LOJQHc2MslcbbFB2+tpzmJo oi3NFk8g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7C-0007GZ-3g; Thu, 12 Nov 2020 21:26:46 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, William Kucharski Subject: [PATCH v4 07/16] mm/filemap: Add mapping_seek_hole_data Date: Thu, 12 Nov 2020 21:26:32 +0000 Message-Id: <20201112212641.27837-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rewrite shmem_seek_hole_data() and move it to filemap.c. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 75 +++++++++++++++++++++++++++++++++++++++++ mm/shmem.c | 74 +++------------------------------------- 3 files changed, 81 insertions(+), 70 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 55e1bff1c4b9..9f1f4ab9612a 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -759,6 +759,8 @@ extern void __delete_from_page_cache(struct page *page, void *shadow); int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask); void delete_from_page_cache_batch(struct address_space *mapping, struct pagevec *pvec); +loff_t mapping_seek_hole_data(struct address_space *, loff_t start, loff_t end, + int whence); /* * Like add_to_page_cache_locked, but used to add newly allocated pages: diff --git a/mm/filemap.c b/mm/filemap.c index e0d9cfa2ae90..ab7103eb7e11 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2586,6 +2586,81 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); +static inline bool page_seek_match(struct page *page, bool seek_data) +{ + if (xa_is_value(page) || PageUptodate(page)) + return seek_data; + return !seek_data; +} + +static inline +unsigned int seek_page_size(struct xa_state *xas, struct page *page) +{ + if (xa_is_value(page)) + return PAGE_SIZE << xa_get_order(xas->xa, xas->xa_index); + return thp_size(page); +} + +/** + * mapping_seek_hole_data - Seek for SEEK_DATA / SEEK_HOLE in the page cache. + * @mapping: Address space to search. + * @start: First byte to consider. + * @end: Limit of search (exclusive). + * @whence: Either SEEK_HOLE or SEEK_DATA. + * + * If the page cache knows which blocks contain holes and which blocks + * contain data, your filesystem can use this function to implement + * SEEK_HOLE and SEEK_DATA. This is useful for filesystems which are + * entirely memory-based such as tmpfs, and filesystems which support + * unwritten extents. + * + * Return: The requested offset on successs, or -ENXIO if @whence specifies + * SEEK_DATA and there is no data after @start. There is an implicit hole + * after @end - 1, so SEEK_HOLE returns @end if all the bytes between @start + * and @end contain data. + */ +loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start, + loff_t end, int whence) +{ + XA_STATE(xas, &mapping->i_pages, start >> PAGE_SHIFT); + pgoff_t max = (end - 1) / PAGE_SIZE; + bool seek_data = (whence == SEEK_DATA); + struct page *page; + + if (end <= start) + return -ENXIO; + + rcu_read_lock(); + while ((page = find_get_entry(&xas, max, XA_PRESENT))) { + loff_t pos = xas.xa_index * PAGE_SIZE; + + if (start < pos) { + if (!seek_data) + goto unlock; + start = pos; + } + + if (page_seek_match(page, seek_data)) + goto unlock; + start = pos + seek_page_size(&xas, page); + put_page(page); + } + rcu_read_unlock(); + + if (seek_data) + return -ENXIO; + goto out; + +unlock: + rcu_read_unlock(); + if (!xa_is_value(page)) + put_page(page); +out: + if (start > end) + return end; + return start; +} + #ifdef CONFIG_MMU #define MMAP_LOTSAMISS (100) /* diff --git a/mm/shmem.c b/mm/shmem.c index c4feb05425f2..6afea99a0dc0 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2694,86 +2694,20 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return retval ? retval : error; } -/* - * llseek SEEK_DATA or SEEK_HOLE through the page cache. - */ -static pgoff_t shmem_seek_hole_data(struct address_space *mapping, - pgoff_t index, pgoff_t end, int whence) -{ - struct page *page; - struct pagevec pvec; - pgoff_t indices[PAGEVEC_SIZE]; - bool done = false; - int i; - - pagevec_init(&pvec); - pvec.nr = 1; /* start small: we may be there already */ - while (!done) { - pvec.nr = find_get_entries(mapping, index, - pvec.nr, pvec.pages, indices); - if (!pvec.nr) { - if (whence == SEEK_DATA) - index = end; - break; - } - for (i = 0; i < pvec.nr; i++, index++) { - if (index < indices[i]) { - if (whence == SEEK_HOLE) { - done = true; - break; - } - index = indices[i]; - } - page = pvec.pages[i]; - if (page && !xa_is_value(page)) { - if (!PageUptodate(page)) - page = NULL; - } - if (index >= end || - (page && whence == SEEK_DATA) || - (!page && whence == SEEK_HOLE)) { - done = true; - break; - } - } - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - pvec.nr = PAGEVEC_SIZE; - cond_resched(); - } - return index; -} - static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) { struct address_space *mapping = file->f_mapping; struct inode *inode = mapping->host; - pgoff_t start, end; - loff_t new_offset; if (whence != SEEK_DATA && whence != SEEK_HOLE) return generic_file_llseek_size(file, offset, whence, MAX_LFS_FILESIZE, i_size_read(inode)); + if (offset < 0) + return -ENXIO; + inode_lock(inode); /* We're holding i_mutex so we can access i_size directly */ - - if (offset < 0 || offset >= inode->i_size) - offset = -ENXIO; - else { - start = offset >> PAGE_SHIFT; - end = (inode->i_size + PAGE_SIZE - 1) >> PAGE_SHIFT; - new_offset = shmem_seek_hole_data(mapping, start, end, whence); - new_offset <<= PAGE_SHIFT; - if (new_offset > offset) { - if (new_offset < inode->i_size) - offset = new_offset; - else if (whence == SEEK_DATA) - offset = -ENXIO; - else - offset = inode->i_size; - } - } - + offset = mapping_seek_hole_data(mapping, offset, inode->i_size, whence); if (offset >= 0) offset = vfs_setpos(file, offset, MAX_LFS_FILESIZE); inode_unlock(inode); From patchwork Thu Nov 12 21:26:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901637 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D8F4697 for ; Thu, 12 Nov 2020 21:26:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DD6782222F for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="f/+BuA+k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DD6782222F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3DF26B0070; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9770B6B0068; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 823676B006E; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 4CD836B005C for ; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DF12E33CD for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-FDA: 77477051064.04.lock11_1c0274b27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id C7A8780039D0 for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-Spam-Summary: 1,0,0,e900c17837812c39,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2559:2562:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3872:4049:4119:4321:4605:5007:6119:6261:6653:7903:8660:8957:9036:9592:10004:11026:11473:11658:11914:12043:12114:12296:12297:12438:12555:12683:12895:13148:13230:13894:14110:14394:21080:21324:21433:21451:21627:21939:21987:21990:30012:30054:30070,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04y8k69ixdf11uweh7h7foy33xy7mop7kry1mtydpnxodw5ik8esq6mz6t7ojhz.gczfs1bnxzgg8s7pyez7d4b51c47qejbb69xghxiirqw1e4y7mzyrz4u8iycbf6.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:416,LUA_SUMMARY:none X-HE-Tag: lock11_1c0274b27309 X-Filterd-Recvd-Size: 8108 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bTrDsmol5aUu0hNNGO8gHhFZiHT6t38oxGw0to6H7hI=; b=f/+BuA+klSCvPBZ5nrqnr6aDig tnqHLooQ2xnBfUL/bh+29e9RMw2V47ZfmV3cvxL7Ldhv16pkZp/7LVF67l3JVezO/pCdL59fSwqWW pBoJwjhNCAedzkvGZc/kh6aQvud1JKehsfneZ+Ygdp3S/mJqk4uWREZOTNs6L+M0nf/W2Zl0XxZMy BVFpA+vPKQrgk9CRFrxOcHLKm4g5Y4Jy5n8fEiy2s1e0l6SoR0DRMdXfJvGOeAraOyGb4Vh3wXGRk 6yx7wubk7J6+jplA8nhom201WsiHPTfJnw5OPB6PuO/sl3IKgrL0/jZ3FUYdOXjSLM5CSfz9O4XXd IDi9MOAg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7C-0007Ge-Tl; Thu, 12 Nov 2020 21:26:46 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 08/16] iomap: Use mapping_seek_hole_data Date: Thu, 12 Nov 2020 21:26:33 +0000 Message-Id: <20201112212641.27837-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Enhance mapping_seek_hole_data() to handle partially uptodate pages and convert the iomap seek code to call it. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/seek.c | 125 +++++------------------------------------------- mm/filemap.c | 37 ++++++++++++-- 2 files changed, 43 insertions(+), 119 deletions(-) diff --git a/fs/iomap/seek.c b/fs/iomap/seek.c index 107ee80c3568..dab1b02eba5b 100644 --- a/fs/iomap/seek.c +++ b/fs/iomap/seek.c @@ -10,122 +10,17 @@ #include #include -/* - * Seek for SEEK_DATA / SEEK_HOLE within @page, starting at @lastoff. - * Returns true if found and updates @lastoff to the offset in file. - */ -static bool -page_seek_hole_data(struct inode *inode, struct page *page, loff_t *lastoff, - int whence) -{ - const struct address_space_operations *ops = inode->i_mapping->a_ops; - unsigned int bsize = i_blocksize(inode), off; - bool seek_data = whence == SEEK_DATA; - loff_t poff = page_offset(page); - - if (WARN_ON_ONCE(*lastoff >= poff + PAGE_SIZE)) - return false; - - if (*lastoff < poff) { - /* - * Last offset smaller than the start of the page means we found - * a hole: - */ - if (whence == SEEK_HOLE) - return true; - *lastoff = poff; - } - - /* - * Just check the page unless we can and should check block ranges: - */ - if (bsize == PAGE_SIZE || !ops->is_partially_uptodate) - return PageUptodate(page) == seek_data; - - lock_page(page); - if (unlikely(page->mapping != inode->i_mapping)) - goto out_unlock_not_found; - - for (off = 0; off < PAGE_SIZE; off += bsize) { - if (offset_in_page(*lastoff) >= off + bsize) - continue; - if (ops->is_partially_uptodate(page, off, bsize) == seek_data) { - unlock_page(page); - return true; - } - *lastoff = poff + off + bsize; - } - -out_unlock_not_found: - unlock_page(page); - return false; -} - -/* - * Seek for SEEK_DATA / SEEK_HOLE in the page cache. - * - * Within unwritten extents, the page cache determines which parts are holes - * and which are data: uptodate buffer heads count as data; everything else - * counts as a hole. - * - * Returns the resulting offset on successs, and -ENOENT otherwise. - */ static loff_t -page_cache_seek_hole_data(struct inode *inode, loff_t offset, loff_t length, - int whence) -{ - pgoff_t index = offset >> PAGE_SHIFT; - pgoff_t end = DIV_ROUND_UP(offset + length, PAGE_SIZE); - loff_t lastoff = offset; - struct pagevec pvec; - - if (length <= 0) - return -ENOENT; - - pagevec_init(&pvec); - - do { - unsigned nr_pages, i; - - nr_pages = pagevec_lookup_range(&pvec, inode->i_mapping, &index, - end - 1); - if (nr_pages == 0) - break; - - for (i = 0; i < nr_pages; i++) { - struct page *page = pvec.pages[i]; - - if (page_seek_hole_data(inode, page, &lastoff, whence)) - goto check_range; - lastoff = page_offset(page) + PAGE_SIZE; - } - pagevec_release(&pvec); - } while (index < end); - - /* When no page at lastoff and we are not done, we found a hole. */ - if (whence != SEEK_HOLE) - goto not_found; - -check_range: - if (lastoff < offset + length) - goto out; -not_found: - lastoff = -ENOENT; -out: - pagevec_release(&pvec); - return lastoff; -} - - -static loff_t -iomap_seek_hole_actor(struct inode *inode, loff_t offset, loff_t length, +iomap_seek_hole_actor(struct inode *inode, loff_t start, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) { + loff_t offset = start; + switch (iomap->type) { case IOMAP_UNWRITTEN: - offset = page_cache_seek_hole_data(inode, offset, length, - SEEK_HOLE); - if (offset < 0) + offset = mapping_seek_hole_data(inode->i_mapping, start, + start + length, SEEK_HOLE); + if (offset == start + length) return length; fallthrough; case IOMAP_HOLE: @@ -164,15 +59,17 @@ iomap_seek_hole(struct inode *inode, loff_t offset, const struct iomap_ops *ops) EXPORT_SYMBOL_GPL(iomap_seek_hole); static loff_t -iomap_seek_data_actor(struct inode *inode, loff_t offset, loff_t length, +iomap_seek_data_actor(struct inode *inode, loff_t start, loff_t length, void *data, struct iomap *iomap, struct iomap *srcmap) { + loff_t offset = start; + switch (iomap->type) { case IOMAP_HOLE: return length; case IOMAP_UNWRITTEN: - offset = page_cache_seek_hole_data(inode, offset, length, - SEEK_DATA); + offset = mapping_seek_hole_data(inode->i_mapping, start, + start + length, SEEK_DATA); if (offset < 0) return length; fallthrough; diff --git a/mm/filemap.c b/mm/filemap.c index ab7103eb7e11..ef7411ea3f91 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2586,11 +2586,36 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); -static inline bool page_seek_match(struct page *page, bool seek_data) +static inline loff_t page_seek_hole_data(struct xa_state *xas, + struct address_space *mapping, struct page *page, + loff_t start, loff_t end, bool seek_data) { + const struct address_space_operations *ops = mapping->a_ops; + size_t offset, bsz = i_blocksize(mapping->host); + if (xa_is_value(page) || PageUptodate(page)) - return seek_data; - return !seek_data; + return seek_data ? start : end; + if (!ops->is_partially_uptodate) + return seek_data ? end : start; + + xas_pause(xas); + rcu_read_unlock(); + lock_page(page); + if (unlikely(page->mapping != mapping)) + goto unlock; + + offset = offset_in_thp(page, start) & ~(bsz - 1); + + do { + if (ops->is_partially_uptodate(page, offset, bsz) == seek_data) + break; + start = (start + bsz) & ~(bsz - 1); + offset += bsz; + } while (offset < thp_size(page)); +unlock: + unlock_page(page); + rcu_read_lock(); + return start; } static inline @@ -2640,9 +2665,11 @@ loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start, start = pos; } - if (page_seek_match(page, seek_data)) + pos += seek_page_size(&xas, page); + start = page_seek_hole_data(&xas, mapping, page, start, pos, + seek_data); + if (start < pos) goto unlock; - start = pos + seek_page_size(&xas, page); put_page(page); } rcu_read_unlock(); From patchwork Thu Nov 12 21:26:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901691 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 186E31391 for ; Thu, 12 Nov 2020 21:28:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 840632222F for ; Thu, 12 Nov 2020 21:27:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bPUApHYo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 840632222F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E5F946B006E; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D94716B0073; Thu, 12 Nov 2020 16:26:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC4296B0072; Thu, 12 Nov 2020 16:26:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 7F0A06B006C for ; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 25AF78249980 for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-FDA: 77477051148.26.balls85_151251b27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id F2D681804B647 for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) X-Spam-Summary: 1,0,0,5fb67fc53a789c5c,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:1:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2393:2553:2559:2562:2693:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4051:4250:4321:5007:6119:6261:6653:7875:8957:9592:10004:11026:11232:11473:11658:11914:12043:12291:12296:12297:12438:12555:12683:12895:13894:14096:14394:21063:21080:21324:21451:21627:21987:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfrnj53knjnbkhppachqud8px6zocpjnn6hjxj3tsdxccs6djpmjfqh6udn96.j9tgrkrwm4w1qssand8ef5dwz3i48amsbzjad44zdnwst85rjw9m6r4xeqtp8dm.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:517,LUA_SUMMARY:none X-HE-Tag: balls85_151251b27309 X-Filterd-Recvd-Size: 10633 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5OIezjcHLj22SXZWV7/dBe4YY1i3XFT69VqGISdM+r0=; b=bPUApHYocKNrU5GqwIQnYCMlhO mDe4dnD2N6Oj5DZ+MZO27jFmp50UQz3Ft0FF41chnTnlHRvhgecYHzKuNzjJtODTE07Ut+ko4t1wD mbJaP/l+5mTZ6gcTQ4IRwr2QPdNiN3xx7F4RQHN92D1l/gOrwxZXU9+/fNbdQdTSlrLtOFSZ/uvMJ /KZl2VVxw9+OBLwZhNgBILKMDT95+5Ux/DwU9nG9bfyhbqnNXSFcCW4GQORQPesO5h8Z5CVHO+rl7 KJfAlq3SCFKRnvcFUaQRkLgq0TvnKgEvElWebZCaMDw/HaCCTxWyAJUwJ9zhOvQNaWap3V9YGQ3Cd kC3bvWAg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7D-0007Gk-Fx; Thu, 12 Nov 2020 21:26:47 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 09/16] mm: Add and use find_lock_entries Date: Thu, 12 Nov 2020 21:26:34 +0000 Message-Id: <20201112212641.27837-10-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We have three functions (shmem_undo_range(), truncate_inode_pages_range() and invalidate_mapping_pages()) which want exactly this function, so add it to filemap.c. Before this patch, shmem_undo_range() would split any compound page which overlaps either end of the range being punched in both the first and second loops through the address space. After this patch, that functionality is left for the second loop, which is arguably more appropriate since the first loop is supposed to run through all the pages quickly, and splitting a page can sleep. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- mm/filemap.c | 57 ++++++++++++++++++++++++++++++++ mm/internal.h | 3 ++ mm/shmem.c | 22 +++---------- mm/truncate.c | 91 +++++++-------------------------------------------- 4 files changed, 76 insertions(+), 97 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ef7411ea3f91..f18c5074865d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1925,6 +1925,63 @@ unsigned find_get_entries(struct address_space *mapping, return ret; } +/** + * find_lock_entries - Find a batch of pagecache entries. + * @mapping: The address_space to search. + * @start: The starting page cache index. + * @end: The final page index (inclusive). + * @pvec: Where the resulting entries are placed. + * @indices: The cache indices of the entries in @pvec. + * + * find_lock_entries() will return a batch of entries from @mapping. + * Swap, shadow and DAX entries are included. Pages are returned + * locked and with an incremented refcount. Pages which are locked by + * somebody else or under writeback are skipped. Only the head page of + * a THP is returned. Pages which are partially outside the range are + * not returned. + * + * The entries have ascending indexes. The indices may not be consecutive + * due to not-present entries, THP pages, pages which could not be locked + * or pages under writeback. + * + * Return: The number of entries which were found. + */ +unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices) +{ + XA_STATE(xas, &mapping->i_pages, start); + struct page *page; + + rcu_read_lock(); + while ((page = find_get_entry(&xas, end, XA_PRESENT))) { + if (!xa_is_value(page)) { + if (page->index < start) + goto put; + VM_BUG_ON_PAGE(page->index != xas.xa_index, page); + if (page->index + thp_nr_pages(page) - 1 > end) + goto put; + if (!trylock_page(page)) + goto put; + if (page->mapping != mapping || PageWriteback(page)) + goto unlock; + } + indices[pvec->nr] = xas.xa_index; + if (!pagevec_add(pvec, page)) + break; + goto next; +unlock: + unlock_page(page); +put: + put_page(page); +next: + if (!xa_is_value(page) && PageTransHuge(page)) + xas_set(&xas, page->index + thp_nr_pages(page)); + } + rcu_read_unlock(); + + return pagevec_count(pvec); +} + /** * find_get_pages_range - gang pagecache lookup * @mapping: The address_space to search diff --git a/mm/internal.h b/mm/internal.h index 93880d460e12..3547fed59d51 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -60,6 +60,9 @@ static inline void force_page_cache_readahead(struct address_space *mapping, force_page_cache_ra(&ractl, &file->f_ra, nr_to_read); } +unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); + /** * page_evictable - test whether a page is evictable * @page: the page to test diff --git a/mm/shmem.c b/mm/shmem.c index 6afea99a0dc0..a4aa762a55f8 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -907,12 +907,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, pagevec_init(&pvec); index = start; - while (index < end) { - pvec.nr = find_get_entries(mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - pvec.pages, indices); - if (!pvec.nr) - break; + while (index < end && find_lock_entries(mapping, index, end - 1, + &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; @@ -927,18 +923,10 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, index, page); continue; } + index += thp_nr_pages(page) - 1; - VM_BUG_ON_PAGE(page_to_pgoff(page) != index, page); - - if (!trylock_page(page)) - continue; - - if ((!unfalloc || !PageUptodate(page)) && - page_mapping(page) == mapping) { - VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) - truncate_inode_page(mapping, page); - } + if (!unfalloc || !PageUptodate(page)) + truncate_inode_page(mapping, page); unlock_page(page); } pagevec_remove_exceptionals(&pvec); diff --git a/mm/truncate.c b/mm/truncate.c index 960edf5803ca..eefd62898db1 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -326,51 +326,19 @@ void truncate_inode_pages_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (index < end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - indices)) { - /* - * Pagevec array has exceptional entries and we may also fail - * to lock some pages. So we store pages that can be deleted - * in a new pagevec. - */ - struct pagevec locked_pvec; - - pagevec_init(&locked_pvec); - for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; - - /* We rely upon deletion not changing page->index */ - index = indices[i]; - if (index >= end) - break; - - if (xa_is_value(page)) - continue; - - if (!trylock_page(page)) - continue; - WARN_ON(page_to_index(page) != index); - if (PageWriteback(page)) { - unlock_page(page); - continue; - } - if (page->mapping != mapping) { - unlock_page(page); - continue; - } - pagevec_add(&locked_pvec, page); - } - for (i = 0; i < pagevec_count(&locked_pvec); i++) - truncate_cleanup_page(mapping, locked_pvec.pages[i]); - delete_from_page_cache_batch(mapping, &locked_pvec); - for (i = 0; i < pagevec_count(&locked_pvec); i++) - unlock_page(locked_pvec.pages[i]); + while (index < end && find_lock_entries(mapping, index, end - 1, + &pvec, indices)) { + index = indices[pagevec_count(&pvec) - 1] + 1; truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + for (i = 0; i < pagevec_count(&pvec); i++) + truncate_cleanup_page(mapping, pvec.pages[i]); + delete_from_page_cache_batch(mapping, &pvec); + for (i = 0; i < pagevec_count(&pvec); i++) + unlock_page(pvec.pages[i]); pagevec_release(&pvec); cond_resched(); - index++; } + if (partial_start) { struct page *page = find_lock_page(mapping, start - 1); if (page) { @@ -539,9 +507,7 @@ static unsigned long __invalidate_mapping_pages(struct address_space *mapping, int i; pagevec_init(&pvec); - while (index <= end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1, - indices)) { + while (find_lock_entries(mapping, index, end, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; @@ -555,39 +521,7 @@ static unsigned long __invalidate_mapping_pages(struct address_space *mapping, page); continue; } - - if (!trylock_page(page)) - continue; - - WARN_ON(page_to_index(page) != index); - - /* Middle of THP: skip */ - if (PageTransTail(page)) { - unlock_page(page); - continue; - } else if (PageTransHuge(page)) { - index += HPAGE_PMD_NR - 1; - i += HPAGE_PMD_NR - 1; - /* - * 'end' is in the middle of THP. Don't - * invalidate the page as the part outside of - * 'end' could be still useful. - */ - if (index > end) { - unlock_page(page); - continue; - } - - /* Take a pin outside pagevec */ - get_page(page); - - /* - * Drop extra pins before trying to invalidate - * the huge page. - */ - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - } + index += thp_nr_pages(page) - 1; ret = invalidate_inode_page(page); unlock_page(page); @@ -601,9 +535,6 @@ static unsigned long __invalidate_mapping_pages(struct address_space *mapping, if (nr_pagevec) (*nr_pagevec)++; } - - if (PageTransHuge(page)) - put_page(page); count += ret; } pagevec_remove_exceptionals(&pvec); From patchwork Thu Nov 12 21:26:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60780697 for ; Thu, 12 Nov 2020 21:27:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0815922227 for ; Thu, 12 Nov 2020 21:27:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="BpevITKR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0815922227 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3ABFB6B006C; Thu, 12 Nov 2020 16:26:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 360FE6B0072; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 165686B0074; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0085.hostedemail.com [216.40.44.85]) by kanga.kvack.org (Postfix) with ESMTP id CFBFE6B006C for ; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 696AE181AEF1A for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-FDA: 77477051148.19.wound99_2c04e4427309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 3DA601AD1B0 for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-Spam-Summary: 1,0,0,e21c4c62adc52231,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1711:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:4117:4321:4605:5007:6261:6653:7875:9592:10004:11026:11658:11914:12043:12296:12297:12438:12555:12895:12986:13894:14181:14394:14721:21080:21324:21433:21451:21611:21627:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfi9869wdbjegj7p5k1bugrosczycmzoeyzjpz4jx64q4m1p81p3n4mbus99c.yfpg58dnmxoq67p1hooq9gajtbc5pw1c4whd8abinffzuktbcwndtscifkgcn8r.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:898,LUA_SUMMARY:none X-HE-Tag: wound99_2c04e4427309 X-Filterd-Recvd-Size: 6090 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tg4uf1yPSb1+NPbluHkPZ5qPAwrm/1+z+EzXPXHIqqs=; b=BpevITKRE81lHU4Xrh0y8Vl+OI l9ohsaRmyZrdPCdtUCoOBe8qoUvtnPjPg0YwrQyc85XA2wJyQA5nMkHwzO5rlUXlV8oJ7BR1PJfVv 6k4H+Cm4zf3g4C+q3bPj1HbmAG0cDBKBjmckvuQXrlfbST2of7GNfH/o1rY6RNaGYJubwa+pVlLAa 51PDIqW2AMERRp0fcV8wHfRui3G9Vbn329LRn6xAaB0gHLlxt7dpjf1juQ0Fsf1Q+gN/Zx/1PSe2r duv4Z9ce/IKfCD2psNiIYSq9Fj+z2/o9rChPVFl33LMwOQ01y4pAy1UAD1d4uTGDBTSm4+lGnrHNp m4lgFH8Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7D-0007H7-Uc; Thu, 12 Nov 2020 21:26:48 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 10/16] mm: Add an 'end' parameter to find_get_entries Date: Thu, 12 Nov 2020 21:26:35 +0000 Message-Id: <20201112212641.27837-11-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This simplifies the callers and leads to a more efficient implementation since the XArray has this functionality already. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 4 ++-- mm/filemap.c | 9 +++++---- mm/shmem.c | 10 ++-------- mm/swap.c | 2 +- 4 files changed, 10 insertions(+), 15 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 9f1f4ab9612a..c7c26a902743 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -449,8 +449,8 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) } unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - unsigned int nr_entries, struct page **entries, - pgoff_t *indices); + pgoff_t end, unsigned int nr_entries, struct page **entries, + pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index f18c5074865d..b3b89a62ab1a 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1870,6 +1870,7 @@ static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, * find_get_entries - gang pagecache lookup * @mapping: The address_space to search * @start: The starting page cache index + * @end: The final page index (inclusive). * @nr_entries: The maximum number of entries * @entries: Where the resulting entries are placed * @indices: The cache indices corresponding to the entries in @entries @@ -1893,9 +1894,9 @@ static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, * * Return: the number of pages and shadow entries which were found. */ -unsigned find_get_entries(struct address_space *mapping, - pgoff_t start, unsigned int nr_entries, - struct page **entries, pgoff_t *indices) +unsigned find_get_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, unsigned int nr_entries, struct page **entries, + pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; @@ -1905,7 +1906,7 @@ unsigned find_get_entries(struct address_space *mapping, return 0; rcu_read_lock(); - while ((page = find_get_entry(&xas, ULONG_MAX, XA_PRESENT))) { + while ((page = find_get_entry(&xas, end, XA_PRESENT))) { /* * Terminate early on finding a THP, to allow the caller to * handle it all at once; but continue if this is hugetlbfs. diff --git a/mm/shmem.c b/mm/shmem.c index a4aa762a55f8..7a62dc967d7d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -913,8 +913,6 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct page *page = pvec.pages[i]; index = indices[i]; - if (index >= end) - break; if (xa_is_value(page)) { if (unfalloc) @@ -967,9 +965,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end) { cond_resched(); - pvec.nr = find_get_entries(mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - pvec.pages, indices); + pvec.nr = find_get_entries(mapping, index, end - 1, + PAGEVEC_SIZE, pvec.pages, indices); if (!pvec.nr) { /* If all gone or hole-punch or unfalloc, we're done */ if (index == start || end != -1) @@ -982,9 +979,6 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct page *page = pvec.pages[i]; index = indices[i]; - if (index >= end) - break; - if (xa_is_value(page)) { if (unfalloc) continue; diff --git a/mm/swap.c b/mm/swap.c index 29220174433b..39be55635ebd 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1102,7 +1102,7 @@ unsigned pagevec_lookup_entries(struct pagevec *pvec, pgoff_t start, unsigned nr_entries, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, nr_entries, + pvec->nr = find_get_entries(mapping, start, ULONG_MAX, nr_entries, pvec->pages, indices); return pagevec_count(pvec); } From patchwork Thu Nov 12 21:26:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901643 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0EB21391 for ; Thu, 12 Nov 2020 21:26:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 36B7522227 for ; Thu, 12 Nov 2020 21:26:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Mlht3xsX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36B7522227 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2287D6B005D; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E0F396B0068; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B72436B006E; Thu, 12 Nov 2020 16:26:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 7B35B6B006C for ; Thu, 12 Nov 2020 16:26:53 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1CC8F1EE6 for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) X-FDA: 77477051106.10.coast10_230db5627309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id DB5D116A0C3 for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) X-Spam-Summary: 1,0,0,4016153d04c32f9c,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3874:4049:4119:4321:4605:5007:6261:6653:7875:7903:9592:10004:11026:11658:11914:12043:12114:12296:12297:12438:12555:12683:12895:12986:13894:14394:21080:21324:21433:21451:21627:21987:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y8pif8x73m9d69tzqrtkdug1wsgycu5o7fp9ahieybdjen5e5uudkp3ukngcu.tcckozbhiezgokb1gora5u6gqdaoxrnpz3nuy8w4z5hf3cxwqehne6en4sqawzz.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:524,LUA_SUMMARY:none X-HE-Tag: coast10_230db5627309 X-Filterd-Recvd-Size: 8469 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=YC6+GAJwlZFwXSFvTpyJewfs7ChCRtgnJTzrVE7HqTA=; b=Mlht3xsXO+OXXrDiIY94EfipvN m1ho5DPy8Pth4aAL18j9U5LwZubRikCMc2tmzM2r1MU1lIf/48LL09WnAUOssXqBHny0i844nH9N8 x1sHE1JY/zeZSPH//OakqfKIbe4hoE1UGj99ATFbCC4cHhZ7frE11jpeHnuTkDuLuQ7K7Gia4GWS2 ep6i2E8USP5ffYDH4t1B1U5DztLW4e3tbGoGxD1QPx/3LcXWB/kcy5+Rs8BKu9aR9BWRoSZ5SnKvT jag/OPdjvATwWYEKIu4Q1Sb5vnliUEa8HkY6soD9xRXD2F+Q1BeG8+ljKc9AjfQTAwJCKqRVjoxcf EvKRTEPQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7E-0007HL-NW; Thu, 12 Nov 2020 21:26:48 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 11/16] mm: Add an 'end' parameter to pagevec_lookup_entries Date: Thu, 12 Nov 2020 21:26:36 +0000 Message-Id: <20201112212641.27837-12-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simplifies the callers and uses the existing functionality in find_get_entries(). We can also drop the final argument of truncate_exceptional_pvec_entries() and simplify the logic in that function. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagevec.h | 5 ++--- mm/swap.c | 8 ++++---- mm/truncate.c | 41 ++++++++++------------------------------- 3 files changed, 16 insertions(+), 38 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index 081d934eda64..4b245592262c 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -26,9 +26,8 @@ struct pagevec { void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, - pgoff_t start, unsigned nr_entries, - pgoff_t *indices); + struct address_space *mapping, pgoff_t start, pgoff_t end, + unsigned nr_entries, pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index 39be55635ebd..5c2c12e3581e 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1078,6 +1078,7 @@ void __pagevec_lru_add(struct pagevec *pvec) * @pvec: Where the resulting entries are placed * @mapping: The address_space to search * @start: The starting entry index + * @end: The highest index to return (inclusive). * @nr_entries: The maximum number of pages * @indices: The cache indices corresponding to the entries in @pvec * @@ -1098,11 +1099,10 @@ void __pagevec_lru_add(struct pagevec *pvec) * found. */ unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, - pgoff_t start, unsigned nr_entries, - pgoff_t *indices) + struct address_space *mapping, pgoff_t start, pgoff_t end, + unsigned nr_entries, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, ULONG_MAX, nr_entries, + pvec->nr = find_get_entries(mapping, start, end, nr_entries, pvec->pages, indices); return pagevec_count(pvec); } diff --git a/mm/truncate.c b/mm/truncate.c index eefd62898db1..dcaee659fe1a 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -57,11 +57,10 @@ static void clear_shadow_entry(struct address_space *mapping, pgoff_t index, * exceptional entries similar to what pagevec_remove_exceptionals does. */ static void truncate_exceptional_pvec_entries(struct address_space *mapping, - struct pagevec *pvec, pgoff_t *indices, - pgoff_t end) + struct pagevec *pvec, pgoff_t *indices) { int i, j; - bool dax, lock; + bool dax; /* Handled by shmem itself */ if (shmem_mapping(mapping)) @@ -75,8 +74,7 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, return; dax = dax_mapping(mapping); - lock = !dax && indices[j] < end; - if (lock) + if (!dax) xa_lock_irq(&mapping->i_pages); for (i = j; i < pagevec_count(pvec); i++) { @@ -88,9 +86,6 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, continue; } - if (index >= end) - continue; - if (unlikely(dax)) { dax_delete_mapping_entry(mapping, index); continue; @@ -99,7 +94,7 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, __clear_shadow_entry(mapping, index, page); } - if (lock) + if (!dax) xa_unlock_irq(&mapping->i_pages); pvec->nr = j; } @@ -329,7 +324,7 @@ void truncate_inode_pages_range(struct address_space *mapping, while (index < end && find_lock_entries(mapping, index, end - 1, &pvec, indices)) { index = indices[pagevec_count(&pvec) - 1] + 1; - truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + truncate_exceptional_pvec_entries(mapping, &pvec, indices); for (i = 0; i < pagevec_count(&pvec); i++) truncate_cleanup_page(mapping, pvec.pages[i]); delete_from_page_cache_batch(mapping, &pvec); @@ -381,8 +376,8 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; for ( ; ; ) { cond_resched(); - if (!pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), indices)) { + if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, + PAGEVEC_SIZE, indices)) { /* If all gone from start onwards, we're done */ if (index == start) break; @@ -390,23 +385,12 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; continue; } - if (index == start && indices[0] >= end) { - /* All gone out of hole to be punched, we're done */ - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - break; - } for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index >= end) { - /* Restart punch to make sure all gone */ - index = start - 1; - break; - } if (xa_is_value(page)) continue; @@ -417,7 +401,7 @@ void truncate_inode_pages_range(struct address_space *mapping, truncate_inode_page(mapping, page); unlock_page(page); } - truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + truncate_exceptional_pvec_entries(mapping, &pvec, indices); pagevec_release(&pvec); index++; } @@ -513,8 +497,6 @@ static unsigned long __invalidate_mapping_pages(struct address_space *mapping, /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index > end) - break; if (xa_is_value(page)) { invalidate_exceptional_entry(mapping, index, @@ -650,16 +632,13 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (index <= end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1, - indices)) { + while (pagevec_lookup_entries(&pvec, mapping, index, end, + PAGEVEC_SIZE, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index > end) - break; if (xa_is_value(page)) { if (!invalidate_exceptional_entry2(mapping, From patchwork Thu Nov 12 21:26:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 723E21391 for ; Thu, 12 Nov 2020 21:27:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 17C732223F for ; Thu, 12 Nov 2020 21:27:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="tsQHj/H6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 17C732223F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BEC6F6B0071; Thu, 12 Nov 2020 16:26:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B353B6B0078; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DB196B0072; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 544B86B0071 for ; Thu, 12 Nov 2020 16:26:55 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id EDF238249980 for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-FDA: 77477051148.29.wing41_1c10b8227309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id D6374180868E8 for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-Spam-Summary: 1,0,0,bf4e0259f5383cdc,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:4321:4605:5007:6261:6653:8603:10004:11026:11232:11658:11914:12043:12114:12296:12297:12438:12555:12895:12986:13894:14181:14394:14721:21080:21324:21451:21627:21987:21990:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04y8ggb3dx5i77qfyuiqorrs83ynkop4kdw3sttg86eyikp9tnbzik78tqcm4np.wghkzop57krj8su98q816g3owqbxzgs5hpmwbrmnezsoix1t7ksmmk3yrn74h4h.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:69,LUA_SUMMARY:none X-HE-Tag: wing41_1c10b8227309 X-Filterd-Recvd-Size: 4250 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=uffLheexwH4YKkYyuwl3ZDoqTJgvhShB/zTgAuL0STg=; b=tsQHj/H6HLq+ZEls4rGNrVmklp TcVPWiwbp3CBY2EcqYYz+OaWLQolgctxTpR6L4F5n+rEVPpQ5SxLUTkZXqXKmsugOUzc41pN22eFC 8esxXmlovmPa1tH8o7ug6dgZy8aeC6uSVPB5XtpMYKOb4LMvlX0CnMPSh/jxpjt+GM2MFgzv4VR2u xYdiSj04gRz4mmwTWdjGkHdvqNYxZC0ATEOVoVUzmmJklxxJ32z0VOjpJvu1JfHSJuOm6OV9LCNre W1UI5pzGjnEdCO2ao0JgH/DDlaVSY+f5MrIJPPLTZcF79PUHi7F282/ax+RV1TACWQeIgMRO67Y4i 8FDI10vA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7F-0007HR-65; Thu, 12 Nov 2020 21:26:49 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 12/16] mm: Remove nr_entries parameter from pagevec_lookup_entries Date: Thu, 12 Nov 2020 21:26:37 +0000 Message-Id: <20201112212641.27837-13-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers want to fetch the full size of the pvec. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagevec.h | 2 +- mm/swap.c | 4 ++-- mm/truncate.c | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index 4b245592262c..ce77724a2ab7 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -27,7 +27,7 @@ void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, - unsigned nr_entries, pgoff_t *indices); + pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index 5c2c12e3581e..9a562f7fd200 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1100,9 +1100,9 @@ void __pagevec_lru_add(struct pagevec *pvec) */ unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, - unsigned nr_entries, pgoff_t *indices) + pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, end, nr_entries, + pvec->nr = find_get_entries(mapping, start, end, PAGEVEC_SIZE, pvec->pages, indices); return pagevec_count(pvec); } diff --git a/mm/truncate.c b/mm/truncate.c index dcaee659fe1a..50d160700b7d 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -377,7 +377,7 @@ void truncate_inode_pages_range(struct address_space *mapping, for ( ; ; ) { cond_resched(); if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, - PAGEVEC_SIZE, indices)) { + indices)) { /* If all gone from start onwards, we're done */ if (index == start) break; @@ -632,8 +632,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (pagevec_lookup_entries(&pvec, mapping, index, end, - PAGEVEC_SIZE, indices)) { + while (pagevec_lookup_entries(&pvec, mapping, index, end, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; From patchwork Thu Nov 12 21:26:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901697 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E64015E6 for ; Thu, 12 Nov 2020 21:28:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8DE352224A for ; Thu, 12 Nov 2020 21:27:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="CHIbihY1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8DE352224A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92ACC6B0072; Thu, 12 Nov 2020 16:26:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B4406B0073; Thu, 12 Nov 2020 16:26:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77C276B0078; Thu, 12 Nov 2020 16:26:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 3DD176B0072 for ; Thu, 12 Nov 2020 16:26:56 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DD1DE3626 for ; Thu, 12 Nov 2020 21:26:55 +0000 (UTC) X-FDA: 77477051190.10.boats92_0d151c227309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id BB68016A0AB for ; Thu, 12 Nov 2020 21:26:55 +0000 (UTC) X-Spam-Summary: 1,0,0,9076505b82d29e85,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:421:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1711:1730:1747:1777:1792:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3872:4117:4321:4605:5007:6261:6653:9592:10004:11026:11232:11658:11914:12043:12296:12297:12438:12555:12895:12986:13894:14096:14181:14394:14721:21080:21324:21433:21451:21611:21627:21796:21990:30034:30036:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfghsd4swb5k15oshiqy4m8fss8ocbgtiipgc4rqpiznk8epoj4rrgytdgqr4.nt8yfqtiajrm1z88z3x8xiikzmtfrcqknzq3x71okg5moqxycoxdu4n9o6yjo8k.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:254,LUA_SUMMARY:none X-HE-Tag: boats92_0d151c227309 X-Filterd-Recvd-Size: 6127 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=MOhcev08Qr0q0WFBcUAlxQv3z5m+M7Jp4y+munP9EsA=; b=CHIbihY1P2M/Ip+chUtSUhnj3r elD6u7bsS9+RzgTnl0LJCsB2RDtvlKb/+FO69euC9vy1E3NlerVYFVjsiEGnTP1A4MWEdMHWHygHD Ndv51Z65hEpInchnaYCGWnWG4KLGe9ZLqfKTKBcyboFB7bk0ORTAmoDL4oDne4UWWwNduDwKrtcuk tp3f+pO5bImAQqt9oneNQYyOXWiubs66lAQ+73plyA/m5OTZUTNw+ElLdntFoQ/S1zRiV28vyBUjX ypR0pp2gkQttKWVfxGupREgDAXSNa2Bq9G3sYKzOJQ8ZekiODCTlFjvA5W3dsslgQmn8diNXHHsQs RuC5wYfw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7F-0007Ha-GX; Thu, 12 Nov 2020 21:26:49 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 13/16] mm: Pass pvec directly to find_get_entries Date: Thu, 12 Nov 2020 21:26:38 +0000 Message-Id: <20201112212641.27837-14-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers of find_get_entries() use a pvec, so pass it directly instead of manipulating it in the caller. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 21 +++++++++------------ mm/shmem.c | 5 ++--- mm/swap.c | 4 +--- 4 files changed, 13 insertions(+), 20 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index c7c26a902743..46d4b1704770 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -449,8 +449,7 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) } unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, unsigned int nr_entries, struct page **entries, - pgoff_t *indices); + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index b3b89a62ab1a..479cbbadd93b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1871,14 +1871,12 @@ static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, * @mapping: The address_space to search * @start: The starting page cache index * @end: The final page index (inclusive). - * @nr_entries: The maximum number of entries - * @entries: Where the resulting entries are placed + * @pvec: Where the resulting entries are placed. * @indices: The cache indices corresponding to the entries in @entries * - * find_get_entries() will search for and return a group of up to - * @nr_entries entries in the mapping. The entries are placed at - * @entries. find_get_entries() takes a reference against any actual - * pages it returns. + * find_get_entries() will search for and return a batch of entries in + * the mapping. The entries are placed in @pvec. find_get_entries() + * takes a reference on any actual pages it returns. * * The search returns a group of mapping-contiguous page cache entries * with ascending indexes. There may be holes in the indices due to @@ -1895,15 +1893,12 @@ static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, * Return: the number of pages and shadow entries which were found. */ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, unsigned int nr_entries, struct page **entries, - pgoff_t *indices) + pgoff_t end, struct pagevec *pvec, pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; unsigned int ret = 0; - - if (!nr_entries) - return 0; + unsigned nr_entries = PAGEVEC_SIZE; rcu_read_lock(); while ((page = find_get_entry(&xas, end, XA_PRESENT))) { @@ -1918,11 +1913,13 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, } indices[ret] = xas.xa_index; - entries[ret] = page; + pvec->pages[ret] = page; if (++ret == nr_entries) break; } rcu_read_unlock(); + + pvec->nr = ret; return ret; } diff --git a/mm/shmem.c b/mm/shmem.c index 7a62dc967d7d..e01457988dd6 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -965,9 +965,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end) { cond_resched(); - pvec.nr = find_get_entries(mapping, index, end - 1, - PAGEVEC_SIZE, pvec.pages, indices); - if (!pvec.nr) { + if (!find_get_entries(mapping, index, end - 1, &pvec, + indices)) { /* If all gone or hole-punch or unfalloc, we're done */ if (index == start || end != -1) break; diff --git a/mm/swap.c b/mm/swap.c index 9a562f7fd200..7cf585223566 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1102,9 +1102,7 @@ unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, end, PAGEVEC_SIZE, - pvec->pages, indices); - return pagevec_count(pvec); + return find_get_entries(mapping, start, end, pvec, indices); } /** From patchwork Thu Nov 12 21:26:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901653 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CF0F697 for ; Thu, 12 Nov 2020 21:27:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 45AF222241 for ; Thu, 12 Nov 2020 21:27:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="WMwLcRRL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 45AF222241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8DE686B0074; Thu, 12 Nov 2020 16:26:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 83DD06B0073; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46EFD6B0074; Thu, 12 Nov 2020 16:26:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 0B9C46B0071 for ; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 95275824999B for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-FDA: 77477051148.08.mass05_2e0894827309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 799E51819E76C for ; Thu, 12 Nov 2020 21:26:54 +0000 (UTC) X-Spam-Summary: 1,0,0,d4bd38b6e95b8a61,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6119:6261:6653:7875:8660:9592:10004:11026:11658:11914:12043:12114:12296:12297:12438:12555:12683:12895:12986:13148:13230:13894:14110:14181:14394:14721:21063:21080:21324:21433:21451:21627:21939:21987:21990:30034:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04ygz774by41jdzx7dnezm4gq5yawyczkkxztcwfp3fu44durmqqwgtgx6pzf1s.qwzdfxcqczp6jsmx5mnh9ecqnwim94ym7dksu68s44b7j9a6heor91gwpqxqhxw.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:69,LUA_SUMMARY:none X-HE-Tag: mass05_2e0894827309 X-Filterd-Recvd-Size: 5778 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=nPDvTHk8PjzTj2XUJS83GH14pU+C7s3G9UCa3yq16Gg=; b=WMwLcRRLIzEY/GjJ+G32FgsHx5 KiaxlnHD6t77xFQx7BG9Nkim1OczitEh18vJdbN5RQSEcy5cpy946x77uW8wHJzzcUrDXtGJnooDn /VK0mHBye057ztpFIfZupOhzLaSeIaLx3CmnDdU7ykPw6xYaPW4jgNe0S4+SVWdTc26UB4JRpbc0m 09HHSiGae29bQ/CP5vrWPYtrof0GN2+umIWSmmhoYCMr9TxGsNCtGEpN57E3mgAc56usPr2rXhHAw HgE+hSvH0QHqfRxw9E4zkvthMYQYXpYcA5NXJ91BD7uLoaIed5wBaueEzQY9HheNRcQ1xmGP7s3R3 oucS1wQA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7G-0007Hj-8A; Thu, 12 Nov 2020 21:26:50 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 14/16] mm: Remove pagevec_lookup_entries Date: Thu, 12 Nov 2020 21:26:39 +0000 Message-Id: <20201112212641.27837-15-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pagevec_lookup_entries() is now just a wrapper around find_get_entries() so remove it and convert all its callers. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagevec.h | 3 --- mm/swap.c | 36 ++---------------------------------- mm/truncate.c | 4 ++-- 3 files changed, 4 insertions(+), 39 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index ce77724a2ab7..a45bea4b4d08 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -25,9 +25,6 @@ struct pagevec { void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); -unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, pgoff_t start, pgoff_t end, - pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index 7cf585223566..14c3bac607a6 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1073,44 +1073,12 @@ void __pagevec_lru_add(struct pagevec *pvec) pagevec_lru_move_fn(pvec, __pagevec_lru_add_fn, NULL); } -/** - * pagevec_lookup_entries - gang pagecache lookup - * @pvec: Where the resulting entries are placed - * @mapping: The address_space to search - * @start: The starting entry index - * @end: The highest index to return (inclusive). - * @nr_entries: The maximum number of pages - * @indices: The cache indices corresponding to the entries in @pvec - * - * pagevec_lookup_entries() will search for and return a group of up - * to @nr_pages pages and shadow entries in the mapping. All - * entries are placed in @pvec. pagevec_lookup_entries() takes a - * reference against actual pages in @pvec. - * - * The search returns a group of mapping-contiguous entries with - * ascending indexes. There may be holes in the indices due to - * not-present entries. - * - * Only one subpage of a Transparent Huge Page is returned in one call: - * allowing truncate_inode_pages_range() to evict the whole THP without - * cycling through a pagevec of extra references. - * - * pagevec_lookup_entries() returns the number of entries which were - * found. - */ -unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, pgoff_t start, pgoff_t end, - pgoff_t *indices) -{ - return find_get_entries(mapping, start, end, pvec, indices); -} - /** * pagevec_remove_exceptionals - pagevec exceptionals pruning * @pvec: The pagevec to prune * - * pagevec_lookup_entries() fills both pages and exceptional radix - * tree entries into the pagevec. This function prunes all + * find_get_entries() fills both pages and XArray value entries (aka + * exceptional entries) into the pagevec. This function prunes all * exceptionals from @pvec without leaving holes, so that it can be * passed on to page-only pagevec operations. */ diff --git a/mm/truncate.c b/mm/truncate.c index 50d160700b7d..68b7630e1fc4 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -376,7 +376,7 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; for ( ; ; ) { cond_resched(); - if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, + if (!find_get_entries(mapping, index, end - 1, &pvec, indices)) { /* If all gone from start onwards, we're done */ if (index == start) @@ -632,7 +632,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (pagevec_lookup_entries(&pvec, mapping, index, end, indices)) { + while (find_get_entries(mapping, index, end, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; From patchwork Thu Nov 12 21:26:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901645 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73C041391 for ; Thu, 12 Nov 2020 21:27:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF6FF22241 for ; Thu, 12 Nov 2020 21:27:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="t+odpWwA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF6FF22241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B064E6B0068; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 99EAE6B0071; Thu, 12 Nov 2020 16:26:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77BB76B006E; Thu, 12 Nov 2020 16:26:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id 377926B006C for ; Thu, 12 Nov 2020 16:26:54 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C23B9181AEF10 for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) X-FDA: 77477051106.07.sky09_5b133aa27309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id A87731803F9A1 for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) X-Spam-Summary: 1,0,0,e27bd87e0c1d4747,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:1:41:69:355:379:541:800:960:966:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:2637:2693:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4384:4385:4395:5007:6119:6261:6653:7875:8603:8957:9036:9592:10004:11026:11232:11473:11658:11914:12043:12291:12295:12296:12297:12438:12555:12683:12895:12986:13141:13161:13229:13230:13894:14394:21080:21324:21451:21627:21809:21987:21990:30012:30054:30064:30069:30070:30075:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfugreoxmm3i75agkcx9crjfzttyc9tyidgyzi7e8wzw3nrtdngu6r6y6xh9m.idqt4o3pp1hr4czo5e3bcndnye3ptpu4164x45pf6rqghf5oah6ihhwrzczqm9m.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:68,LUA_SUMMARY:none X-HE-Tag: sky09_5b133aa27309 X-Filterd-Recvd-Size: 13121 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=Vcc2Gm3awZbAsTWopBxfrDFjg5Xf0IYed41wGhvKh4w=; b=t+odpWwArKJCoToaTQXc1TtYwJ vnPXlIV04BRYtWjzQcbQAe+dC06/sDjllkr88i5zIt4qYXW4YC+S2vlZUlPko4mHXGR85ywUxCP7D jbFMULCWPXlsAZeeZEK8XjTZ4vJW+A2G8mY9SymRRLakj1QN/N8quhrUcmWfm3Qb0LZ+uq2aKEpWc zjy9pUEBVTeFog0Hq17OFicyoBgDTrRL88m8BkRRABuzflAs1Odtl0i+gAwskUUFsyjg1QwW89vhT B/5W+4QNHp6wg5ssBWWKIB0Xm/Ai1Yt7NS9jznrTMhHsKahb1gaHB6E8kCACROcR/mbN4xfSzTPMp J8FEZNIg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7G-0007Hr-Nt; Thu, 12 Nov 2020 21:26:50 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 15/16] mm/truncate,shmem: Handle truncates that split THPs Date: Thu, 12 Nov 2020 21:26:40 +0000 Message-Id: <20201112212641.27837-16-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Handle THP splitting in the parts of the truncation functions which already handle partial pages. Factor all that code out into a new function called truncate_inode_partial_page(). We lose the easy 'bail out' path if a truncate or hole punch is entirely within a single page. We can add some more complex logic to restore the optimisation if it proves to be worthwhile. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- mm/internal.h | 1 + mm/shmem.c | 97 ++++++++++++++--------------------------- mm/truncate.c | 118 +++++++++++++++++++++++++++++++------------------- 3 files changed, 108 insertions(+), 108 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 3547fed59d51..cb7487efa856 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -618,4 +618,5 @@ struct migration_target_control { gfp_t gfp_mask; }; +bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end); #endif /* __MM_INTERNAL_H */ diff --git a/mm/shmem.c b/mm/shmem.c index e01457988dd6..25fe257e56c0 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -857,32 +857,6 @@ void shmem_unlock_mapping(struct address_space *mapping) } } -/* - * Check whether a hole-punch or truncation needs to split a huge page, - * returning true if no split was required, or the split has been successful. - * - * Eviction (or truncation to 0 size) should never need to split a huge page; - * but in rare cases might do so, if shmem_undo_range() failed to trylock on - * head, and then succeeded to trylock on tail. - * - * A split can only succeed when there are no additional references on the - * huge page: so the split below relies upon find_get_entries() having stopped - * when it found a subpage of the huge page, without getting further references. - */ -static bool shmem_punch_compound(struct page *page, pgoff_t start, pgoff_t end) -{ - if (!PageTransCompound(page)) - return true; - - /* Just proceed to delete a huge page wholly within the range punched */ - if (PageHead(page) && - page->index >= start && page->index + HPAGE_PMD_NR <= end) - return true; - - /* Try to split huge page, so we can truly punch the hole or truncate */ - return split_huge_page(page) >= 0; -} - /* * Remove range of pages and swap entries from page cache, and free them. * If !unfalloc, truncate or punch hole; if unfalloc, undo failed fallocate. @@ -894,13 +868,13 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct shmem_inode_info *info = SHMEM_I(inode); pgoff_t start = (lstart + PAGE_SIZE - 1) >> PAGE_SHIFT; pgoff_t end = (lend + 1) >> PAGE_SHIFT; - unsigned int partial_start = lstart & (PAGE_SIZE - 1); - unsigned int partial_end = (lend + 1) & (PAGE_SIZE - 1); struct pagevec pvec; pgoff_t indices[PAGEVEC_SIZE]; + struct page *page; long nr_swaps_freed = 0; pgoff_t index; int i; + bool partial_end; if (lend == -1) end = -1; /* unsigned, so actually very big */ @@ -910,7 +884,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end && find_lock_entries(mapping, index, end - 1, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; index = indices[i]; @@ -933,33 +907,37 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, index++; } - if (partial_start) { - struct page *page = NULL; - shmem_getpage(inode, start - 1, &page, SGP_READ); - if (page) { - unsigned int top = PAGE_SIZE; - if (start > end) { - top = partial_end; - partial_end = 0; - } - zero_user_segment(page, partial_start, top); - set_page_dirty(page); - unlock_page(page); - put_page(page); + partial_end = ((lend + 1) % PAGE_SIZE) > 0; + page = NULL; + shmem_getpage(inode, lstart >> PAGE_SHIFT, &page, SGP_READ); + if (page) { + bool same_page; + + page = thp_head(page); + same_page = lend < page_offset(page) + thp_size(page); + if (same_page) + partial_end = false; + set_page_dirty(page); + if (!truncate_inode_partial_page(page, lstart, lend)) { + start = page->index + thp_nr_pages(page); + if (same_page) + end = page->index; } + unlock_page(page); + put_page(page); + page = NULL; } - if (partial_end) { - struct page *page = NULL; + + if (partial_end) shmem_getpage(inode, end, &page, SGP_READ); - if (page) { - zero_user_segment(page, 0, partial_end); - set_page_dirty(page); - unlock_page(page); - put_page(page); - } + if (page) { + page = thp_head(page); + set_page_dirty(page); + if (!truncate_inode_partial_page(page, lstart, lend)) + end = page->index; + unlock_page(page); + put_page(page); } - if (start >= end) - return; index = start; while (index < end) { @@ -975,7 +953,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, continue; } for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; index = indices[i]; if (xa_is_value(page)) { @@ -1000,18 +978,9 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, break; } VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) - truncate_inode_page(mapping, page); - else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { - /* Wipe the page and don't get stuck */ - clear_highpage(page); - flush_dcache_page(page); - set_page_dirty(page); - if (index < - round_up(start, HPAGE_PMD_NR)) - start = index + 1; - } + truncate_inode_page(mapping, page); } + index = page->index + thp_nr_pages(page) - 1; unlock_page(page); } pagevec_remove_exceptionals(&pvec); diff --git a/mm/truncate.c b/mm/truncate.c index 68b7630e1fc4..288781e41a7b 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -224,6 +224,53 @@ int truncate_inode_page(struct address_space *mapping, struct page *page) return 0; } +/* + * Handle partial (transparent) pages. The page may be entirely within the + * range if a split has raced with us. If not, we zero the part of the + * page that's within the [start, end] range, and then split the page if + * it's a THP. split_page_range() will discard pages which now lie beyond + * i_size, and we rely on the caller to discard pages which lie within a + * newly created hole. + * + * Returns false if THP splitting failed so the caller can avoid + * discarding the entire page which is stubbornly unsplit. + */ +bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end) +{ + loff_t pos = page_offset(page); + unsigned int offset, length; + + if (pos < start) + offset = start - pos; + else + offset = 0; + length = thp_size(page); + if (pos + length <= (u64)end) + length = length - offset; + else + length = end + 1 - pos - offset; + + wait_on_page_writeback(page); + if (length == thp_size(page)) { + truncate_inode_page(page->mapping, page); + return true; + } + + /* + * We may be zeroing pages we're about to discard, but it avoids + * doing a complex calculation here, and then doing the zeroing + * anyway if the page split fails. + */ + zero_user(page, offset, length); + + cleancache_invalidate_page(page->mapping, page); + if (page_has_private(page)) + do_invalidatepage(page, offset, length); + if (!PageTransHuge(page)) + return true; + return split_huge_page(page) == 0; +} + /* * Used to get rid of pages on hardware memory corruption. */ @@ -288,20 +335,16 @@ void truncate_inode_pages_range(struct address_space *mapping, { pgoff_t start; /* inclusive */ pgoff_t end; /* exclusive */ - unsigned int partial_start; /* inclusive */ - unsigned int partial_end; /* exclusive */ struct pagevec pvec; pgoff_t indices[PAGEVEC_SIZE]; pgoff_t index; int i; + struct page * page; + bool partial_end; if (mapping->nrpages == 0 && mapping->nrexceptional == 0) goto out; - /* Offsets within partial pages */ - partial_start = lstart & (PAGE_SIZE - 1); - partial_end = (lend + 1) & (PAGE_SIZE - 1); - /* * 'start' and 'end' always covers the range of pages to be fully * truncated. Partial pages are covered with 'partial_start' at the @@ -334,48 +377,35 @@ void truncate_inode_pages_range(struct address_space *mapping, cond_resched(); } - if (partial_start) { - struct page *page = find_lock_page(mapping, start - 1); - if (page) { - unsigned int top = PAGE_SIZE; - if (start > end) { - /* Truncation within a single page */ - top = partial_end; - partial_end = 0; - } - wait_on_page_writeback(page); - zero_user_segment(page, partial_start, top); - cleancache_invalidate_page(mapping, page); - if (page_has_private(page)) - do_invalidatepage(page, partial_start, - top - partial_start); - unlock_page(page); - put_page(page); + partial_end = ((lend + 1) % PAGE_SIZE) > 0; + page = find_lock_head(mapping, lstart >> PAGE_SHIFT); + if (page) { + bool same_page = lend < page_offset(page) + thp_size(page); + if (same_page) + partial_end = false; + if (!truncate_inode_partial_page(page, lstart, lend)) { + start = page->index + thp_nr_pages(page); + if (same_page) + end = page->index; } + unlock_page(page); + put_page(page); + page = NULL; } - if (partial_end) { - struct page *page = find_lock_page(mapping, end); - if (page) { - wait_on_page_writeback(page); - zero_user_segment(page, 0, partial_end); - cleancache_invalidate_page(mapping, page); - if (page_has_private(page)) - do_invalidatepage(page, 0, - partial_end); - unlock_page(page); - put_page(page); - } + + if (partial_end) + page = find_lock_head(mapping, end); + if (page) { + if (!truncate_inode_partial_page(page, lstart, lend)) + end = page->index; + unlock_page(page); + put_page(page); } - /* - * If the truncation happened within a single page no pages - * will be released, just zeroed, so we can bail out now. - */ - if (start >= end) - goto out; index = start; - for ( ; ; ) { + while (index < end) { cond_resched(); + if (!find_get_entries(mapping, index, end - 1, &pvec, indices)) { /* If all gone from start onwards, we're done */ @@ -387,7 +417,7 @@ void truncate_inode_pages_range(struct address_space *mapping, } for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; @@ -396,7 +426,7 @@ void truncate_inode_pages_range(struct address_space *mapping, continue; lock_page(page); - WARN_ON(page_to_index(page) != index); + index = page->index + thp_nr_pages(page) - 1; wait_on_page_writeback(page); truncate_inode_page(mapping, page); unlock_page(page); From patchwork Thu Nov 12 21:26:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11901701 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A43D697 for ; Thu, 12 Nov 2020 21:28:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A5AAB2223F for ; Thu, 12 Nov 2020 21:27:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="crwQX+of" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5AAB2223F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 26C3A6B0073; Thu, 12 Nov 2020 16:26:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A4146B0078; Thu, 12 Nov 2020 16:26:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E35846B007B; Thu, 12 Nov 2020 16:26:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id A5C506B0078 for ; Thu, 12 Nov 2020 16:26:56 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 54CFE1EE6 for ; Thu, 12 Nov 2020 21:26:56 +0000 (UTC) X-FDA: 77477051232.11.chalk75_420814927309 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 378C9180F8B81 for ; Thu, 12 Nov 2020 21:26:56 +0000 (UTC) X-Spam-Summary: 1,0,0,f0b41a0b8f35ee92,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2689:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4031:4117:4321:4605:5007:6119:6261:6653:8957:9592:10004:11026:11658:11914:12043:12296:12297:12438:12555:12683:12895:12986:13894:14096:14181:14394:14721:21063:21080:21324:21451:21627:21740:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yf7ddom3aw8rmnuws36jo4zai7doc3yb3yh3omj9h6s5axy35bawkk5yteux8.j437n7xs1dgsmbhcq9qnngbardm67ca86s9akkezckmbby83jfpkkob9fikx8we.4-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:205,LUA_SUMMARY:none X-HE-Tag: chalk75_420814927309 X-Filterd-Recvd-Size: 6353 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Thu, 12 Nov 2020 21:26:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=qbzmVFagUCXYcU5rqtSiOvPsAwsTj6kZyDh1iXpm+g8=; b=crwQX+ofqsb4dwJIDejthD3rlI DfM1w6S4JFiJhDoILV7BLJ8eA/dmU2eThfVU39XCsCt23dHYcQlwEWw1EIKOnK6lp1fDK9mQaElvX A9/hA1zBksSTeN0qL4XJIdgBUSS/CMxV6eqidkNLja4FT+ciQzk38m622FhtcyOc8Uoo/YguzCf4w +z1pPgjU0XhG/Zq5hcbrE3oQa7CkaK5hPqRk1gMLI3XldOygpSoHx/6diRSxUhNxWMuRd/7mdTdjZ ZndubeNT/0NmVYgvpWR1LSOzUS5QiIoySacMtm8rz+Yaepr/jS4xKWHfCs0d17D9qV6wj1dlZLwlC jFrOHkXg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kdK7H-0007Hx-A1; Thu, 12 Nov 2020 21:26:51 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , akpm@linux-foundation.org, hughd@google.com, hch@lst.de, hannes@cmpxchg.org, yang.shi@linux.alibaba.com, dchinner@redhat.com, linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v4 16/16] mm/filemap: Return only head pages from find_get_entries Date: Thu, 12 Nov 2020 21:26:41 +0000 Message-Id: <20201112212641.27837-17-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201112212641.27837-1-willy@infradead.org> References: <20201112212641.27837-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers now expect head (and base) pages, and can handle multiple head pages in a single batch, so make find_get_entries() behave that way. Also take the opportunity to make it use the pagevec infrastructure instead of open-coding how pvecs behave. This has the side-effect of being able to append to a pagevec with existing contents, although we don't make use of that functionality anywhere yet. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski Reviewed-by: Christoph Hellwig --- include/linux/pagemap.h | 2 -- mm/filemap.c | 36 ++++++++---------------------------- mm/internal.h | 2 ++ 3 files changed, 10 insertions(+), 30 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 46d4b1704770..65ef8db8eaab 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -448,8 +448,6 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) return head + (index & (thp_nr_pages(head) - 1)); } -unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index 479cbbadd93b..f8c294905e8d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1878,49 +1878,29 @@ static inline struct page *find_get_entry(struct xa_state *xas, pgoff_t max, * the mapping. The entries are placed in @pvec. find_get_entries() * takes a reference on any actual pages it returns. * - * The search returns a group of mapping-contiguous page cache entries - * with ascending indexes. There may be holes in the indices due to - * not-present pages. + * The entries have ascending indexes. The indices may not be consecutive + * due to not-present entries or THPs. * * Any shadow entries of evicted pages, or swap entries from * shmem/tmpfs, are included in the returned array. * - * If it finds a Transparent Huge Page, head or tail, find_get_entries() - * stops at that page: the caller is likely to have a better way to handle - * the compound page as a whole, and then skip its extent, than repeatedly - * calling find_get_entries() to return all its tails. - * - * Return: the number of pages and shadow entries which were found. + * Return: The number of entries which were found. */ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, pgoff_t end, struct pagevec *pvec, pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; - unsigned int ret = 0; - unsigned nr_entries = PAGEVEC_SIZE; rcu_read_lock(); while ((page = find_get_entry(&xas, end, XA_PRESENT))) { - /* - * Terminate early on finding a THP, to allow the caller to - * handle it all at once; but continue if this is hugetlbfs. - */ - if (!xa_is_value(page) && PageTransHuge(page) && - !PageHuge(page)) { - page = find_subpage(page, xas.xa_index); - nr_entries = ret + 1; - } - - indices[ret] = xas.xa_index; - pvec->pages[ret] = page; - if (++ret == nr_entries) + indices[pvec->nr] = xas.xa_index; + if (!pagevec_add(pvec, page)) break; } rcu_read_unlock(); - pvec->nr = ret; - return ret; + return pagevec_count(pvec); } /** @@ -1939,8 +1919,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, * not returned. * * The entries have ascending indexes. The indices may not be consecutive - * due to not-present entries, THP pages, pages which could not be locked - * or pages under writeback. + * due to not-present entries, THPs, pages which could not be locked or + * pages under writeback. * * Return: The number of entries which were found. */ diff --git a/mm/internal.h b/mm/internal.h index cb7487efa856..1f137a5d66bb 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -60,6 +60,8 @@ static inline void force_page_cache_readahead(struct address_space *mapping, force_page_cache_ra(&ractl, &file->f_ra, nr_to_read); } +unsigned find_get_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, pgoff_t end, struct pagevec *pvec, pgoff_t *indices);