From patchwork Mon Oct 26 04:13:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855655 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8E9961C for ; Mon, 26 Oct 2020 04:14:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 999CF2073A for ; Mon, 26 Oct 2020 04:14:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="p47NmDV4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 999CF2073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6AB966B0072; Mon, 26 Oct 2020 00:14:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 659DC6B0073; Mon, 26 Oct 2020 00:14:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DA6B6B0074; Mon, 26 Oct 2020 00:14:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 170EA6B0072 for ; Mon, 26 Oct 2020 00:14:23 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B18E98249980 for ; Mon, 26 Oct 2020 04:14:22 +0000 (UTC) X-FDA: 77412759564.05.stove16_1b185cd27270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 98BD61801AB42 for ; Mon, 26 Oct 2020 04:14:22 +0000 (UTC) X-Spam-Summary: 1,0,0,b2d3ed3392b1d8fd,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2553:2559:2562:2895:2898:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:5007:6119:6261:6653:7875:10004:11026:11658:11914:12043:12050:12296:12297:12438:12555:12895:13894:14181:14394:14721:21080:21324:21451:21627:21990:30034:30054:30055:30064:30070:30090:30091,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfkmhaej6r7dazugu33sqh1jdgoocrnehca1wuc9hids499hfu4d3zak9jhg3.cujy6umi1w77g9jxnds3yheqp34gu4u8s7wygmwicp489t1mfdpda3o579r5esp.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:27,LUA_SUMMARY:none X-HE-Tag: stove16_1b185cd27270 X-Filterd-Recvd-Size: 3899 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=S4uj6+LHbvxU3x1hyEQHv26ToiNAm0/3JPZsqUbH6YY=; b=p47NmDV4nDqyPquS0ChnyJEk42 OnWEUFsDTQ6ib7FbZtMxGRjnaQuuU02tA7CzkpNg6TxoIooBAGNqbPsZQb/z8UQWKhA2xJidAyZKp rif+IzLiDaht+LIyINpZaNbDUESpC1R5tvJnoU9frbWn7RiXFxk0qAZjlrm6cEJNasAeqWg0YIZts Sc7t0PsVthpztR+SAl3JAtpK5r1sZUA2sCK5O8Rt0rcfieHWWP0l5aLntEp5syE7w7JhuB/JwlGP/ 1/UmaG/cBKE5fxuNO0rvXUuRmTUDU0vwblPmjHEd+NH0iAmOy5bHVN5GkmtuPGs02aOvoPYEbEQiB c/ljcpuQ==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWtta-0006Zj-Fu; Mon, 26 Oct 2020 04:14:10 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 01/12] mm: Make pagecache tagged lookups return only head pages Date: Mon, 26 Oct 2020 04:13:57 +0000 Message-Id: <20201026041408.25230-2-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Pagecache tags are used for dirty page writeback. Since dirtiness is tracked on a per-THP basis, we only want to return the head page rather than each subpage of a tagged page. All the filesystems which use huge pages today are in-memory, so there are no tagged huge pages today. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- mm/filemap.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index d5e7c2029d16..edde5dc0d28f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2066,7 +2066,7 @@ unsigned find_get_pages_contig(struct address_space *mapping, pgoff_t index, EXPORT_SYMBOL(find_get_pages_contig); /** - * find_get_pages_range_tag - find and return pages in given range matching @tag + * find_get_pages_range_tag - Find and return head pages matching @tag. * @mapping: the address_space to search * @index: the starting page index * @end: The final page index (inclusive) @@ -2074,8 +2074,8 @@ EXPORT_SYMBOL(find_get_pages_contig); * @nr_pages: the maximum number of pages * @pages: where the resulting pages are placed * - * Like find_get_pages, except we only return pages which are tagged with - * @tag. We update @index to index the next page for the traversal. + * Like find_get_pages(), except we only return head pages which are tagged + * with @tag. We update @index to index the next page for the traversal. * * Return: the number of pages which were found. */ @@ -2109,9 +2109,9 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, if (unlikely(page != xas_reload(&xas))) goto put_page; - pages[ret] = find_subpage(page, xas.xa_index); + pages[ret] = page; if (++ret == nr_pages) { - *index = xas.xa_index + 1; + *index = page->index + thp_nr_pages(page); goto out; } continue; From patchwork Mon Oct 26 04:13:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855661 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C54261C for ; Mon, 26 Oct 2020 04:14:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 08B842224A for ; Mon, 26 Oct 2020 04:14:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qZPapEVv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 08B842224A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 746F36B0075; Mon, 26 Oct 2020 00:14:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6D0C56B007B; Mon, 26 Oct 2020 00:14:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56F076B007D; Mon, 26 Oct 2020 00:14:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0253.hostedemail.com [216.40.44.253]) by kanga.kvack.org (Postfix) with ESMTP id 24B536B0075 for ; Mon, 26 Oct 2020 00:14:35 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B3FBC180AD815 for ; Mon, 26 Oct 2020 04:14:34 +0000 (UTC) X-FDA: 77412760068.05.shake20_1f123e327270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 9C8501801AB42 for ; Mon, 26 Oct 2020 04:14:34 +0000 (UTC) X-Spam-Summary: 1,0,0,6fda1570b90d212c,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2553:2559:2562:2691:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3871:3872:3874:4321:5007:6261:6653:7996:10004:11026:11658:11914:12043:12114:12294:12296:12297:12438:12555:12895:12986:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21324:21451:21627:21987:21990:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfkw5wsgsn4nsr6sz1z9ibdo31qypfzs3974d5tn51qmtagjx5i5djc6ctebc.riy5iib3qquofj4pcsnrjku4axrf66g6za896bgknqtenw6zxpzxwsfdh9rdqk6.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: shake20_1f123e327270 X-Filterd-Recvd-Size: 3489 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=77sJtHQNYX1BOg5wgRb+5x5Ey0reVR4xYsh2AEFFWs0=; b=qZPapEVvLMGtwwMKEZdKqalBea fGV4x8P4djRsrOx3mNJZvcBgYPg7mNnZrIWZU99SlP5VJdUyYYS19diHZGGf3BeRSSI10jEDY1RpL Qe4GJy/AoXIsG8Xn/MVS6QggESPXQdgFmmhcxMEix+AQ1QMIktgxHpqURuUh7HIizZNRs5FH54Z52 L9707duJAdRnye1BdWuqfd448PP0dJm3Ld/JeLSVXprY+EJFh82ug/HxbyUpI9dk1BF62TkiK0wDO HQHBk9LWD8JxWGrWYo8ZxlzWOjxaX6kqYTWYu+tco4HwQLlKG3Gzk5jA1RsKzgCnq1AH4oyQytXab yFzhYOyg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWtta-0006Zn-NA; Mon, 26 Oct 2020 04:14:10 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 02/12] mm/shmem: Use pagevec_lookup in shmem_unlock_mapping Date: Mon, 26 Oct 2020 04:13:58 +0000 Message-Id: <20201026041408.25230-3-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The comment shows that the reason for using find_get_entries() is now stale; find_get_pages() will not return 0 if it hits a consecutive run of swap entries, and I don't believe it has since 2011. pagevec_lookup() is a simpler function to use than find_get_pages(), so use it instead. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- mm/shmem.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 537c137698f8..a33972126b60 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -842,7 +842,6 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma) void shmem_unlock_mapping(struct address_space *mapping) { struct pagevec pvec; - pgoff_t indices[PAGEVEC_SIZE]; pgoff_t index = 0; pagevec_init(&pvec); @@ -850,16 +849,8 @@ void shmem_unlock_mapping(struct address_space *mapping) * Minor point, but we might as well stop if someone else SHM_LOCKs it. */ while (!mapping_unevictable(mapping)) { - /* - * Avoid pagevec_lookup(): find_get_pages() returns 0 as if it - * has finished, if it hits a row of PAGEVEC_SIZE swap entries. - */ - pvec.nr = find_get_entries(mapping, index, - PAGEVEC_SIZE, pvec.pages, indices); - if (!pvec.nr) + if (!pagevec_lookup(&pvec, mapping, &index)) break; - index = indices[pvec.nr - 1] + 1; - pagevec_remove_exceptionals(&pvec); check_move_unevictable_pages(&pvec); pagevec_release(&pvec); cond_resched(); From patchwork Mon Oct 26 04:13:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2FB6361C for ; Mon, 26 Oct 2020 04:14:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E85CD20773 for ; Mon, 26 Oct 2020 04:14:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="QH09f85p" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E85CD20773 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C51DF6B0070; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BD7AC6B0071; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC8316B0072; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0223.hostedemail.com [216.40.44.223]) by kanga.kvack.org (Postfix) with ESMTP id 66D8C6B0071 for ; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 096E93624 for ; Mon, 26 Oct 2020 04:14:20 +0000 (UTC) X-FDA: 77412759480.10.brick94_12058a427270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id DD22B16A0BE for ; Mon, 26 Oct 2020 04:14:19 +0000 (UTC) X-Spam-Summary: 1,0,0,a0a35ccfc3b18052,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4117:4250:4321:5007:6261:6653:7875:8957:9592:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12683:12895:12986:13161:13229:13894:14110:14181:14394:14721:21080:21324:21451:21611:21627:21990:30012:30054:30055:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yfuy1h5eie3dpnfsaxohqhx6itxyc1c4kxfpx6yf6w5sb6ahcqaiag7huuf3e.xpai679dfzrfwrkfmh43ydr6zd55osyrodxzu181mkksasa9mieq4udzd9f71x8.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: brick94_12058a427270 X-Filterd-Recvd-Size: 6559 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=RRSaMGClXkysIFfJsu4KHWmeCgAPKYKCtm1cMXLEJ0o=; b=QH09f85pBq7GFer39wr1BLQzn7 iifWOhtWMlWu2buN18tedzflvnjTqtSHe70Jn9aXBFswbO6sdolkWn/8/sQ+KB//DLkzw5eMG8Zno QH/3PCrxtqPo22+Q2IVnnBpF6yidJwccpVDY4qnkK1SkeXfvzi4xO6yVLf1Ac9SEQEZ1lyhNxfuVh ndZ/K6pdtcJvjM5oPOcySCTLXVfb05clLySy9QGjVzSNxcyMvgm+RDlebZeBhtEDUgQ89fK4XjZ95 7LzaoEVGld9eurHp7DU9kYKPYCNeNwaHBze/SSrouPAL1h5XTJ2MpqkaCuKfBqfLv1J1vnz0wPikN Xlec8N/g==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWtta-0006Zr-SU; Mon, 26 Oct 2020 04:14:10 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 03/12] mm/filemap: Add helper for finding pages Date: Mon, 26 Oct 2020 04:13:59 +0000 Message-Id: <20201026041408.25230-4-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a lot of common code in find_get_entries(), find_get_pages_range() and find_get_pages_range_tag(). Factor out xas_find_get_entry() which simplifies all three functions. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- mm/filemap.c | 98 +++++++++++++++++++++++----------------------------- 1 file changed, 43 insertions(+), 55 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index edde5dc0d28f..00eaed59e797 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1856,6 +1856,43 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t index, } EXPORT_SYMBOL(pagecache_get_page); +static inline struct page *xas_find_get_entry(struct xa_state *xas, + pgoff_t max, xa_mark_t mark) +{ + struct page *page; + +retry: + if (mark == XA_PRESENT) + page = xas_find(xas, max); + else + page = xas_find_marked(xas, max, mark); + + if (xas_retry(xas, page)) + goto retry; + /* + * A shadow entry of a recently evicted page, a swap + * entry from shmem/tmpfs or a DAX entry. Return it + * without attempting to raise page count. + */ + if (!page || xa_is_value(page)) + return page; + + if (!page_cache_get_speculative(page)) + goto reset; + + /* Has the page moved or been split? */ + if (unlikely(page != xas_reload(xas))) { + put_page(page); + goto reset; + } + VM_BUG_ON_PAGE(!thp_contains(page, xas->xa_index), page); + + return page; +reset: + xas_reset(xas); + goto retry; +} + /** * find_get_entries - gang pagecache lookup * @mapping: The address_space to search @@ -1895,42 +1932,21 @@ unsigned find_get_entries(struct address_space *mapping, return 0; rcu_read_lock(); - xas_for_each(&xas, page, ULONG_MAX) { - if (xas_retry(&xas, page)) - continue; - /* - * A shadow entry of a recently evicted page, a swap - * entry from shmem/tmpfs or a DAX entry. Return it - * without attempting to raise page count. - */ - if (xa_is_value(page)) - goto export; - - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - + while ((page = xas_find_get_entry(&xas, ULONG_MAX, XA_PRESENT))) { /* * Terminate early on finding a THP, to allow the caller to * handle it all at once; but continue if this is hugetlbfs. */ - if (PageTransHuge(page) && !PageHuge(page)) { + if (!xa_is_value(page) && PageTransHuge(page) && + !PageHuge(page)) { page = find_subpage(page, xas.xa_index); nr_entries = ret + 1; } -export: + indices[ret] = xas.xa_index; entries[ret] = page; if (++ret == nr_entries) break; - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } rcu_read_unlock(); return ret; @@ -1969,30 +1985,16 @@ unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, return 0; rcu_read_lock(); - xas_for_each(&xas, page, end) { - if (xas_retry(&xas, page)) - continue; + while ((page = xas_find_get_entry(&xas, end, XA_PRESENT))) { /* Skip over shadow, swap and DAX entries */ if (xa_is_value(page)) continue; - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - pages[ret] = find_subpage(page, xas.xa_index); if (++ret == nr_pages) { *start = xas.xa_index + 1; goto out; } - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } /* @@ -2091,9 +2093,7 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, return 0; rcu_read_lock(); - xas_for_each_marked(&xas, page, end, tag) { - if (xas_retry(&xas, page)) - continue; + while ((page = xas_find_get_entry(&xas, end, tag))) { /* * Shadow entries should never be tagged, but this iteration * is lockless so there is a window for page reclaim to evict @@ -2102,23 +2102,11 @@ unsigned find_get_pages_range_tag(struct address_space *mapping, pgoff_t *index, if (xa_is_value(page)) continue; - if (!page_cache_get_speculative(page)) - goto retry; - - /* Has the page moved or been split? */ - if (unlikely(page != xas_reload(&xas))) - goto put_page; - pages[ret] = page; if (++ret == nr_pages) { *index = page->index + thp_nr_pages(page); goto out; } - continue; -put_page: - put_page(page); -retry: - xas_reset(&xas); } /* From patchwork Mon Oct 26 04:14:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855653 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27F2692C for ; Mon, 26 Oct 2020 04:14:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7664D2073A for ; Mon, 26 Oct 2020 04:14:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="SX+zpd7v" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7664D2073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 79A526B0071; Mon, 26 Oct 2020 00:14:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7217B6B0072; Mon, 26 Oct 2020 00:14:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 525126B0073; Mon, 26 Oct 2020 00:14:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id 1FAE46B0071 for ; Mon, 26 Oct 2020 00:14:22 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A952A181AC9CB for ; Mon, 26 Oct 2020 04:14:21 +0000 (UTC) X-FDA: 77412759522.15.alarm49_480b90327270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 8B61D1814B0C1 for ; Mon, 26 Oct 2020 04:14:21 +0000 (UTC) X-Spam-Summary: 1,0,0,ee95c1b82c441203,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4049:4118:4321:4605:5007:6261:6653:7875:8957:9036:9592:10004:11026:11473:11658:11914:12043:12114:12291:12296:12297:12438:12555:12683:12895:12986:13894:14096:14110:14394:21080:21324:21451:21627:21987:21990:30003:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yfyxz36juzyfbyu647bzgcs9pk8oc1n8m6n1i7whjbsantp6kddrcxjc6gwwk.gjz9do3161edho45zjguzgs1ujndaux1rc9rzsatpptmzihdho373bihje444d9.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: alarm49_480b90327270 X-Filterd-Recvd-Size: 7990 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=MWcn/Nhf9sR48o2zboUwMXLYP6nYp5d/8r8AJss6SUs=; b=SX+zpd7vntl7+TRHG3Xx0HdxWy dgdXvI69NjlzgIBoyJYaDRojtTlzb8QpoGhBvhButticsnPIDC6Qj1N+ubjszj8yreojIdux5fPsF 68dZNddppah/zu3uuk5v/X/z+PuA8UO4u4Vxw/2xZNhDdWuTe9DX4SXim55NaSC4aj4BbVDJrskqy C74ZbFSMvMfGZ3qFh4uVk1jthEu5YcMF+hPo4JrCbzaH6hTvH8zutOH2B4AinnN+iHojfpifcJKyC avZQQrk6u9Izv3EYCsyA67OdqUo9iNqX78jLTkSDKSykEeiMnGGCyI83XWlamuVCsd+Ekz43RTr/i ns9vugpg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttb-0006Zw-2E; Mon, 26 Oct 2020 04:14:11 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, William Kucharski Subject: [PATCH v3 04/12] mm/filemap: Add mapping_seek_hole_data Date: Mon, 26 Oct 2020 04:14:00 +0000 Message-Id: <20201026041408.25230-5-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rewrite shmem_seek_hole_data() and move it to filemap.c. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 76 +++++++++++++++++++++++++++++++++++++++++ mm/shmem.c | 72 +++----------------------------------- 3 files changed, 82 insertions(+), 68 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index c77b7c31b2e4..5f3e829c91fd 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -760,6 +760,8 @@ extern void __delete_from_page_cache(struct page *page, void *shadow); int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask); void delete_from_page_cache_batch(struct address_space *mapping, struct pagevec *pvec); +loff_t mapping_seek_hole_data(struct address_space *, loff_t start, loff_t end, + int whence); /* * Like add_to_page_cache_locked, but used to add newly allocated pages: diff --git a/mm/filemap.c b/mm/filemap.c index 00eaed59e797..3a55d258d9f2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2526,6 +2526,82 @@ generic_file_read_iter(struct kiocb *iocb, struct iov_iter *iter) } EXPORT_SYMBOL(generic_file_read_iter); +static inline loff_t page_seek_hole_data(struct page *page, + loff_t start, loff_t end, bool seek_data) +{ + if (xa_is_value(page) || PageUptodate(page)) + return seek_data ? start : end; + return seek_data ? end : start; +} + +static inline +unsigned int seek_page_size(struct xa_state *xas, struct page *page) +{ + if (xa_is_value(page)) + return PAGE_SIZE << xa_get_order(xas->xa, xas->xa_index); + return thp_size(page); +} + +/** + * mapping_seek_hole_data - Seek for SEEK_DATA / SEEK_HOLE in the page cache. + * @mapping: Address space to search. + * @start: First byte to consider. + * @end: Limit of search (exclusive). + * @whence: Either SEEK_HOLE or SEEK_DATA. + * + * If the page cache knows which blocks contain holes and which blocks + * contain data, your filesystem can use this function to implement + * SEEK_HOLE and SEEK_DATA. This is useful for filesystems which are + * entirely memory-based such as tmpfs, and filesystems which support + * unwritten extents. + * + * Return: The requested offset on successs, or -ENXIO if @whence specifies + * SEEK_DATA and there is no data after @start. There is an implicit hole + * after @end - 1, so SEEK_HOLE returns @end if all the bytes between @start + * and @end contain data. + */ +loff_t mapping_seek_hole_data(struct address_space *mapping, loff_t start, + loff_t end, int whence) +{ + XA_STATE(xas, &mapping->i_pages, start >> PAGE_SHIFT); + pgoff_t max = (end - 1) / PAGE_SIZE; + bool seek_data = (whence == SEEK_DATA); + struct page *page; + + if (end <= start) + return -ENXIO; + + rcu_read_lock(); + while ((page = xas_find_get_entry(&xas, max, XA_PRESENT))) { + loff_t pos = xas.xa_index * PAGE_SIZE; + + if (start < pos) { + if (!seek_data) + goto unlock; + start = pos; + } + + pos += seek_page_size(&xas, page); + start = page_seek_hole_data(page, start, pos, seek_data); + if (start < pos) + goto unlock; + } + rcu_read_unlock(); + + if (seek_data) + return -ENXIO; + goto out; + +unlock: + rcu_read_unlock(); + if (!xa_is_value(page)) + put_page(page); +out: + if (start > end) + return end; + return start; +} + #ifdef CONFIG_MMU #define MMAP_LOTSAMISS (100) /* diff --git a/mm/shmem.c b/mm/shmem.c index a33972126b60..726cede653f5 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2670,85 +2670,21 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return retval ? retval : error; } -/* - * llseek SEEK_DATA or SEEK_HOLE through the page cache. - */ -static pgoff_t shmem_seek_hole_data(struct address_space *mapping, - pgoff_t index, pgoff_t end, int whence) -{ - struct page *page; - struct pagevec pvec; - pgoff_t indices[PAGEVEC_SIZE]; - bool done = false; - int i; - - pagevec_init(&pvec); - pvec.nr = 1; /* start small: we may be there already */ - while (!done) { - pvec.nr = find_get_entries(mapping, index, - pvec.nr, pvec.pages, indices); - if (!pvec.nr) { - if (whence == SEEK_DATA) - index = end; - break; - } - for (i = 0; i < pvec.nr; i++, index++) { - if (index < indices[i]) { - if (whence == SEEK_HOLE) { - done = true; - break; - } - index = indices[i]; - } - page = pvec.pages[i]; - if (page && !xa_is_value(page)) { - if (!PageUptodate(page)) - page = NULL; - } - if (index >= end || - (page && whence == SEEK_DATA) || - (!page && whence == SEEK_HOLE)) { - done = true; - break; - } - } - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - pvec.nr = PAGEVEC_SIZE; - cond_resched(); - } - return index; -} - static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) { struct address_space *mapping = file->f_mapping; struct inode *inode = mapping->host; - pgoff_t start, end; - loff_t new_offset; if (whence != SEEK_DATA && whence != SEEK_HOLE) return generic_file_llseek_size(file, offset, whence, MAX_LFS_FILESIZE, i_size_read(inode)); + if (offset < 0) + return -ENXIO; + inode_lock(inode); /* We're holding i_mutex so we can access i_size directly */ - if (offset < 0 || offset >= inode->i_size) - offset = -ENXIO; - else { - start = offset >> PAGE_SHIFT; - end = (inode->i_size + PAGE_SIZE - 1) >> PAGE_SHIFT; - new_offset = shmem_seek_hole_data(mapping, start, end, whence); - new_offset <<= PAGE_SHIFT; - if (new_offset > offset) { - if (new_offset < inode->i_size) - offset = new_offset; - else if (whence == SEEK_DATA) - offset = -ENXIO; - else - offset = inode->i_size; - } - } + offset = mapping_seek_hole_data(mapping, offset, inode->i_size, whence); if (offset >= 0) offset = vfs_setpos(file, offset, MAX_LFS_FILESIZE); From patchwork Mon Oct 26 04:14:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855659 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43F5061C for ; Mon, 26 Oct 2020 04:14:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F3DFE2224A for ; Mon, 26 Oct 2020 04:14:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="EGs1p8sF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3DFE2224A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 41D6F6B0073; Mon, 26 Oct 2020 00:14:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1F05F6B0075; Mon, 26 Oct 2020 00:14:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DA0F6B007B; Mon, 26 Oct 2020 00:14:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id BE1156B0074 for ; Mon, 26 Oct 2020 00:14:27 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 73F6E3624 for ; Mon, 26 Oct 2020 04:14:27 +0000 (UTC) X-FDA: 77412759774.12.flesh31_1617bf027270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 57C0E1800EC4D for ; Mon, 26 Oct 2020 04:14:27 +0000 (UTC) X-Spam-Summary: 1,0,0,5a207f2c6b303d75,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:1:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2393:2553:2559:2562:2693:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4051:4250:4321:5007:6119:6261:6653:7875:8957:9592:10004:11026:11232:11473:11658:11914:12043:12291:12296:12297:12438:12555:12683:12895:12986:13894:14096:14394:21063:21080:21324:21451:21627:21987:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y8ihx3ugyx9915t8t3miipsui4rycpjnn6hjxj7myjx74w6rjpmjfqh6udn96.j9tgrkrwm4w1qssaru5b5yak1ji48amsbzj4r4cnokybf85rjw9m6ri4nkoafn9.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: flesh31_1617bf027270 X-Filterd-Recvd-Size: 10741 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=D4HUkF63j7GrFkU84geEXagljZ4wF1EtZdc5p5/p3pw=; b=EGs1p8sFgOKa6qgZad9xlY0Cju i1oJNd/8uuYAxaImSQmGz4DGuVqT09wW762qsH/nnroe2ZS0R1o26nrzOdp2FVfx0VHNWzG/8TlRm jwXNvNwf+Xe6k88yOc5VJ6DpiuZZdSJz1zVVzZvaQfxSVH8V8ZVeFfo4QSDQVwPjuSlUdB+skjl7U dIw0m+gE/vDoQz03YAYSi0TIc/dV1zgAEeFTak4YvqwZTzww0ZpUm1dxlUCQuyDD9YY2S1kpQG7OY WA8lbigI6x04OVb29uo675bCb7Yt+3yKRDBV3RQWa0/1XOTbTD03ESIHQKtDDtXsFBtrvj0uDXmCN eDr4RiRA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttb-0006a3-8x; Mon, 26 Oct 2020 04:14:11 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 05/12] mm: Add and use find_lock_entries Date: Mon, 26 Oct 2020 04:14:01 +0000 Message-Id: <20201026041408.25230-6-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We have three functions (shmem_undo_range(), truncate_inode_pages_range() and invalidate_mapping_pages()) which want exactly this function, so add it to filemap.c. Before this patch, shmem_undo_range() would split any compound page which overlaps either end of the range being punched in both the first and second loops through the address space. After this patch, that functionality is left for the second loop, which is arguably more appropriate since the first loop is supposed to run through all the pages quickly, and splitting a page can sleep. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- mm/filemap.c | 57 ++++++++++++++++++++++++++++++++ mm/internal.h | 2 ++ mm/shmem.c | 22 +++---------- mm/truncate.c | 91 +++++++-------------------------------------------- 4 files changed, 75 insertions(+), 97 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3a55d258d9f2..9a33d1b8cef6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1952,6 +1952,63 @@ unsigned find_get_entries(struct address_space *mapping, return ret; } +/** + * find_lock_entries - Find a batch of pagecache entries. + * @mapping: The address_space to search. + * @start: The starting page cache index. + * @end: The final page index (inclusive). + * @pvec: Where the resulting entries are placed. + * @indices: The cache indices of the entries in @pvec. + * + * find_lock_entries() will return a batch of entries from @mapping. + * Swap, shadow and DAX entries are included. Pages are returned + * locked and with an incremented refcount. Pages which are locked by + * somebody else or under writeback are skipped. Only the head page of + * a THP is returned. Pages which are partially outside the range are + * not returned. + * + * The entries have ascending indexes. The indices may not be consecutive + * due to not-present entries, THP pages, pages which could not be locked + * or pages under writeback. + * + * Return: The number of entries which were found. + */ +unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices) +{ + XA_STATE(xas, &mapping->i_pages, start); + struct page *page; + + rcu_read_lock(); + while ((page = xas_find_get_entry(&xas, end, XA_PRESENT))) { + if (!xa_is_value(page)) { + if (page->index < start) + goto put; + VM_BUG_ON_PAGE(page->index != xas.xa_index, page); + if (page->index + thp_nr_pages(page) - 1 > end) + goto put; + if (!trylock_page(page)) + goto put; + if (page->mapping != mapping || PageWriteback(page)) + goto unlock; + } + indices[pvec->nr] = xas.xa_index; + if (!pagevec_add(pvec, page)) + break; + goto next; +unlock: + unlock_page(page); +put: + put_page(page); +next: + if (!xa_is_value(page) && PageTransHuge(page)) + xas_set(&xas, page->index + thp_nr_pages(page)); + } + rcu_read_unlock(); + + return pagevec_count(pvec); +} + /** * find_get_pages_range - gang pagecache lookup * @mapping: The address_space to search diff --git a/mm/internal.h b/mm/internal.h index c43ccdddb0f6..8d79f4d21eaf 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -62,6 +62,8 @@ static inline void force_page_cache_readahead(struct address_space *mapping, struct page *find_get_entry(struct address_space *mapping, pgoff_t index); struct page *find_lock_entry(struct address_space *mapping, pgoff_t index); +unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); /** * page_evictable - test whether a page is evictable diff --git a/mm/shmem.c b/mm/shmem.c index 726cede653f5..ef34271cad2d 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -907,12 +907,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, pagevec_init(&pvec); index = start; - while (index < end) { - pvec.nr = find_get_entries(mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - pvec.pages, indices); - if (!pvec.nr) - break; + while (index < end && find_lock_entries(mapping, index, end - 1, + &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; @@ -927,18 +923,10 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, index, page); continue; } + index += thp_nr_pages(page) - 1; - VM_BUG_ON_PAGE(page_to_pgoff(page) != index, page); - - if (!trylock_page(page)) - continue; - - if ((!unfalloc || !PageUptodate(page)) && - page_mapping(page) == mapping) { - VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) - truncate_inode_page(mapping, page); - } + if (!unfalloc || !PageUptodate(page)) + truncate_inode_page(mapping, page); unlock_page(page); } pagevec_remove_exceptionals(&pvec); diff --git a/mm/truncate.c b/mm/truncate.c index 18cec39a9f53..3c6b6d5a0046 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -326,51 +326,19 @@ void truncate_inode_pages_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (index < end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - indices)) { - /* - * Pagevec array has exceptional entries and we may also fail - * to lock some pages. So we store pages that can be deleted - * in a new pagevec. - */ - struct pagevec locked_pvec; - - pagevec_init(&locked_pvec); - for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; - - /* We rely upon deletion not changing page->index */ - index = indices[i]; - if (index >= end) - break; - - if (xa_is_value(page)) - continue; - - if (!trylock_page(page)) - continue; - WARN_ON(page_to_index(page) != index); - if (PageWriteback(page)) { - unlock_page(page); - continue; - } - if (page->mapping != mapping) { - unlock_page(page); - continue; - } - pagevec_add(&locked_pvec, page); - } - for (i = 0; i < pagevec_count(&locked_pvec); i++) - truncate_cleanup_page(mapping, locked_pvec.pages[i]); - delete_from_page_cache_batch(mapping, &locked_pvec); - for (i = 0; i < pagevec_count(&locked_pvec); i++) - unlock_page(locked_pvec.pages[i]); + while (index < end && find_lock_entries(mapping, index, end - 1, + &pvec, indices)) { + index = indices[pagevec_count(&pvec) - 1] + 1; truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + for (i = 0; i < pagevec_count(&pvec); i++) + truncate_cleanup_page(mapping, pvec.pages[i]); + delete_from_page_cache_batch(mapping, &pvec); + for (i = 0; i < pagevec_count(&pvec); i++) + unlock_page(pvec.pages[i]); pagevec_release(&pvec); cond_resched(); - index++; } + if (partial_start) { struct page *page = find_lock_page(mapping, start - 1); if (page) { @@ -539,9 +507,7 @@ unsigned long __invalidate_mapping_pages(struct address_space *mapping, int i; pagevec_init(&pvec); - while (index <= end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1, - indices)) { + while (find_lock_entries(mapping, index, end, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; @@ -555,39 +521,7 @@ unsigned long __invalidate_mapping_pages(struct address_space *mapping, page); continue; } - - if (!trylock_page(page)) - continue; - - WARN_ON(page_to_index(page) != index); - - /* Middle of THP: skip */ - if (PageTransTail(page)) { - unlock_page(page); - continue; - } else if (PageTransHuge(page)) { - index += HPAGE_PMD_NR - 1; - i += HPAGE_PMD_NR - 1; - /* - * 'end' is in the middle of THP. Don't - * invalidate the page as the part outside of - * 'end' could be still useful. - */ - if (index > end) { - unlock_page(page); - continue; - } - - /* Take a pin outside pagevec */ - get_page(page); - - /* - * Drop extra pins before trying to invalidate - * the huge page. - */ - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - } + index += thp_nr_pages(page) - 1; ret = invalidate_inode_page(page); unlock_page(page); @@ -601,9 +535,6 @@ unsigned long __invalidate_mapping_pages(struct address_space *mapping, if (nr_pagevec) (*nr_pagevec)++; } - - if (PageTransHuge(page)) - put_page(page); count += ret; } pagevec_remove_exceptionals(&pvec); From patchwork Mon Oct 26 04:14:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855663 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2A6792C for ; Mon, 26 Oct 2020 04:14:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 52F592225C for ; Mon, 26 Oct 2020 04:14:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YP8CA8T0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 52F592225C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 990FF6B0074; Mon, 26 Oct 2020 00:14:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9196C6B0075; Mon, 26 Oct 2020 00:14:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B9076B007B; Mon, 26 Oct 2020 00:14:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 457FD6B0074 for ; Mon, 26 Oct 2020 00:14:30 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D6FA58249980 for ; Mon, 26 Oct 2020 04:14:29 +0000 (UTC) X-FDA: 77412759858.26.bell57_231687d27270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id BA34D1804B65A for ; Mon, 26 Oct 2020 04:14:29 +0000 (UTC) X-Spam-Summary: 1,0,0,b26e4b2f40366fe7,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1711:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:4117:4321:4605:5007:6261:6653:7875:9592:10004:11026:11658:11914:12043:12296:12297:12438:12555:12895:12986:13894:14181:14394:14721:21080:21324:21433:21451:21627:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yr79b3pjhmrdxe4q4bassajcw4qocmzoeyzjpz4jx64q4m1pdhymf6ews1z5n.dgfjyxdnmxoq67p1hooq9gajtyq4cqtzkped8abinffz1ub6ksa7hjq5i7ohpsj.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: bell57_231687d27270 X-Filterd-Recvd-Size: 6140 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=DLWB4TNbNLy+vkO6OI9SCJlRakguX3c3DViMXWpue3Y=; b=YP8CA8T0rij4qO56RnbwxvhtfQ Ot5Cy3FbS062Jt0im9h7XCoMlLQkxBkqf4IWbY+x8qNyVKsWQhEytO4Q+86JuHwy9CnyUtck2xTBi SZ5C7nI+T9ysi5m+qSddHtAdsSEGvQNKqLaAowWFBxboDf3PgY8mREObTilfMMNhYV5vVrarkqX/g lPtWTaSjQjUkABJZx9Jo69qy+ExvRgnzOanbG3AZ1KkSrAKBQ52HdqIHfEOjSQ5lDCqwzVKsZW+d9 1clJATtpMkWPRXo+cRNEMhuvJSS2+YAQu8+9Tw1bxPPubxcaVpI3CDjH+cSb/v7oJe9SxOpuiuRjR B/tZX1CA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttb-0006a6-Eu; Mon, 26 Oct 2020 04:14:11 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 06/12] mm: Add an 'end' parameter to find_get_entries Date: Mon, 26 Oct 2020 04:14:02 +0000 Message-Id: <20201026041408.25230-7-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This simplifies the callers and leads to a more efficient implementation since the XArray has this functionality already. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagemap.h | 4 ++-- mm/filemap.c | 9 +++++---- mm/shmem.c | 10 ++-------- mm/swap.c | 2 +- 4 files changed, 10 insertions(+), 15 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 5f3e829c91fd..5b425f666bc5 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -450,8 +450,8 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) } unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - unsigned int nr_entries, struct page **entries, - pgoff_t *indices); + pgoff_t end, unsigned int nr_entries, struct page **entries, + pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index 9a33d1b8cef6..6ed2422426d2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1897,6 +1897,7 @@ static inline struct page *xas_find_get_entry(struct xa_state *xas, * find_get_entries - gang pagecache lookup * @mapping: The address_space to search * @start: The starting page cache index + * @end: The final page index (inclusive). * @nr_entries: The maximum number of entries * @entries: Where the resulting entries are placed * @indices: The cache indices corresponding to the entries in @entries @@ -1920,9 +1921,9 @@ static inline struct page *xas_find_get_entry(struct xa_state *xas, * * Return: the number of pages and shadow entries which were found. */ -unsigned find_get_entries(struct address_space *mapping, - pgoff_t start, unsigned int nr_entries, - struct page **entries, pgoff_t *indices) +unsigned find_get_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, unsigned int nr_entries, struct page **entries, + pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; @@ -1932,7 +1933,7 @@ unsigned find_get_entries(struct address_space *mapping, return 0; rcu_read_lock(); - while ((page = xas_find_get_entry(&xas, ULONG_MAX, XA_PRESENT))) { + while ((page = xas_find_get_entry(&xas, end, XA_PRESENT))) { /* * Terminate early on finding a THP, to allow the caller to * handle it all at once; but continue if this is hugetlbfs. diff --git a/mm/shmem.c b/mm/shmem.c index ef34271cad2d..27b93b738ea0 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -913,8 +913,6 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct page *page = pvec.pages[i]; index = indices[i]; - if (index >= end) - break; if (xa_is_value(page)) { if (unfalloc) @@ -967,9 +965,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end) { cond_resched(); - pvec.nr = find_get_entries(mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), - pvec.pages, indices); + pvec.nr = find_get_entries(mapping, index, end - 1, + PAGEVEC_SIZE, pvec.pages, indices); if (!pvec.nr) { /* If all gone or hole-punch or unfalloc, we're done */ if (index == start || end != -1) @@ -982,9 +979,6 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct page *page = pvec.pages[i]; index = indices[i]; - if (index >= end) - break; - if (xa_is_value(page)) { if (unfalloc) continue; diff --git a/mm/swap.c b/mm/swap.c index 47a47681c86b..9b0836cda971 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1099,7 +1099,7 @@ unsigned pagevec_lookup_entries(struct pagevec *pvec, pgoff_t start, unsigned nr_entries, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, nr_entries, + pvec->nr = find_get_entries(mapping, start, ULONG_MAX, nr_entries, pvec->pages, indices); return pagevec_count(pvec); } From patchwork Mon Oct 26 04:14:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855643 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0432A1130 for ; Mon, 26 Oct 2020 04:14:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 94D752073A for ; Mon, 26 Oct 2020 04:14:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="NBdVZ0Im" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94D752073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 12C796B0062; Mon, 26 Oct 2020 00:14:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0B2CD6B0068; Mon, 26 Oct 2020 00:14:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0CA16B006C; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id C51BD6B0062 for ; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5C7103624 for ; Mon, 26 Oct 2020 04:14:17 +0000 (UTC) X-FDA: 77412759354.27.teeth37_6104df727270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 4356C3D67F for ; Mon, 26 Oct 2020 04:14:17 +0000 (UTC) X-Spam-Summary: 1,0,0,324c86e043ef2881,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1730:1747:1777:1792:2393:2553:2559:2562:2898:3138:3139:3140:3141:3142:3503:3504:3865:3866:3867:3868:3870:3874:4049:4119:4321:4605:5007:6261:6653:7875:7903:9592:10004:11026:11658:11914:12043:12114:12296:12297:12438:12555:12683:12895:12986:13894:14394:21080:21324:21433:21451:21627:21987:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y87ndpcukb367xu646g8cmr1gm1ocu5o7fp9ahieybdjen5e5uudkp3ukngcu.tcckozdxkrgszzeuwqu1q91bx5aoxrnpz3nuy8w4z5hf3cxwqehne6en4sqawzz.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: teeth37_6104df727270 X-Filterd-Recvd-Size: 8522 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=e4FXtqfuCUGViBP5Rkju2Ey7iwc/YYNQDs91qQkDRLc=; b=NBdVZ0Ima1mHOQuNzV/DpcWYz/ 8lYx9OlJF0IZgo0p68XInbUT9vxN1gcZI9zu3O/1hcU35DvrqGQoREmihU2H/keME/NeAEWFLSN4d 3bpqt0mMTbkE7xy/O8DBfktx47FU9YzzVI9b7fVz+KceZBFNPNM/TFhzpZ88L1cvXOjDSHPpLSgE5 G6d2XcExIMZPrNKCGES2wSHzfNMPMhj6I5mSuUK5D8DS0K3syuo6oqZmHXT+4AfnYfpiWH+OGi2E4 r3Q9wR3LBRItMmOiFQDoo98066pMXHqfWYNvbJFqklGpHbJgNZTJB/voJDQ4YBO2KrVR3FAI2uSV5 IUzQHBJg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttb-0006aA-Ll; Mon, 26 Oct 2020 04:14:11 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 07/12] mm: Add an 'end' parameter to pagevec_lookup_entries Date: Mon, 26 Oct 2020 04:14:03 +0000 Message-Id: <20201026041408.25230-8-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simplifies the callers and uses the existing functionality in find_get_entries(). We can also drop the final argument of truncate_exceptional_pvec_entries() and simplify the logic in that function. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagevec.h | 5 ++--- mm/swap.c | 8 ++++---- mm/truncate.c | 41 ++++++++++------------------------------- 3 files changed, 16 insertions(+), 38 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index 081d934eda64..4b245592262c 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -26,9 +26,8 @@ struct pagevec { void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, - pgoff_t start, unsigned nr_entries, - pgoff_t *indices); + struct address_space *mapping, pgoff_t start, pgoff_t end, + unsigned nr_entries, pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index 9b0836cda971..d4d0c54d6ec9 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1075,6 +1075,7 @@ void __pagevec_lru_add(struct pagevec *pvec) * @pvec: Where the resulting entries are placed * @mapping: The address_space to search * @start: The starting entry index + * @end: The highest index to return (inclusive). * @nr_entries: The maximum number of pages * @indices: The cache indices corresponding to the entries in @pvec * @@ -1095,11 +1096,10 @@ void __pagevec_lru_add(struct pagevec *pvec) * found. */ unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, - pgoff_t start, unsigned nr_entries, - pgoff_t *indices) + struct address_space *mapping, pgoff_t start, pgoff_t end, + unsigned nr_entries, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, ULONG_MAX, nr_entries, + pvec->nr = find_get_entries(mapping, start, end, nr_entries, pvec->pages, indices); return pagevec_count(pvec); } diff --git a/mm/truncate.c b/mm/truncate.c index 3c6b6d5a0046..ec43312f4756 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -57,11 +57,10 @@ static void clear_shadow_entry(struct address_space *mapping, pgoff_t index, * exceptional entries similar to what pagevec_remove_exceptionals does. */ static void truncate_exceptional_pvec_entries(struct address_space *mapping, - struct pagevec *pvec, pgoff_t *indices, - pgoff_t end) + struct pagevec *pvec, pgoff_t *indices) { int i, j; - bool dax, lock; + bool dax; /* Handled by shmem itself */ if (shmem_mapping(mapping)) @@ -75,8 +74,7 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, return; dax = dax_mapping(mapping); - lock = !dax && indices[j] < end; - if (lock) + if (!dax) xa_lock_irq(&mapping->i_pages); for (i = j; i < pagevec_count(pvec); i++) { @@ -88,9 +86,6 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, continue; } - if (index >= end) - continue; - if (unlikely(dax)) { dax_delete_mapping_entry(mapping, index); continue; @@ -99,7 +94,7 @@ static void truncate_exceptional_pvec_entries(struct address_space *mapping, __clear_shadow_entry(mapping, index, page); } - if (lock) + if (!dax) xa_unlock_irq(&mapping->i_pages); pvec->nr = j; } @@ -329,7 +324,7 @@ void truncate_inode_pages_range(struct address_space *mapping, while (index < end && find_lock_entries(mapping, index, end - 1, &pvec, indices)) { index = indices[pagevec_count(&pvec) - 1] + 1; - truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + truncate_exceptional_pvec_entries(mapping, &pvec, indices); for (i = 0; i < pagevec_count(&pvec); i++) truncate_cleanup_page(mapping, pvec.pages[i]); delete_from_page_cache_batch(mapping, &pvec); @@ -381,8 +376,8 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; for ( ; ; ) { cond_resched(); - if (!pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE), indices)) { + if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, + PAGEVEC_SIZE, indices)) { /* If all gone from start onwards, we're done */ if (index == start) break; @@ -390,23 +385,12 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; continue; } - if (index == start && indices[0] >= end) { - /* All gone out of hole to be punched, we're done */ - pagevec_remove_exceptionals(&pvec); - pagevec_release(&pvec); - break; - } for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index >= end) { - /* Restart punch to make sure all gone */ - index = start - 1; - break; - } if (xa_is_value(page)) continue; @@ -417,7 +401,7 @@ void truncate_inode_pages_range(struct address_space *mapping, truncate_inode_page(mapping, page); unlock_page(page); } - truncate_exceptional_pvec_entries(mapping, &pvec, indices, end); + truncate_exceptional_pvec_entries(mapping, &pvec, indices); pagevec_release(&pvec); index++; } @@ -513,8 +497,6 @@ unsigned long __invalidate_mapping_pages(struct address_space *mapping, /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index > end) - break; if (xa_is_value(page)) { invalidate_exceptional_entry(mapping, index, @@ -650,16 +632,13 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (index <= end && pagevec_lookup_entries(&pvec, mapping, index, - min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1, - indices)) { + while (pagevec_lookup_entries(&pvec, mapping, index, end, + PAGEVEC_SIZE, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; - if (index > end) - break; if (xa_is_value(page)) { if (!invalidate_exceptional_entry2(mapping, From patchwork Mon Oct 26 04:14:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855647 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69B0861C for ; Mon, 26 Oct 2020 04:14:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 199FB2073A for ; Mon, 26 Oct 2020 04:14:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Jsdh2v+F" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 199FB2073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 96C276B006C; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8EF9B6B006E; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 790DF6B0070; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 48BB16B006E for ; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C51E08249980 for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) X-FDA: 77412759396.07.verse29_250b96927270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id AEAFE1803F9A1 for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) X-Spam-Summary: 1,0,0,ca6e90fe9acc0667,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1542:1711:1730:1747:1777:1792:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:4321:4605:5007:6261:6653:8603:10004:11026:11232:11658:11914:12043:12114:12296:12297:12438:12555:12895:12986:13894:14181:14394:14721:21080:21324:21451:21627:21987:21990:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04ygeufqh3w34gss7g73ebu8ax3kiyp4kdw3sttg86eyikp9tnbzik78tqcm4np.wghkzop57krj8su98q816g3owqbxzgs5hpmwbrmnezsoix1t7ksmmk3yrn74h4h.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: verse29_250b96927270 X-Filterd-Recvd-Size: 4310 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ETX87MTgKWaxDq/ZgR/LhP6+qskX119fCC7yru+WEmE=; b=Jsdh2v+F+pr6rgZoxRjGGsj5wV D1Td4x3LapVeO8HSYlNht5MphssVq8ny0Pt9tghNDTP6FeZVBDhLnjHXhFHPb2wff77Oy4a+U0S4X YHzrBB/798krwPh+9IGmFunPMM4ufXVbmQ7tTMkyBRuD85Sy5Bi79XCpPvxuuwAvJ9rwGo7nUx//B xAkXq7yO8jKcyuWfKIKENtvGDiaILEk5z9G6Kih/u34Qsq28IxnZ0zvxP6Gk9/V5gHDeCKDbCmcCz kec30sF+PfzSJs4NT956c7x27kGFQjQXTcMwn82f+aby7dBe61PTclt5uQ8+hFx8R7kCJ/wbgMvje Kbka90Pw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttb-0006aO-US; Mon, 26 Oct 2020 04:14:11 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 08/12] mm: Remove nr_entries parameter from pagevec_lookup_entries Date: Mon, 26 Oct 2020 04:14:04 +0000 Message-Id: <20201026041408.25230-9-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers want to fetch the full size of the pvec. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagevec.h | 2 +- mm/swap.c | 4 ++-- mm/truncate.c | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index 4b245592262c..ce77724a2ab7 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -27,7 +27,7 @@ void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, - unsigned nr_entries, pgoff_t *indices); + pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index d4d0c54d6ec9..31653152f55d 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1097,9 +1097,9 @@ void __pagevec_lru_add(struct pagevec *pvec) */ unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, - unsigned nr_entries, pgoff_t *indices) + pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, end, nr_entries, + pvec->nr = find_get_entries(mapping, start, end, PAGEVEC_SIZE, pvec->pages, indices); return pagevec_count(pvec); } diff --git a/mm/truncate.c b/mm/truncate.c index ec43312f4756..c4a3c9d6a0c1 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -377,7 +377,7 @@ void truncate_inode_pages_range(struct address_space *mapping, for ( ; ; ) { cond_resched(); if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, - PAGEVEC_SIZE, indices)) { + indices)) { /* If all gone from start onwards, we're done */ if (index == start) break; @@ -632,8 +632,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (pagevec_lookup_entries(&pvec, mapping, index, end, - PAGEVEC_SIZE, indices)) { + while (pagevec_lookup_entries(&pvec, mapping, index, end, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; From patchwork Mon Oct 26 04:14:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855677 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 376C6139F for ; Mon, 26 Oct 2020 04:31:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9ED0207C4 for ; Mon, 26 Oct 2020 04:31:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="GoZMtAwS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9ED0207C4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 25F266B0062; Mon, 26 Oct 2020 00:31:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1E7526B0068; Mon, 26 Oct 2020 00:31:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0891B6B006C; Mon, 26 Oct 2020 00:31:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CE0FE6B0062 for ; Mon, 26 Oct 2020 00:31:33 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5F6E3180AD815 for ; Mon, 26 Oct 2020 04:31:33 +0000 (UTC) X-FDA: 77412802866.18.bun70_290127727270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 451CC100ED0E0 for ; Mon, 26 Oct 2020 04:31:33 +0000 (UTC) X-Spam-Summary: 1,0,0,a7a7f3d284943637,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:421:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1711:1730:1747:1777:1792:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3872:4117:4321:4605:5007:6261:6653:9592:10004:11026:11232:11658:11914:12043:12296:12297:12438:12555:12895:12986:13894:14096:14181:14394:14721:21080:21324:21433:21451:21627:21796:21990:30034:30036:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04yg75whgjwg9nbz8c8u9igg46m1soc4fb1u5wftrqpiznk8epdgywjr5878msp.5yqbyftiajrm1z88hrzeoo49smtfrcqknzq3x71okg5moqxycoxdu4n9o6yjo8k.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: bun70_290127727270 X-Filterd-Recvd-Size: 6174 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TvkNUGXQnu6homFCzqrBDp2K7EKV5Qz0vG6C5SIadzM=; b=GoZMtAwShbDz/UdPB7GONHqlxS 16xcp7dbelp88o4XGj7mXnDm270KD3vGPMpulzzCsqzyfkCA8RM6oAzU2HCmfOyVmap9SVjKrUQpt owJDpEy3NGPbcEKsXBt8vlvKdCgXdjnflRd0oXWaqptg5gsYbeIxSXF0iIfO+CqG4xYY2LKRytFMP SppNmlCQl2XQl8JVkd8kxzkXFiG2zV5SHHWcLdjXdA33dzwahfWhMmYtv5FA5bN4vRxs49iSdyzhf L2Mr4x1tgag3UIFvMaYRMYfJPMhefabcTO624uv6plWPIscJWAq8TMIR7lrj5lFkBS8OmTE89rZBc Rsoi7k+Q==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttc-0006aV-5n; Mon, 26 Oct 2020 04:14:12 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 09/12] mm: Pass pvec directly to find_get_entries Date: Mon, 26 Oct 2020 04:14:05 +0000 Message-Id: <20201026041408.25230-10-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers of find_get_entries() use a pvec, so pass it directly instead of manipulating it in the caller. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 21 +++++++++------------ mm/shmem.c | 5 ++--- mm/swap.c | 4 +--- 4 files changed, 13 insertions(+), 20 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 5b425f666bc5..f0bbe29de732 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -450,8 +450,7 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) } unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, unsigned int nr_entries, struct page **entries, - pgoff_t *indices); + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index 6ed2422426d2..e0fa943011d8 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1898,14 +1898,12 @@ static inline struct page *xas_find_get_entry(struct xa_state *xas, * @mapping: The address_space to search * @start: The starting page cache index * @end: The final page index (inclusive). - * @nr_entries: The maximum number of entries - * @entries: Where the resulting entries are placed + * @pvec: Where the resulting entries are placed. * @indices: The cache indices corresponding to the entries in @entries * - * find_get_entries() will search for and return a group of up to - * @nr_entries entries in the mapping. The entries are placed at - * @entries. find_get_entries() takes a reference against any actual - * pages it returns. + * find_get_entries() will search for and return a batch of entries in + * the mapping. The entries are placed in @pvec. find_get_entries() + * takes a reference on any actual pages it returns. * * The search returns a group of mapping-contiguous page cache entries * with ascending indexes. There may be holes in the indices due to @@ -1922,15 +1920,12 @@ static inline struct page *xas_find_get_entry(struct xa_state *xas, * Return: the number of pages and shadow entries which were found. */ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, unsigned int nr_entries, struct page **entries, - pgoff_t *indices) + pgoff_t end, struct pagevec *pvec, pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; unsigned int ret = 0; - - if (!nr_entries) - return 0; + unsigned nr_entries = PAGEVEC_SIZE; rcu_read_lock(); while ((page = xas_find_get_entry(&xas, end, XA_PRESENT))) { @@ -1945,11 +1940,13 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, } indices[ret] = xas.xa_index; - entries[ret] = page; + pvec->pages[ret] = page; if (++ret == nr_entries) break; } rcu_read_unlock(); + + pvec->nr = ret; return ret; } diff --git a/mm/shmem.c b/mm/shmem.c index 27b93b738ea0..7880a245ac32 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -965,9 +965,8 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end) { cond_resched(); - pvec.nr = find_get_entries(mapping, index, end - 1, - PAGEVEC_SIZE, pvec.pages, indices); - if (!pvec.nr) { + if (!find_get_entries(mapping, index, end - 1, &pvec, + indices)) { /* If all gone or hole-punch or unfalloc, we're done */ if (index == start || end != -1) break; diff --git a/mm/swap.c b/mm/swap.c index 31653152f55d..0c5659c05e6e 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1099,9 +1099,7 @@ unsigned pagevec_lookup_entries(struct pagevec *pvec, struct address_space *mapping, pgoff_t start, pgoff_t end, pgoff_t *indices) { - pvec->nr = find_get_entries(mapping, start, end, PAGEVEC_SIZE, - pvec->pages, indices); - return pagevec_count(pvec); + return find_get_entries(mapping, start, end, pvec, indices); } /** From patchwork Mon Oct 26 04:14:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855641 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E6B161C for ; Mon, 26 Oct 2020 04:14:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BD74E22247 for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ey6vT/qV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD74E22247 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6A84D6B005D; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 631406B0062; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F8A46B0068; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id 217BD6B005D for ; Mon, 26 Oct 2020 00:14:17 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B32641EE6 for ; Mon, 26 Oct 2020 04:14:16 +0000 (UTC) X-FDA: 77412759312.24.flock03_42060b027270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 971FE1A4A0 for ; Mon, 26 Oct 2020 04:14:16 +0000 (UTC) X-Spam-Summary: 1,0,0,96097479c2b27621,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6119:6261:6653:7875:8660:9592:10004:11026:11658:11914:12043:12114:12296:12297:12438:12555:12683:12895:13148:13230:13894:14110:14181:14394:14721:21063:21080:21324:21433:21451:21627:21939:21987:21990:30034:30054:30064:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04yrsm8bpyey4ektp3t865fosm66gyczkkxztcwfp3fu44durmqqwgtgx6pzf1s.qwzdfxcqczp6jsmx5mnh9ecqnwim94ym7dksu68s44b7j9a6heor91gwpqxqhxw.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:235,LUA_SUMMARY:none X-HE-Tag: flock03_42060b027270 X-Filterd-Recvd-Size: 5839 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=9bKO5BckbV2VgZEPlLHdw6VBrK4Lqq5F1PqASZdfsOM=; b=Ey6vT/qVSvLtYtAoug5KdK8/6G 8W/I0yNq/9wpoa/ehFJwTBHli/jCgivAtVJwE/IMsAwnpPXMOfTsy3SCpCSFreHy/QxbmTKH4sOOr GWZ3ZF6zacfETzOZIrnLy08dmDVrPB8DtSc7kAcxat4huRs1tEkOaYII8Rh/FWny/nV+0UWqdUTdt dyZUy9tTisfmD451aembuaGjkr6RI58W4TIhlCfDTAwfqjpMDyBjIToe9qdVUEMwkchh49TZmURQf 4VSv+/NcRllyqi1PDGk97f2hoLelt0hsp4Xna6idX1cNXtC+Am4nWkBlVNavwGdjqWDXCox97PFCk lIISRdvg==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttc-0006ac-Dx; Mon, 26 Oct 2020 04:14:12 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 10/12] mm: Remove pagevec_lookup_entries Date: Mon, 26 Oct 2020 04:14:06 +0000 Message-Id: <20201026041408.25230-11-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: pagevec_lookup_entries() is now just a wrapper around find_get_entries() so remove it and convert all its callers. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagevec.h | 3 --- mm/swap.c | 36 ++---------------------------------- mm/truncate.c | 4 ++-- 3 files changed, 4 insertions(+), 39 deletions(-) diff --git a/include/linux/pagevec.h b/include/linux/pagevec.h index ce77724a2ab7..a45bea4b4d08 100644 --- a/include/linux/pagevec.h +++ b/include/linux/pagevec.h @@ -25,9 +25,6 @@ struct pagevec { void __pagevec_release(struct pagevec *pvec); void __pagevec_lru_add(struct pagevec *pvec); -unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, pgoff_t start, pgoff_t end, - pgoff_t *indices); void pagevec_remove_exceptionals(struct pagevec *pvec); unsigned pagevec_lookup_range(struct pagevec *pvec, struct address_space *mapping, diff --git a/mm/swap.c b/mm/swap.c index 0c5659c05e6e..8f5623336eb3 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1070,44 +1070,12 @@ void __pagevec_lru_add(struct pagevec *pvec) pagevec_lru_move_fn(pvec, __pagevec_lru_add_fn, NULL); } -/** - * pagevec_lookup_entries - gang pagecache lookup - * @pvec: Where the resulting entries are placed - * @mapping: The address_space to search - * @start: The starting entry index - * @end: The highest index to return (inclusive). - * @nr_entries: The maximum number of pages - * @indices: The cache indices corresponding to the entries in @pvec - * - * pagevec_lookup_entries() will search for and return a group of up - * to @nr_pages pages and shadow entries in the mapping. All - * entries are placed in @pvec. pagevec_lookup_entries() takes a - * reference against actual pages in @pvec. - * - * The search returns a group of mapping-contiguous entries with - * ascending indexes. There may be holes in the indices due to - * not-present entries. - * - * Only one subpage of a Transparent Huge Page is returned in one call: - * allowing truncate_inode_pages_range() to evict the whole THP without - * cycling through a pagevec of extra references. - * - * pagevec_lookup_entries() returns the number of entries which were - * found. - */ -unsigned pagevec_lookup_entries(struct pagevec *pvec, - struct address_space *mapping, pgoff_t start, pgoff_t end, - pgoff_t *indices) -{ - return find_get_entries(mapping, start, end, pvec, indices); -} - /** * pagevec_remove_exceptionals - pagevec exceptionals pruning * @pvec: The pagevec to prune * - * pagevec_lookup_entries() fills both pages and exceptional radix - * tree entries into the pagevec. This function prunes all + * find_get_entries() fills both pages and XArray value entries (aka + * exceptional entries) into the pagevec. This function prunes all * exceptionals from @pvec without leaving holes, so that it can be * passed on to page-only pagevec operations. */ diff --git a/mm/truncate.c b/mm/truncate.c index c4a3c9d6a0c1..a96e44a5ce59 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -376,7 +376,7 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; for ( ; ; ) { cond_resched(); - if (!pagevec_lookup_entries(&pvec, mapping, index, end - 1, + if (!find_get_entries(mapping, index, end - 1, &pvec, indices)) { /* If all gone from start onwards, we're done */ if (index == start) @@ -632,7 +632,7 @@ int invalidate_inode_pages2_range(struct address_space *mapping, pagevec_init(&pvec); index = start; - while (pagevec_lookup_entries(&pvec, mapping, index, end, indices)) { + while (find_get_entries(mapping, index, end, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { struct page *page = pvec.pages[i]; From patchwork Mon Oct 26 04:14:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855645 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 519CC92C for ; Mon, 26 Oct 2020 04:14:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F042B2076A for ; Mon, 26 Oct 2020 04:14:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="JRw2pV5G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F042B2076A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 39AC06B0068; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 34F856B006C; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EE276B006E; Mon, 26 Oct 2020 00:14:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id E6BB86B0068 for ; Mon, 26 Oct 2020 00:14:18 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 880FA181AC9CB for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) X-FDA: 77412759396.04.hour43_0f09de527270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 6F85880051BD for ; Mon, 26 Oct 2020 04:14:18 +0000 (UTC) X-Spam-Summary: 1,0,0,b6a5959a754478f6,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:1:41:69:355:379:541:800:960:966:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1605:1730:1747:1777:1792:2196:2199:2393:2553:2559:2562:2637:2693:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4384:4385:4395:5007:6119:6261:6653:7875:8603:8957:9036:9592:10004:11026:11232:11473:11658:11914:12043:12291:12295:12296:12297:12438:12555:12683:12895:12986:13141:13161:13229:13230:13894:14394:21080:21324:21451:21627:21809:21987:21990:30012:30054:30064:30069:30070:30075:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-62.8.15.100 64.201.201.201;04y8i55ujbmj4i6r1hjiojdpuognryc9tyidgyzi7e8wzw3nrtuburftwi79h9m.idqt4o3pp1hr4czo5e3bcndnye3ptpu4164x45pf6rqghf5oah6ihhwrzczqm9m.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: hour43_0f09de527270 X-Filterd-Recvd-Size: 13117 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=KHbD78qZkx3YwH8ABS51sTB6IGE5SHiJWiP9NqEd/II=; b=JRw2pV5GSQnTF7Pr8QdBfrQxhE dJIjeeHMbV7aQdmEAqgnkTD8T7xcoulCpHfGmE0nRaArwsPq/WNyNopmNB3EqqTE38ENzoBcHCJ30 F9nNkvL2J7orCooS2hf3Al+CGeJGgoRTyf4xHdfvSoitYarf1Y/CJj4BULRvXMFwdQcPvLonIryld OUx18Gr889lMBf+1lJG+052fDLEPNKi2D93yTsoa0D8+Jcvx15YrXAXFQmWl8ezvLVadu+B1dL+gg 0G1b7EEG9o1YcdyImJXTwJ2DeMeHetcdoEoJSlqp+OgCY1KiyhTBT/j3L2+3PVrB84El815Owe+zn tp9ZrESA==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttc-0006al-OW; Mon, 26 Oct 2020 04:14:12 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, William Kucharski Subject: [PATCH v3 11/12] mm/truncate,shmem: Handle truncates that split THPs Date: Mon, 26 Oct 2020 04:14:07 +0000 Message-Id: <20201026041408.25230-12-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Handle THP splitting in the parts of the truncation functions which already handle partial pages. Factor all that code out into a new function called truncate_inode_partial_page(). We lose the easy 'bail out' path if a truncate or hole punch is entirely within a single page. We can add some more complex logic to restore the optimisation if it proves to be worthwhile. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Jan Kara --- mm/internal.h | 1 + mm/shmem.c | 97 ++++++++++++++--------------------------- mm/truncate.c | 118 +++++++++++++++++++++++++++++++------------------- 3 files changed, 108 insertions(+), 108 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 8d79f4d21eaf..194572e1ab49 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -620,4 +620,5 @@ struct migration_target_control { gfp_t gfp_mask; }; +bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end); #endif /* __MM_INTERNAL_H */ diff --git a/mm/shmem.c b/mm/shmem.c index 7880a245ac32..bcb4ecaa5949 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -857,32 +857,6 @@ void shmem_unlock_mapping(struct address_space *mapping) } } -/* - * Check whether a hole-punch or truncation needs to split a huge page, - * returning true if no split was required, or the split has been successful. - * - * Eviction (or truncation to 0 size) should never need to split a huge page; - * but in rare cases might do so, if shmem_undo_range() failed to trylock on - * head, and then succeeded to trylock on tail. - * - * A split can only succeed when there are no additional references on the - * huge page: so the split below relies upon find_get_entries() having stopped - * when it found a subpage of the huge page, without getting further references. - */ -static bool shmem_punch_compound(struct page *page, pgoff_t start, pgoff_t end) -{ - if (!PageTransCompound(page)) - return true; - - /* Just proceed to delete a huge page wholly within the range punched */ - if (PageHead(page) && - page->index >= start && page->index + HPAGE_PMD_NR <= end) - return true; - - /* Try to split huge page, so we can truly punch the hole or truncate */ - return split_huge_page(page) >= 0; -} - /* * Remove range of pages and swap entries from page cache, and free them. * If !unfalloc, truncate or punch hole; if unfalloc, undo failed fallocate. @@ -894,13 +868,13 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, struct shmem_inode_info *info = SHMEM_I(inode); pgoff_t start = (lstart + PAGE_SIZE - 1) >> PAGE_SHIFT; pgoff_t end = (lend + 1) >> PAGE_SHIFT; - unsigned int partial_start = lstart & (PAGE_SIZE - 1); - unsigned int partial_end = (lend + 1) & (PAGE_SIZE - 1); struct pagevec pvec; pgoff_t indices[PAGEVEC_SIZE]; + struct page *page; long nr_swaps_freed = 0; pgoff_t index; int i; + bool partial_end; if (lend == -1) end = -1; /* unsigned, so actually very big */ @@ -910,7 +884,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, while (index < end && find_lock_entries(mapping, index, end - 1, &pvec, indices)) { for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; index = indices[i]; @@ -933,33 +907,37 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, index++; } - if (partial_start) { - struct page *page = NULL; - shmem_getpage(inode, start - 1, &page, SGP_READ); - if (page) { - unsigned int top = PAGE_SIZE; - if (start > end) { - top = partial_end; - partial_end = 0; - } - zero_user_segment(page, partial_start, top); - set_page_dirty(page); - unlock_page(page); - put_page(page); + partial_end = ((lend + 1) % PAGE_SIZE) > 0; + page = NULL; + shmem_getpage(inode, lstart >> PAGE_SHIFT, &page, SGP_READ); + if (page) { + bool same_page; + + page = thp_head(page); + same_page = lend < page_offset(page) + thp_size(page); + if (same_page) + partial_end = false; + set_page_dirty(page); + if (!truncate_inode_partial_page(page, lstart, lend)) { + start = page->index + thp_nr_pages(page); + if (same_page) + end = page->index; } + unlock_page(page); + put_page(page); + page = NULL; } - if (partial_end) { - struct page *page = NULL; + + if (partial_end) shmem_getpage(inode, end, &page, SGP_READ); - if (page) { - zero_user_segment(page, 0, partial_end); - set_page_dirty(page); - unlock_page(page); - put_page(page); - } + if (page) { + page = thp_head(page); + set_page_dirty(page); + if (!truncate_inode_partial_page(page, lstart, lend)) + end = page->index; + unlock_page(page); + put_page(page); } - if (start >= end) - return; index = start; while (index < end) { @@ -975,7 +953,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, continue; } for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; index = indices[i]; if (xa_is_value(page)) { @@ -1000,18 +978,9 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, break; } VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) - truncate_inode_page(mapping, page); - else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { - /* Wipe the page and don't get stuck */ - clear_highpage(page); - flush_dcache_page(page); - set_page_dirty(page); - if (index < - round_up(start, HPAGE_PMD_NR)) - start = index + 1; - } + truncate_inode_page(mapping, page); } + index = page->index + thp_nr_pages(page) - 1; unlock_page(page); } pagevec_remove_exceptionals(&pvec); diff --git a/mm/truncate.c b/mm/truncate.c index a96e44a5ce59..11ef90d7e3af 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -224,6 +224,53 @@ int truncate_inode_page(struct address_space *mapping, struct page *page) return 0; } +/* + * Handle partial (transparent) pages. The page may be entirely within the + * range if a split has raced with us. If not, we zero the part of the + * page that's within the [start, end] range, and then split the page if + * it's a THP. split_page_range() will discard pages which now lie beyond + * i_size, and we rely on the caller to discard pages which lie within a + * newly created hole. + * + * Returns false if THP splitting failed so the caller can avoid + * discarding the entire page which is stubbornly unsplit. + */ +bool truncate_inode_partial_page(struct page *page, loff_t start, loff_t end) +{ + loff_t pos = page_offset(page); + unsigned int offset, length; + + if (pos < start) + offset = start - pos; + else + offset = 0; + length = thp_size(page); + if (pos + length <= (u64)end) + length = length - offset; + else + length = end + 1 - pos - offset; + + wait_on_page_writeback(page); + if (length == thp_size(page)) { + truncate_inode_page(page->mapping, page); + return true; + } + + /* + * We may be zeroing pages we're about to discard, but it avoids + * doing a complex calculation here, and then doing the zeroing + * anyway if the page split fails. + */ + zero_user(page, offset, length); + + cleancache_invalidate_page(page->mapping, page); + if (page_has_private(page)) + do_invalidatepage(page, offset, length); + if (!PageTransHuge(page)) + return true; + return split_huge_page(page) == 0; +} + /* * Used to get rid of pages on hardware memory corruption. */ @@ -288,20 +335,16 @@ void truncate_inode_pages_range(struct address_space *mapping, { pgoff_t start; /* inclusive */ pgoff_t end; /* exclusive */ - unsigned int partial_start; /* inclusive */ - unsigned int partial_end; /* exclusive */ struct pagevec pvec; pgoff_t indices[PAGEVEC_SIZE]; pgoff_t index; int i; + struct page * page; + bool partial_end; if (mapping->nrpages == 0 && mapping->nrexceptional == 0) goto out; - /* Offsets within partial pages */ - partial_start = lstart & (PAGE_SIZE - 1); - partial_end = (lend + 1) & (PAGE_SIZE - 1); - /* * 'start' and 'end' always covers the range of pages to be fully * truncated. Partial pages are covered with 'partial_start' at the @@ -334,48 +377,35 @@ void truncate_inode_pages_range(struct address_space *mapping, cond_resched(); } - if (partial_start) { - struct page *page = find_lock_page(mapping, start - 1); - if (page) { - unsigned int top = PAGE_SIZE; - if (start > end) { - /* Truncation within a single page */ - top = partial_end; - partial_end = 0; - } - wait_on_page_writeback(page); - zero_user_segment(page, partial_start, top); - cleancache_invalidate_page(mapping, page); - if (page_has_private(page)) - do_invalidatepage(page, partial_start, - top - partial_start); - unlock_page(page); - put_page(page); + partial_end = ((lend + 1) % PAGE_SIZE) > 0; + page = find_lock_head(mapping, lstart >> PAGE_SHIFT); + if (page) { + bool same_page = lend < page_offset(page) + thp_size(page); + if (same_page) + partial_end = false; + if (!truncate_inode_partial_page(page, lstart, lend)) { + start = page->index + thp_nr_pages(page); + if (same_page) + end = page->index; } + unlock_page(page); + put_page(page); + page = NULL; } - if (partial_end) { - struct page *page = find_lock_page(mapping, end); - if (page) { - wait_on_page_writeback(page); - zero_user_segment(page, 0, partial_end); - cleancache_invalidate_page(mapping, page); - if (page_has_private(page)) - do_invalidatepage(page, 0, - partial_end); - unlock_page(page); - put_page(page); - } + + if (partial_end) + page = find_lock_head(mapping, end); + if (page) { + if (!truncate_inode_partial_page(page, lstart, lend)) + end = page->index; + unlock_page(page); + put_page(page); } - /* - * If the truncation happened within a single page no pages - * will be released, just zeroed, so we can bail out now. - */ - if (start >= end) - goto out; index = start; - for ( ; ; ) { + while (index < end) { cond_resched(); + if (!find_get_entries(mapping, index, end - 1, &pvec, indices)) { /* If all gone from start onwards, we're done */ @@ -387,7 +417,7 @@ void truncate_inode_pages_range(struct address_space *mapping, } for (i = 0; i < pagevec_count(&pvec); i++) { - struct page *page = pvec.pages[i]; + page = pvec.pages[i]; /* We rely upon deletion not changing page->index */ index = indices[i]; @@ -396,7 +426,7 @@ void truncate_inode_pages_range(struct address_space *mapping, continue; lock_page(page); - WARN_ON(page_to_index(page) != index); + index = page->index + thp_nr_pages(page) - 1; wait_on_page_writeback(page); truncate_inode_page(mapping, page); unlock_page(page); From patchwork Mon Oct 26 04:14:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11855649 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A04B561C for ; Mon, 26 Oct 2020 04:14:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4745A2073A for ; Mon, 26 Oct 2020 04:14:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YcfZjdNN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4745A2073A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 724956B006E; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5C3196B0070; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 483846B0071; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0228.hostedemail.com [216.40.44.228]) by kanga.kvack.org (Postfix) with ESMTP id 109BD6B006E for ; Mon, 26 Oct 2020 00:14:20 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A474C180AD815 for ; Mon, 26 Oct 2020 04:14:19 +0000 (UTC) X-FDA: 77412759438.17.cook88_320218027270 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 8BF25180D0180 for ; Mon, 26 Oct 2020 04:14:19 +0000 (UTC) X-Spam-Summary: 1,0,0,78760c361c786138,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2393:2553:2559:2562:2689:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4031:4117:4321:4605:5007:6119:6261:6653:8957:9592:10004:11026:11658:11914:12043:12296:12297:12438:12555:12683:12895:13894:14096:14181:14394:14721:21063:21080:21324:21451:21627:21740:21990:30034:30054:30064:30070:30090,0,RBL:90.155.50.34:@infradead.org:.lbl8.mailshell.net-64.201.201.201 62.8.15.100;04ygxaboq3ht7rqqzzwe5aqdmy3wzoc3yb3yh3omj9h6s5axy35bawkk5yteux8.j437n7xs1dgsmbhcq9qnngbardm67ca86s99ax8rs4yoiy83jfpkkors3t87pmx.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:144,LUA_SUMMARY:none X-HE-Tag: cook88_320218027270 X-Filterd-Recvd-Size: 6503 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 26 Oct 2020 04:14:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=kpUczdeKYi1+38xZwHEXixHpdmrDrWMrdTwGd2Ba8x0=; b=YcfZjdNN1aDe/LcHqu+I3/RUD7 ZB/2qcxmUEd0z/Z+O9YaxXNnWUummKleEY3HmDXPF9Z9OOL/EfnAFqBUH5T9hRUaOQTScouC78Aq3 xw8jSrszyqXqn+fwCOXxEUMcOyXHpzE8nFgzbwL/2K7eVO4HPakPOm+rnQy6p2csVx+4pPDsObLRm OZKyyMBUt4CJHQbpNVhyAGYa00xjzg9iaQ0UP/sx8iI2v8g0Ng5KC4SMhk37oLj49g8y97KRZo4Y9 keDM8o968FrKmDot2d98jRh6+ozGlQarsJz4VWaUjUssDqRLKqQCDReXLZw7XEU3VvvmQOfpsEwMJ prUEKttw==; Received: from willy by casper.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1kWttd-0006as-1o; Mon, 26 Oct 2020 04:14:13 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , linux-fsdevel@vger.kernel.org, Andrew Morton , Hugh Dickins , Johannes Weiner , Yang Shi , Dave Chinner , linux-kernel@vger.kernel.org, Jan Kara , William Kucharski Subject: [PATCH v3 12/12] mm/filemap: Return only head pages from find_get_entries Date: Mon, 26 Oct 2020 04:14:08 +0000 Message-Id: <20201026041408.25230-13-willy@infradead.org> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20201026041408.25230-1-willy@infradead.org> References: <20201026041408.25230-1-willy@infradead.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All callers now expect head (and base) pages, and can handle multiple head pages in a single batch, so make find_get_entries() behave that way. Also take the opportunity to make it use the pagevec infrastructure instead of open-coding how pvecs behave. This has the side-effect of being able to append to a pagevec with existing contents, although we don't make use of that functionality anywhere yet. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Jan Kara Reviewed-by: William Kucharski --- include/linux/pagemap.h | 2 -- mm/filemap.c | 36 ++++++++---------------------------- mm/internal.h | 2 ++ 3 files changed, 10 insertions(+), 30 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index f0bbe29de732..8938c64f418b 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -449,8 +449,6 @@ static inline struct page *find_subpage(struct page *head, pgoff_t index) return head + (index & (thp_nr_pages(head) - 1)); } -unsigned find_get_entries(struct address_space *mapping, pgoff_t start, - pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_get_pages_range(struct address_space *mapping, pgoff_t *start, pgoff_t end, unsigned int nr_pages, struct page **pages); diff --git a/mm/filemap.c b/mm/filemap.c index e0fa943011d8..a117718ec1a8 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1905,49 +1905,29 @@ static inline struct page *xas_find_get_entry(struct xa_state *xas, * the mapping. The entries are placed in @pvec. find_get_entries() * takes a reference on any actual pages it returns. * - * The search returns a group of mapping-contiguous page cache entries - * with ascending indexes. There may be holes in the indices due to - * not-present pages. + * The entries have ascending indexes. The indices may not be consecutive + * due to not-present entries or THPs. * * Any shadow entries of evicted pages, or swap entries from * shmem/tmpfs, are included in the returned array. * - * If it finds a Transparent Huge Page, head or tail, find_get_entries() - * stops at that page: the caller is likely to have a better way to handle - * the compound page as a whole, and then skip its extent, than repeatedly - * calling find_get_entries() to return all its tails. - * - * Return: the number of pages and shadow entries which were found. + * Return: The number of entries which were found. */ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, pgoff_t end, struct pagevec *pvec, pgoff_t *indices) { XA_STATE(xas, &mapping->i_pages, start); struct page *page; - unsigned int ret = 0; - unsigned nr_entries = PAGEVEC_SIZE; rcu_read_lock(); while ((page = xas_find_get_entry(&xas, end, XA_PRESENT))) { - /* - * Terminate early on finding a THP, to allow the caller to - * handle it all at once; but continue if this is hugetlbfs. - */ - if (!xa_is_value(page) && PageTransHuge(page) && - !PageHuge(page)) { - page = find_subpage(page, xas.xa_index); - nr_entries = ret + 1; - } - - indices[ret] = xas.xa_index; - pvec->pages[ret] = page; - if (++ret == nr_entries) + indices[pvec->nr] = xas.xa_index; + if (!pagevec_add(pvec, page)) break; } rcu_read_unlock(); - pvec->nr = ret; - return ret; + return pagevec_count(pvec); } /** @@ -1966,8 +1946,8 @@ unsigned find_get_entries(struct address_space *mapping, pgoff_t start, * not returned. * * The entries have ascending indexes. The indices may not be consecutive - * due to not-present entries, THP pages, pages which could not be locked - * or pages under writeback. + * due to not-present entries, THPs, pages which could not be locked or + * pages under writeback. * * Return: The number of entries which were found. */ diff --git a/mm/internal.h b/mm/internal.h index 194572e1ab49..5aca7d7bc57c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -62,6 +62,8 @@ static inline void force_page_cache_readahead(struct address_space *mapping, struct page *find_get_entry(struct address_space *mapping, pgoff_t index); struct page *find_lock_entry(struct address_space *mapping, pgoff_t index); +unsigned find_get_entries(struct address_space *mapping, pgoff_t start, + pgoff_t end, struct pagevec *pvec, pgoff_t *indices); unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, pgoff_t end, struct pagevec *pvec, pgoff_t *indices);