From patchwork Fri May 29 02:58:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 11577619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52E6A13B4 for ; Fri, 29 May 2020 03:37:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1EBFF207D3 for ; Fri, 29 May 2020 03:37:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bqWGL6nN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1EBFF207D3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 364D48001A; Thu, 28 May 2020 23:37:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3158280010; Thu, 28 May 2020 23:37:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22B888001A; Thu, 28 May 2020 23:37:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 0AF4980010 for ; Thu, 28 May 2020 23:37:34 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C0FE7180AD815 for ; Fri, 29 May 2020 03:37:33 +0000 (UTC) X-FDA: 76868346786.28.scale90_41fc6898e601f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id AB7116D99 for ; Fri, 29 May 2020 03:37:33 +0000 (UTC) X-Spam-Summary: 2,0,0,ff71f23c11e69557,d41d8cd98f00b204,willy@infradead.org,,RULES_HIT:2:41:69:355:379:541:800:960:966:968:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1606:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2731:2898:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4118:4250:4384:4385:4395:5007:6119:6261:6653:7576:7875:8603:8660:8957:9592:10004:11026:11473:11658:11914:12043:12291:12295:12296:12297:12438:12555:12683:12895:13141:13148:13230:13894:14394:21080:21324:21451:21627:21939:21990:30054:30070,0,RBL:198.137.202.133:@infradead.org:.lbl8.mailshell.net-62.8.0.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fn,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: scale90_41fc6898e601f X-Filterd-Recvd-Size: 7451 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 May 2020 03:37:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=6hOSY0AepZPUXeoiFiLtItvOqrCqHbgM6bZ40Vh0p/o=; b=bqWGL6nNFUxBWrAdJ2lQJ33NZs +Ij15Zop82OumlIIQf82UWNjsZz6a2dCf9eydJMCvNSQ+gFZE4YErya6iwCoIntUJwSKVSnYKrNUW 8eLeZwVhrf16NQmKm4H/9evP1ew8g2ZGT3CNOKRtDQCNxpRpqDBKIIp9hAMMuRxXh4mpzk7ZeIdzW pMFIY8+o/IahjAQTt3xz0SYEHf1cS6c59CV3ZSYPsBgCi0VLCryUukwx03C4g5ZcH9X6MMM9h/1S/ AMeIOQi9Ah65VyY/0pwzGtGvzr7v8Xf2c+zvJ96s+OsGtgg+hKeKhcn4L836bOzhVa4Pn96iUEfI1 kq2zqr2A==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1jeVE3-0008T2-Nl; Fri, 29 May 2020 02:58:27 +0000 From: Matthew Wilcox To: linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v5 30/39] mm: Handle truncates that split large pages Date: Thu, 28 May 2020 19:58:15 -0700 Message-Id: <20200529025824.32296-31-willy@infradead.org> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200529025824.32296-1-willy@infradead.org> References: <20200529025824.32296-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: AB7116D99 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Move shmem_punch_compound() to truncate.c and rename it to punch_thp(). Change its arguments to loff_t to make calling do_invalidatepage() easier. Call it when we find a THP in the cache. Signed-off-by: Matthew Wilcox (Oracle) --- mm/internal.h | 2 ++ mm/shmem.c | 30 ++-------------------------- mm/truncate.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 57 insertions(+), 30 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5efb13d5c226..3090e22b984d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -604,4 +604,6 @@ static inline bool is_migrate_highatomic_page(struct page *page) void setup_zone_pageset(struct zone *zone); extern struct page *alloc_new_node_page(struct page *page, unsigned long node); + +bool punch_thp(struct page *page, loff_t start, loff_t end); #endif /* __MM_INTERNAL_H */ diff --git a/mm/shmem.c b/mm/shmem.c index d722eb830317..a4d4e817a33f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -788,32 +788,6 @@ void shmem_unlock_mapping(struct address_space *mapping) } } -/* - * Check whether a hole-punch or truncation needs to split a huge page, - * returning true if no split was required, or the split has been successful. - * - * Eviction (or truncation to 0 size) should never need to split a huge page; - * but in rare cases might do so, if shmem_undo_range() failed to trylock on - * head, and then succeeded to trylock on tail. - * - * A split can only succeed when there are no additional references on the - * huge page: so the split below relies upon find_get_entries() having stopped - * when it found a subpage of the huge page, without getting further references. - */ -static bool shmem_punch_compound(struct page *page, pgoff_t start, pgoff_t end) -{ - if (!PageTransCompound(page)) - return true; - - /* Just proceed to delete a huge page wholly within the range punched */ - if (PageHead(page) && - page->index >= start && page->index + HPAGE_PMD_NR <= end) - return true; - - /* Try to split huge page, so we can truly punch the hole or truncate */ - return split_huge_page(page) >= 0; -} - /* * Remove range of pages and swap entries from page cache, and free them. * If !unfalloc, truncate or punch hole; if unfalloc, undo failed fallocate. @@ -867,7 +841,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, if ((!unfalloc || !PageUptodate(page)) && page_mapping(page) == mapping) { VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) + if (punch_thp(page, lstart, lend)) truncate_inode_page(mapping, page); } unlock_page(page); @@ -950,7 +924,7 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, loff_t lend, break; } VM_BUG_ON_PAGE(PageWriteback(page), page); - if (shmem_punch_compound(page, start, end)) + if (punch_thp(page, lstart, lend)) truncate_inode_page(mapping, page); else { /* Wipe the page and don't get stuck */ diff --git a/mm/truncate.c b/mm/truncate.c index dad384a4dc6d..59cae74b152d 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -229,6 +229,55 @@ int truncate_inode_page(struct address_space *mapping, struct page *page) return 0; } +/* + * Check whether a hole-punch or truncation needs to split a huge page, + * returning true if no split was required, or the split has been + * successful. + * + * Eviction (or truncation to 0 size) should never need to split a huge + * page; but in rare cases might do so, if shmem_undo_range() failed to + * trylock on head, and then succeeded to trylock on tail. + * + * A split can only succeed when there are no additional references on + * the huge page: so the split below relies upon find_get_entries() + * having stopped when it found a subpage of the huge page, without + * getting further references. + */ +bool punch_thp(struct page *page, loff_t start, loff_t end) +{ + struct page *head = compound_head(page); + loff_t pos = page_offset(head); + unsigned int offset, length; + + if (!PageTransCompound(page)) + return true; + + if (pos < start) + offset = start - pos; + else + offset = 0; + length = thp_size(head); + if (pos + length < end) + length = length - offset; + else + length = end - pos - offset; + + /* Just proceed to delete a huge page wholly within the range punched */ + if (length == thp_size(head)) + return true; + + /* + * We're going to split the page into order-0 pages. Tell the + * filesystem which range of the page is going to be punched out + * so it can discard unnecessary private data. + */ + if (page_has_private(head)) + do_invalidatepage(head, offset, length); + + /* Try to split huge page, so we can truly punch the hole or truncate */ + return split_huge_page(page) >= 0; +} + /* * Used to get rid of pages on hardware memory corruption. */ @@ -359,7 +408,8 @@ void truncate_inode_pages_range(struct address_space *mapping, unlock_page(page); continue; } - pagevec_add(&locked_pvec, page); + if (punch_thp(page, lstart, lend)) + pagevec_add(&locked_pvec, page); } for (i = 0; i < pagevec_count(&locked_pvec); i++) truncate_cleanup_page(mapping, locked_pvec.pages[i]); @@ -446,7 +496,8 @@ void truncate_inode_pages_range(struct address_space *mapping, lock_page(page); WARN_ON(page_to_index(page) != index); wait_on_page_writeback(page); - truncate_inode_page(mapping, page); + if (punch_thp(page, lstart, lend)) + truncate_inode_page(mapping, page); unlock_page(page); } truncate_exceptional_pvec_entries(mapping, &pvec, indices, end);