From patchwork Wed Oct 6 02:18:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Wang X-Patchwork-Id: 12538355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FBADC4332F for ; Wed, 6 Oct 2021 02:18:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 25B7661207 for ; Wed, 6 Oct 2021 02:18:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 25B7661207 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 875D36B006C; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84D166B0071; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B0BD6B0074; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 53A826B006C for ; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id ED4A12DECA for ; Wed, 6 Oct 2021 02:18:44 +0000 (UTC) X-FDA: 78664404168.37.7431EEC Received: from out30-44.freemail.mail.aliyun.com (out30-44.freemail.mail.aliyun.com [115.124.30.44]) by imf25.hostedemail.com (Postfix) with ESMTP id 20CD8B000D3A for ; Wed, 6 Oct 2021 02:18:42 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R141e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04407;MF=rongwei.wang@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Uqi9lTZ_1633486717; Received: from localhost.localdomain(mailfrom:rongwei.wang@linux.alibaba.com fp:SMTPD_---0Uqi9lTZ_1633486717) by smtp.aliyun-inc.com(127.0.0.1); Wed, 06 Oct 2021 10:18:38 +0800 From: Rongwei Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, willy@infradead.org, song@kernel.org, william.kucharski@oracle.com, hughd@google.com Subject: [PATCH v3 v3 1/2] mm, thp: lock filemap when truncating page cache Date: Wed, 6 Oct 2021 10:18:36 +0800 Message-Id: <20211006021837.59721-2-rongwei.wang@linux.alibaba.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211006021837.59721-1-rongwei.wang@linux.alibaba.com> References: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> <20211006021837.59721-1-rongwei.wang@linux.alibaba.com> MIME-Version: 1.0 Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf25.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.44 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 20CD8B000D3A X-Stat-Signature: mp6znhgo7o4rmw34twrht95z941bwm5n X-HE-Tag: 1633486722-828326 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Transparent huge page has supported read-only non-shmem files. The file- backed THP is collapsed by khugepaged and truncated when written (for shared libraries). However, there is a race when multiple writers truncate the same page cache concurrently. In that case, subpage(s) of file THP can be revealed by find_get_entry in truncate_inode_pages_range, which will trigger PageTail BUG_ON in truncate_inode_page, as follows. [40326.247034] page:000000009e420ff2 refcount:1 mapcount:0 mapping:0000000000000000 index:0x7ff pfn:0x50c3ff [40326.247041] head:0000000075ff816d order:9 compound_mapcount:0 compound_pincount:0 [40326.247046] flags: 0x37fffe0000010815(locked|uptodate|lru|arch_1|head) [40326.247051] raw: 37fffe0000000000 fffffe0013108001 dead000000000122 dead000000000400 [40326.247053] raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000 [40326.247055] head: 37fffe0000010815 fffffe001066bd48 ffff000404183c20 0000000000000000 [40326.247057] head: 0000000000000600 0000000000000000 00000001ffffffff ffff000c0345a000 [40326.247058] page dumped because: VM_BUG_ON_PAGE(PageTail(page)) [40326.247077] ------------[ cut here ]------------ [40326.247080] kernel BUG at mm/truncate.c:213! [40326.280581] Internal error: Oops - BUG: 0 [#1] SMP [40326.281077] Modules linked in: xfs(E) libcrc32c(E) rfkill(E) ... [40326.285130] CPU: 14 PID: 11394 Comm: check_madvise_d Kdump: ... [40326.286202] Hardware name: ECS, BIOS 0.0.0 02/06/2015 [40326.286968] pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--) [40326.287584] pc : truncate_inode_page+0x64/0x70 [40326.288040] lr : truncate_inode_page+0x64/0x70 [40326.288498] sp : ffff80001b60b900 [40326.288837] x29: ffff80001b60b900 x28: 00000000000007ff [40326.289377] x27: ffff80001b60b9a0 x26: 0000000000000000 [40326.289943] x25: 000000000000000f x24: ffff80001b60b9a0 [40326.290485] x23: ffff80001b60ba18 x22: ffff0001e0999ea8 [40326.291027] x21: ffff0000c21db300 x20: ffffffffffffffff [40326.291566] x19: fffffe001310ffc0 x18: 0000000000000020 [40326.292106] x17: 0000000000000000 x16: 0000000000000000 [40326.292655] x15: ffff0000c21db960 x14: 3030306666666620 [40326.293197] x13: 6666666666666666 x12: 3130303030303030 [40326.293746] x11: ffff8000117b69b8 x10: 00000000ffff8000 [40326.294313] x9 : ffff80001012690c x8 : 0000000000000000 [40326.294851] x7 : ffff8000114f69b8 x6 : 0000000000017ffd [40326.295392] x5 : ffff0007fffbcbc8 x4 : ffff80001b60b5c0 [40326.295942] x3 : 0000000000000001 x2 : 0000000000000000 [40326.296497] x1 : 0000000000000000 x0 : 0000000000000000 [40326.297047] Call trace: [40326.297304] truncate_inode_page+0x64/0x70 [40326.297724] truncate_inode_pages_range+0x550/0x7e4 [40326.298251] truncate_pagecache+0x58/0x80 [40326.298662] do_dentry_open+0x1e4/0x3c0 [40326.299052] vfs_open+0x38/0x44 [40326.299377] do_open+0x1f0/0x310 [40326.299709] path_openat+0x114/0x1dc [40326.300077] do_filp_open+0x84/0x134 [40326.300444] do_sys_openat2+0xbc/0x164 [40326.300825] __arm64_sys_openat+0x74/0xc0 [40326.301236] el0_svc_common.constprop.0+0x88/0x220 [40326.301723] do_el0_svc+0x30/0xa0 [40326.302089] el0_svc+0x20/0x30 [40326.302404] el0_sync_handler+0x1a4/0x1b0 [40326.302814] el0_sync+0x180/0x1c0 [40326.303157] Code: aa0103e0 900061e1 910ec021 9400d300 (d4210000) [40326.303775] ---[ end trace f70cdb42cb7c2d42 ]--- [40326.304244] Kernel panic - not syncing: Oops - BUG: Fatal exception This patch mainly to lock filemap when one enter truncate_pagecache(), avoiding truncating the same page cache concurrently. Fixes: eb6ecbed0aa2 ("mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs") Suggested-by: Matthew Wilcox Tested-by: Song Liu Signed-off-by: Xu Yu Signed-off-by: Rongwei Wang --- fs/open.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/open.c b/fs/open.c index daa324606a41..9ec3cfca3b1a 100644 --- a/fs/open.c +++ b/fs/open.c @@ -856,8 +856,11 @@ static int do_dentry_open(struct file *f, * of THPs into the page cache will fail. */ smp_mb(); - if (filemap_nr_thps(inode->i_mapping)) + if (filemap_nr_thps(inode->i_mapping)) { + filemap_invalidate_lock(inode->i_mapping); truncate_pagecache(inode, 0); + filemap_invalidate_unlock(inode->i_mapping); + } } return 0; From patchwork Wed Oct 6 02:18:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Wang X-Patchwork-Id: 12538357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2343FC433EF for ; Wed, 6 Oct 2021 02:18:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6BD5661350 for ; Wed, 6 Oct 2021 02:18:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6BD5661350 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id B35FC6B0073; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABB646B0071; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FEF86B0075; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 5E9256B0071 for ; Tue, 5 Oct 2021 22:18:45 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1A10D18095546 for ; Wed, 6 Oct 2021 02:18:45 +0000 (UTC) X-FDA: 78664404210.32.8A677A6 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by imf21.hostedemail.com (Postfix) with ESMTP id 23277D03A0CD for ; Wed, 6 Oct 2021 02:18:42 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R811e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04395;MF=rongwei.wang@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0Uqi9lTZ_1633486717; Received: from localhost.localdomain(mailfrom:rongwei.wang@linux.alibaba.com fp:SMTPD_---0Uqi9lTZ_1633486717) by smtp.aliyun-inc.com(127.0.0.1); Wed, 06 Oct 2021 10:18:39 +0800 From: Rongwei Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, willy@infradead.org, song@kernel.org, william.kucharski@oracle.com, hughd@google.com Subject: [PATCH v3 v3 2/2] mm, thp: bail out early in collapse_file for writeback page Date: Wed, 6 Oct 2021 10:18:37 +0800 Message-Id: <20211006021837.59721-3-rongwei.wang@linux.alibaba.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211006021837.59721-1-rongwei.wang@linux.alibaba.com> References: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> <20211006021837.59721-1-rongwei.wang@linux.alibaba.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 23277D03A0CD X-Stat-Signature: hmbzxhk44bek7zzmnpwdokq8pbmgjzym Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf21.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.42 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com X-HE-Tag: 1633486722-692054 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently collapse_file does not explicitly check PG_writeback, instead, page_has_private and try_to_release_page are used to filter writeback pages. This does not work for xfs with blocksize equal to or larger than pagesize, because in such case xfs has no page->private. This makes collapse_file bail out early for writeback page. Otherwise, xfs end_page_writeback will panic as follows. [ 6411.448211] page:fffffe00201bcc80 refcount:0 mapcount:0 mapping:ffff0003f88c86a8 index:0x0 pfn:0x84ef32 [ 6411.448304] aops:xfs_address_space_operations [xfs] ino:30000b7 dentry name:"libtest.so" [ 6411.448312] flags: 0x57fffe0000008027(locked|referenced|uptodate|active|writeback) [ 6411.448317] raw: 57fffe0000008027 ffff80001b48bc28 ffff80001b48bc28 ffff0003f88c86a8 [ 6411.448321] raw: 0000000000000000 0000000000000000 00000000ffffffff ffff0000c3e9a000 [ 6411.448324] page dumped because: VM_BUG_ON_PAGE(((unsigned int) page_ref_count(page) + 127u <= 127u)) [ 6411.448327] page->mem_cgroup:ffff0000c3e9a000 [ 6411.448340] ------------[ cut here ]------------ [ 6411.448343] kernel BUG at include/linux/mm.h:1212! [ 6411.449288] Internal error: Oops - BUG: 0 [#1] SMP [ 6411.449786] Modules linked in: [ 6411.449790] BUG: Bad page state in process khugepaged pfn:84ef32 [ 6411.450143] xfs(E) [ 6411.450459] page:fffffe00201bcc80 refcount:0 mapcount:0 mapping:0 index:0x0 pfn:0x84ef32 [ 6411.451361] libcrc32c(E) rfkill(E) aes_ce_blk(E) crypto_simd(E) ... [ 6411.451387] CPU: 25 PID: 0 Comm: swapper/25 Kdump: loaded Tainted: ... [ 6411.451389] pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--) [ 6411.451393] pc : end_page_writeback+0x1c0/0x214 [ 6411.451394] lr : end_page_writeback+0x1c0/0x214 [ 6411.451395] sp : ffff800011ce3cc0 [ 6411.451396] x29: ffff800011ce3cc0 x28: 0000000000000000 [ 6411.451398] x27: ffff000c04608040 x26: 0000000000000000 [ 6411.451399] x25: ffff000c04608040 x24: 0000000000001000 [ 6411.451401] x23: ffff0003f88c8530 x22: 0000000000001000 [ 6411.451403] x21: ffff0003f88c8530 x20: 0000000000000000 [ 6411.451404] x19: fffffe00201bcc80 x18: 0000000000000030 [ 6411.451406] x17: 0000000000000000 x16: 0000000000000000 [ 6411.451407] x15: ffff000c018f9760 x14: ffffffffffffffff [ 6411.451409] x13: ffff8000119d72b0 x12: ffff8000119d6ee3 [ 6411.451410] x11: ffff8000117b69b8 x10: 00000000ffff8000 [ 6411.451412] x9 : ffff800010617534 x8 : 0000000000000000 [ 6411.451413] x7 : ffff8000114f69b8 x6 : 000000000000000f [ 6411.451415] x5 : 0000000000000000 x4 : 0000000000000000 [ 6411.451416] x3 : 0000000000000400 x2 : 0000000000000000 [ 6411.451418] x1 : 0000000000000000 x0 : 0000000000000000 [ 6411.451420] Call trace: [ 6411.451421] end_page_writeback+0x1c0/0x214 [ 6411.451424] iomap_finish_page_writeback+0x13c/0x204 [ 6411.451425] iomap_finish_ioend+0xe8/0x19c [ 6411.451426] iomap_writepage_end_bio+0x38/0x50 [ 6411.451427] bio_endio+0x168/0x1ec [ 6411.451430] blk_update_request+0x278/0x3f0 [ 6411.451432] blk_mq_end_request+0x34/0x15c [ 6411.451435] virtblk_request_done+0x38/0x74 [virtio_blk] [ 6411.451437] blk_done_softirq+0xc4/0x110 [ 6411.451439] __do_softirq+0x128/0x38c [ 6411.451441] __irq_exit_rcu+0x118/0x150 [ 6411.451442] irq_exit+0x1c/0x30 [ 6411.451445] __handle_domain_irq+0x8c/0xf0 [ 6411.451446] gic_handle_irq+0x84/0x108 [ 6411.451447] el1_irq+0xcc/0x180 [ 6411.451448] arch_cpu_idle+0x18/0x40 [ 6411.451450] default_idle_call+0x4c/0x1a0 [ 6411.451453] cpuidle_idle_call+0x168/0x1e0 [ 6411.451454] do_idle+0xb4/0x104 [ 6411.451455] cpu_startup_entry+0x30/0x9c [ 6411.451458] secondary_start_kernel+0x104/0x180 [ 6411.451460] Code: d4210000 b0006161 910c8021 94013f4d (d4210000) [ 6411.451462] ---[ end trace 4a88c6a074082f8c ]--- [ 6411.451464] Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt Fixes: eb6ecbed0aa2 ("mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs") Suggested-by: Yang Shi Signed-off-by: Xu Yu Signed-off-by: Rongwei Wang Reviewed-by: Matthew Wilcox (Oracle) Reviewed-by: Yang Shi --- mm/khugepaged.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 045cc579f724..48de4e1b0783 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1763,6 +1763,10 @@ static void collapse_file(struct mm_struct *mm, filemap_flush(mapping); result = SCAN_FAIL; goto xa_unlocked; + } else if (PageWriteback(page)) { + xas_unlock_irq(&xas); + result = SCAN_FAIL; + goto xa_unlocked; } else if (trylock_page(page)) { get_page(page); xas_unlock_irq(&xas); @@ -1798,7 +1802,8 @@ static void collapse_file(struct mm_struct *mm, goto out_unlock; } - if (!is_shmem && PageDirty(page)) { + if (!is_shmem && (PageDirty(page) || + PageWriteback(page))) { /* * khugepaged only works on read-only fd, so this * page is dirty because it hasn't been flushed