From patchwork Mon Sep 6 12:11:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Wang X-Patchwork-Id: 12476897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAA7BC433EF for ; Mon, 6 Sep 2021 12:12:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 86B3360724 for ; Mon, 6 Sep 2021 12:12:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 86B3360724 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 50B7E6B0071; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A9D1900002; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D4ED6B0071; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 137D1900003 for ; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A7F1882499A8 for ; Mon, 6 Sep 2021 12:12:07 +0000 (UTC) X-FDA: 78557035494.16.2A1A928 Received: from out30-42.freemail.mail.aliyun.com (out30-42.freemail.mail.aliyun.com [115.124.30.42]) by imf22.hostedemail.com (Postfix) with ESMTP id 1E2311907 for ; Mon, 6 Sep 2021 12:12:05 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R901e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04420;MF=rongwei.wang@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0UnRyn5B_1630930321; Received: from localhost.localdomain(mailfrom:rongwei.wang@linux.alibaba.com fp:SMTPD_---0UnRyn5B_1630930321) by smtp.aliyun-inc.com(127.0.0.1); Mon, 06 Sep 2021 20:12:02 +0800 From: Rongwei Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, cfijalkovich@google.com, song@kernel.org, william.kucharski@oracle.com, hughd@google.com Subject: [PATCH 1/2] mm, thp: check page mapping when truncating page cache Date: Mon, 6 Sep 2021 20:11:59 +0800 Message-Id: <20210906121200.57905-2-rongwei.wang@linux.alibaba.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> References: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> MIME-Version: 1.0 Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.42 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com; dmarc=pass (policy=none) header.from=alibaba.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1E2311907 X-Stat-Signature: wnra56zmbceqm5q3ez3q9qoouubm38sk X-HE-Tag: 1630930325-162621 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Transparent huge page has supported read-only non-shmem files. The file- backed THP is collapsed by khugepaged and truncated when written (for shared libraries). However, there is race in two possible places. 1) multiple writers truncate the same page cache concurrently; 2) collapse_file rolls back when writer truncates the page cache; In both cases, subpage(s) of file THP can be revealed by find_get_entry in truncate_inode_pages_range, which will trigger PageTail BUG_ON in truncate_inode_page, as follows. [40326.247034] page:000000009e420ff2 refcount:1 mapcount:0 mapping:0000000000000000 index:0x7ff pfn:0x50c3ff [40326.247041] head:0000000075ff816d order:9 compound_mapcount:0 compound_pincount:0 [40326.247046] flags: 0x37fffe0000010815(locked|uptodate|lru|arch_1|head) [40326.247051] raw: 37fffe0000000000 fffffe0013108001 dead000000000122 dead000000000400 [40326.247053] raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000 [40326.247055] head: 37fffe0000010815 fffffe001066bd48 ffff000404183c20 0000000000000000 [40326.247057] head: 0000000000000600 0000000000000000 00000001ffffffff ffff000c0345a000 [40326.247058] page dumped because: VM_BUG_ON_PAGE(PageTail(page)) [40326.247077] ------------[ cut here ]------------ [40326.247080] kernel BUG at mm/truncate.c:213! [40326.280581] Internal error: Oops - BUG: 0 [#1] SMP [40326.281077] Modules linked in: xfs(E) libcrc32c(E) rfkill(E) aes_ce_blk(E) crypto_simd(E) cryptd(E) aes_ce_cipher(E) crct10dif_ce(E) ghash_ce(E) sha1_ce(E) uio_pdrv_genirq(E) uio(E) nfsd(E) vfat(E) fat(E) auth_rpcgss(E) nfs_acl(E) lockd(E) grace(E) sunrpc(E) sch_fq_codel(E) ip_tables(E) ext4(E) mbcache(E) jbd2(E) virtio_net(E) net_failover(E) virtio_blk(E) failover(E) sha2_ce(E) sha256_arm64(E) virtio_mmio(E) virtio_pci(E) virtio_ring(E) virtio(E) [40326.285130] CPU: 14 PID: 11394 Comm: check_madvise_d Kdump: loaded Tainted: G W E 5.10.46-hugetext+ #55 [40326.286202] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 0.0.0 02/06/2015 [40326.286968] pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--) [40326.287584] pc : truncate_inode_page+0x64/0x70 [40326.288040] lr : truncate_inode_page+0x64/0x70 [40326.288498] sp : ffff80001b60b900 [40326.288837] x29: ffff80001b60b900 x28: 00000000000007ff [40326.289377] x27: ffff80001b60b9a0 x26: 0000000000000000 [40326.289943] x25: 000000000000000f x24: ffff80001b60b9a0 [40326.290485] x23: ffff80001b60ba18 x22: ffff0001e0999ea8 [40326.291027] x21: ffff0000c21db300 x20: ffffffffffffffff [40326.291566] x19: fffffe001310ffc0 x18: 0000000000000020 [40326.292106] x17: 0000000000000000 x16: 0000000000000000 [40326.292655] x15: ffff0000c21db960 x14: 3030306666666620 [40326.293197] x13: 6666666666666666 x12: 3130303030303030 [40326.293746] x11: ffff8000117b69b8 x10: 00000000ffff8000 [40326.294313] x9 : ffff80001012690c x8 : 0000000000000000 [40326.294851] x7 : ffff8000114f69b8 x6 : 0000000000017ffd [40326.295392] x5 : ffff0007fffbcbc8 x4 : ffff80001b60b5c0 [40326.295942] x3 : 0000000000000001 x2 : 0000000000000000 [40326.296497] x1 : 0000000000000000 x0 : 0000000000000000 [40326.297047] Call trace: [40326.297304] truncate_inode_page+0x64/0x70 [40326.297724] truncate_inode_pages_range+0x550/0x7e4 [40326.298251] truncate_pagecache+0x58/0x80 [40326.298662] do_dentry_open+0x1e4/0x3c0 [40326.299052] vfs_open+0x38/0x44 [40326.299377] do_open+0x1f0/0x310 [40326.299709] path_openat+0x114/0x1dc [40326.300077] do_filp_open+0x84/0x134 [40326.300444] do_sys_openat2+0xbc/0x164 [40326.300825] __arm64_sys_openat+0x74/0xc0 [40326.301236] el0_svc_common.constprop.0+0x88/0x220 [40326.301723] do_el0_svc+0x30/0xa0 [40326.302089] el0_svc+0x20/0x30 [40326.302404] el0_sync_handler+0x1a4/0x1b0 [40326.302814] el0_sync+0x180/0x1c0 [40326.303157] Code: aa0103e0 900061e1 910ec021 9400d300 (d4210000) [40326.303775] ---[ end trace f70cdb42cb7c2d42 ]--- [40326.304244] Kernel panic - not syncing: Oops - BUG: Fatal exception This checks the page mapping and retries when subpage of file THP is found, in truncate_inode_pages_range. Fixes: eb6ecbed0aa2 ("mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs") Signed-off-by: Xu Yu Signed-off-by: Rongwei Wang Reported-by: kernel test robot --- mm/filemap.c | 7 ++++++- mm/truncate.c | 17 ++++++++++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index dae481293..a3af2ec 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2093,7 +2093,6 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, if (!xa_is_value(page)) { if (page->index < start) goto put; - VM_BUG_ON_PAGE(page->index != xas.xa_index, page); if (page->index + thp_nr_pages(page) - 1 > end) goto put; if (!trylock_page(page)) @@ -2102,6 +2101,12 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t start, goto unlock; VM_BUG_ON_PAGE(!thp_contains(page, xas.xa_index), page); + /* + * We can find and get head page of file THP with + * non-head index. The head page should have already + * be truncated with page->mapping reset to NULL. + */ + VM_BUG_ON_PAGE(page->index != xas.xa_index, page); } indices[pvec->nr] = xas.xa_index; if (!pagevec_add(pvec, page)) diff --git a/mm/truncate.c b/mm/truncate.c index 714eaf1..8c59c00 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -319,7 +319,8 @@ void truncate_inode_pages_range(struct address_space *mapping, index = start; while (index < end && find_lock_entries(mapping, index, end - 1, &pvec, indices)) { - index = indices[pagevec_count(&pvec) - 1] + 1; + index = indices[pagevec_count(&pvec) - 1] + + thp_nr_pages(pvec.pages[pagevec_count(&pvec) - 1]); truncate_exceptional_pvec_entries(mapping, &pvec, indices); for (i = 0; i < pagevec_count(&pvec); i++) truncate_cleanup_page(pvec.pages[i]); @@ -391,6 +392,20 @@ void truncate_inode_pages_range(struct address_space *mapping, if (xa_is_value(page)) continue; + /* + * Already truncated? We can find and get subpage + * of file THP, of which the head page is truncated. + * + * In addition, another race will be avoided, where + * collapse_file rolls back when writer truncates the + * page cache. + */ + if (page_mapping(page) != mapping) { + /* Restart to make sure all gone */ + index = start - 1; + continue; + } + lock_page(page); WARN_ON(page_to_index(page) != index); wait_on_page_writeback(page); From patchwork Mon Sep 6 12:12:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Wang X-Patchwork-Id: 12476895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AB94C433F5 for ; Mon, 6 Sep 2021 12:12:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9491B60F45 for ; Mon, 6 Sep 2021 12:12:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9491B60F45 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 2E1326B0075; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 260356B0073; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10045900002; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0211.hostedemail.com [216.40.44.211]) by kanga.kvack.org (Postfix) with ESMTP id 029606B0071 for ; Mon, 6 Sep 2021 08:12:08 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9C2848249980 for ; Mon, 6 Sep 2021 12:12:07 +0000 (UTC) X-FDA: 78557035494.10.8F6E37C Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by imf23.hostedemail.com (Postfix) with ESMTP id 74B6290000A0 for ; Mon, 6 Sep 2021 12:12:06 +0000 (UTC) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R841e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=rongwei.wang@linux.alibaba.com;NM=1;PH=DS;RN=7;SR=0;TI=SMTPD_---0UnRyn5B_1630930321; Received: from localhost.localdomain(mailfrom:rongwei.wang@linux.alibaba.com fp:SMTPD_---0UnRyn5B_1630930321) by smtp.aliyun-inc.com(127.0.0.1); Mon, 06 Sep 2021 20:12:03 +0800 From: Rongwei Wang To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, cfijalkovich@google.com, song@kernel.org, william.kucharski@oracle.com, hughd@google.com Subject: [PATCH 2/2] mm, thp: bail out early in collapse_file for writeback page Date: Mon, 6 Sep 2021 20:12:00 +0800 Message-Id: <20210906121200.57905-3-rongwei.wang@linux.alibaba.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> References: <20210906121200.57905-1-rongwei.wang@linux.alibaba.com> MIME-Version: 1.0 Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=alibaba.com; spf=pass (imf23.hostedemail.com: domain of rongwei.wang@linux.alibaba.com designates 115.124.30.132 as permitted sender) smtp.mailfrom=rongwei.wang@linux.alibaba.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 74B6290000A0 X-Stat-Signature: rt8ewupheznf4za1ogi4p5pj9mmtpn16 X-HE-Tag: 1630930326-858289 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently collapse_file does not explicitly check PG_writeback, instead, page_has_private and try_to_release_page are used to filter writeback pages. This does not work for xfs with blocksize equal to or larger than pagesize, because in such case xfs has no page->private. This makes collapse_file bail out early for writeback page. Otherwise, xfs end_page_writeback will panic as follows. [ 6411.448211] page:fffffe00201bcc80 refcount:0 mapcount:0 mapping:ffff0003f88c86a8 index:0x0 pfn:0x84ef32 [ 6411.448304] aops:xfs_address_space_operations [xfs] ino:30000b7 dentry name:"libtest.so" [ 6411.448312] flags: 0x57fffe0000008027(locked|referenced|uptodate|active|writeback) [ 6411.448317] raw: 57fffe0000008027 ffff80001b48bc28 ffff80001b48bc28 ffff0003f88c86a8 [ 6411.448321] raw: 0000000000000000 0000000000000000 00000000ffffffff ffff0000c3e9a000 [ 6411.448324] page dumped because: VM_BUG_ON_PAGE(((unsigned int) page_ref_count(page) + 127u <= 127u)) [ 6411.448327] page->mem_cgroup:ffff0000c3e9a000 [ 6411.448340] ------------[ cut here ]------------ [ 6411.448343] kernel BUG at include/linux/mm.h:1212! [ 6411.449288] Internal error: Oops - BUG: 0 [#1] SMP [ 6411.449786] Modules linked in: [ 6411.449790] BUG: Bad page state in process khugepaged pfn:84ef32 [ 6411.450143] xfs(E) [ 6411.450459] page:fffffe00201bcc80 refcount:0 mapcount:0 mapping:0 index:0x0 pfn:0x84ef32 [ 6411.451361] libcrc32c(E) rfkill(E) aes_ce_blk(E) crypto_simd(E) cryptd(E) aes_ce_cipher(E) crct10dif_ce(E) ghash_ce(E) sha1_ce(E) uio_pdrv_genirq(E) uio(E) vfat(E) nfsd(E) fat(E) auth_rpcgss(E) nfs_acl(E) lockd(E) grace(E) sunrpc(E) sch_fq_codel(E) ip_tables(E) ext4(E) mbcache(E) jbd2(E) virtio_net(E) net_failover(E) virtio_blk(E) failover(E) sha2_ce(E) sha256_arm64(E) virtio_mmio(E) virtio_pci(E) virtio_ring(E) virtio(E) [ 6411.451387] CPU: 25 PID: 0 Comm: swapper/25 Kdump: loaded Tainted: G W E [ 6411.451389] pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--) [ 6411.451393] pc : end_page_writeback+0x1c0/0x214 [ 6411.451394] lr : end_page_writeback+0x1c0/0x214 [ 6411.451395] sp : ffff800011ce3cc0 [ 6411.451396] x29: ffff800011ce3cc0 x28: 0000000000000000 [ 6411.451398] x27: ffff000c04608040 x26: 0000000000000000 [ 6411.451399] x25: ffff000c04608040 x24: 0000000000001000 [ 6411.451401] x23: ffff0003f88c8530 x22: 0000000000001000 [ 6411.451403] x21: ffff0003f88c8530 x20: 0000000000000000 [ 6411.451404] x19: fffffe00201bcc80 x18: 0000000000000030 [ 6411.451406] x17: 0000000000000000 x16: 0000000000000000 [ 6411.451407] x15: ffff000c018f9760 x14: ffffffffffffffff [ 6411.451409] x13: ffff8000119d72b0 x12: ffff8000119d6ee3 [ 6411.451410] x11: ffff8000117b69b8 x10: 00000000ffff8000 [ 6411.451412] x9 : ffff800010617534 x8 : 0000000000000000 [ 6411.451413] x7 : ffff8000114f69b8 x6 : 000000000000000f [ 6411.451415] x5 : 0000000000000000 x4 : 0000000000000000 [ 6411.451416] x3 : 0000000000000400 x2 : 0000000000000000 [ 6411.451418] x1 : 0000000000000000 x0 : 0000000000000000 [ 6411.451420] Call trace: [ 6411.451421] end_page_writeback+0x1c0/0x214 [ 6411.451424] iomap_finish_page_writeback+0x13c/0x204 [ 6411.451425] iomap_finish_ioend+0xe8/0x19c [ 6411.451426] iomap_writepage_end_bio+0x38/0x50 [ 6411.451427] bio_endio+0x168/0x1ec [ 6411.451430] blk_update_request+0x278/0x3f0 [ 6411.451432] blk_mq_end_request+0x34/0x15c [ 6411.451435] virtblk_request_done+0x38/0x74 [virtio_blk] [ 6411.451437] blk_done_softirq+0xc4/0x110 [ 6411.451439] __do_softirq+0x128/0x38c [ 6411.451441] __irq_exit_rcu+0x118/0x150 [ 6411.451442] irq_exit+0x1c/0x30 [ 6411.451445] __handle_domain_irq+0x8c/0xf0 [ 6411.451446] gic_handle_irq+0x84/0x108 [ 6411.451447] el1_irq+0xcc/0x180 [ 6411.451448] arch_cpu_idle+0x18/0x40 [ 6411.451450] default_idle_call+0x4c/0x1a0 [ 6411.451453] cpuidle_idle_call+0x168/0x1e0 [ 6411.451454] do_idle+0xb4/0x104 [ 6411.451455] cpu_startup_entry+0x30/0x9c [ 6411.451458] secondary_start_kernel+0x104/0x180 [ 6411.451460] Code: d4210000 b0006161 910c8021 94013f4d (d4210000) [ 6411.451462] ---[ end trace 4a88c6a074082f8c ]--- [ 6411.451464] Kernel panic - not syncing: Oops - BUG: Fatal exception in interrupt Fixes: eb6ecbed0aa2 ("mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs") Signed-off-by: Xu Yu Signed-off-by: Rongwei Wang --- mm/khugepaged.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 045cc57..529b117 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1798,7 +1798,8 @@ static void collapse_file(struct mm_struct *mm, goto out_unlock; } - if (!is_shmem && PageDirty(page)) { + if (!is_shmem && (PageDirty(page) || + PageWriteback(page))) { /* * khugepaged only works on read-only fd, so this * page is dirty because it hasn't been flushed