From patchwork Thu Nov 25 03:12:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gang Li X-Patchwork-Id: 12638409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD11DC433F5 for ; Thu, 25 Nov 2021 03:13:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F01DB6B0074; Wed, 24 Nov 2021 22:13:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB1AC6B0075; Wed, 24 Nov 2021 22:13:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D789F6B007B; Wed, 24 Nov 2021 22:13:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0132.hostedemail.com [216.40.44.132]) by kanga.kvack.org (Postfix) with ESMTP id C523B6B0074 for ; Wed, 24 Nov 2021 22:13:07 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8545C82D5A6B for ; Thu, 25 Nov 2021 03:12:57 +0000 (UTC) X-FDA: 78845980794.20.55BD62F Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by imf11.hostedemail.com (Postfix) with ESMTP id BC167F000201 for ; Thu, 25 Nov 2021 03:12:55 +0000 (UTC) Received: by mail-pg1-f179.google.com with SMTP id m15so3882545pgu.11 for ; Wed, 24 Nov 2021 19:12:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=glg1j47E6Aumck1TvU69ObionOgNID+ISN0rKvu33AA=; b=yn2UNbxlYFY2bGnEcm0Uitdr39Sk/YbEfknDVPLdKL15PNdiHKWj+rAcIrHIr47CbU j3K4EakXwUpgxzau9nnhiOWabJ7xYZtJh9P1k+iE1nreuTjM5cWD9piyc8sEOsvIMcwG uKRIx1qT4FHE6YeUJraue0Wu+zZUMTMRpAmA+aASFZ8asatw8s7nltStdPAaGZwNeeFO ETCqGVHsxsrJmpAKSyVnDkfw14n7MjEo3vr0aYJGnyd8AvyxGISpxbg8D/aAUES+y1wm Y7oRrBR2NVhCAescMY0ZOIl4a7QvNm74hN1lziUgU9cH3K1rlqPDb5CjUwp4MGMnKJ23 E7IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=glg1j47E6Aumck1TvU69ObionOgNID+ISN0rKvu33AA=; b=SFTGNl2WFd9s8obyWzLzV0ZNOMoIiCIZiGzXKmmpOEnniVJEuHIFLPc8ukeFwEu14S DtEgo2H0QHz3cSpIHPADcBWYXU81zrasn6Pm2H4Gcf+m/KV06CgHKrl1qZZGj+yS0WQN mYTOohn2rhQULeMOEp4aQv9gSt3JEprRgEKkFNCbTCoV4OAQxw1QOzjY5DBJSK+GNzgE IPXV0djPhtUd7MqYv1UkbLAoshBCBqdi/zudDTFJbhHKuQJy5U6+Un1Cf+ZHGZKJYrft fZBnOTdqmREZ2/s9RxdwM1B0eVYgheTvrTsrRdsEWpHHyMN0hLp35Kkeco8faYQaaTGN ebDQ== X-Gm-Message-State: AOAM531L3jN3TGZPuApvrYqCMfGnVDX7dE9crZA25oSRWNCT7InEnsf5 HBMEDkm2S0VqWqZwBBby8LnMRw== X-Google-Smtp-Source: ABdhPJwsDUQx2tG9dyc5T5xwcOQ8UU5feP0PsguUtZmWM6A25NY8Er/dsWohJHQqW0f+aMQUhvJooQ== X-Received: by 2002:a63:1a4e:: with SMTP id a14mr13854390pgm.273.1637809974247; Wed, 24 Nov 2021 19:12:54 -0800 (PST) Received: from C02FT5A6MD6R.bytedance.net ([61.120.150.76]) by smtp.gmail.com with ESMTPSA id k2sm1238755pfc.9.2021.11.24.19.12.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 24 Nov 2021 19:12:53 -0800 (PST) From: Gang Li To: Hugh Dickins , Andrew Morton , "Kirill A. Shutemov" Cc: stable@vger.kernel.org, songmuchun@bytedance.com, Gang Li , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4] shmem: fix a race between shmem_unused_huge_shrink and shmem_evict_inode Date: Thu, 25 Nov 2021 11:12:43 +0800 Message-Id: <20211125031244.89848-1-ligang.bdlg@bytedance.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: BC167F000201 X-Stat-Signature: wwoec4bajzyeg9es6egpifsh7qodzr3m Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=yn2UNbxl; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf11.hostedemail.com: domain of ligang.bdlg@bytedance.com designates 209.85.215.179 as permitted sender) smtp.mailfrom=ligang.bdlg@bytedance.com X-Rspamd-Server: rspam02 X-HE-Tag: 1637809975-823434 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch fixes a data race in commit 779750d20b93 ("shmem: split huge pages beyond i_size under memory pressure"). Here are call traces causing race: Call Trace 1: shmem_unused_huge_shrink+0x3ae/0x410 ? __list_lru_walk_one.isra.5+0x33/0x160 super_cache_scan+0x17c/0x190 shrink_slab.part.55+0x1ef/0x3f0 shrink_node+0x10e/0x330 kswapd+0x380/0x740 kthread+0xfc/0x130 ? mem_cgroup_shrink_node+0x170/0x170 ? kthread_create_on_node+0x70/0x70 ret_from_fork+0x1f/0x30 Call Trace 2: shmem_evict_inode+0xd8/0x190 evict+0xbe/0x1c0 do_unlinkat+0x137/0x330 do_syscall_64+0x76/0x120 entry_SYSCALL_64_after_hwframe+0x3d/0xa2 A simple explanation: Image there are 3 items in the local list (@list). In the first traversal, A is not deleted from @list. 1) A->B->C ^ | pos (leave) In the second traversal, B is deleted from @list. Concurrently, A is deleted from @list through shmem_evict_inode() since last reference counter of inode is dropped by other thread. Then the @list is corrupted. 2) A->B->C ^ ^ | | evict pos (drop) We should make sure the inode is either on the global list or deleted from any local list before iput(). Fixed by moving inodes back to global list before we put them. Fixes: 779750d20b93 ("shmem: split huge pages beyond i_size under memory pressure") Signed-off-by: Gang Li --- mm/shmem.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 9023103ee7d8..e6ccb2a076ff 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -569,7 +569,6 @@ static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo, /* inode is about to be evicted */ if (!inode) { list_del_init(&info->shrinklist); - removed++; goto next; } @@ -577,12 +576,12 @@ static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo, if (round_up(inode->i_size, PAGE_SIZE) == round_up(inode->i_size, HPAGE_PMD_SIZE)) { list_move(&info->shrinklist, &to_remove); - removed++; goto next; } list_move(&info->shrinklist, &list); next: + sbinfo->shrinklist_len--; if (!--batch) break; } @@ -602,7 +601,7 @@ static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo, inode = &info->vfs_inode; if (nr_to_split && split >= nr_to_split) - goto leave; + goto move_back; page = find_get_page(inode->i_mapping, (inode->i_size & HPAGE_PMD_MASK) >> PAGE_SHIFT); @@ -616,38 +615,43 @@ static unsigned long shmem_unused_huge_shrink(struct shmem_sb_info *sbinfo, } /* - * Leave the inode on the list if we failed to lock - * the page at this time. + * Move the inode on the list back to shrinklist if we failed + * to lock the page at this time. * * Waiting for the lock may lead to deadlock in the * reclaim path. */ if (!trylock_page(page)) { put_page(page); - goto leave; + goto move_back; } ret = split_huge_page(page); unlock_page(page); put_page(page); - /* If split failed leave the inode on the list */ + /* If split failed move the inode on the list back to shrinklist */ if (ret) - goto leave; + goto move_back; split++; drop: list_del_init(&info->shrinklist); - removed++; -leave: + goto put; +move_back: + /* + * Make sure the inode is either on the global list or deleted from + * any local list before iput() since it could be deleted in another + * thread once we put the inode (then the local list is corrupted). + */ + spin_lock(&sbinfo->shrinklist_lock); + list_move(&info->shrinklist, &sbinfo->shrinklist); + sbinfo->shrinklist_len++; + spin_unlock(&sbinfo->shrinklist_lock); +put: iput(inode); } - spin_lock(&sbinfo->shrinklist_lock); - list_splice_tail(&list, &sbinfo->shrinklist); - sbinfo->shrinklist_len -= removed; - spin_unlock(&sbinfo->shrinklist_lock); - return split; }