From patchwork Tue Jul 2 07:40:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13719066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AB11C30658 for ; Tue, 2 Jul 2024 07:41:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04EC26B0098; Tue, 2 Jul 2024 03:41:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F198F6B009B; Tue, 2 Jul 2024 03:41:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D935F6B009E; Tue, 2 Jul 2024 03:41:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B754A6B0098 for ; Tue, 2 Jul 2024 03:41:13 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5374A141F06 for ; Tue, 2 Jul 2024 07:41:13 +0000 (UTC) X-FDA: 82294016826.26.4530BD2 Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by imf03.hostedemail.com (Postfix) with ESMTP id 8CCC020018 for ; Tue, 2 Jul 2024 07:41:11 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sW41Nsvb; spf=pass (imf03.hostedemail.com: domain of hughd@google.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719906054; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=v96APnDdVOJx5oTshVzUPV7cP/8jCbffCJX62A3mfo8=; b=j/++4oHYA4pWw//1xmh3UC0oeVV7ENXWTBKP7+J0w1PKaEQUJ6hfx65krw5mh+1vPEdhSN XOZyGpkmgNKmTDquGzjd3CY/DaAN2+ld6hEDk3ZRAckqvtVBn32TQdVxPEdVjg9kB8tES3 Nmf66f5NuYJsn13/VH1bAbSexLoZ3wQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=sW41Nsvb; spf=pass (imf03.hostedemail.com: domain of hughd@google.com designates 209.85.219.173 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719906054; a=rsa-sha256; cv=none; b=6G/RLuYER2s52tSe8JjQJjf2MDbRpPd3GEcMlNe7mtuJ34A8jIlU1diEeaudg8RSjj7fxf +wNdQLelgoNc7oviYkg+KmVmz+9NbH80XJoYCTziPaApjUpIqK9ZUS9+lFSyvbYPKaKG94 /zhvmouniAtsr+9g3WtQQSqzXV6+B2o= Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-e02748b2402so4193336276.0 for ; Tue, 02 Jul 2024 00:41:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1719906070; x=1720510870; darn=kvack.org; h=mime-version:message-id:subject:cc:to:from:date:from:to:cc:subject :date:message-id:reply-to; bh=v96APnDdVOJx5oTshVzUPV7cP/8jCbffCJX62A3mfo8=; b=sW41NsvbmVAhUheRtI9cRdln5HV6OQgjxgz1obvoLmeKYCI0X//PmIILmk6x5jqd2R Z1hXirNcYZUnSwiUYp+dTLIxVlu9cnANUTfBz42pAD9y4tBDOcOiZck2K+8qiv9xFZpK lGh3ANWmlHQg9+W7m/ljXKIryyrcCobVjUUS1xUkkm43S5+z9OcrVhzEtZeaFTryob5X iPQQaLsceSecTlfQup6M5/x8YE2Xtv/1Li379t9taX/ZZVNalXjbPtvt4RlGq0kQz3pb IfWjQKIvnsj9vP/wi1CjCs/SaAWE0/H0DXqvsvslN1jShM8bnbR/djkllNIUN5475htm VAsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719906070; x=1720510870; h=mime-version:message-id:subject:cc:to:from:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=v96APnDdVOJx5oTshVzUPV7cP/8jCbffCJX62A3mfo8=; b=eZLmPxg3smpAZY+ob+bN2E6cLeRrGttBS+h5YHgPMkpi27+Ofswm9ZDvsjsBLRMvvo EQv8kS2XODDH5TlKZOi/yLfP1A/t3KQeFplopdgkeaT3LCuIjOnG35JvAbRh3Se+3FmM TYhKnWxm+pBRyMeq54Orr1C9yw//h6KZDpKn+qt6FwL/aMC2pG7Jn0xpkn23TZ9s6bhm qDc8VLvOnN2INoe/h0u1O07h1IwKNh/oTKCjF7DsOHOCJIogz1OfoS9Fk/Swsl+zCq+L xJS1l7vRHIOqMDev//kds8mGqVdIrM8HLUFH1VHQ6yeanYaNYbYx4O4HeCoj7PuQBRhC XFeg== X-Forwarded-Encrypted: i=1; AJvYcCX+4KuxXRb/6Ybr3VyF4wb99UlJ1fozT4YFNqpbvqTyKddIb8vPF8+Y4uGjx0m/beFRyJeAdnEkrYGsoOLW7rHrwjc= X-Gm-Message-State: AOJu0YzCqsJtVY4yJ7tGFTeL/UJtakncoCd1gdEQ8MSSfTN2p5sBcH04 MLgsc5+7LjaREycaltzn4jBfJXtz6mvXGqCyRZCH0FEPg6sXGUco2/KtB1SUOQ== X-Google-Smtp-Source: AGHT+IEcCHJR7/DobPvgviabErk0cRA95wcHifggrrRYP7p9mHLzm5PGv7GjbQBauYxFT8hwA3QF+w== X-Received: by 2002:a25:8811:0:b0:e03:4e08:c93c with SMTP id 3f1490d57ef6-e036eaf4975mr8118729276.13.1719906070304; Tue, 02 Jul 2024 00:41:10 -0700 (PDT) Received: from darker.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e039c802626sm125563276.9.2024.07.02.00.41.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jul 2024 00:41:09 -0700 (PDT) Date: Tue, 2 Jul 2024 00:40:55 -0700 (PDT) From: Hugh Dickins To: Andrew Morton cc: Baolin Wang , Nhat Pham , Yang Shi , Zi Yan , Barry Song , Kefeng Wang , David Hildenbrand , Matthew Wilcox , Hugh Dickins , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH hotfix] mm: fix crashes from deferred split racing folio migration Message-ID: <29c83d1a-11ca-b6c9-f92e-6ccb322af510@google.com> MIME-Version: 1.0 X-Stat-Signature: uky4sxs5cdehje6mqxrw4mgwhuwedfzm X-Rspam-User: X-Rspamd-Queue-Id: 8CCC020018 X-Rspamd-Server: rspam02 X-HE-Tag: 1719906071-660729 X-HE-Meta: U2FsdGVkX1/DKoBbrmkmldw03fqlWY0H8sG8MTehdmpX5ZpYyryQiRj5W9illTysOG0nF50Rq/kZDqUZXbrEtziZCXoOLJscm60fClI4wTuemc7v0JnHpx/t16gjDJg1vfQ7WIr3tSxJ60WZDVQFkjVtBJfq55uVYGE+UGc+cPfbmtFD3XpHglet+TtpzxqDg0/msxJ1pokTpbnx54QjH4V4N/DHzC2c79mJaDCrpbliZFL53y0fvUOzfAaNf5VxB6yqY4AWan9RJqCwBydjOYa48oP8+hpVfJZvCiJ2hjby9PcRutrIOV7UMZuT5wP50TGvotPP9dO9B1UiKP5cVqn3W6SzIi8RFr1O1YVBCOjh0CP9K3qpCA7GWO0c6qKY4m9SFmI2Hmue95OFtr96uvTh9g3MNmV0HyC/mlBQ050iqAE7cnUy1VgI5e8jyR14WuQTKy0HbdXBHEix6+M6BzIVjJW9EhUTK1fWp6vxWUQsA2VmXXbPkjjaaKLMVMyY0tr2F6khoRXw6CVFt07Rf4ruSd0UM2LGh2hVSpfALU1B2SFk3ulYUkGkiq+hq+ZPhorFBc9PNud1baj2BQn1CbdARbA2beIekRRLzqH5un1EuLERClKBrH9eJudXUW0XdOltSjKViQhNcfy/u6fqmCJxokSnHL5G/B3wAaA+Rjb7BmDiVqOWTdYcKlN0J0FkDOHzi5p+HlD/dDnVlRnd5rKALYvUC8L4AQ4F5NqEOPg3FbYQT7QEYr8fG2/e/hnASAXcS8gtHEgJpMwNEc0J/h0Bk3M9nkrUwRqlRtLhE1p6dXr6QhnEOZE4K+LdSC6ldrEvRJpTex/t4rWM0NZ70WxHfinhMsy1ji6vjoUaogwl9GlaGtOfDM9Qk2dTlhPn0SicuRsvivtr9D6CBA0Qk+uIeoKm0EUOoFdR1/uA4jG+uor+OHhZxZbe3/aMIIM2gYUCbGpkda+Q4evxv61 Bg5ANk1Z TWptkl6CMoCeEUfPdo+PXxH15dtJeabfyb2v59teAlwkdxsB0B/FRPLoJJqjLkgPgunXoYda+pXuLlkYm3nSqxWGnNeSxlNdqbDh1Gc4W+OXdSpuXss3k3BPWkXpGH+cBHXd7/B9lM6JRNjdYbE2OX6ZHIqA4Ja1E7biIusproiWTzjUUgzcBZnpWVJs/sql/B1fJ0dHHzdGnRVO4dFxzwQqP/g+GqTCOndeI0NJBpyDgY3bm48CZEllLtzPw30sUUlbROpeGQ+ifcZXAvfg8K2B1ec72UFRD4ysELoT8g8QtHkR2+QutCB1JvkfmoldOFL5R8JpZxdWQ+qrEo/4Qvw9Xrjj9Ai4/qr+yMpXGKVwbSglRfnm/+N7u7/nIBvQga1GN X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Even on 6.10-rc6, I've been seeing elusive "Bad page state"s (often on flags when freeing, yet the flags shown are not bad: PG_locked had been set and cleared??), and VM_BUG_ON_PAGE(page_ref_count(page) == 0)s from deferred_split_scan()'s folio_put(), and a variety of other BUG and WARN symptoms implying double free by deferred split and large folio migration. 6.7 commit 9bcef5973e31 ("mm: memcg: fix split queue list crash when large folio migration") was right to fix the memcg-dependent locking broken in 85ce2c517ade ("memcontrol: only transfer the memcg data for migration"), but missed a subtlety of deferred_split_scan(): it moves folios to its own local list to work on them without split_queue_lock, during which time folio->_deferred_list is not empty, but even the "right" lock does nothing to secure the folio and the list it is on. Fortunately, deferred_split_scan() is careful to use folio_try_get(): so folio_migrate_mapping() can avoid the race by folio_undo_large_rmappable() while the old folio's reference count is temporarily frozen to 0 - adding such a freeze in the !mapping case too (originally, folio lock and unmapping and no swap cache left an anon folio unreachable, so no freezing was needed there: but the deferred split queue offers a way to reach it). Fixes: 9bcef5973e31 ("mm: memcg: fix split queue list crash when large folio migration") Signed-off-by: Hugh Dickins Cc: stable@vger.kernel.org Reviewed-by: Baolin Wang Reviewed-by: Zi Yan Signed-off-by: Hugh Dickins Reviewed-by: Zi Yan --- This patch against 6.10-rc6: Kefeng has commits in the mm-tree which which will need adjustment to go over this, but we can both check the result. I have wondered whether just reverting 85ce2c517ade and its subsequent fixups would be better: but that would be a bigger job, and probably not the right choice. mm/memcontrol.c | 11 ----------- mm/migrate.c | 13 +++++++++++++ 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 71fe2a95b8bd..8f2f1bb18c9c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7823,17 +7823,6 @@ void mem_cgroup_migrate(struct folio *old, struct folio *new) /* Transfer the charge and the css ref */ commit_charge(new, memcg); - /* - * If the old folio is a large folio and is in the split queue, it needs - * to be removed from the split queue now, in case getting an incorrect - * split queue in destroy_large_folio() after the memcg of the old folio - * is cleared. - * - * In addition, the old folio is about to be freed after migration, so - * removing from the split queue a bit earlier seems reasonable. - */ - if (folio_test_large(old) && folio_test_large_rmappable(old)) - folio_undo_large_rmappable(old); old->memcg_data = 0; } diff --git a/mm/migrate.c b/mm/migrate.c index 20cb9f5f7446..a8c6f466e33a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -415,6 +415,15 @@ int folio_migrate_mapping(struct address_space *mapping, if (folio_ref_count(folio) != expected_count) return -EAGAIN; + /* Take off deferred split queue while frozen and memcg set */ + if (folio_test_large(folio) && + folio_test_large_rmappable(folio)) { + if (!folio_ref_freeze(folio, expected_count)) + return -EAGAIN; + folio_undo_large_rmappable(folio); + folio_ref_unfreeze(folio, expected_count); + } + /* No turning back from here */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; @@ -433,6 +442,10 @@ int folio_migrate_mapping(struct address_space *mapping, return -EAGAIN; } + /* Take off deferred split queue while frozen and memcg set */ + if (folio_test_large(folio) && folio_test_large_rmappable(folio)) + folio_undo_large_rmappable(folio); + /* * Now we know that no one else is looking at the folio: * no turning back from here.