From patchwork Thu Jun 27 20:17:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13715003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65ADEC2BD09 for ; Thu, 27 Jun 2024 20:17:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B53DC6B00AB; Thu, 27 Jun 2024 16:17:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADD756B00AC; Thu, 27 Jun 2024 16:17:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9562B6B00AD; Thu, 27 Jun 2024 16:17:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6DFBB6B00AB for ; Thu, 27 Jun 2024 16:17:42 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 08411A4268 for ; Thu, 27 Jun 2024 20:17:42 +0000 (UTC) X-FDA: 82277779164.22.845FAFF Received: from mail-vk1-f174.google.com (mail-vk1-f174.google.com [209.85.221.174]) by imf16.hostedemail.com (Postfix) with ESMTP id 31B13180011 for ; Thu, 27 Jun 2024 20:17:39 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Nx6+ioz2; spf=pass (imf16.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719519450; a=rsa-sha256; cv=none; b=EGynf3vWIrWIB986cm778v77lOfDdSS9HqN1NoN3QwmVJPCTJtZgCMXDWHA5x3lOmj2Ceb jfNVUBolf3v5/DdZKl8DF3qdhM9W+MJDUTFRYvrlcC41t6bKNa/5tSTxEgOQHdkp8eUeUR opIIm3gktmzradWJH4BGztPu8uHZ4OU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Nx6+ioz2; spf=pass (imf16.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719519450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SfWTtRmgo1E8I0TtuekVypKcxx4UC0A4xdyehBHx0NE=; b=oTwSHFj+42PChx1s/pAr5GNoTZRDy1Jo+fs0Um3m/uBS2vTM8gu2Qk6FKYvQzfgKmb/e1d VA7KcLLznxBXLNa6wGASHMMtzeohL8bjgAhUwTM1jvVHwsrYsTpdUszT0i6sUYTxw5pHYE pn+nl9e19tR2hsYnazdpuZG0Ookx8V8= Received: by mail-vk1-f174.google.com with SMTP id 71dfb90a1353d-4ef33a09a3aso2433932e0c.2 for ; Thu, 27 Jun 2024 13:17:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719519458; x=1720124258; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SfWTtRmgo1E8I0TtuekVypKcxx4UC0A4xdyehBHx0NE=; b=Nx6+ioz2YfPAPv8/gWjy+NLRvLRTLe72GES4u9QtJCxczS5lV7guyg5G03+7LB8z8T /YdAfX2GbMMmlb5FuDcPpHoeH559H3yTgxXxOJFZYpDEHa4j9RnpriC8C8hf502XKWpA aeKo4b8J4V0TRnU+kbMIZN8+Yh5ccAjwSbTJx24qq56APycPo+LQbjAB7OgzXAM+6X6X EmDik1XO2y9llqhyScTOPnQM++p5ufmvi7sXKVLbM0CgIezOaDQOYI3IYCZmayTIXl/S hE/MeYRebtWFcaPRZTr5XSHDEseaDWO6PGKM3qE6OZ0w2HZMbmAJpd8PCErpEgfyax39 bpzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719519458; x=1720124258; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SfWTtRmgo1E8I0TtuekVypKcxx4UC0A4xdyehBHx0NE=; b=B/kHgouLsqVZOAWJS8wsYawGArTspK3n9ha6FB4sAk3uZgNzOrJxcExyZ7OTrgsFWO 3VkTOzrDCR0xAh4S7IVpHM83U60Qcvgt9OGnbg8d3hxxcEYRLTkU6AqnHhNDqnF0dWEe mZ+fX6gwZ0PYvmT8Quikm5YiEU3MIih/l3qYDA4cx5S0iTY3/hdxAXO06httwwYFZxse Q11ktQZmm8kEpJGVQ1vYVt5RemTIVFXzoCcW2LmwgPjscHfUlmbsxw1M0lr4ElKGu4tB t4jYHWUXQ6mFPLXo2rx8ybtAck3Ao1qJk9izjrsOzDgi21szF6CLs5fPDfSmsftLbL3Z EdAA== X-Forwarded-Encrypted: i=1; AJvYcCUWQGhShq7FQRIktMVoS/QPYCFv0LTuGB6Vl2SNdTg16cdEdpTEwBCnZvf1AFpbIB4ifDrrJ3xK0fxVjkF4gDGSCxg= X-Gm-Message-State: AOJu0YyE3A8pqFCxspXT1WkpXbOU9kmddAaJxHoVdyUBajzcrMPjO9dz GzQv0n6LUWD8wzsLRIMyskYd1knRScrXB0pTNkMbr27XU7w5Axmu X-Google-Smtp-Source: AGHT+IHR0tAYDBylwArb4Spu8TCeyDyzv0vW6zgLAt6R2VTvPr0551huRj1vxUJvSv/B9E8w/0zdIw== X-Received: by 2002:a05:6122:4592:b0:4d4:21cc:5f4f with SMTP id 71dfb90a1353d-4ef6a73950emr12317938e0c.11.1719519457919; Thu, 27 Jun 2024 13:17:37 -0700 (PDT) Received: from localhost (fwdproxy-nao-115.fbsv.net. [2a03:2880:23ff:73::face:b00c]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b59e609b9asm1691706d6.106.2024.06.27.13.17.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Jun 2024 13:17:37 -0700 (PDT) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, willy@infradead.org, david@redhat.com, ryan.roberts@arm.com, ying.huang@intel.com, viro@zeniv.linux.org.uk, kasong@tencent.com, yosryahmed@google.com, shakeel.butt@linux.dev, linux-fsdevel@vger.kernel.org Subject: [PATCH] cachestat: do not flush stats in recency check Date: Thu, 27 Jun 2024 13:17:37 -0700 Message-ID: <20240627201737.3506959-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <000000000000f71227061bdf97e0@google.com> References: <000000000000f71227061bdf97e0@google.com> MIME-Version: 1.0 X-Stat-Signature: u6qrbdkm5qz5osbkjnad9h7fyozezuf1 X-Rspamd-Queue-Id: 31B13180011 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719519458-780487 X-HE-Meta: U2FsdGVkX1+8lfKZwZOE2H6o9RK+wYgRUJS7gUNqvguZu1PFDL1Q4gSglU0DhPsHFNlf01d7DYrcA4dWu2QPWmLLStPN7uDLBhI9mo26IfBZtO3KqdxrOeCpI2LQc3iybvICUSsQYNrBZ2Zzg0qNTsvAvPJkXKEc9Whh+anZQafOKFnnIJKR+WYA1R6xM142hJcxwyJe6OkJHO+3gJaziwgpHHMJ/A4rd1O2aqT8VidToPZYpYQvQP/UVUd2J5IzsMWl3fFuS405NY1y9DVkao2bbwmwkjhM9GjP7ygkRYYpMMO1j+n02aulubR0r23xAES2aSN++mPZ7atpXGIUm2k73botvCosexwubt4U68f/YqDrN/iP1WKgnbdiC5OJwTJ9aliC2TwrBcmNaGZ/HHLRcQiqw7vPnTaVjs10o6f+IS/Iqbt2kuSC1gW4qe8HLKxvOlD4tjBnOjCNomggLWueE1OuSVZrWURTIve2/4L8JGlwSsej9qIFvPGyGV7hlY3o9KitROHW+awDU0jucKVac4zXLF4WUKGIupqfrm+Fl7EhhcxA0PJktLdhgJO6iItGBKK0FCwQPmr6oi3O0zDYFrjzrVyTxWQuMK9MVTOOHPSdGGFAt15RIUEuF5nW6AHe4WzhfpR0xgvCYwryI2v9gijyIQzDiYR91RNwsIkt7rDCPtrAOyR8X+w8iLAPCmogs8eGmBJKIM5kQedZpTpAgPdA6IoWOr7G/C0AtMpFWw9QNX33Kk9Dc5lQYVezNr+yc/hu0z0tgqfNlbPqGSEggP6O5Ic8zp1DDkIlld6hpfx6mwQf0QkMblCjWOlfBwiLteoCSrmL6dxtDPHm0ukSkbui80HxSklJe+721ksiESsJSQKJVgQ4lLSUoulCVI4fPJLRuYOuE/HFWaLT9lHRoEEwSJA9ZGYAmceeQX7FhCqld5JWvEOUAdCVKEpA4dOg9cqPv3Jszp7U0fm CjFRkKUd qr8utYEUKExJKdgWXvrdN+C+isLvtlKR+peKKlYr0g/zZBnnHcZ19XlgvY6tpx4JmDLRw6iLwYNgr7adn/EPc7cT4k+1iUEIE+xmkriWeH71O9rTSpLqybuXS/ti0KDUU4Fgw3UTBlRSNXERAN4TfMcBxnXKmTp/Quq1v6hodgfdOvPoVw99HTAULYsBoJdF1tUXWpfGZLS0TdaRz23DgIY7oWHwo4vxYRBuVzGEM0c5eFv4fxWZK/Hdhto+Czu0Nhvo7omHZhEOpt0eyI4phCv0KYgiFZ3vsPiA0oJvl0K4ggsIybRyTGF4LTiPK/3vXWLb0Lk5FkhPlFgDt/dviaKGLhRlr3jcG9nLAkhIjs+tqAt+6XBAxYaSSux7PL+3G5OxWtv/DNv+iH7g4r7vktFnFhADujlPpEME7an4mw0/95BOy0s05u68rXPtPyKsM7Zi4oKFWT/Vzf3m5AunlEkDoFkifTc3lVgc7wO3Ys9gThkg3GVfVYZKuiw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: syzbot detects that cachestat() is flushing stats, which can sleep, in its RCU read section (see [1]). This is done in the workingset_test_recent() step (which checks if the folio's eviction is recent). Move the stat flushing step to before the RCU read section of cachestat, and skip stat flushing during the recency check. [1]: https://lore.kernel.org/cgroups/000000000000f71227061bdf97e0@google.com/ Reported-by: syzbot+b7f13b2d0cc156edf61a@syzkaller.appspotmail.com Closes: https://lore.kernel.org/cgroups/000000000000f71227061bdf97e0@google.com/ Debugged-by: Johannes Weiner Suggested-by: Johannes Weiner Signed-off-by: Nhat Pham Fixes: b00684722262 ("mm: workingset: move the stats flush into workingset_test_recent()") Cc: stable@vger.kernel.org # v6.8+ Acked-by: Johannes Weiner Acked-by: Shakeel Butt --- include/linux/swap.h | 3 ++- mm/filemap.c | 5 ++++- mm/workingset.c | 14 +++++++++++--- 3 files changed, 17 insertions(+), 5 deletions(-) base-commit: a5c6fededf806aba1ff9b0f01278f7d089da5725 diff --git a/include/linux/swap.h b/include/linux/swap.h index bd450023b9a4..e685e93ba354 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -354,7 +354,8 @@ static inline swp_entry_t page_swap_entry(struct page *page) } /* linux/mm/workingset.c */ -bool workingset_test_recent(void *shadow, bool file, bool *workingset); +bool workingset_test_recent(void *shadow, bool file, bool *workingset, + bool flush); void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages); void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg); void workingset_refault(struct folio *folio, void *shadow); diff --git a/mm/filemap.c b/mm/filemap.c index fedefb10d947..298485d4b992 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4248,6 +4248,9 @@ static void filemap_cachestat(struct address_space *mapping, XA_STATE(xas, &mapping->i_pages, first_index); struct folio *folio; + /* Flush stats (and potentially sleep) outside the RCU read section. */ + mem_cgroup_flush_stats_ratelimited(NULL); + rcu_read_lock(); xas_for_each(&xas, folio, last_index) { int order; @@ -4311,7 +4314,7 @@ static void filemap_cachestat(struct address_space *mapping, goto resched; } #endif - if (workingset_test_recent(shadow, true, &workingset)) + if (workingset_test_recent(shadow, true, &workingset, false)) cs->nr_recently_evicted += nr_pages; goto resched; diff --git a/mm/workingset.c b/mm/workingset.c index c22adb93622a..a2b28e356e68 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -412,10 +412,12 @@ void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg) * @file: whether the corresponding folio is from the file lru. * @workingset: where the workingset value unpacked from shadow should * be stored. + * @flush: whether to flush cgroup rstat. * * Return: true if the shadow is for a recently evicted folio; false otherwise. */ -bool workingset_test_recent(void *shadow, bool file, bool *workingset) +bool workingset_test_recent(void *shadow, bool file, bool *workingset, + bool flush) { struct mem_cgroup *eviction_memcg; struct lruvec *eviction_lruvec; @@ -467,10 +469,16 @@ bool workingset_test_recent(void *shadow, bool file, bool *workingset) /* * Flush stats (and potentially sleep) outside the RCU read section. + * + * Note that workingset_test_recent() itself might be called in RCU read + * section (for e.g, in cachestat) - these callers need to skip flushing + * stats (via the flush argument). + * * XXX: With per-memcg flushing and thresholding, is ratelimiting * still needed here? */ - mem_cgroup_flush_stats_ratelimited(eviction_memcg); + if (flush) + mem_cgroup_flush_stats_ratelimited(eviction_memcg); eviction_lruvec = mem_cgroup_lruvec(eviction_memcg, pgdat); refault = atomic_long_read(&eviction_lruvec->nonresident_age); @@ -558,7 +566,7 @@ void workingset_refault(struct folio *folio, void *shadow) mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); - if (!workingset_test_recent(shadow, file, &workingset)) + if (!workingset_test_recent(shadow, file, &workingset, true)) return; folio_set_active(folio);