From patchwork Sat May 4 07:30:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanchu Xie X-Patchwork-Id: 13653797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 618C9C25B5C for ; Sat, 4 May 2024 07:30:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D871F6B008C; Sat, 4 May 2024 03:30:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D398B6B0092; Sat, 4 May 2024 03:30:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD8546B0093; Sat, 4 May 2024 03:30:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9EF806B008C for ; Sat, 4 May 2024 03:30:50 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3E5688127C for ; Sat, 4 May 2024 07:30:50 +0000 (UTC) X-FDA: 82079891460.14.01DF3C0 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf19.hostedemail.com (Postfix) with ESMTP id 7C82D1A0008 for ; Sat, 4 May 2024 07:30:47 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=RHgtqvgw; spf=pass (imf19.hostedemail.com: domain of 3JuQ1ZgcKCP03zfshmzlttlqj.htrqnsz2-rrp0fhp.twl@flex--yuanchu.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3JuQ1ZgcKCP03zfshmzlttlqj.htrqnsz2-rrp0fhp.twl@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714807847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/qRNmge9VPkattIC8VKnofvZP4QSD6CRqtGoSzsf/P0=; b=8DZvDO76KrlXccQ2A3JtSOwGBdnovS+UUuemZG6Cx/aJ4b/baEyBXrRL+G/fM6OufviuIe YkCDiWzT5BdE38mGWcJxk4JlBQhcLEkqvQgJsi6ZphGGBVddDFKlcdOcGGgaBgGTJZq55H KMO64+zwzD4QAEdZDntlgP0yk9WMPDw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=RHgtqvgw; spf=pass (imf19.hostedemail.com: domain of 3JuQ1ZgcKCP03zfshmzlttlqj.htrqnsz2-rrp0fhp.twl@flex--yuanchu.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3JuQ1ZgcKCP03zfshmzlttlqj.htrqnsz2-rrp0fhp.twl@flex--yuanchu.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714807847; a=rsa-sha256; cv=none; b=gLkWiRIBt0eDBUITZQskD2hwXc5b7dPCvtHl01NbNKLoU+Km9m+sioGeqaId3Ta1GUp1Dg aWDne8SHZebu1xClD7WSzf3J6oDCiVYs18apnPgJoJ9LAQShMw4VouHq1NhhrjL0L6KGEB B+KOMR591tmm4kOBjiWWlcqBHMcdA5g= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc743cc50a6so531591276.2 for ; Sat, 04 May 2024 00:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714807846; x=1715412646; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/qRNmge9VPkattIC8VKnofvZP4QSD6CRqtGoSzsf/P0=; b=RHgtqvgwEr78XC2R34LFGftHdkNr1wbBHbAyQJ9t67ao9t4ypF9ansFEyaiNpusiMt 2FpPjKRST1+2n1vE9hzZvIdI6Pk9HhQHmlFX8LbEVEyZ7qFvkJUiJul4Ue+pA7UMWQhx qNxTvDRhEdFZL1CA43j6VXePRqBIcuXJGS88PYZIUYrAHEiQOGVTxDkbM3RoS5VH0Xo0 LFZLNq1ivFb2TVycuxja9YBK/8vZ8Es9/3vh7Mzl3ODl1QsrUCb40VwyGbDLlEW7DiB3 +fW8Wqa6TXkRT4E4+4V0uldpLMu8wPJiVkdRIzIBii0UYbvg9QG0ydCmTHyN9+MYK/M/ kDyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714807846; x=1715412646; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/qRNmge9VPkattIC8VKnofvZP4QSD6CRqtGoSzsf/P0=; b=wuodLrYxU4TfCkTLWQBbXU9EvmIFG1fXDvWQipP1oNVx8yyHnFsWJHBtSrolaOP8o8 0vt1XkCJ9aQlbL6jylZBkz0B3kwv9QS25lVTpwrcZvaaR7WqVeP2IQLkKgaLiMQn96Lx q31Em0PFUTdYMnVtqL+kUd8kSXqPZs89fDJq/98fd+mFMhW4/Dwoarw5mtDEL8CSvNbY mY/JNei6aI7MvTEfP6yKsw+mtDa4Qc2LyTyWlLu5qGKOyewjYXcFp4ZA98Enaquv4Ku/ WK2sOB3X4mk8t32H7NB1JHy9YWANYmQYj14ZMETO2f8a7Ue1a7VOXxh93KSSWQrYmTDw isEw== X-Forwarded-Encrypted: i=1; AJvYcCUeSiREqdZS9EiVcv2TG66h/9EXp1t1q4jBUkI17jyjbZwdgOeTgh69S9b9l9rXzcVqNjunMxjSV+jgsLHZCSRxWbU= X-Gm-Message-State: AOJu0Ywz69pTHmsidjEkdi8WBO2mrUrDJroMcQ7h3Z4eToWZIC8RctRl 8XVnpDJLQEAOZx++avz0BeBr6zLPaUdADlP1imtodzTtKBdMYqsotBPHirKDYZE6IfArOJ6V3bD nCTzfMQ== X-Google-Smtp-Source: AGHT+IE9krkIkSbJnvwj2NL+PSHxSR2wSCv1KndjNbNT83C6ldeF/Rb3CC7Y0V2+f15W7Gb9F6OShS6x5yDQ X-Received: from yuanchu-desktop.svl.corp.google.com ([2620:15c:2a3:200:da8f:bd07:9977:eb21]) (user=yuanchu job=sendgmr) by 2002:a05:6902:726:b0:dd9:2a64:e98a with SMTP id l6-20020a056902072600b00dd92a64e98amr544178ybt.9.1714807846444; Sat, 04 May 2024 00:30:46 -0700 (PDT) Date: Sat, 4 May 2024 00:30:05 -0700 In-Reply-To: <20240504073011.4000534-1-yuanchu@google.com> Mime-Version: 1.0 References: <20240504073011.4000534-1-yuanchu@google.com> X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <20240504073011.4000534-2-yuanchu@google.com> Subject: [PATCH v1 1/7] mm: multi-gen LRU: ignore non-leaf pmd_young for force_scan=true From: Yuanchu Xie To: David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying Cc: Kalesh Singh , Wei Xu , David Rientjes , Greg Kroah-Hartman , "Rafael J. Wysocki" , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Shuah Khan , Yosry Ahmed , Matthew Wilcox , Sudarshan Rajagopalan , Kairui Song , "Michael S. Tsirkin" , Vasily Averin , Nhat Pham , Miaohe Lin , Qi Zheng , Abel Wu , "Vishal Moola (Oracle)" , Kefeng Wang , Yuanchu Xie , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org X-Stat-Signature: qjb4wo8nwdo9ze4matymn6euyo4ennji X-Rspamd-Queue-Id: 7C82D1A0008 X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1714807847-244329 X-HE-Meta: U2FsdGVkX1/FYBkr9oblU7aX7GKEQ9nthTvnArqmTdCcmfW4+1pvFl5nFbDDi+1+mBDZhDHpRcSeMQrxi9QECDFzPxj0P27VqWSdHif1qX3JDK6RFMDuZDAEpfEP4cZtAm+7WQPpl4xG3AzmZsoUarf4IgAzyVWoNnzcuWSH7e4TFfZXX0I8M2DyMw1b09XujA7IiJWy630wFZs7RfOWAFh984wMLMT8otrx04U9Kk+KKvQxe4KEBjmCtmioFWnaNmTRlZPxjG9ujhuYvCFQj4Fo2hWpDH85HX/6p1ZsKgDyIcJ9V0CC5oNOy9sEmq74Ob7AmI+E1pLTUU5aufCRmUTKtUDILAtGvQ0oC31XkjpyPlX4IFOJv+HtmhAsO4GgrOKKqZEv5pd7bbgflMl/O5LqE8Yjsv3nBhofGtMb5FZU3oloDGG9J2cLUivJPre6UTc6uGbQn75k2MMIRyOY96EYijMeNyRMFdW1zvB60yZzUdfyaXWqzypLvNINQgysReAxNXXkZ5e+5ImrTEJw4H3TmlK0BsALuAHVIEdglTkgqDPTQY538232j1UdfRLDVrkb+TvEvLlxtcLmpsJN2gmuq/TpMnLO595QJ4Q2QLh6O17CVSc5ZJwa/fmEDDuPF03LeDCaaSMWD1MgAOhCJQsz/gmeiR0C9hYgOJANV0mBqzrAqrrCZov0+iBM3LDrGXR+BMSiby4+dwiJi1zYQeUpLSWe7vnnqqQjuTNsR/esyBoFsyuN20Zy6VuElAD08ECxPKjnLRFD5K5Qt/lJ4DZ0FvqJlu1qiu0OwVWSHlNKhGcgE+WzSohrrngSh0lom4+a5fh19bg9LufuakkRIfoVguhifOWWBnuxcAAMLK1bAglE3ppf8or0Nh2wsp1DfeBRgNxbVfR9lzrSUckI7G3xv765Dr0NCg7UGW0kAJwfOGXA/SZiWtO/2O+/TrJ6bGWTcXBYNnKiE44I9dv XOZytxu4 fhKH6oaRYYReTW+JYVNTeAVXoh4YHCXBuYYP947px7f8e/taphFlBbwm8qGvzNNN8zewu5dmaDiJrY5lh3GxpSffPaSpC5EbUGZ3djwQB3sFj+nuq0TgkqazmgOCzGB8ffkEAuqnliNUD2gruEDWXaJsTSYec7gnE8KtrC7h/IGLgKPh+RWScdiGUVKik3+04maMD9K1ha1C7Ied/O9e/cK6tTJoFAhDmPH2tjGAsSRgHKB01dxu/A9to06aHnC3dkSjme9C47aCSvRpJgepArR8DukpBgOF54q4f+ikN0n4IOVLP9bKrZAkyd14dOpdI8yj+U6dJhxvD4NZPGTdejZ3WaJ0WZy6eD4vgijjAJDfx/V5lMtGiiQXB0sc+6QhRioGLyuiUegwXF696N3rbm/jllGI7nWBZBe33xN2oxPK5SO0WcDDz+2RlOnp6JBXbSN92uUVVKzUTYQYSi3Jr52/BO66C7Y60VgymNbDjfBjAi+vnt2VcQR9AsnsOPpxigBI0WiHc9x48YxBZa9FOXPbJpXdj5Aw8y6I4SlnPOtc+qJIGlhfiH4pbj5tHSMMVNyJ2HGNafYgogqd+o6QqH1u0Z+ipN/UNSLWyN/rHx+Z7+eUUNaSikpW2fzph1HKV72lScGR/KgxVPgFM8YhbLyjarIyfkY3cHoEnL7xXhFvbSb7SL8VB0ikGRgfCIA8HcbisUK0S7y18iaQky8OcQKdpPgQVjTcqfbww X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When non-leaf pmd accessed bits are available, MGLRU page table walks can clear the non-leaf pmd accessed bit and ignore the accessed bit on the pte if it's on a different node, skipping a generation update as well. If another scan occurrs on the same node as said skipped pte. the non-leaf pmd accessed bit might remain cleared and the pte accessed bits won't be checked. While this is sufficient for reclaim-driven aging, where the goal is to select a reasonably cold page, the access can be missed when aging proactively for workingset estimation of a of a node/memcg. In more detail, get_pfn_folio returns NULL if the folio's nid != node under scanning, so the page table walk skips processing of said pte. Now the pmd_young flag on this pmd is cleared, and if none of the pte's are accessed before another scan occurrs on the folio's node, the pmd_young check fails and the pte accessed bit is skipped. Since force_scan disables various other optimizations, we check force_scan to ignore the non-leaf pmd accessed bit. Signed-off-by: Yuanchu Xie --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 4f9c854ce6cc..1a7c7d537db6 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3522,7 +3522,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk->mm_stats[MM_NONLEAF_TOTAL]++; - if (should_clear_pmd_young()) { + if (!walk->force_scan && should_clear_pmd_young()) { if (!pmd_young(val)) continue;