From patchwork Tue Jun 4 02:05:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanchu Xie X-Patchwork-Id: 13684568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65DC6C27C50 for ; Tue, 4 Jun 2024 02:06:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9DE26B008A; Mon, 3 Jun 2024 22:06:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D28126B008C; Mon, 3 Jun 2024 22:06:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B06986B0092; Mon, 3 Jun 2024 22:06:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8B8186B008A for ; Mon, 3 Jun 2024 22:06:12 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A74D614105F for ; Tue, 4 Jun 2024 02:06:11 +0000 (UTC) X-FDA: 82191566142.21.646E293 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf08.hostedemail.com (Postfix) with ESMTP id D4306160010 for ; Tue, 4 Jun 2024 02:06:09 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=a9ll87XC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3kHZeZgcKCDElhNaPUhTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yuanchu.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3kHZeZgcKCDElhNaPUhTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yuanchu.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717466769; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TjWUm/jwk/hpgAY6bYVVdFyPbNpQM4PAlbOY34rPVUc=; b=4VQF/TnGWPpoWAj2sYlbQxu1JCF689l0T+ROYlX4GVLKq9gxLGqGu9qbksNpz0Wymip2NI qwEqhfoIJDwJRNkmw0pI9J5WfwVUyoxDb/nxqNtCzwTUtYc37J5jVZeI7JM9DAi0nfqZyr PKeMHVRhAwGuh4/HtGDDn4u+HPIrmmw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=a9ll87XC; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of 3kHZeZgcKCDElhNaPUhTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yuanchu.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3kHZeZgcKCDElhNaPUhTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--yuanchu.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717466769; a=rsa-sha256; cv=none; b=UB48ogHjcXkhqzC4m+36Lj3Wzzlu82AAGPPuY6K7o5VxSlKUlJ0JS7ezS83PQKR9vSMG2b bsXrTubP4Vwdv9oR3vsxfOlhXMvtWrUMaZ/AX6IW+UseMNEpt/NOS25fMH3AwNxWrMMn9v LVHCJwJPg10PIFt/1r3xzTGCDQ8Hb/0= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-62a27e501d4so10189357b3.3 for ; Mon, 03 Jun 2024 19:06:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717466769; x=1718071569; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=TjWUm/jwk/hpgAY6bYVVdFyPbNpQM4PAlbOY34rPVUc=; b=a9ll87XCq7zHX8sERFkDcdedNXWFWKIvN/8ut98u2R0Sap32+oLdFlrbpC/0Wq5qpw /Gauklq7o8Ez13JsD6KtO5RD+CXJbDfKe9hHTgSMl2NKa1uayfhMhl8fltMTOVRL4e/F tLHLosqf7l7seV7BsZYGZM9NAb27BxJ0fM49ITgWhvDzKkmHL82bv7O+la817kYGWMac GxohaEW8RtdGo1TdaL8vAUYGT7ylqZPw8pYQl3uVFOTuBqBsV5ErgjGPQK0vZEO1AbUp zYd2/siRR8+ly21WR4RPryhQh5YAxrD23ylLVPLN0YdAjkhHb5j4dydsd2qzLUehFkZs oCrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717466769; x=1718071569; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=TjWUm/jwk/hpgAY6bYVVdFyPbNpQM4PAlbOY34rPVUc=; b=bs1qdPsxTtUh6jHxqRbZ4pb3kNmNinZ6+PdXvK2ODK6UcS50u7l5vriB1B5mEW4vUW J1Kh7aNr0UcdUQdTOHPhBk3ImFFOQqh58KttAd2pKSbuTewUMRDWF+si6BiXcYjLUczG jKJ/XPEbnJRa3c7zO1FLeHdLZqbx8SH0JD9Qs1jSyNc2hKva84CrrlAW3aN7Mkd8Lpl/ m9xiHnaQIn+3dPe+HyW64pvY7MEgPcLbJ5f1KGX1oE3uzwxAnsAVetYP0q/tM6V2itRp YaNZUFjwjHthotAX1uzzpOVnHg3uXRlHmBcN0SQkoUr3V+pweIbHTx+8bDHfVJpY1uGK RGpg== X-Forwarded-Encrypted: i=1; AJvYcCUKzVhZzvrXUWzqyo1WNdu8yN/p6XQU82V9nzATT6SLTZMJfXVxlD0HITzzzjVWxa81dibDGLniBdftHdNeQ4XOIGs= X-Gm-Message-State: AOJu0YzOQBn14RMX/JBxlNzm4/0sr6ZxTTb9pA1hAVV+wu0Fmf42FR6u dto/yKVLosTHMQyE/wgY/cwxGNzMxBsOLk6P07tVUZbXOlanA7av/Y9VfkYwzdlBD77UnSccicz daBwTqg== X-Google-Smtp-Source: AGHT+IEDfb+3JEeKpiUuy5LGS6SnXp9v414pUptf89nU5rjiLVpz5pdRpN0kM4+F2bCDbN9n10wQa2F2tJMo X-Received: from yuanchu-desktop.svl.corp.google.com ([2620:15c:2a3:200:367f:7387:3dd2:73f1]) (user=yuanchu job=sendgmr) by 2002:a05:6902:110d:b0:df1:d00c:130c with SMTP id 3f1490d57ef6-dfa73bf59bcmr1176974276.5.1717466768619; Mon, 03 Jun 2024 19:06:08 -0700 (PDT) Date: Mon, 3 Jun 2024 19:05:42 -0700 In-Reply-To: <20240604020549.1017540-1-yuanchu@google.com> Mime-Version: 1.0 References: <20240604020549.1017540-1-yuanchu@google.com> X-Mailer: git-send-email 2.45.1.467.gbab1589fc0-goog Message-ID: <20240604020549.1017540-2-yuanchu@google.com> Subject: [PATCH v2 1/8] mm: multi-gen LRU: ignore non-leaf pmd_young for force_scan=true From: Yuanchu Xie To: David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying , Muhammad Usama Anjum Cc: Kalesh Singh , Wei Xu , David Rientjes , Greg Kroah-Hartman , "Rafael J. Wysocki" , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Shuah Khan , Yosry Ahmed , Matthew Wilcox , Sudarshan Rajagopalan , Kairui Song , "Michael S. Tsirkin" , Vasily Averin , Nhat Pham , Miaohe Lin , Qi Zheng , Abel Wu , "Vishal Moola (Oracle)" , Kefeng Wang , Yuanchu Xie , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org X-Rspamd-Queue-Id: D4306160010 X-Stat-Signature: jfc5da6zr6wshoyxeddw8du8pemoyh3e X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1717466769-904864 X-HE-Meta: U2FsdGVkX1/TNQL6/nDaPCPWU2XlccudoSE2xinjS5tQwFrKGW2kEHRhU2Ne0SW0AYeOpMd3T846RAbse+ivcKk0Q6xlE9IhfFqgZrJ+8vY7kVVhxiOkdFeEz/IjzsiQGuIfOFRFgqQTx7dxb1XiR6rbRetNENac81eXd7Lo1o7Zu+pbq6UHMMfcRgAQDBqjiuPcdT+XjAuf7ywJTnrE49htEgjn0FIwCHVLWvQVhfucX+Ll5lBN1FEvMpX5W9oIOmb6VUezWGqUYVyPLFQZ/TJZuW9dCkVvlxpI/gD2hWhQamgSqDkUQ2/q24v47D6SVrIzKBGisxkMuoDbS2qVm7gAZO7vE1+ehur1FUp/PktdjN6XnJ2Pcr+XQRSG14mVK3XxH88w5R8aKQIS5r5b4ONBsBPp61HxSBvvYIr49bkaSTCjksy2dIG8d3jK3TvSTw5XWFstAubzVDFzZ0Y4sMbHj+5mt7p2VQvPr9YIqEorhKPD3j18z4GOjsE00nyq5RtbX9YPndw4vS4kkLRUFzxIBSVzswJbowWrvOVhw3xyixlO3N2Ud1jklkzW2DaoRAYnYMj65FR1aaftTROkQTYUYWz9708SFCOSxj7HmdyFrgId80UPXXpCW/X/N3niwq6/7iI0CeWdNn8EzMaZvQ1mSkFrD3JsSFtjgon6XTPNArA9MvRFr0WzzUz4VriTNOrpWlRjuqdik40dV3e7S1/QQoO/wSgU7IX/5NxgDC9f2x8O3FeVyV7NKrT0YvyG1hWRt9Wfrr9lRGWSFIhIUu3+qXKdGPzDUH6lhJ2t5U37whOkmJAvSjO7T3r8t0ujhXEqwZvzgbrfYOzjUfnQkyi719Adu1O0X/yg7M1ItD5fpuJ3FCHkw0uGBvVvveqzHBWNfVTOPpCcpTwWl5HJDu/IDmVWVcBeB8mV9L7sQVpIa+/xqh4BHVaqyMnMYyeHx25vidkoaimUxhCQzrM 1Er3AjBF shZPawHXJK7jdcyh47m3eXtUA1H4f/Nx1gapN4KRdflYZK9WRpcF+KQdDgeklx5PrurCmitZX9iWfuYmjporxdLwxjQJGLvOlgcDa9T12Iasi0t20LOJFsLaTGrrhwJyYV7nwd1ufyEKxc7a84c7Ly/+uMJSqcdAkutjPDrEId4C8LDATIWbEg7xIkjXqJZ7YYiMHGx9XnwN32SLWlQLTKQi2t8uXikEQ4h0uek4VsdiuhiDKvFTrSZWNGsZD1vffIBuu+tw8esS45ZlU4I/aYADGgcP9V4KtGlac8MWQA4uCrxs2c0+epwJdDHqFvrQ2ZNZWOHxzAocqpmUkEWm5jlpXDy4O+O+jTtDUKUf81L1yxg34rnrarSycnWsm7swJBA/agTt1tJ0/K5ZaLUe5QpBhJglsHBY6wdceXYzNidp85IgppCNeMUKJ5GlN4uBQitNQXpFqPxb2cPOL04ra4EIev7GpfNDF30w0ElufC7U5OSlbQbyqOMZj0JFpK3KMRoqjaQrtND2fM3ZwxNEuKlxOXA2Og8geyvY0GTUaFfKk48WI0diwiITkYszb4CzArNqWU05SytMSmX7wVT1zYEiieWQOCHRfezcPSCSNJ8yewsqG07ysMY17hl9toHh4LqHbB0uDLgIB3V5qIak3p4jmWJzVs6xmpswiTJCGq1Qz59kyFQZAaBTmXO6+dAiE6/jaIoIR69kgW7N9rwsBYkYp3MkaG7rYgOi3SenVLRUHUjU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When non-leaf pmd accessed bits are available, MGLRU page table walks can clear the non-leaf pmd accessed bit and ignore the accessed bit on the pte if it's on a different node, skipping a generation update as well. If another scan occurrs on the same node as said skipped pte. the non-leaf pmd accessed bit might remain cleared and the pte accessed bits won't be checked. While this is sufficient for reclaim-driven aging, where the goal is to select a reasonably cold page, the access can be missed when aging proactively for workingset estimation of a of a node/memcg. In more detail, get_pfn_folio returns NULL if the folio's nid != node under scanning, so the page table walk skips processing of said pte. Now the pmd_young flag on this pmd is cleared, and if none of the pte's are accessed before another scan occurrs on the folio's node, the pmd_young check fails and the pte accessed bit is skipped. Since force_scan disables various other optimizations, we check force_scan to ignore the non-leaf pmd accessed bit. Signed-off-by: Yuanchu Xie --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d55e8d07ffc4..73f3718b33f7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3548,7 +3548,7 @@ static void walk_pmd_range(pud_t *pud, unsigned long start, unsigned long end, walk->mm_stats[MM_NONLEAF_TOTAL]++; - if (should_clear_pmd_young()) { + if (!walk->force_scan && should_clear_pmd_young()) { if (!pmd_young(val)) continue;