From patchwork Sat Oct 19 01:29:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13842526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A7FBD3E194 for ; Sat, 19 Oct 2024 01:29:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C5156B00A5; Fri, 18 Oct 2024 21:29:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 874A06B00A6; Fri, 18 Oct 2024 21:29:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 73CA56B00A8; Fri, 18 Oct 2024 21:29:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 578816B00A5 for ; Fri, 18 Oct 2024 21:29:47 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6C8C41203CE for ; Sat, 19 Oct 2024 01:29:35 +0000 (UTC) X-FDA: 82688619552.06.0066C12 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf12.hostedemail.com (Postfix) with ESMTP id 9E74F40014 for ; Sat, 19 Oct 2024 01:29:39 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XyV+WeTw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3hwsTZwoKCCoPZNUaMNZUTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--jthoughton.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3hwsTZwoKCCoPZNUaMNZUTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--jthoughton.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729301223; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=YuCuzNJhXO6NMFow4//rMdfOZm/3Ta56iibLMueqV4o=; b=gK0quyrZwXKInLGeo5kjzVeUwXlWgijrCDytZ4DiRURIgEvhIIlph05h6zVJc9zoj0up3O Xl1imSciHiP2fmJFg0TQt0dmXMJQdU3SN088xTyWtMxfsQh+o0AS/nXkzZOyylZfv7rBoj aSd9n2HN6zWT5o020WUCV2sp2iroEUA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729301223; a=rsa-sha256; cv=none; b=TyrdDOfB2L+CungCT7eSW+Ud49ewptQmhDf3Ak69UVPjUQMSpPzNXY0QQzxMeY1w2B56Oc 6IbgBNlnKvuHZd3ar6LBQb3s58fBOM3UpTEZ5qAfj09yoQr4Yj8lW3znp1CNJpe87OOtOf ef206jjjOiWiaM+CrED138T8v3pWUMA= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XyV+WeTw; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 3hwsTZwoKCCoPZNUaMNZUTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--jthoughton.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3hwsTZwoKCCoPZNUaMNZUTMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--jthoughton.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e2954ada861so4308255276.3 for ; Fri, 18 Oct 2024 18:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729301384; x=1729906184; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=YuCuzNJhXO6NMFow4//rMdfOZm/3Ta56iibLMueqV4o=; b=XyV+WeTwfj/aNHrE7f9hDeWVzOAZw3Ym2vYqqS4PI72mlB5L3UN+FY9SQtoLrEagtk TrVHznFHMXkzO+gWPNUrOHGZGGZJCVJSF0WCgZ7iUOz+uFCd8d4mHQ6xu8cWWwSCZ+hp xCVUSg64lgRzxUR6ifZuMp4ThccegQwQI/ZmmaxEqLHIsSz+NdaqvG0ijGfV3DwPexSR 8NHydOz3FJF7NZTSe7QT/kU6295rMiRCOTu8trITKf8cPD3VpR2SyG0ge3/slbEUAn9D YXrnqjcFbTK+rS4y3Ifo6RYO4ShSKa4J2ivsLPIj0QFmuN5MDelbE0rL9qROVhkyiuh4 3mWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729301384; x=1729906184; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=YuCuzNJhXO6NMFow4//rMdfOZm/3Ta56iibLMueqV4o=; b=Ya3tQ8de4oQnzxmdhItHNyRJvx4sZG8Owq0FNFKTH4btSjpmxEC6Gm519sqL49cEE7 L1a6DMNEnKP20YcBgCZF1B/dlOb6dmNQxZlBhkIs36Z9uYILCbrq9XUX/m8TNNGXoQqj 69O5rhORq0f6AWuhN+SBdeS8D6/CDcDcLwvhxS/eRAYXJx5BnLibqSm3BCsNTaPj1zEQ KbGEkOORZPJWp8WbVvL4E/tLEqAJHu1gAsRUEyQi8wdiiXLogmxgX2xddZBw62qpSFKl 5zEXUg/VEzLy9HJuHCApW8ATM5G1UNrH/LsySYfGL/4VPv58cGHqVxdrJ8UL5jeUYIuk UNGg== X-Forwarded-Encrypted: i=1; AJvYcCXpsU7/mDPRSliX0Jhrz0TU59V1ulGoDfnQRvLaA93FBXAdFgKYIzHfBwxySv1e1x/f4jK3Z8udHA==@kvack.org X-Gm-Message-State: AOJu0YxGUpRlnBaGHhc1HGqRcqOOvP6J5PKsQUaTWazbZuu12EwlXEpP rquhJV5EN0su2HfxjEI1LsftvKyXlISBJyGBdOf6dQvTaQ8oNfmWq4cJA6IfXfFBtuOGwEvxpb9 F0rjT5EVMSfNiP9KtMQ== X-Google-Smtp-Source: AGHT+IFmq0/1EPi5496WrRokUU9WA5t7S1WG8rkiw/MIseZ9iZLjY0dGBCILI69xXidmtmdxWEdpgYhtiPqAdLd2 X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:13d:fb22:ac12:a84b]) (user=jthoughton job=sendgmr) by 2002:a25:81d0:0:b0:e28:eaba:356a with SMTP id 3f1490d57ef6-e2bb16d53e4mr14075276.9.1729301383632; Fri, 18 Oct 2024 18:29:43 -0700 (PDT) Date: Sat, 19 Oct 2024 01:29:37 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.47.0.105.g07ac214952-goog Message-ID: <20241019012940.3656292-1-jthoughton@google.com> Subject: [PATCH 0/2] mm: multi-gen LRU: Have secondary MMUs participate in MM_WALK From: James Houghton To: Andrew Morton Cc: Sean Christopherson , Paolo Bonzini , David Matlack , David Rientjes , James Houghton , Oliver Upton , David Stevens , Yu Zhao , Wei Xu , Axel Rasmussen , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9E74F40014 X-Stat-Signature: 1okn3ze4m19us34ok54y7gsth5gy17ty X-Rspam-User: X-HE-Tag: 1729301379-647146 X-HE-Meta: U2FsdGVkX1/br5Ua0LI9GIn73yi1qCorhdTDzH9KRJ46NS4qb4Pqh1B2r1Gp4lo4ne0kgXWaZZxAK8wUMpqH+9mPeR6fADdjEsW6UstH1z2eiRbasG70DfVAzFOdHzPqxwbNkENIEGqo7YNTnVhIQyjVp+IirEhvlfctu0iwgWyT2XME1247eR+nn+dQjp1HDSt7GZFh3KCJfmU3723bRPowVI2SAI4sRnj73sbedgzYZxttKvP5P1vT0TtIFSY4D7NoROJEiLkXRxFYLq9M4jixSWDx8/QyIfb+4ZiZXTmFBEJQehZZYgWxEeStO3mtcShfilXsLysY9nXtCVitAR36k1ASDG5hGSWYH1zd0lPWcmH11pUtMy5Of83pfRhOITMplrD7nLoI85jei7bNIi/g6qPeJYKWnnjmUMokaL0kDph7RYXGjHnf1T1bmNnYCaFHfmpppFK18BLgh4GZAnC2VcS7j2pswpjO8HtzrzynBNJQncBfQ77bLl8ikxpQFEPCDnw4vGJ3jB7bYedvALqAuGKmRdIweNRfDKWaT/MCYGWMyup+tIGRVFECUPpeAixyfdYsxfbowQu3K9YdDIiACbp7cAgltv5FXvLibnV750rUct5X3zX8Lwb00RmWwlxuvSuikC7ye9jI7S4sqiwfnTHAK/Dji/vyZqBxpmV2Ah9Jy/rfOeLpCyDN4EbSE7MqUkfgVCPYvR5WzzgO5tr58Ba5YlTkge+IOxZU/UkCts2Bf/w416SuB/tGlBKBj7W6hCkDqUKuiVZ3r0bi1oSriOjmBjVBJmTWl7VJiLJ234BwaHtQ/U4WBtF0K+wKL/0XNoyXwz3fTjxnFqlekaY3C8KmaNYcdnj75UWu/m0ZAih4onCD4DvTW3GETs6pnmDbt73ij5ZVKE+YD7fXL3eeiB+uSyyLoTXTr5wR+MKAp1pLHuS2BzYV38RE6kvITR/yUdZA17UHtoyePh6 LUQ0c8Dy lNmWFIizNSAKaNLCvr/mBZ+PC0Fuph3cs1p154U8YnTfMGU/co6AJNVeXbvN/FR0g36/JCjrrqUwqfZyGCqqDYiZ2VJQFJN6CbqlCDKGi5zHfs3PD7M/4owfkxtiNWxb0tgKqSFlZO0x+EKCWvs1u/MJrjmhhe+r46kj0OIJXBBwbAhh996/J6qM/J+V/pn5ZK0lMYgiNZwR/0YrdZ18hL4z5wD1FTnVqTzpB6qF1tkJVk5lh3H2631brB9/uD7IsYUTPqfYX5Vse8m3V+65XJ3AQ/JXNO6JkT5Q/dql/pzytznRXgyBYc1BzzIvjcvUp2e+u/cQjtX0oZP5Fgww4oOOpjf+IcS+xTsgQFcCr7tgLjLwTHDID/BDS7x3mGsa4F5sWWG8t4mP3yHPB0eOZDKxVhTxkhnU7ArxrFN1XhMaK/tKB9cTUXqKjD+UOlMNe7i/EcAvcpo2neRKCNSrEi34gfBeFu5izMLTlp7TmgrFIfjv5hYr6DUFTtISPIoWKttYNa5xHk7UGRo/LKXzQfONxArNgfX92iZytfLYKokVKsI4kgXqo7oTzreQRdrmwiBmaSchtkzmsnqzG/JCL2+3AJPGtOPn8ok2ZPaQUN/FsBoNg5I6q/6eXLlhyZYEZK7qJuz4YG+iWcDI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Today, the MM_WALK capability causes MGLRU to clear the young bit from PMDs and PTEs during the page table walk before eviction, but MGLRU does not call the clear_young() MMU notifier in this case. By not calling this notifier, the MM walk takes less time/CPU, but it causes pages that are accessed mostly through KVM / secondary MMUs to appear younger than they should be. We do call the clear_young() notifier today, but only when attempting to evict the page, so we end up clearing young/accessed information less frequently for secondary MMUs than for mm PTEs, and therefore they appear younger and are less likely to be evicted. Therefore, memory that is *not* being accessed mostly by KVM will be evicted *more* frequently, worsening performance. ChromeOS observed a tab-open latency regression when enabling MGLRU with a setup that involved running a VM: Tab-open latency histogram (ms) Version p50 mean p95 p99 max base 1315 1198 2347 3454 10319 mglru 2559 1311 7399 12060 43758 fix 1119 926 2470 4211 6947 This series replaces the final non-selftest patchs from this series[1], which introduced a similar change (and a new MMU notifier) with KVM optimizations. I'll send a separate series (to Sean and Paolo) for the KVM optimizations. This series also makes proactive reclaim with MGLRU possible for KVM memory. I have verified that this functions correctly with the selftest from [1], but given that that test is a KVM selftest, I'll send it with the rest of the KVM optimizations later. Andrew, let me know if you'd like to take the test now anyway. [1]: https://lore.kernel.org/linux-mm/20240926013506.860253-18-jthoughton@google.com/ Yu Zhao (2): mm: multi-gen LRU: remove MM_LEAF_OLD and MM_NONLEAF_TOTAL stats mm: multi-gen LRU: use {ptep,pmdp}_clear_young_notify() include/linux/mmzone.h | 7 ++- mm/rmap.c | 9 ++-- mm/vmscan.c | 105 +++++++++++++++++++++-------------------- 3 files changed, 60 insertions(+), 61 deletions(-) base-commit: b5d43fad926a3f542cd06f3c9d286f6f489f7129