From patchwork Wed Jan 4 22:29:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13089149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA1A4C46467 for ; Wed, 4 Jan 2023 22:29:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 47A648E0003; Wed, 4 Jan 2023 17:29:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 42A748E0001; Wed, 4 Jan 2023 17:29:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F2678E0003; Wed, 4 Jan 2023 17:29:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2058D8E0001 for ; Wed, 4 Jan 2023 17:29:49 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D378AAB309 for ; Wed, 4 Jan 2023 22:29:48 +0000 (UTC) X-FDA: 80318560056.01.AAEADB9 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf17.hostedemail.com (Postfix) with ESMTP id 3A7F340010 for ; Wed, 4 Jan 2023 22:29:47 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="OWeh/xJa"; spf=pass (imf17.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672871387; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=hwLcsupm9nci6pKlBz0MxnkaaRMflnQODrI1MEXSAsGp8sXgbJGb0LmVfM5y33kmC6RbPM 805TdMnwH9micIUjiPaVFFaQx78+AaOV7LnZluZznnp6uoi1hmjXHTlQRPxuWjpvr2vUk1 JUFNldRMleQjlgo+Zaatt/vsZjkHwc4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="OWeh/xJa"; spf=pass (imf17.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672871387; a=rsa-sha256; cv=none; b=ORHP+BhzcV+5Qsc9BNX1MII09qOmb3y7Cw4UK40kI/mUBgE0D/eWXdh0Cp3SDdZn/KzL8G IfX4fHoHqmyBau0sjLv6QDJBN4wojA1KfQ/iIZddkg/CpKzoEPjcgBFpyQQy7dt0crcCpw z4pf+lQgdjx2el4h8jb8l3ZtzLM1a4I= Received: by mail-pl1-f170.google.com with SMTP id c6so4214811pls.4 for ; Wed, 04 Jan 2023 14:29:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=OWeh/xJasix7ucatObuWl3DONvbabYvNQFmjbkxy4TsWrHQ6sgQ4Q1RrAkB4fHWo+R QqcOI5wHYusIxypkiUE8QA01I5znwb5MuEjGOOxYlGRJXgJ+t/Y07y47zlCa+bDecmCa jRLDzay7BhzAr/uX7Bq2XoBrcJNGjwJAn35htNsjVF1VEApIDR1mSs2kCG3nZpWp7gOg Bv65mwAtANq3K4PBT7gLkZyUIoth4O9joCJSfs0ZH5tVn9bhQrv8h6OrMlNoPYCDJn8L sPOIgorTOJPWjB04nmHqo48GY30cOLowMhG9Ue3oL4PXSXkMHjI0kIOTZs49MzFZYsTj Zy4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=eRGO7fAEjGOu+diTkrfmVs4wVusuWWNxWDbWndUpk+YrTiibyMrDoawT8FzIuUuK7A ZJQictkVmdN06WMEQtQgvJ/u4k8v1e5lvOc7G1fiUxu1Xh/3IOyJndUTMD/udL7ZyKJ2 psudPZrQwWGsKjK7jdxHfCrbyxxofBekrmUtsptR3ExW14No2ZZBAmm7jeGZgpS4EaX0 GwXgQW9c/mXF6fOqCoO13Imq2oL8XUzEmxJvR9dmF111YGNaQjKKg8n9X7dSgPNzjL0A cKKFVjX8OPrv921HXhygfTeNwXiQJNqzWCM5oGPiY4dporTmMWdaf/dUFApAkcnL+6YU pR8w== X-Gm-Message-State: AFqh2kqinPYz2AuixUfDiCeCMBGAtFoVNWGLIwfZEKENvGnaJRJjZRNQ HTI5Ga0hW8Ze/Goafm1HfuE= X-Google-Smtp-Source: AMrXdXummmRPdLPDCZJnf16qcRjCFT8Ft0zu6vTjKxFqQng10LGXjfMissTbYkzRjIio/50ljKvhOg== X-Received: by 2002:a17:902:848d:b0:191:1e89:35de with SMTP id c13-20020a170902848d00b001911e8935demr52352601plo.9.1672871386139; Wed, 04 Jan 2023 14:29:46 -0800 (PST) Received: from localhost (fwdproxy-prn-012.fbsv.net. [2a03:2880:ff:c::face:b00c]) by smtp.gmail.com with ESMTPSA id im22-20020a170902bb1600b00192944e3650sm15456333plb.268.2023.01.04.14.29.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Jan 2023 14:29:45 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, kernel-team@meta.com Subject: [PATCH] workingset: fix confusion around eviction vs refault container Date: Wed, 4 Jan 2023 14:29:44 -0800 Message-Id: <20230104222944.2380117-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3A7F340010 X-Stat-Signature: 6xtxfcr86uaxs43dmthk8rxx8fd8dncd X-Rspam-User: X-HE-Tag: 1672871387-476431 X-HE-Meta: U2FsdGVkX19XhSVPOeJrhHc8Wpx/bLbfblcETGlUx61Rp5779j2AS6+P10JNICM9Cnph8FC/dQ4LXPmbVVFQDldlgTwPqyEbCejohzAXPXHjKOU80JbKDrOE5PcRCELBPbGriMw8RcKHWEDr3+Aafx6fV9Fir6UBnLfU6QjTGGFpoX8BH5z2PSABTIuJMJCvJ3pumjcMhHR1u3mt2+ZGFmx8cFh3mPegUhPb6UPhchJ+a3TiijMWC1Ta4IprEGwhKlAb1W4Z67rqnue9rB7IIxHPoYLkGLwsRBKVAvofE+HUQZydRlHI/5AJ7UwaJ8n3eWtrEFPNWFbm6OM3DBsIXjIthujvXJjgK780n53GuZjb9gz0ibr1e1d4el3nQJhQc7bn6Vd/Ar0eZgReOhp0emrWyB/woRcxa7JArk18aCnC290gwTd+dXRJG9uNN/1RPBbnhthGLzvpQEr0uEWUQp49ogUJgADC175tSmj4SlTPcdTOxEfPLuI4guZVPXRz9EyaF2hXXdjEQ7SEcFp46o3eUw7G2hQ/vLKxYIr3Tplj7li81lISCTucIjWBGkm8u58FlUJoQ8eCNGiq31EtpN1a2qndaRR3e9ed55qxo7eDgcKLu2VY+PDMTpAN/4xmYzlJ3sgPBLitaDqO97BYcNxWuBtH2tFX7HckQ9c06JjQV7NLaAgof3xVAj6LFw/VW8+PmjOfHhvauVMt4bWwXlHVAOUfL0zVx/tzi4Nkw5ehWup77AlJmiXSjyCyU4SMRhXpTKYZWktVeSDnDh8XJbwALzhtD5VO7I5OiQA5vWcAP5OIGuZvvS3vEgVw33BdvQevpj/Fm3Wx9q9Rw5upQw65kvWoHM5tFNkdjgHbZhH4nQAgUrvtZX0RKPcS0rCdzQlTqZYff3le+8LYWNVeZfR2cTqXlIyttKrRD0+yuTUcDxkNVs3VAhTQsmsIWJVK7XOxGnhHu1p1DVJHnsu 235oSh9P 4Bm2wtk+DIVHhqpcD5qAZ5HxsdFC6xTltqBg0MFjUewBsuBmSCCsR0uTL9ehGtCy6EDU6trR+RXTsbPuxOxPmzIA5m0uqzw2myKvAg5S/oVFxbFNHSYgRLROs83bW6LX2Js1L X-Bogosity: Ham, tests=bogofilter, spamicity=0.026992, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {