From patchwork Tue Jul 25 18:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13326950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7C23EB64DD for ; Tue, 25 Jul 2023 18:57:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 636A46B0080; Tue, 25 Jul 2023 14:57:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E7456B0081; Tue, 25 Jul 2023 14:57:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AF1F6B0082; Tue, 25 Jul 2023 14:57:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3C8316B0080 for ; Tue, 25 Jul 2023 14:57:56 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F1EB6140E8F for ; Tue, 25 Jul 2023 18:57:55 +0000 (UTC) X-FDA: 81051043710.01.F367A88 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf10.hostedemail.com (Postfix) with ESMTP id 14247C001D for ; Tue, 25 Jul 2023 18:57:52 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=epCKqrGF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690311473; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=znBcrDJlu2vhy2NP1qwf2fUju20MlCI6lGcj+tnBX+8=; b=cr+EBuL5GP6IoG9xVIKwOs9NqZtVrbJRoJ3k9fR1GBqb6utF1qZuZa6yXUOWXMYcfr0F5d Qwng18t6HfiCWWwWAWRV2qbd+en5RqkZNZ9VUHXtSXpssusNBaxULJEXFdurVBwqiXUduG Jj8M4SHvYGhypGv1LUyu9gMsVN/fn0g= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=epCKqrGF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690311473; a=rsa-sha256; cv=none; b=B6pSm4PDJmTzCCp4k/9AZnslU9+oz+ZLQq/FyBugOTrW0H2pHK/uniXD3eXE5SSpb+j5dz jO+KeYi5aHXZlnlJ60hABRI29IhbUNq4ThjsGm6F17DSERz5af7pKa8YPxxlKuQyEBiTF5 goZreN01+rr4zj2BESJQ3slRCMs34p4= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1b9cdef8619so37167145ad.0 for ; Tue, 25 Jul 2023 11:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690311471; x=1690916271; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=znBcrDJlu2vhy2NP1qwf2fUju20MlCI6lGcj+tnBX+8=; b=epCKqrGFvJFObLRdi8TOkSMjEfFMJntgFRngtZomoX0L3Eu+EfMiBkW5of9rNQsmna JKMizQFy5MgVXtHcBWBkgQc2edojoHH2JAZvm8wLDT4rcpHgaxU5a/af2I0qW39xD/e1 kMAMz+QHcF/l23aPNtwQ1kEel2G03vEimI7WJwrdJpbgF+GhDijdCSY4CVXYXmvpUrY8 FLGWVLdrE+aeIbtWDla0nWdbmmGhwS72eJKBL38n8adu3eZICSXQ/4rwKuh9OP46KeId GOAc6b32Tav7vTZXIa0s9wDygiWbZw9PTW5ehh+FVEv30u+DH59Vfr19X6ugM+jsrG8l sxMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690311471; x=1690916271; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=znBcrDJlu2vhy2NP1qwf2fUju20MlCI6lGcj+tnBX+8=; b=X7JojBnhWIs5q2Pv9/99o60c9CY0Mds81xOC+B6cGxRPWHfQ7AI9FclV07KjOXIjE0 WIGESsugz3q9cn3INcszfElC1REYhMzDE7aXsB9oL5S8zFW89aPFlYEyJtcEliehaRzG BmXEKNid4ON4bVPH/IWgfiBkw6KA30dBYsK8gTxPBOsV5epASWuIJRm9gk2Ti+Q0C0Nh 7WzfgZVW0CvU3tDh+datwh/JX2S55TRa2UcRXZ0jbvPNuixGsbDcuAC05BpiET2ftvHM qvqmbmW6VxgmaluOw9HQl0x7MqopRyYn7DqJjvXWtQ8+rnQGYTkfKlW/coAu49eN7iN2 SQ+A== X-Gm-Message-State: ABy/qLYuR4aEvMtzV3dP1GDxmFeqqNM/Hqytunx4k436+fDjFSWh5GDA iSwIy5pKz4+6iFsxFvjRbEUwPhEgx9SOqg1tLFnOew== X-Google-Smtp-Source: APBJJlGqhdPZvsK0xai2Q554lWRYz9xrPShpCSnAtyuNX/JGRlE5azgc8XlpPZBkizCXQg8xzZAgmw== X-Received: by 2002:a17:903:1105:b0:1bb:3a7:6af7 with SMTP id n5-20020a170903110500b001bb03a76af7mr48512plh.23.1690311471261; Tue, 25 Jul 2023 11:57:51 -0700 (PDT) Received: from KASONG-MB2.tencent.com ([240e:306:1582:8dc2:70ef:4d19:edfe:c3a3]) by smtp.gmail.com with ESMTPSA id j17-20020a170902c3d100b001b03f208323sm11443150plj.64.2023.07.25.11.57.47 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 25 Jul 2023 11:57:50 -0700 (PDT) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Yu Zhao , Roman Gushchin , Johannes Weiner , Michal Hocko , Hugh Dickins , Nhat Pham , Yuanchu Xie , Suren Baghdasaryan , "T . J . Mercier" , Kairui Song Subject: [RFC PATCH 3/4] lru_gen: convert avg_total and avg_refaulted to atomic Date: Wed, 26 Jul 2023 02:57:31 +0800 Message-ID: <20230725185733.43929-4-ryncsn@gmail.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230725185733.43929-1-ryncsn@gmail.com> References: <20230725185733.43929-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 14247C001D X-Stat-Signature: kzoeqt3qw178wbobynzo71fy4rgzp6pi X-Rspam-User: X-HE-Tag: 1690311472-813734 X-HE-Meta: U2FsdGVkX1+x/Nf+BL9ksPMhjrxM2MuasHTiLydpWPLeSpUJdKmplfU45HF8GuEW5ibT8NdcQcp15EHLNIOusPwyr+j9rhBcTzrsNj/EBigZJNjlqDpqOAkHkP+zsiI0y/iZuGlqttSnSdYa5Csfq1Du2iF2O2UzMkuY8/POztKwwsGNLCvkMNJcEAVeARniAWgmmMdZHEb9i+ok0B3Gnmds2JQbp3TbQVf5Uw/CwDywjjFSNO4KT8yVvhkuXOE4w456k2yR2ts2xDGr+S6RzH03/A20+GJEu7IEPGiyeaxjW2MRBB7SNDsSabIo+Rj4BRHiVFEqZgkqg/Hg6cE6kzBCMEYDxiAGD9zhsODSLjxxAk5Kg0idJ9rGzynuznLtjfhYw+t3itwjDNoEHiNbjTFBhRvWP86rJE9yAP6uF2QLFKcmjjySGwmw/9pvHun/BwKt0anqGCaordY/V3thtCL2htRJsCrXhXkOdI2EwPe4W2fAbrSoKsj9qjXnYeW8iqJEBmZVucoGGzIgz39e9R8eyc+Ub31C+++RGys7tXovcTlhIA/eAw4k4gFM1nemwGsJ1JdMlQ7S3FJi2fk78bsakTa7iL/RqNxulq3D7A/hih6mY0p0ZuWWIB05aSqp4tyk0D7cZZJGwOxkg+UDjxXTW7Ei/VTAKWe0RuIlZkoOFmwP1ck5BjetXDQUO1cjBZYgcwvQRH9/vSwIT8480DAJeL5COMGmNYuffNY5Ok9pyJ38YuBFZ/hO7pBTHYhoVPCb3+QeaKeVs8a+X968+pQk4V52C8rTeDPIyn3ND1p3V8rohoi/8ORT2EsWkkqFL6P3GgvKO+Chu0QkvFVa2jXrwAtYzJKdIwSJEG8sWe34MFkaXnE7doCceuhIHsXAbnzEULT0Be/VrNdZ59IrWfnMig0E9pJJ/Av/FCdPwfjVYeKQkmZH5kCUNyo6I/cAeV4C3DaRPscurkhSSS5 DtM7KX8e wmzE21mPIVXMAicCWRcWWuJSaM4k/txZdUgIldcYcatQS2NhED77w9/wjz+uCHqwHz8l9bWoi1uRXVOrKX1/IpvNUB0wb21/Kofldr7kbbco2llLFzRv92bQJ7rlSwDg6O7CSRSl9/K9IhGsLYhAPYr1Yi2R3Sv78TCYJ092Ev7Di42Av0c+IBnXYqcYrIfi1Gq3oy/kroQ4rcbkjVDwrMpg53WA6CxF5zpMtAKQ0+mqrQBR6oPqwsNQ2UdIGy5RnIukgVYrSJrJVM51WlgEvftZWLnVFLgwcqU453FZQuj4vdHsvWfxeUR9sWPSybyOAJbKZsViRGr9DkgWOYZq1CpqE6Cg24w8qikchIYDkwBypSdx2lSXxximX2JiQODV6fjiV2bwC4uvj43DeE8HJGyNG8rMDqK7hsN/StnNeBXWFtR8kqo8rlsOKKk78LOI7k4V9u0OH9bJbqe/g0XQjNysKnuSjvysZGk9L/2bihgUUSKuWVmycTbjqsdwbmarrI2W/M7bEciqntanUbQjZruTy01o6MLA/vbUHrGEjhXHTlFkvdrEHMTi9PECyMrzVnAPY0aqfw5BHsQVbAJzlx3wYyQnIu4nAVs66sn6jGVyALSyeikpnxCknxs+tXxHgLHwUuEC77XV5pntv9mBVgkx43ok744Ui1SSrDWKtn4OhfZjCEpjcVbTQAA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000875, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kairui Song No feature change, prepare for later patch. Signed-off-by: Kairui Song --- include/linux/mmzone.h | 4 ++-- mm/vmscan.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 5e50b78d58ea..4ab6bedd3c5b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -425,9 +425,9 @@ struct lru_gen_folio { /* the multi-gen LRU sizes, eventually consistent */ long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; /* the exponential moving average of refaulted */ - unsigned long avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; /* the exponential moving average of evicted+protected */ - unsigned long avg_total[ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t avg_total[ANON_AND_FILE][MAX_NR_TIERS]; /* the first tier doesn't need protection, hence the minus one */ unsigned long protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1]; /* can be modified without holding the LRU lock */ diff --git a/mm/vmscan.c b/mm/vmscan.c index e7906f7fdc77..d34817795c70 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3705,9 +3705,9 @@ static void read_ctrl_pos(struct lruvec *lruvec, int type, int tier, int gain, struct lru_gen_folio *lrugen = &lruvec->lrugen; int hist = lru_hist_from_seq(lrugen->min_seq[type]); - pos->refaulted = lrugen->avg_refaulted[type][tier] + + pos->refaulted = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - pos->total = lrugen->avg_total[type][tier] + + pos->total = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) pos->total += lrugen->protected[hist][type][tier - 1]; @@ -3732,15 +3732,15 @@ static void reset_ctrl_pos(struct lruvec *lruvec, int type, bool carryover) if (carryover) { unsigned long sum; - sum = lrugen->avg_refaulted[type][tier] + + sum = atomic_long_read(&lrugen->avg_refaulted[type][tier]) + atomic_long_read(&lrugen->refaulted[hist][type][tier]); - WRITE_ONCE(lrugen->avg_refaulted[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_refaulted[type][tier], sum / 2); - sum = lrugen->avg_total[type][tier] + + sum = atomic_long_read(&lrugen->avg_total[type][tier]) + atomic_long_read(&lrugen->evicted[hist][type][tier]); if (tier) sum += lrugen->protected[hist][type][tier - 1]; - WRITE_ONCE(lrugen->avg_total[type][tier], sum / 2); + atomic_long_set(&lrugen->avg_total[type][tier], sum / 2); } if (clear) { @@ -5869,8 +5869,8 @@ static void lru_gen_seq_show_full(struct seq_file *m, struct lruvec *lruvec, if (seq == max_seq) { s = "RT "; - n[0] = READ_ONCE(lrugen->avg_refaulted[type][tier]); - n[1] = READ_ONCE(lrugen->avg_total[type][tier]); + n[0] = atomic_long_read(&lrugen->avg_refaulted[type][tier]); + n[1] = atomic_long_read(&lrugen->avg_total[type][tier]); } else if (seq == min_seq[type] || NR_HIST_GENS > 1) { s = "rep"; n[0] = atomic_long_read(&lrugen->refaulted[hist][type][tier]);