From patchwork Wed Oct 18 08:21:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13426607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A0F1CDB482 for ; Wed, 18 Oct 2023 08:22:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B19DF8D0144; Wed, 18 Oct 2023 04:22:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AC8738D0016; Wed, 18 Oct 2023 04:22:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96A1E8D0144; Wed, 18 Oct 2023 04:22:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 83C9F8D0016 for ; Wed, 18 Oct 2023 04:22:17 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 55D0412017A for ; Wed, 18 Oct 2023 08:22:17 +0000 (UTC) X-FDA: 81357889914.13.D804635 Received: from APC01-SG2-obe.outbound.protection.outlook.com (mail-sgaapc01on2137.outbound.protection.outlook.com [40.107.215.137]) by imf27.hostedemail.com (Postfix) with ESMTP id 576874000D for ; Wed, 18 Oct 2023 08:22:13 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=MjIb9KPM; spf=pass (imf27.hostedemail.com: domain of link@vivo.com designates 40.107.215.137 as permitted sender) smtp.mailfrom=link@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697617334; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EWIhVH8kG9NLSbfhfkLvuBEveomuvEk4IFEU8DuIXio=; b=57E8S8aDABk/9A0BZPBV5/M6jjPqffcZoKcHiuMxJN+eq0Fi/HeSRLG2f732du6wkU50gb o/iMhRhWX67aO64ZM/5rffM7CjpbFUhC2MqNv86mCWbKPzUZ0hu3fFfrAFMCm5Rywf0OGq Ccn5RTn3QVRTfUxuS5+gQubBHte6F0w= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1697617334; a=rsa-sha256; cv=pass; b=zPZH74damUWv7eFi0rfG4yeZUMGSSXm1I5yqObSGroKmrN6OjTkk3NBQmk0Cs2sTbchfOO gUJqFQ7Kj/tB64hCBXO0D0Jpcc/FJL4cX1s4lIcqQvagcGul2bCwW3ufkIQodoiaADNuZW WkN5RtmpymhsSiwx3tvNQcHMU+Q8y5Y= ARC-Authentication-Results: i=2; imf27.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=MjIb9KPM; spf=pass (imf27.hostedemail.com: domain of link@vivo.com designates 40.107.215.137 as permitted sender) smtp.mailfrom=link@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dJNjtDbq5MY8+/8k53ArckBxifmZ3KLitaHfWuxF7Nz3cOU+QjGYAk5m0wFCitNS3M8GSfOezR3YzGbwOpzQzsMczdWV21Mc5KCgIuOFPAFMVwM+IeR7I1E+IBnWz/TKijbCAuSWc9gUV1XYkaV4W6EvVdmWCpMpRGE1W0vbS+t3FCwXT5MGBCwzAaO5MLGgHt4bvnqPT37uX5t9SB1aNgQwFim5wyqYCr9Raq5uUgbCAk371n7CH3T3U1+MyxgKrfwnUcvOmtyeQVaRA9EWuJzsO8lX+WcuI6QKDZQi///G/wsm0jJux2R221KkZMxL5G9FcWGcmTwDSrIhkmERbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EWIhVH8kG9NLSbfhfkLvuBEveomuvEk4IFEU8DuIXio=; b=HoBI4C5m4aE6SBXmEpXbVG+5nDYOYPQoDTMVFYO9jWmNd0rXgqDco8BCrTOybW0rOvDHaP7CmGgqSJ6rOFQj0HFDVjSvLCi91zpUUi3x6E4zVA6CrHCVLpx44nAlmqaXxBMaMRdJ3Wwezad2D7zEMSekGiaXUPLyJf98KIloCfObu3Eno8gs7OQyJFM2TGr6mS+yzgUvPAdcJ1yr2jfrAZNwfXvrHc364iXERIIAB5Ow89FzCtSP39pJqQBgZq2lRI9d2ht8cSs/QuBZInuD4EuBmx2GzUxozcIwNsufpXPtP4rus+ZqcBXIAujpN7pMDKB4D1Ozp7tFeUj0GKEZtg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EWIhVH8kG9NLSbfhfkLvuBEveomuvEk4IFEU8DuIXio=; b=MjIb9KPMWPUYy+bMJuJpjnkuL420kv/3z3pEzhwh8SAjHMqqKmt/y+Tjb8LgoNT6QlWDIKhTuaAwVfKiTP2vr8yZATvWYUio4lA1R8dgTspcSIXPariaag6SvQrqXub8w/rwOK8q99B69S2+ULL+ymD64ZtkliXS14n3F5J1vi87A8joyQSemM9Pzydh2IrQJGGgYgZkYMb7dtctvJH3boF34We9FPgBRbbkAHp7UHuzpgRk28AVu2p5JyRrU9dnV2XhypmwEpPZNQ+OAxF/6l5yDV+hX4dJlLo0BHTB8qU6cPsz2seR4LxaHDtc1zNGTHMcwubLm51R1I/CidfgLQ== Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by PSAPR06MB4391.apcprd06.prod.outlook.com (2603:1096:301:82::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.22; Wed, 18 Oct 2023 08:22:08 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::40ac:5701:4617:f503]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::40ac:5701:4617:f503%4]) with mapi id 15.20.6907.021; Wed, 18 Oct 2023 08:22:08 +0000 From: Huan Yang To: Yu Zhao , Steven Rostedt , Masami Hiramatsu , Andrew Morton , Huan Yang , Suren Baghdasaryan , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org Cc: opensource.kernel@vivo.com Subject: [PATCH 2/2] mm: multi-gen lru: fix stat count Date: Wed, 18 Oct 2023 16:21:00 +0800 Message-Id: <20231018082104.3918770-3-link@vivo.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231018082104.3918770-1-link@vivo.com> References: <20231018082104.3918770-1-link@vivo.com> X-ClientProxiedBy: SG2PR04CA0185.apcprd04.prod.outlook.com (2603:1096:4:14::23) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|PSAPR06MB4391:EE_ X-MS-Office365-Filtering-Correlation-Id: 9545bd03-848c-4418-0106-08dbcfb351cc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jUXaTwskgInwOhmnvEmDReDAfqDVqzr1IwWmWfcoWsvlRGiNX4DdFKXyC0cUpqsDU1/L7wAWXGXxSWaoKes/+WY8NKHpK38lOUkf6H7vWwxuJxcxa5+Sq7tM1lcI2SWHrtTLtw7g+88YXsYpdDAkLnAMRAUxfPopSL/6FK3LTCayh3Je90ngSWQabyYR0ViikbfA294kwBAPR1BOC5NDeKjt+21sXewM+5Evl10Lrq+VZx5GcFqvB00cnV42EPtlyTK7hl+chsVZ0yVVYJ7PgOf5xGcs+bW2bqoeSM8u1mavoNI/PhHng88meKc0/9230F37vj1OAyJ5qgAYsgpmwuDC78dT4sdZ2xZX5erfFe+ibcHsHPA3jj5W6gSWIU7aZt6gkmnNJkn9XUge2xpC6lzlRPb/UkUBUR4v/v6EcMq+GvMc2cseGS8ITq3Kj3IWoCDzWiRTr3HPmW+kvhrFK7yIRDw8GeZOsi12N1lZLtk5zoclxO6TTX4e6MExk9okVudUHfRgEK20WvtkHlSPF1ZUolv5JnQ0wXxM44tQf1ou11k7lGP6fzNApK+R78WUz4niOhOiJL0YQ7526uwKD7TrjipdDkxdA79no2KRhF81PVI7so0jEFGTmu4wLLuBikc4qjppQ1GQEILok2/gYg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(376002)(136003)(366004)(396003)(346002)(39850400004)(230922051799003)(64100799003)(451199024)(186009)(1800799009)(7416002)(4326008)(8676002)(8936002)(5660300002)(2906002)(86362001)(36756003)(41300700001)(38100700002)(6506007)(83380400001)(6666004)(52116002)(6512007)(6486002)(107886003)(316002)(478600001)(1076003)(26005)(921005)(110136005)(38350700005)(66476007)(2616005)(66946007)(66556008);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1mnGyRBonP7nfJzGGd0Um/FvTMR58vdZaxRHqIIFzgsilsKyy5clAdGYSP1dhvgZwe4sCfCVXTq8JZ27b+qjTHILP5JFbK3s5ZV4qggD0iS/AHRNWwQSW+FxnzmogV3PxCcXzGiI5KjTknsvMDYNP5pMjT/E1waJx1+fTtHV2oNNpGsLOuQk1RGLsldoyweJoLFLTF/ofHzyGxLzZpkfVhT1AGLciZWiybkUkiqXGqVinDB80iaVQQ/1dwznJFQq8A4IlJgie3sEmAkMMsANTT/9xGAvdYezSZ114BofuBXwaul0yOkSMZfBX9z29b/8tSPjdnqfXjB0Xxtm9gw6wRePyopGHnZNNr9tcxCCojrBI5aGdEamZbLCCuWKuDO8bEtXNfFlwEG3OqUEARRYX2jGwvr1hoj5ceX9zohH3JIeFQDOD1N2c+YkZZUYrBUgkTvIZqlukvhjEJmrrsc/mTOmvz8m9LLPwQ/VzkC6wHUX0EBN9W26wWtDewB/Mq6RVOHe2I2KtyfhYBNvGVKhpKsyLY+8cJUYMkLaaAIWiCK2GqREghKJEBhn0gMGLVa1RocUoAtsHm2IxN6DTJ9mvZWi/GyGe3NUdAiABohIRWF3zn/Igfgi0yHAL1msVdm3CB5P7kmXlsUj3AFP/oFzzx3vRa7zf03owoIXsrUf8kHBca+T7WJ9L/YR101sb7P/wA6XV1SpufKFKpa5tINUWV804zq1kwQHjbR0M/7TND8Swmi3e22mQYR/oLcnMln8suqut0kP0JkX2BsEs1xNWPsz4BIqi0rkBGMsBl2tAGdTFKnIoybqFCOVJF2ZddBbLcikk6vvV3n6nT68lDQL1gMCi/5ErV/J5KTt9hVihk06Rd6FjxHootOd5Beh2zyGIZA7vVzIUCR/lE3NPjJLZ4ooQ0LTICL5y526mtyQA/MTaQn7CWFF7O9I+5uepj8CdPT7X6wEn7aQ7L/FB1C4Fg2tFVIDpHZR6yA1vzkFywmKONvwRzoSBqflx/PAvSm1MDGsZjtCGOTWWhmwzXxz4fDBJc1nmqRwltsM5mdGbkFTH83EeblFu8Qpi8ESswdFs7f1WHkzYVSmo5YKc8JkyS70WtJ31EOFUuxQzpw3IYW3yMRYXLh2BSfe4fw0v6ChAkJl1LEfEntQI7aCMbeHa+fqfeJuHvKzeShQXqmy9PgrkLh+QZYugvEjS4npEwYqXcvNoGpPsDFvLHnLa6dSxXmkEpB/O7ZxqVCih79zJxqMYTonxev19RPF6VFuiL5nMVPZjsyd+SGWNx5bhaiIqG7yN0gbNJZMeF4P/oCzDaoQY1mB3kb229llzSewyzoHsbTs6JQYDuFY0cPzqEaXJOpgcdVyjeYLaiiN03x8rGHh2BkjnU5RBwJ4n4eK/iPbB1AgdJTDnqrjAJSt7y38Wunn141Gi9JClhM1Mrn06uhg449iq75GqZLrvWtjLOFXfujGBxsubClRIqYMAJgyzKsRMu46o+8m/z+WyFIV9uIS8MtXpFdZEZr0Q6haNoJi8WwrRA0f7kg4zFeZfdw1FattIfA41MX+k1rH7eZA03YoG96EKSYTaSkxP9Fm06/9 X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9545bd03-848c-4418-0106-08dbcfb351cc X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Oct 2023 08:22:08.2941 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: humrRbJhtzRz7mHw3ImA04YdvnTUCHIFeVefqEAMtiz6oeqaw/URu+bmPYOWCWlYnOX1btJyeZPTSBYDOvXzJg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PSAPR06MB4391 X-Rspamd-Queue-Id: 576874000D X-Rspam-User: X-Stat-Signature: exunbz6cr6184758hpzdn1hzho1hs4c8 X-Rspamd-Server: rspam03 X-HE-Tag: 1697617333-544531 X-HE-Meta: U2FsdGVkX19136g5LIGpCyNJ3sTEM2+Vwcjq80QhlYj2L8Qu0yBOE6uorNut/q1uCvXGh0ZV3bHzwRZPenN9KNsQn5+5WVWXKjB2An5a7WAuTlEB86pwWBeKRUKu5DMPvzVK+SEMUQSgOuIiHY+WF3NHQOBfrUvYxk+Dhow0IS9LzqdSPFcnmu69iaqArmc6lZ6U6/70jP9pnS1XL3YS3dgdFPlaIo/BbOYwUycLcLPt4wRISjfUYSDod70OYTNYsoZPPrdi3eQG4BCxQ19fhIfvEL8+uow7y7JlBakLk7xH+Yt8IyQShh2CTNWiScgW2fCyKxXH0bD8Aa+gKksiR4//hHfsIWtFnAvLd8DpCqRFFmmjqxZ2mgWzLeZq1g09MhyguWW0XEMW0VZUkAsn9ZJ7fM89uP3vrKv32arQ3Hp2bc4UZk/0RtlXaLdbwENo47ArlTVI4OW/3cFg+fmZRjo05fxx3JgmdE59B2eyiBaqlQmrB0eYvC6rIYKGF4tXQsjjZFLGl2LlviesDGVsicWLZZSqqTKk1cQYoAYdYT0FPdYBNayp0hd/WeCwFAosEaTB3UsAZJ23GYSOOxu7gKsms+xQQIDEkjUEKxUDJYVTcHvZr3fG344IT115C01GpHAW8W13i1gIqQQVS2h8RpDW4M7lCSyZTRJqKmbIMqS5Ot0eM8vtmxPeQmqTtj3osKlrs+NG4ALiJYuO7URUfX/a0jOOsCc99ihtRG/bIy6KrvU7AqTwwusvxkX2d6EgytxhSGQWvYI2DUv/HhRhP7wHjV4dJjaPamwdG+rDDWR48EcWb2YxIz0rxQtvJ7gij3ZiInz5QctO0B+ec94tZpqq6eP75WhlLOxQjyFHBJM0w3VElns4ELnNDDglR/9TxtPVWNxiA9pYpUlKVMvBSaKaH5snIz5YsHWZVgxHNpec7ZSKURc6iyYh6WP2+Faenoc7uGriHEY1DbesELK uRo05jni eW7bSEesgXyo0trerFwHvMQrfiUrRd5m0kXisXaAgtLFBhcRDr1kc3Zr8JHv4BxMveiz2A59Ej8dlPTpGu9i14tqRGNxqhWrZFsP1COpAGnssOK+KT2D10CtHDpdrD+CGdK4Q0WD/ovQZyzfgirDiwi2kL3Mxk/grWVYVsotlrx2CqINMxX4DzD5QHiceODH5MtnK1Zlvb+nmjSvk+uPb1FYARaQOvyJBQReNHE9evKbOVtQM78AScZV/2w2ZvSDXGOzR1vhyXpFskHFJVF5YhuINHv8ZIcdy1j/wQU9797G2qofWqWegvBeYI5TtIStU798clAEnM6f3l8UA+bwQQv9CQQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000440, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For multi-gen lru reclaim in evict_folios, like shrink_inactive_list, gather folios which isolate to reclaim, and invoke shirnk_folio_list. But, when complete shrink, it not gather shrink reclaim stat into sc, we can't get info like nr_dirty\congested in reclaim, and then control writeback, dirty number and mark as LRUVEC_CONGESTED, or just bpf trace shrink and get correct sc stat. This patch fix this by simple copy code from shrink_inactive_list when end of shrink list. Signed-off-by: Huan Yang --- mm/vmscan.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 21099b9f21e0..88d1d586aea5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4593,6 +4593,41 @@ static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, int swap */ nr_taken = sc->nr_scanned - nr_taken; + /* + * If dirty folios are scanned that are not queued for IO, it + * implies that flushers are not doing their job. This can + * happen when memory pressure pushes dirty folios to the end of + * the LRU before the dirty limits are breached and the dirty + * data has expired. It can also happen when the proportion of + * dirty folios grows not through writes but through memory + * pressure reclaiming all the clean cache. And in some cases, + * the flushers simply cannot keep up with the allocation + * rate. Nudge the flusher threads in case they are asleep. + */ + if (unlikely(stat.nr_unqueued_dirty == nr_taken)) { + wakeup_flusher_threads(WB_REASON_VMSCAN); + /* + * For cgroupv1 dirty throttling is achieved by waking up + * the kernel flusher here and later waiting on folios + * which are in writeback to finish (see shrink_folio_list()). + * + * Flusher may not be able to issue writeback quickly + * enough for cgroupv1 writeback throttling to work + * on a large system. + */ + if (!writeback_throttling_sane(sc)) + reclaim_throttle(pgdat, VMSCAN_THROTTLE_WRITEBACK); + } + + sc->nr.dirty += stat.nr_dirty; + sc->nr.congested += stat.nr_congested; + sc->nr.unqueued_dirty += stat.nr_unqueued_dirty; + sc->nr.writeback += stat.nr_writeback; + sc->nr.immediate += stat.nr_immediate; + sc->nr.taken += nr_taken; + if (type) + sc->nr.file_taken += nr_taken; + sc->nr_reclaimed += total_reclaimed; trace_mm_vmscan_lru_shrink_inactive(pgdat->node_id, nr_taken, total_reclaimed, &stat,