From patchwork Wed Aug 2 02:56:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kalesh Singh X-Patchwork-Id: 13337539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1AE5C001DF for ; Wed, 2 Aug 2023 02:56:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5704428011B; Tue, 1 Aug 2023 22:56:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FBC1280112; Tue, 1 Aug 2023 22:56:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3741E28011B; Tue, 1 Aug 2023 22:56:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2329F280112 for ; Tue, 1 Aug 2023 22:56:23 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F0FFD14063A for ; Wed, 2 Aug 2023 02:56:22 +0000 (UTC) X-FDA: 81077651004.23.23FD8AA Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 34D5CC000D for ; Wed, 2 Aug 2023 02:56:20 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Jrr8nzKX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 31MXJZAsKCLMdTeXlalbgZaZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kaleshsingh.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=31MXJZAsKCLMdTeXlalbgZaZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kaleshsingh.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690944981; a=rsa-sha256; cv=none; b=FkO0FV945Fw7002NY/chA8VXs8YEHm4OCvPxMf4Iwjtux9uJWJzNRTJPjsLwEn4Aal/Pvp XyeimAfnxg8kSb75+snEXMaEh6d0seL0mmoSAaLlLG5XKj9xJEcKjznRkKP3QtAd/mKkyb eWwgsocPZrp2HLv07sWHxCJQOXkohRs= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=Jrr8nzKX; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 31MXJZAsKCLMdTeXlalbgZaZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kaleshsingh.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=31MXJZAsKCLMdTeXlalbgZaZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kaleshsingh.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690944981; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J6cxPzKelBRogs1RcqlDIpKMeY4RfiCOB4MpGgBN/Dg=; b=yGcNw8Cg8GeZ2dQ6EwxAv/U9qGPZz93vdqH95Qdi0pvEcIjpcbXoy0fjwp3nVHQ3IayyR3 iW/BiEkXqkOGaycUU4qQhzbdlwp5updfTX4/7S2e0SkfqiD4nkKMvTtNeNQhoap+vcY3L8 1GE08qJjzzGvZ5DXgqgDG5RuCJZUqhg= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-c647150c254so860890276.1 for ; Tue, 01 Aug 2023 19:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1690944980; x=1691549780; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=J6cxPzKelBRogs1RcqlDIpKMeY4RfiCOB4MpGgBN/Dg=; b=Jrr8nzKXxGfH9rZ05coei/Z5XkXPQKYffsRvq0JJPMkpPWKSPML6OO06eZDIof3ZxE FqR3adURv8HbiY0CSWCrzORDmomvxAk0WLRY9cf18uCIkdSyJjl/VvbdCY4q9Wy7Oc4w +pskRXdmBEFfcC42q/5pqN3PxM0TzWr4gK4sE9xc5wu50UZc34qQkFTahmwhRGZJVytQ iQt96NsYT5jF6oFb+S9xqxKcAjWxbIwls+nZQCaZUiFG50rjkxYUVoWP/KSzWH+XfZJr RVj4ctwi+migEpHkjEJI3k5GUHjwOtKhv4c0gSFmtgVSST9QuZmYcyIVrwx+qWlaHlyG UnUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690944980; x=1691549780; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J6cxPzKelBRogs1RcqlDIpKMeY4RfiCOB4MpGgBN/Dg=; b=Vkai1KPBWnYWPETN7UQ30RyX93lR+f/sxZXmct11ykczTt2i2kLbqV5mWKzSvMPU8F 9YzP8o1qUkwCS1av0auuO4yI8WqQomPHZvZuun65SNTkqh17t5EMsspavM9QuLuoDaQP C+3kArCShEC7QSEcG3TseTMUw4q3Y5ODRT0ZU+GicR7J+X6fCCSpHeW7fkIFXLQDY8+t J9i+Qi/1uyphC5TF4SyfcLsr5ZRCQvpimoTJnOpaD/x/eC3I9a8pQdrph7dvxi5gC6Bj nz+us1mGNXEbF42ZyG/cUcXN8+1sLLELszFa8ivP3JiY7dgfnJrY9Ua4WEkNcvfE8kMK F3gA== X-Gm-Message-State: ABy/qLZAUXFCWwxNf0maZD8J6fva+SPhHgqbAwl6jj4FuxlLr4ItdJGO ljk/M9jImLLVhM2UjC74Hz+Q2iPNBkoTMze2Ag== X-Google-Smtp-Source: APBJJlHvG4mc4jsYVOJYfZ7h42Tv/EwdyD/HOeVXUVvtcmr8y48s75g7rEDYvVAnOWGuPl5UYIq68sSDD86PJyMPYA== X-Received: from kalesh.mtv.corp.google.com ([2620:15c:211:201:aa5f:ff75:318:255]) (user=kaleshsingh job=sendgmr) by 2002:a25:b084:0:b0:bd5:dc2d:9d7f with SMTP id f4-20020a25b084000000b00bd5dc2d9d7fmr167401ybj.4.1690944980290; Tue, 01 Aug 2023 19:56:20 -0700 (PDT) Date: Tue, 1 Aug 2023 19:56:03 -0700 In-Reply-To: <20230802025606.346758-1-kaleshsingh@google.com> Mime-Version: 1.0 References: <20230802025606.346758-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230802025606.346758-2-kaleshsingh@google.com> Subject: [PATCH v2 2/3] mm-unstable: Multi-gen LRU: Avoid race in inc_min_seq() From: Kalesh Singh To: yuzhao@google.com, akpm@linux-foundation.org Cc: surenb@google.com, android-mm@google.com, kernel-team@android.com, Kalesh Singh , stable@vger.kernel.org, Aneesh Kumar K V , Matthias Brugger , AngeloGioacchino Del Regno , "Jan Alexander Steffens (heftig)" , Oleksandr Natalenko , Brian Geffon , Steven Barrett , Suleiman Souhlal , Qi Zheng , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 34D5CC000D X-Stat-Signature: ojgnap87xn7ffgpg8a1ua9fgmt97kpxi X-HE-Tag: 1690944980-662320 X-HE-Meta: U2FsdGVkX19fON8PBReDNzTG0aT1M/IdOGgCpd19YJ4FNhnW3QsVS3Jao8eRto0ENHbr9MJx4LJsyASTXphEvbwgcGh9m8SDWkJ5ukO23F6aMbmc7OOOOhZBYMa+/XA88UcRzqVYxWaYRohQNx4tRc30Q0X6Lj0opT/XQ29HCQnMLQBb7kiGE8QM/IpdXOlHsD5iFcJ08yIFmU4zL6gD//VrBWc2OTf+3CfQVuqB4zCFpva5IvEz2Mgmn7YhQIyr8KYeIGexL9HrXMHDrvvd8JuPK2kfgDh9T2XT8fMt8NcmnScok/vxeFRk2SYe/5Eg6iGkChrmTeQQGrdrx+ndiJgLPzx2hIIbeKsQgSznqHg0xbRarW5JDzkhn3Q8jJk02jKgNMzvlO3Ctzlj5w/yW9SEz5RXQdwHihjG6dz2jyNRTldsc76UtiVuSlcJ9yUpezEvCJFdgfc/yB1J1mz7r3zCp//K1RZMTVPnFwbDSKMDMv9AznvIrNtLDOZSzdKGRBpdmHH7FYi+nKI9ZcnNRToaMKOU2pQ4uLJxHHXxWhXbPhyPnGjFmH33tJb3Ft0WqKsy1oV3YxLnFXa3C2oQ6t/xrVfDPnXQxVj+tufdzCG69sENvgoiepjQl6LyBh/HgxZGrTwWXZoO2FhdsY81j3RRdkxwLNL9Yf6pYrRc6cZy7e0wchn/Nwx8Y1SqtP/IY0217bzF1dWx/GjfiuPwAdvZYEETNcFzcVpYFvXBleVeeT6ZFZDpIgsiXddlx7lTK1el0Vk4fdcIE2kG5cIdA2oXEaq++BFF7sLGpI7GC3viwEfwcJoTtAupoxT6v6UpeZgJXoj3pRusyoyaqrpJ5CQFQFS+WfWlWNmwVl2OiytxrlLWLHtkTvzrZh6VppwV1xUOmL56l8pKOk8v838jbYtGwtasXsW+izFPHLiRlpVFMuFIQA9s1sTj1DyzjlAMVPJ1IeFCgTG3XQrdqO2 3OvpEN+b WNN7HwcrrDeFK2Ie4Pd2Dz2D7G8vW9AC2wAvFiZAdZ0JSt+QgQUyaHLMbdaNbO+5wZ1HlmnEzgjYOetbP9fGbRgYK762LrIC52J/Y/007yuA9cdsx5UUn0HSyl0mc0MyFmP4dd3Ft2wEdN/owd0Nj0Xf0M6FVmVQn4OSPjw5may+2kRhQcW/ILNBuWrj+mAuvDJ0z1nx54W1GAEpNKSNEeJlY5eegXQYx1ODKjOcjwhrRZQ/MURr0rjkbsMwwooVeCvm6zfY5seX8RBDD3cD68JJpfnIJYy3ggXqtA9SM5pi/ILoqjF8Q3+KYaQCCvq01woU4uRi7CPUIhSdqyVV500Qkz8CEywghcyzv5h3COB9KgmLpDLOx3wk7mo+qRmqB3YOeoaQqoxCoLCwNMmqCSWgA1iqTNW+ufGbspecjFfIOEXv1CUh0xgPBMXVIoSLl7e91ocoVdinPzapMEOvKsXBzPqmnI90JJW/xsNap0ZiHSQU7fBY2qBbaWQ+5lWOalDXJ/Mp1MxHk5BZUHNmxRn3Q4Rcat1zYbhZWDHQFzHFzqjcvO+4CAqTM/9xJx4BNRl0mJJfGChB0u1nhwmIBwGqUw9x+NdAkPcbGFqllvR5YqJo78Zyizs1Sk8Plc5UjvphqObhQznFhJteiJ8vfA2MJEl52AN30ZkffCOle4kqfDvD8HDSi4Rw7cJ1gij3+kjSUj5ntDzwd9EsFoI/OlddC/0RSpm7+fNOj7HKxIpyFaH+lNfnKTU1SVADBK3ibeNzzKI0Ic49/XqiXhZ8D2DSQJDQt+RwhAtlwymeNTDte/YM79LlhN7qhqAv1nRgEN24a0P8EUCu6xCkwc3btTE4uLP7VTtmVWiSYC9JtvS8ixSQK8e8nMzEo3/scAm3huO/GhsdFBXfsaZgIrI/LSrylC9EP/j1lc95Y9xZgCCDXWF8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: inc_max_seq() will try to inc_min_seq() if nr_gens == MAX_NR_GENS. This is because the generations are reused (the last oldest now empty generation will become the next youngest generation). inc_min_seq() is retried until successful, dropping the lru_lock and yielding the CPU on each failure, and retaking the lock before trying again: while (!inc_min_seq(lruvec, type, can_swap)) { spin_unlock_irq(&lruvec->lru_lock); cond_resched(); spin_lock_irq(&lruvec->lru_lock); } However, the initial condition that required incrementing the min_seq (nr_gens == MAX_NR_GENS) is not retested. This can change by another call to inc_max_seq() from run_aging() with force_scan=true from the debugfs interface. Since the eviction stalls when the nr_gens == MIN_NR_GENS, avoid unnecessarily incrementing the min_seq by rechecking the number of generations before each attempt. This issue was uncovered in previous discussion on the list by Yu Zhao and Aneesh Kumar [1]. [1] https://lore.kernel.org/linux-mm/CAOUHufbO7CaVm=xjEb1avDhHVvnC8pJmGyKcFf2iY_dpf+zR3w@mail.gmail.com/ Fixes: d6c3af7d8a2b ("mm: multi-gen LRU: debugfs interface") Cc: stable@vger.kernel.org Cc: Yu Zhao Cc: Aneesh Kumar K V Cc: Andrew Morton Signed-off-by: Kalesh Singh --- Changes in v2: - Add Fixes tag and cc stable mm/vmscan.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 489a4fc7d9b1..6eecd291756c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4439,7 +4439,7 @@ static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) int prev, next; int type, zone; struct lru_gen_folio *lrugen = &lruvec->lrugen; - +restart: spin_lock_irq(&lruvec->lru_lock); VM_WARN_ON_ONCE(!seq_is_valid(lruvec)); @@ -4450,11 +4450,12 @@ static void inc_max_seq(struct lruvec *lruvec, bool can_swap, bool force_scan) VM_WARN_ON_ONCE(!force_scan && (type == LRU_GEN_FILE || can_swap)); - while (!inc_min_seq(lruvec, type, can_swap)) { - spin_unlock_irq(&lruvec->lru_lock); - cond_resched(); - spin_lock_irq(&lruvec->lru_lock); - } + if (inc_min_seq(lruvec, type, can_swap)) + continue; + + spin_unlock_irq(&lruvec->lru_lock); + cond_resched(); + goto restart; } /*