From patchwork Thu Jul 11 02:13:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13729930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED36FC3DA41 for ; Thu, 11 Jul 2024 02:13:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7BC376B0095; Wed, 10 Jul 2024 22:13:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 76B936B0093; Wed, 10 Jul 2024 22:13:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E5A86B0095; Wed, 10 Jul 2024 22:13:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3D7546B0092 for ; Wed, 10 Jul 2024 22:13:28 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BEAB9140327 for ; Thu, 11 Jul 2024 02:13:27 +0000 (UTC) X-FDA: 82325850054.07.3B3E55B Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf08.hostedemail.com (Postfix) with ESMTP id 12CDE160017 for ; Thu, 11 Jul 2024 02:13:25 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=D1d71zuc; spf=pass (imf08.hostedemail.com: domain of 3xD-PZgYKCLszv0ibphpphmf.dpnmjovy-nnlwbdl.psh@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3xD-PZgYKCLszv0ibphpphmf.dpnmjovy-nnlwbdl.psh@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720663990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Q2KA7HqAEDJDFjninnbueVy/cjuPGw3T2Kp1IRta9cE=; b=cOGJOlu8mxlTM3E3ZUOCSnoy/FNVHNdpisdUc74QW+qp6RMLsTIHSx5abJHVfnPERRxT+c XvEVXKFCN3QunvilcYNeSrC+zzY7O1G/InRfTwx0PdEwiD2I0iMIgLKLDMKSHLAMgq0ZZ5 xE0vc5BIQZQoZQSkam4y++bi72MgQOs= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=D1d71zuc; spf=pass (imf08.hostedemail.com: domain of 3xD-PZgYKCLszv0ibphpphmf.dpnmjovy-nnlwbdl.psh@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3xD-PZgYKCLszv0ibphpphmf.dpnmjovy-nnlwbdl.psh@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720663990; a=rsa-sha256; cv=none; b=zeKCXsVtVzIqnTDnyz97V7Q1OC5SIloCqD58lKnPbBqt2Qh/xs1B0hXT1s7+fra8Xq92gG Yf14yTMcM5QQuwGUjGGbKJNsEGABQnNDPJGaQFzHHe4Op6HyR/53/r2AG3gmPj83Y2G+t/ u85wPdOmf7b6o5vgauZY1hXNtbnho3Q= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6525230ceeaso6929857b3.2 for ; Wed, 10 Jul 2024 19:13:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664005; x=1721268805; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Q2KA7HqAEDJDFjninnbueVy/cjuPGw3T2Kp1IRta9cE=; b=D1d71zuc+W/Oyq+hORKxYX1wYM9Ot2XYms6kNoQebqR+yYgdQjiabp+G0tSVNnahSi UZeGOXKOsTnKbPmgvH5JTeon4G4q53rPw/07KyMYkoI1S+3dhzRx7V+eO+6ZuQH7VOK+ QcQ0IojFozma148+vPOWvgJsgP7nJxDaSFBBeySSRkiqLtsA52Paf3zjsnHA5zIlrrw5 xkLybLoRlOGzcUhsoc07ejDks92Kxxv4SUUDmFKjs3bVDgRX6EnSvUjBP9rn7FwOzVTi 6k6km1M65ZzBfdXFcvyd40s/p5KcLQqOr0GNCt3fy3VFx/ekyuvLYny5kN/yPz7M1J2p Yckg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664005; x=1721268805; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Q2KA7HqAEDJDFjninnbueVy/cjuPGw3T2Kp1IRta9cE=; b=a7IYj05lxPPWSZYPBIqWxSfOz36FzmgClXS2pYuWsvNiU4ixYG2WDmWxFNV+pksS7j p2NZfeVXHmD7FfVGIlkKf5JRawIZdUfwxapHluo3MOEu19b2swFphHehEJb9Q99MKoQj dzITp6xLDdCmBlIlJZvmaz08TdvA/CQQp8YcTxVeh0BCuFw8/C/O6Bq0Gv6y5g1jsxDn 6QEqYyXdZXjojnjRBISi8Ddphb8Zi9r18u038/o8b3c+g2KM88no596lqB2FH9IXAKHE +FAitBtyTQ/8Pmopkxz7+/eziNAPn74n8rmZrJGTuSQzaB1v02+ZiVGgJLpjjaPYY1zE EeMg== X-Gm-Message-State: AOJu0Yywb+Aj0+bakfN6R+iLstmY+WpqtKLljtlg+ueFxj8jbyeHI2Jc IR6Fxhbihb+Qf5GOixY7n45M/LKYKgMdPJuGj1z1HKoTgPds6oG3x3zwl6Rr+OGopc4VNi3VAAz 4GA== X-Google-Smtp-Source: AGHT+IH/Rp/IjlPFZgIAEzdNYzL8aBM88d5tP9iD3BV755kvnBakj7CsVrD7R8NFWV17wOfNsmdxA2dCn38= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:1701:b0:e02:c478:c8b9 with SMTP id 3f1490d57ef6-e041b1449e9mr15176276.12.1720664004897; Wed, 10 Jul 2024 19:13:24 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:13 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-2-yuzhao@google.com> Subject: [PATCH mm-unstable v1 1/5] mm/swap: reduce indentation level From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 12CDE160017 X-Stat-Signature: szgs6z3nyhggbbdugxitf86enbygfjh7 X-HE-Tag: 1720664005-809998 X-HE-Meta: U2FsdGVkX19cxpRqEBESHi64Xl/2xkj8ex7V7YBFUOZxYvqde4m3k6fJxbNxMebNcIGtiwahSfl7uSO/JJWKx4aHd6PrabKNQiK5fYkuP13St4P9AW5++FomU9W14pQ7ts56xLVoDdgXTB7isqFWe9FTH9lVU8CgF3/4Q50NNYJHj3xVW3sz1kLsUwLpFHmokwWFrC9vlWvq6yyCbVL1HlU5QT+JR0IkhwLZVIkCwQ8BvV9NQoEf56tTvwTciDtRTQat6qgAzyquLwP9N54VrCy1Y/mPJDpRQcvNI8EOtC274IH6pFHhc7WSyHTcO+FGHitDLT5MdZCfhSC8RisxqMnaqjQU6aM7tfmm9kWLQE2ywJH1pArwnJnSkvIQtl+mWFaJ/j2qmRhwUlGMv0ep6lBMdyN6Y1uZd23sFVXUv7KRvj7x9OV05HW7CTCObeJiYMGHtjp9bz2KYZABPeoNGJx9pR5TbtIlXJuqAbE9U7b7glET+kJSfs6uNVikEaWNh5nrQb9d4IT9Q0+Ed40mOe+Kn2Knmvom+F+IY+22jCoOHnxNlf9H7pjfiwolUXJMLYxccX/aZGjzQniGxzjWbwiA2iqd8UXZOKtaLGXDnvpNLoIJUVZGiLEgUav1sVr3ZAIcZOB2pb8kVCdM4q5k+U0+unQXibnf3ZB7t35iY4Y7tiBC6vWphwYqKnLqoVrPWQ7wNuBor9jnenQqOAZPWUJnsB/Ntl0+wXtQL47DMrUSzlygCfFmzY3ABwmefVOhnkji44M+QmCGkEe+z+S22nmDjTDtxngkLt9/9xl49o1W8P1dScSKoZZYhtamFikE08+aaut7/CC7w0nRx1uvqLfbamCk/SeCUOz5093+lRq63R6UXYktf6gr4/JOlD4fwjY8zthGFRQlhaI42B5UZmOoiN35yISTfCgz7Xf5zPhjW5Er9yYEcN4wtHE3+nlHMr1AVUglCJxE591G2BL yuJtxNCR O6U5Fzyp4a4bYsjqrCLBabWmH8Wg6druq5X25j2HDmc/MlVUO+i3AdCc7vLxcviNSKfP2qDmtEstGb24x1GFTh17yeLzYbKGBXvLKvbFDpqeaZPaZNbY/mGJjL2ahzGKPnbzuNpJcKlpLnGTBBuS/jqCjwyTF522iBtjptpuo10H9n8b9KN1k31AKFowt6H/jyIZWRRiQ/b48ukagcnyv3fDz+HT4NwNG+VMzsP57liqs9NWSjgbUnbds9Ew9ZmQqwFbjAkuGdmPl5XqyvcdJFbSsZT1Po1l8wlBWO1+WvKd0BeH+LSDOqMPbs8FrM3DLN1T0/AnEj9dU2syq6MwloShWKmZLqkDgKx1GnKkpwmuPbyKu1Weahkp33US1J7M5Zkij+ovwR1geEnnAjWpSGknxbHQlWViGvseatQjjnqJolz6U3+imik2anHMRw0nwSuBGLVXMn1OiiYwkvRFuHFcDFRKyNQf7ARY+0NVqEJ9I4AAgndjE9eIsnS9dmALtVAOJ6ninGWpnMgs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Reduce indentation level by returning directly when there is no cleanup needed, i.e., if (condition) { | if (condition) { do_this(); | do_this(); return; | return; } else { | } do_that(); | } | do_that(); and if (condition) { | if (!condition) do_this(); | return; do_that(); | } | do_this(); return; | do_that(); Presumably the old style became repetitive as the result of copy and paste. Signed-off-by: Yu Zhao --- mm/swap.c | 209 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 109 insertions(+), 100 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 9caf6b017cf0..952e4aac6eb1 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -117,7 +117,9 @@ void __folio_put(struct folio *folio) if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); return; - } else if (folio_test_hugetlb(folio)) { + } + + if (folio_test_hugetlb(folio)) { free_huge_folio(folio); return; } @@ -228,17 +230,19 @@ static void folio_batch_add_and_move(struct folio_batch *fbatch, if (folio_batch_add(fbatch, folio) && !folio_test_large(folio) && !lru_cache_disabled()) return; + folio_batch_move_lru(fbatch, move_fn); } static void lru_move_tail_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_unevictable(folio)) { - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - lruvec_add_folio_tail(lruvec, folio); - __count_vm_events(PGROTATED, folio_nr_pages(folio)); - } + if (folio_test_unevictable(folio)) + return; + + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + lruvec_add_folio_tail(lruvec, folio); + __count_vm_events(PGROTATED, folio_nr_pages(folio)); } /* @@ -250,22 +254,23 @@ static void lru_move_tail_fn(struct lruvec *lruvec, struct folio *folio) */ void folio_rotate_reclaimable(struct folio *folio) { - if (!folio_test_locked(folio) && !folio_test_dirty(folio) && - !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; - unsigned long flags; + struct folio_batch *fbatch; + unsigned long flags; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_locked(folio) || folio_test_dirty(folio) || + folio_test_unevictable(folio)) + return; - local_lock_irqsave(&lru_rotate.lock, flags); - fbatch = this_cpu_ptr(&lru_rotate.fbatch); - folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock_irqsave(&lru_rotate.lock, flags); + fbatch = this_cpu_ptr(&lru_rotate.fbatch); + folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); + local_unlock_irqrestore(&lru_rotate.lock, flags); } void lru_note_cost(struct lruvec *lruvec, bool file, @@ -328,18 +333,19 @@ void lru_note_cost_refault(struct folio *folio) static void folio_activate_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_active(folio) && !folio_test_unevictable(folio)) { - long nr_pages = folio_nr_pages(folio); - - lruvec_del_folio(lruvec, folio); - folio_set_active(folio); - lruvec_add_folio(lruvec, folio); - trace_mm_lru_activate(folio); - - __count_vm_events(PGACTIVATE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGACTIVATE, - nr_pages); - } + long nr_pages = folio_nr_pages(folio); + + if (folio_test_active(folio) || folio_test_unevictable(folio)) + return; + + + lruvec_del_folio(lruvec, folio); + folio_set_active(folio); + lruvec_add_folio(lruvec, folio); + trace_mm_lru_activate(folio); + + __count_vm_events(PGACTIVATE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGACTIVATE, nr_pages); } #ifdef CONFIG_SMP @@ -353,20 +359,21 @@ static void folio_activate_drain(int cpu) void folio_activate(struct folio *folio) { - if (!folio_test_active(folio) && !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_active(folio) || folio_test_unevictable(folio)) + return; - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.activate); - folio_batch_add_and_move(fbatch, folio, folio_activate_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch = this_cpu_ptr(&cpu_fbatches.activate); + folio_batch_add_and_move(fbatch, folio, folio_activate_fn); + local_unlock(&cpu_fbatches.lock); } #else @@ -378,12 +385,13 @@ void folio_activate(struct folio *folio) { struct lruvec *lruvec; - if (folio_test_clear_lru(folio)) { - lruvec = folio_lruvec_lock_irq(folio); - folio_activate_fn(lruvec, folio); - unlock_page_lruvec_irq(lruvec); - folio_set_lru(folio); - } + if (!folio_test_clear_lru(folio)) + return; + + lruvec = folio_lruvec_lock_irq(folio); + folio_activate_fn(lruvec, folio); + unlock_page_lruvec_irq(lruvec); + folio_set_lru(folio); } #endif @@ -610,41 +618,41 @@ static void lru_deactivate_file_fn(struct lruvec *lruvec, struct folio *folio) static void lru_deactivate_fn(struct lruvec *lruvec, struct folio *folio) { - if (!folio_test_unevictable(folio) && (folio_test_active(folio) || lru_gen_enabled())) { - long nr_pages = folio_nr_pages(folio); + long nr_pages = folio_nr_pages(folio); - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - folio_clear_referenced(folio); - lruvec_add_folio(lruvec, folio); + if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_gen_enabled())) + return; - __count_vm_events(PGDEACTIVATE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, - nr_pages); - } + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + folio_clear_referenced(folio); + lruvec_add_folio(lruvec, folio); + + __count_vm_events(PGDEACTIVATE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages); } static void lru_lazyfree_fn(struct lruvec *lruvec, struct folio *folio) { - if (folio_test_anon(folio) && folio_test_swapbacked(folio) && - !folio_test_swapcache(folio) && !folio_test_unevictable(folio)) { - long nr_pages = folio_nr_pages(folio); + long nr_pages = folio_nr_pages(folio); - lruvec_del_folio(lruvec, folio); - folio_clear_active(folio); - folio_clear_referenced(folio); - /* - * Lazyfree folios are clean anonymous folios. They have - * the swapbacked flag cleared, to distinguish them from normal - * anonymous folios - */ - folio_clear_swapbacked(folio); - lruvec_add_folio(lruvec, folio); + if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || + folio_test_swapcache(folio) || folio_test_unevictable(folio)) + return; - __count_vm_events(PGLAZYFREE, nr_pages); - __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, - nr_pages); - } + lruvec_del_folio(lruvec, folio); + folio_clear_active(folio); + folio_clear_referenced(folio); + /* + * Lazyfree folios are clean anonymous folios. They have + * the swapbacked flag cleared, to distinguish them from normal + * anonymous folios + */ + folio_clear_swapbacked(folio); + lruvec_add_folio(lruvec, folio); + + __count_vm_events(PGLAZYFREE, nr_pages); + __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, nr_pages); } /* @@ -726,21 +734,21 @@ void deactivate_file_folio(struct folio *folio) */ void folio_deactivate(struct folio *folio) { - if (!folio_test_unevictable(folio) && (folio_test_active(folio) || - lru_gen_enabled())) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_gen_enabled())) + return; - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate); + folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); + local_unlock(&cpu_fbatches.lock); } /** @@ -752,21 +760,22 @@ void folio_deactivate(struct folio *folio) */ void folio_mark_lazyfree(struct folio *folio) { - if (folio_test_anon(folio) && folio_test_swapbacked(folio) && - !folio_test_swapcache(folio) && !folio_test_unevictable(folio)) { - struct folio_batch *fbatch; + struct folio_batch *fbatch; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } + if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || + folio_test_swapcache(folio) || folio_test_unevictable(folio)) + return; - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); - local_unlock(&cpu_fbatches.lock); + folio_get(folio); + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return; } + + local_lock(&cpu_fbatches.lock); + fbatch = this_cpu_ptr(&cpu_fbatches.lru_lazyfree); + folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); + local_unlock(&cpu_fbatches.lock); } void lru_add_drain(void) From patchwork Thu Jul 11 02:13:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13729931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 727A3C3DA49 for ; Thu, 11 Jul 2024 02:13:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E73AB6B0092; Wed, 10 Jul 2024 22:13:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E229F6B0093; Wed, 10 Jul 2024 22:13:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C506F6B0096; Wed, 10 Jul 2024 22:13:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9732B6B0092 for ; Wed, 10 Jul 2024 22:13:30 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3D573160134 for ; Thu, 11 Jul 2024 02:13:30 +0000 (UTC) X-FDA: 82325850180.13.A9C629F Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf18.hostedemail.com (Postfix) with ESMTP id 746C51C0011 for ; Thu, 11 Jul 2024 02:13:28 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Fz1SxUgJ; spf=pass (imf18.hostedemail.com: domain of 3xz-PZgYKCL42y3lesksskpi.gsqpmry1-qqozego.svk@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3xz-PZgYKCL42y3lesksskpi.gsqpmry1-qqozego.svk@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720663977; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Mz1u87ev8DYclZ/2H0TBqU+M5VpYJmHKMXbGaF77G5w=; b=jRyO3ohHOSIMvYrtp2oFKngFPI0AnqxQdgewYvDu4jQs/3i+H3XICDcCRpeE7aVgpBz6sQ a7Zp79dorR+IB4A3NgLT2Y3jn9RScxr4HO+mj1is4Phf5ruKSG3XJA0CkDMCo/QpmnkVKB Wcz0gNJJFYqBJcK1iHPtq4SD0LkfNas= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720663977; a=rsa-sha256; cv=none; b=jVe93hPDHTvGDUuAr5UFtWxsyLoUEWNgTj/JDnM7DdOkqzZ4G58fBlKhlZyaHbvFrrXn0k r5SfGeYoWZPc/k3zcHp8FwEeuYFeRXzEizjhtFEJN/GC3TzBLJjqsg+QXyTXNN8SHH2mPe ZgyEHylnoP0bU5mRXBHREfcCIk/B9WQ= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Fz1SxUgJ; spf=pass (imf18.hostedemail.com: domain of 3xz-PZgYKCL42y3lesksskpi.gsqpmry1-qqozego.svk@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3xz-PZgYKCL42y3lesksskpi.gsqpmry1-qqozego.svk@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e03a434e33cso689979276.3 for ; Wed, 10 Jul 2024 19:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664007; x=1721268807; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Mz1u87ev8DYclZ/2H0TBqU+M5VpYJmHKMXbGaF77G5w=; b=Fz1SxUgJtVLDLMcfrGm6f/6XlO0Of9Kq6dROcfhqr9QuTiVrODs3iQb4HH1JjQjwnj Ncl6UnhlsUY7F0EwCUBKTbH/U7KcQzggDHRJyZaClGMDlQubxzp3R2HvzXfry1tOvyMW Ria4D1KBLeRkIhmKz9ksoez4T+2eo4knKCcwBFRIIzcvThnFMEI7AeBT4y1WGd5YT84x 6HUMoix6aV/KZkoj7len/V91wFl/xwGAHw+3N/h+++GzmHDWn8FzfpEYGXqHOIleaJlZ G89LtgTL0PlWG5W0tw079Z+LREkS7H9vZBhlV0+Cl4EPKGJ/UByUgRBl5rWKFzA8QSBf U8pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664007; x=1721268807; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Mz1u87ev8DYclZ/2H0TBqU+M5VpYJmHKMXbGaF77G5w=; b=AS8Bar4eyEpPbt0D9lTzUszQMzBb9wysZ+8ckp7Jjap9Ch+aYcaSXLmtDRixnBhdIr E0FJ7ifVERMg4EgnmUQFGqHedVe9akfuETcmmPFYa/VtugnnWHUc8iY2J7niECbsyMAf 0HlCK0rb9RZjGhJQcyS7kC2DY2SRkNHXnI7GDM9c1R15uzwM6BX0JiIgTOeBnkGuIro2 IRShKGLiVjHiMnOoAPeY32H82ns4zJfia11IAFRiPv3/aC/a415tA5XHEglklHd9K+G0 4RnaPJqBxWGNA6GwiLbpns4JInKGD8NhLmikKhVcAIUVVC47JLWd2rrQCxUYa/Ywjm69 5mKQ== X-Gm-Message-State: AOJu0Yw9gEnH+jBTY1W6uy7eKbekJcHawdQzBys/ghWx/8hGWNIQrICk I/r/OvDa1qYR6TM/ccT07KmZS40yLx6YUQPCVm+AcqCldt4fnlOdW+aXUJUwQngI6D/qqCf/QeP 0kw== X-Google-Smtp-Source: AGHT+IEFx7WEmcVLtwhV7MLkiuoJz3OGus58zTubJgwlkdvY2QBLN+8BvOJfH7aoBGbgNkKVhQae3bL0CsI= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:729:b0:e01:f998:fea2 with SMTP id 3f1490d57ef6-e041b14337emr15713276.11.1720664007468; Wed, 10 Jul 2024 19:13:27 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:14 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-3-yuzhao@google.com> Subject: [PATCH mm-unstable v1 2/5] mm/swap: rename cpu_fbatches->activate From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Queue-Id: 746C51C0011 X-Stat-Signature: zi794eg7qpfbcxbnxhcq3fbpyugn1zwi X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1720664008-614174 X-HE-Meta: U2FsdGVkX1/720s/eW+FWneuwF6M6yTrzGasOLrCsiR9gN6lfH5XmBFSpQZ+7nMd6bUbXFt8HADpceh1kXdxJf7i5yMCG0nUnfqW40bwBe971haa6yIoSQ6rGgI2C+k85KRmTgGNbUe9jqpWkYZgRtYtWfhtSGKU8+kVx7lTPdNZMTtaq/uHcBf9nhV0yeStHmL9EmcWA4I+28xwYw4FOX5KfzwDOr8St6cYpuPkHdlTBa3trG5TbU5i4VTcX9k8bSFYPKF0YbxLRz/eH9YUcaryz+4fseHn6QbFTHUqd8AToqMoUuwGgxtqZyDqmEskAA+ZfO88JnmXgVvh2XpsR3fr9N3q7oH+wtYI54A9AFpm37hQSSULLlOMwvwcjTmGvci14DN3RnOJuhT6ZKZlB/MJ9M1Vf/BrI1PvZJp8rJTKoSrsYwU22yhTH3/8KcBTbq/OBPhW3aoS06+v2m8WUBN2DwVVcm+OlcgXtYFwq1ML8LBblNAbdfQNvcX4641XTB9SGGow4XM2XMlE8ctp1nueC6SGLcxy8hQQe+tA4kKN1coPIrmgQeMHb4SnQcXkhSQuj2ag3OZtHArKB7qpL+zIRXfQhIwlEwBOIc+Yi+m/7UrDWTuHHnnThMeMjeisWZ+d7JXSfq1/J9L3uKsYzcKR88LJ1Jyy9qVfR7jT85FPm+EfTEz4zaUp1iUa1LiTVrfEQDK3+1tXrd4N28mb70JjJQ7Q7Hb5Unyj76DXcpfofpb/1eql/xtPVadx4j5onacE7RA/b0js0kbJ5dzdecIEOJOCgFIkpPXsy6C59wDYvEecuYsoR50R7Hryf56J5/IVbZWfkIL5EpYeTX1dfwufEG78hsYYaO/IPjQLsBcep5LPLgeJVelCdMuIL6XWI3AeNyOXP4uo6BvE+x3xIx4VKHwt2AcEwTGqeBANy4wiyfn7OT4TN/OsFull5kT06xzHyNtX+HEYJ0bub/+ nXcT2EZK zbMppt3Z51snj2AO8Mey9YaF0NARdYT3TCCpKuOP+6Gzax/NLoYhy8mNmTWCeAKqnN4pXrfXSlapHImV5nZCIOImpaV9bT//vwDrXHneLWp0oqK4A87dTaBVqeSeNM8EqZB8zA3hooWdpqFRaoLNaO0TrixHTHrF6j25Bshg9FITvLdQphYiOF8qX1nijeSNPAsZmOMluPhIvlwFdsZ7TWR2gYrFdjci3YRmSXjmrob5no/9LDt6qtNBI9L1a5z8pVC5KohfAu+zMNxJzZLso0z1hjf6wQ37OI9bPsBF/NX267me1GU8hJlKXv6X4v2fTvzrcaINDa5+LWScNJ8b8lf8ZwkVN18UGtrIqzgQhOtL1yBGPhUeixw2dTrMxCAfYsp3N8XtwXaopacbFt4zY0cyR/zI9gWPNJtdfOrK8/YY1+BlSqa3bVtODAdTmSbzz4H2+yrJBEsqtJppJ45eOcS5BxsRM358RkJl+hMCPkqG7VF+I9M6irji6HkbhTiqpIdsnDKqAadetJyr9HGnStfeyZ4/dqxi7GKaFBNAVd9kRnaMTgHuC21udtg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rename cpu_fbatches->activate to cpu_fbatches->lru_activate, and its handler folio_activate_fn() to lru_activate() so that all the boilerplate can be removed at the end of this series. Signed-off-by: Yu Zhao --- mm/swap.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 952e4aac6eb1..e4745b88a964 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -67,7 +67,7 @@ struct cpu_fbatches { struct folio_batch lru_deactivate; struct folio_batch lru_lazyfree; #ifdef CONFIG_SMP - struct folio_batch activate; + struct folio_batch lru_activate; #endif }; static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) = { @@ -331,7 +331,7 @@ void lru_note_cost_refault(struct folio *folio) folio_nr_pages(folio), 0); } -static void folio_activate_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_activate(struct lruvec *lruvec, struct folio *folio) { long nr_pages = folio_nr_pages(folio); @@ -351,10 +351,10 @@ static void folio_activate_fn(struct lruvec *lruvec, struct folio *folio) #ifdef CONFIG_SMP static void folio_activate_drain(int cpu) { - struct folio_batch *fbatch = &per_cpu(cpu_fbatches.activate, cpu); + struct folio_batch *fbatch = &per_cpu(cpu_fbatches.lru_activate, cpu); if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, folio_activate_fn); + folio_batch_move_lru(fbatch, lru_activate); } void folio_activate(struct folio *folio) @@ -371,8 +371,8 @@ void folio_activate(struct folio *folio) } local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.activate); - folio_batch_add_and_move(fbatch, folio, folio_activate_fn); + fbatch = this_cpu_ptr(&cpu_fbatches.lru_activate); + folio_batch_add_and_move(fbatch, folio, lru_activate); local_unlock(&cpu_fbatches.lock); } @@ -389,7 +389,7 @@ void folio_activate(struct folio *folio) return; lruvec = folio_lruvec_lock_irq(folio); - folio_activate_fn(lruvec, folio); + lru_activate(lruvec, folio); unlock_page_lruvec_irq(lruvec); folio_set_lru(folio); } @@ -490,7 +490,7 @@ void folio_mark_accessed(struct folio *folio) } else if (!folio_test_active(folio)) { /* * If the folio is on the LRU, queue it for activation via - * cpu_fbatches.activate. Otherwise, assume the folio is in a + * cpu_fbatches.lru_activate. Otherwise, assume the folio is in a * folio_batch, mark it active and it'll be moved to the active * LRU on the next drain. */ @@ -829,7 +829,7 @@ static bool cpu_needs_drain(unsigned int cpu) folio_batch_count(&fbatches->lru_deactivate_file) || folio_batch_count(&fbatches->lru_deactivate) || folio_batch_count(&fbatches->lru_lazyfree) || - folio_batch_count(&fbatches->activate) || + folio_batch_count(&fbatches->lru_activate) || need_mlock_drain(cpu) || has_bh_in_lru(cpu, NULL); } From patchwork Thu Jul 11 02:13:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13729932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11D45C3DA41 for ; Thu, 11 Jul 2024 02:13:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59B966B0096; Wed, 10 Jul 2024 22:13:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FD936B0098; Wed, 10 Jul 2024 22:13:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D8766B0099; Wed, 10 Jul 2024 22:13:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0DDBB6B0096 for ; Wed, 10 Jul 2024 22:13:33 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BD91C1202A6 for ; Thu, 11 Jul 2024 02:13:32 +0000 (UTC) X-FDA: 82325850264.13.B57CD00 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf28.hostedemail.com (Postfix) with ESMTP id 09FF0C000C for ; Thu, 11 Jul 2024 02:13:30 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="Wb/uPt/U"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3yj-PZgYKCME516ohvnvvnsl.jvtspu14-ttr2hjr.vyn@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3yj-PZgYKCME516ohvnvvnsl.jvtspu14-ttr2hjr.vyn@flex--yuzhao.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720663984; a=rsa-sha256; cv=none; b=Wk/skJ33Pa1svY3yaI1ROxGHrDNqFm+8G+9Gi3JI/odUVXcxuUvHEuFb7p9BTPMgTOkU/Q fNgi/huYKiIfOWUk9yfYmLcdVLjQZb6PGJ2JdfSlSpuajmkvFYYbd8hT7T1kkzC30Bhv7p x6Mnlr6wpCZhew2LtVLVjbKbUyp6g8A= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="Wb/uPt/U"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3yj-PZgYKCME516ohvnvvnsl.jvtspu14-ttr2hjr.vyn@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3yj-PZgYKCME516ohvnvvnsl.jvtspu14-ttr2hjr.vyn@flex--yuzhao.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720663984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WhNUH7SAP/MNFXjYcDX49dOz4gKNgHUZEXTQwt2QQyA=; b=eoc0mhOPKSKk2yaDLZB5EZ/JL8EQZ+L+hl9VkjKMlryL7LOD+J3njQEEBMxFSokbIS7/uy kg/aZH94mos3b4oYnjewAQAWSvFNUNI6Oo0XfBQH9DOWPyUGpIUG8SJXWmv/H3EtVaTNTI eaX3wewQi7wTBgQ9sJXfSg07tqg83F4= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e035f7b5976so889977276.0 for ; Wed, 10 Jul 2024 19:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664010; x=1721268810; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WhNUH7SAP/MNFXjYcDX49dOz4gKNgHUZEXTQwt2QQyA=; b=Wb/uPt/UA2vib45JkW404UaHaR+kxMwYFXdpJ8EqSNGs2ApWoM4lDUYLzxsFWyuhFS drMUoM4+eleHOtlIyEE9+vwg0+sJ1QyKTf1zRv0RlWw86NzYpABGvGqyldm0Z9MZjT6r M/sJV0uhKcxr0kKMy+ak5of1yEQDaBTSjJ20JNOs0Mh12aB5fDcDLyRGU8b1l4znsdbi 7p+5CThjfIjDRE9uuEMtGquoXBf+AMMnwjWMHVxshyFKH2HlOAmU/1VSmISa5jNR6F7K SDgjHRLE/fpPwxnCeyZ2QTQMPi2u+lFXnSRczvSkGieCbLX7PmwnpgQqMYCdoDDmFOE3 nwYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664010; x=1721268810; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WhNUH7SAP/MNFXjYcDX49dOz4gKNgHUZEXTQwt2QQyA=; b=E3sETzz1GYJM7cl9fLC78l95rvFx2LF8Gpw2RdonLaqwlOLzg0FgHkEdu7eNnoGkxu SKj6uCZN6ubvIQu96FaaMC0fkEjbXJYkCRC+jg6r96kcAFkkH7kEHFrEvRcLnSYYYOFj KCkf70gNQsvGD1HKqk0dcO7MxhPChZNbEoLJBy7IxZg7BiyGUNrtScakPDSIg+oodOVf /YE69kJZJ4nhUiAmnBEH3mAHeY6s0Il0wyjNz/SMe+VeTliMRb+zxRt7wHBrjPLMtx88 AlJjjaYJtBeFn9maLeovNPUj5IzuJJGs7UwRDBMuprOKVuCbsmuuh+OcZLcHmd5ms8xZ Fvbg== X-Gm-Message-State: AOJu0YwuCDi91XqsSGEQAafFwuewjd0jliaTsRBWYKyf7n6ypr817ArR b5C0LB1yem+fPfezSwBA1GgCA+ABzyPaVaooa9kxsnJyhtdmdF/5+6pBtxapn5QRYO+PJhjXWqG MDw== X-Google-Smtp-Source: AGHT+IGkIwS4bVyfM5ByoYpmZQq2iqjOHx1IN9uZ+1eS+2/SoQ8AfYNeitOzBddCLqNu0G2MAu021nNjM8U= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:32a6:b0:e03:61bb:6032 with SMTP id 3f1490d57ef6-e0577fa164amr20670276.1.1720664010014; Wed, 10 Jul 2024 19:13:30 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:15 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-4-yuzhao@google.com> Subject: [PATCH mm-unstable v1 3/5] mm/swap: fold lru_rotate into cpu_fbatches From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 09FF0C000C X-Stat-Signature: qs8kkpsud1181k98u47n88r4ymt6fosr X-Rspam-User: X-HE-Tag: 1720664010-539159 X-HE-Meta: U2FsdGVkX1+u7LSGO9lxcUi6u+hdBqxEtG7i55Gn1k6y3VKEYbg6l6Snht+QOLtKipMlfZnYeg0cRmLKEB66vvRrJ6JSYSHVafnuQThDDjuHL13VDwan2uDKrn595DbIoFl5tpMKN84XeKE1j6W2BTEOgPfkt3koSb0zcEE0WFoew/QNSTAdpE38etdMtLMuU+MtDwcWSCOsjeHYmsKvoLcYI0VIGj7kBFCiDvTu8XUJ3D2VcrjMfDMEHbxPtbpdI+2T4Ge/yS4QpXizDKc20+sFtLATf9BziT82Cq6yeBIqoVB0MkR+LnEJcASL8juEfOi01QUpxf7RrYSWmWXMK2xhTCpomHHZWx1seJMoaNmBnzS6IjKRTUEPrGIYUUomAWKV//lrlFrFxAMCPhS/OLIMePaMw4lMyb6hB1nAfwDa3YeCxjKMLE1xHdfDesAX9U48qWkNX2cOZhBT5etXI7eGeMj0uOTZeyB3RsU5KgcbbpmfHFF1hTMZ+9zDXr9oTyB9XuxcU795903dCeiMySuTnjREZ5pP3rW5xyqjuy4+eoA/LK0EHggrUWBkIUMNRl9lSGKTnVdZz0h+tmn/hqzJygtskjw1LYfBss/ybmgE3fM9+0Rsrg9wew9V+G57Z3iwpmCeMsmixXv23qFmy+W81UciMHmR5n9HNbp8RGrYPcf2HB3qt97lK234gAXLrMEkQvxv9m2rdH9Vp7dGvWXp8p+IBVbk5E506RzN1Rs/tYxkDqaUCnz2AATwuFhxAN1UDqThtVe4hOS8OoBpFmcHflXLqaIj/gqltN3ge+M+GS+96RarGGyerP+4DbpgaungGcNL9W3NmBZ+giEM9efIIM3Fmj8mZMFEvkb/Q9IVNM2AdaZVG+GiUxS4GYMZhUSNI8FRBG/8j2oElICXn4mQZQqow6yjdesnNnwl3v7EPQfUNYe2jdiKWQsVY4YqVCaJvPZx+nUAFA9sKk6 UFY7N+CA ddBov9XYqvr93sPrVgK/W/BaUi0WeCDPhWsUPk6ub8jq0qjj1RNLYJpatuAM47aOtLIvxeYzMWdUoH3pY5jOniSzD/fhF/PJdeyrXF7W5R8vZEpCXFR/9dTw8/8DX1nY13Svt++LqcU2ODO8AHCPZ/+0NgzEUpCa6b4LSde6pz8H6HvckumetbJQOFSU67g6Nv4Ug2bAGwhNZKnbdeb0hAnRo/L8WA8cbgSDsWIMFHd1e3XoWBhSP8OZUm2L/VqMkqvSZdJ8CbVEZZX3vOSp+9DRcvoNLIqk0kbQz1QQFqzYm7FMSMwloiBRicqbhZKjsvmcuTiWMw9VINHuYnMxU9ucaThE3PXq56teQ1DBcXzRbw3DSr0IB0BoUgdvg8nGZ79MkM90WwLLiWHGpl39gdZudlyK0dyXJGEavCKZtBHazcEoDsWkeCV4ZRnKPG8f9QYUoYATAVQLsQAo8I0XyY9SrIrnrkw4BU4ESDElpxS4fe0y/ownw8BvvLTkOVD3DDB8GSlgo/r/qFyas1pQgrTM/mHDYW/21ZEPAskIm+IrH53yLGWJDHRAKnQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Fold lru_rotate into cpu_fbatches, and rename the folio_batch and the lock protecting it to lru_move_tail and lock_irq respectively so that all the boilerplate can be removed at the end of this series. Also remove data_race() around folio_batch_count(), which is out of place: all folio_batch_count() calls on remote cpu_fbatches are subject to data_race(), and therefore data_race() should be inside folio_batch_count(). Signed-off-by: Yu Zhao --- mm/swap.c | 36 ++++++++++++++++-------------------- 1 file changed, 16 insertions(+), 20 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index e4745b88a964..774ae9eab1e6 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -47,20 +47,11 @@ int page_cluster; const int page_cluster_max = 31; -/* Protecting only lru_rotate.fbatch which requires disabling interrupts */ -struct lru_rotate { - local_lock_t lock; - struct folio_batch fbatch; -}; -static DEFINE_PER_CPU(struct lru_rotate, lru_rotate) = { - .lock = INIT_LOCAL_LOCK(lock), -}; - -/* - * The following folio batches are grouped together because they are protected - * by disabling preemption (and interrupts remain enabled). - */ struct cpu_fbatches { + /* + * The following folio batches are grouped together because they are protected + * by disabling preemption (and interrupts remain enabled). + */ local_lock_t lock; struct folio_batch lru_add; struct folio_batch lru_deactivate_file; @@ -69,9 +60,14 @@ struct cpu_fbatches { #ifdef CONFIG_SMP struct folio_batch lru_activate; #endif + /* Protecting the following batches which require disabling interrupts */ + local_lock_t lock_irq; + struct folio_batch lru_move_tail; }; + static DEFINE_PER_CPU(struct cpu_fbatches, cpu_fbatches) = { .lock = INIT_LOCAL_LOCK(lock), + .lock_irq = INIT_LOCAL_LOCK(lock_irq), }; static void __page_cache_release(struct folio *folio, struct lruvec **lruvecp, @@ -267,10 +263,10 @@ void folio_rotate_reclaimable(struct folio *folio) return; } - local_lock_irqsave(&lru_rotate.lock, flags); - fbatch = this_cpu_ptr(&lru_rotate.fbatch); + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); + fbatch = this_cpu_ptr(&cpu_fbatches.lru_move_tail); folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } void lru_note_cost(struct lruvec *lruvec, bool file, @@ -668,15 +664,15 @@ void lru_add_drain_cpu(int cpu) if (folio_batch_count(fbatch)) folio_batch_move_lru(fbatch, lru_add_fn); - fbatch = &per_cpu(lru_rotate.fbatch, cpu); + fbatch = &fbatches->lru_move_tail; /* Disabling interrupts below acts as a compiler barrier. */ if (data_race(folio_batch_count(fbatch))) { unsigned long flags; /* No harm done if a racing interrupt already did this */ - local_lock_irqsave(&lru_rotate.lock, flags); + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); folio_batch_move_lru(fbatch, lru_move_tail_fn); - local_unlock_irqrestore(&lru_rotate.lock, flags); + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } fbatch = &fbatches->lru_deactivate_file; @@ -825,7 +821,7 @@ static bool cpu_needs_drain(unsigned int cpu) /* Check these in order of likelihood that they're not zero */ return folio_batch_count(&fbatches->lru_add) || - data_race(folio_batch_count(&per_cpu(lru_rotate.fbatch, cpu))) || + folio_batch_count(&fbatches->lru_move_tail) || folio_batch_count(&fbatches->lru_deactivate_file) || folio_batch_count(&fbatches->lru_deactivate) || folio_batch_count(&fbatches->lru_lazyfree) || From patchwork Thu Jul 11 02:13:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13729933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C03BC3DA41 for ; Thu, 11 Jul 2024 02:13:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7C186B0099; Wed, 10 Jul 2024 22:13:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD84D6B009A; Wed, 10 Jul 2024 22:13:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A54DD6B009C; Wed, 10 Jul 2024 22:13:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7ED036B0099 for ; Wed, 10 Jul 2024 22:13:35 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2C2FB1602F6 for ; Thu, 11 Jul 2024 02:13:35 +0000 (UTC) X-FDA: 82325850390.28.83C02FB Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf02.hostedemail.com (Postfix) with ESMTP id 73FAB80003 for ; Thu, 11 Jul 2024 02:13:33 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=gIwEu0h5; spf=pass (imf02.hostedemail.com: domain of 3zD-PZgYKCMM738qjxpxxpun.lxvurw36-vvt4jlt.x0p@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3zD-PZgYKCMM738qjxpxxpun.lxvurw36-vvt4jlt.x0p@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720663970; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6RCNHIc0/84Yq5SbDOD6xZQS6zwZu2n7K7QhObKOUEk=; b=JpcU6SB2QRhFrWmClmtjtmK79y2PEfkLgW08HXbYWPJicoE01YTDnIeNJoBccWYutXwOiw SWd56NOPWjgfbrU1T54Z5qBLPcegGxBbW+YTqht/W0o57jxQNHPLarh4gdTkx87sknNSgF cTaPl9/X1bxDAK2C0qv12+tva5FYfhE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=gIwEu0h5; spf=pass (imf02.hostedemail.com: domain of 3zD-PZgYKCMM738qjxpxxpun.lxvurw36-vvt4jlt.x0p@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3zD-PZgYKCMM738qjxpxxpun.lxvurw36-vvt4jlt.x0p@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720663970; a=rsa-sha256; cv=none; b=btRJpmkSDB6zQEFFvU7v/jz8+Q90p9+OXgG0+HTrOgfNwoxSM2crixtBb7KXMDCqRh53C8 ZGImYWL9W/qumCSX3drmkvqM30Ea2kIcwzZlYGy1BZQsC+G6cbTsyzRtz5T61MAOrLemsz VrD0XhAEOCG20I3ww7Dd350xiV2ZSFg= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e036440617fso710017276.1 for ; Wed, 10 Jul 2024 19:13:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664012; x=1721268812; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6RCNHIc0/84Yq5SbDOD6xZQS6zwZu2n7K7QhObKOUEk=; b=gIwEu0h5X6ZykD+T7+JLq1hB4fWRGc1cRirc5tfsu3l1DbXPKxnUYlIMaSik+5XsCG Hp+mE3rCf+H+TdcC26N13tYGbbHKQu5AK0214tiSQkEZ9+rJJ47RYEHutQQr1f/0KEZf 6/fV7WAviCZq6ROXI97ifQ+73gKxLzsCd0FyfzwWJHSu1OoTUITVq2fjHwFYES5lw/yS ZCtda5+kgSp2SBfMHi7RipFdS6xVNLMW4+o6cl/jHLEjHZCv/tsNxGPlPDZkY3affDcL ZN2vbdnNN8A0oIQ4wvFud/UkoYQ5Kky4Mc1+Rn42q++9tioycYGnKQfGY9YW2WkCtUfB eoaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664012; x=1721268812; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6RCNHIc0/84Yq5SbDOD6xZQS6zwZu2n7K7QhObKOUEk=; b=J4pxeOm1sX/rNWoCDaxTHZgvz75Qnf52e4NTrbWvnEKpOxfW2A58WRcNYjrYFIwCRB KZYalB87EbY4KIyhZxLo8mUuiuC897jcmc6oHFPunFtRLkj/X3nMFgS9VeD2f8ALdlrZ cxfUq1a27yF1vTCB3GM53+Z0dDaNlRt18CG7GKVKsaFZsq13W7Rv1LtlIOUyY4zVF5/d H5r6tWfFN50ZOUxexEW6DisLtjU279mFBOerg2gxvpmQbwEmHRkXXwVHuKdgGyQZn8MN 0CdzILdvHGcu2gaHkXElAr2ugY/4PHCDLslDdK2wnnFLsDjiJZ4oB33hCcwOqGzK9D77 XNzQ== X-Gm-Message-State: AOJu0YwkYiXirqcl040MucQEtXQG4k9OEZIOIDYM37MQp+dKxVdWGuOC 3Eu/c+QL+DJLb8ZW4fD7iSvmo+k6lJsAUOFJclel7khtdOQVVfnAAYgOYPytUy1HMbaeQzCa6RE Jlw== X-Google-Smtp-Source: AGHT+IGs4gFVi/kxatEGxz/B1lm3LDZuehH0XgpGsn3CIFnEqcrCXAfD8VgVg/nDjjGVQUAHTzXmxtBJ/OQ= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:2101:b0:e03:31ec:8a24 with SMTP id 3f1490d57ef6-e041b17dfb5mr397954276.8.1720664012653; Wed, 10 Jul 2024 19:13:32 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:16 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-5-yuzhao@google.com> Subject: [PATCH mm-unstable v1 4/5] mm/swap: remove remaining _fn suffix From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 73FAB80003 X-Stat-Signature: dke8t938sakhfgggoqrmxrmwd8hzsi1y X-Rspam-User: X-HE-Tag: 1720664013-992600 X-HE-Meta: U2FsdGVkX18YEm5XgFSYckj2mTT4Un+ucFUQCNxjy++ZxHoNkx1ONXB9rtfaLqKLYCIV46AhXlsvcDci9HIP3y442uiYnuARzehTVhiZmEky+mykL1p/5cZu/Bz+jVZJW7VX1yE5CsvqkUwnNpfJyUhVyM9Qe8qktOnJ4Jr0AUeU91N3bwqfjW3gbCIvQ9VlzZltVR8ZoHfC3rDeOPjcwvH9heDZcCxVKKZcNuabFX+zCifp0aJki9IFIsusmmaLZrGxx2WKGLz1uzmUBVcYd8asit39C7GYpjq8nV9nSmVUhn1WoEManRSNOY2uzCV9eyHOQh02faGFnO6DkhbBpvPMDfA896R12aLxZrgjHfvu9UMO8BRZ1iuhkrQ3RA5CZGlou/7/7367AhC8ZgDuZ6pz1dZOOsL0VlwDjyd4dZoccF/TT8Kd32Tj35hoXrkLocR/QtzLiJhENrr8CscCvVpQpbtarTs09c5c5vMmUEtihlY0BPPo7QN/ivnrwub3Xi9vq8HGlRE5v06+0a+uwdprvh+NCiVpFdWIunU9IDtE9bAtW47CrvJqxYBlAozJlF5clLq/JF1V1yVZzc59eRO6PIQz9lxm+nGKboukBH31UbLvdPSQIQoWI05yluXzVNI9F+9zFockGaKIU1RykVYekSxGZVFwAFhbA/fToxOchySvR2EFS1BVdY/jLUhLk0h12uirEBW5RU2xWOwpzsWUvcaa78CJZLiyhIcC11OFrWHIza273S76OxRW7ObdUb9Tj5h9AYJyPLobKTGFTmNYQDv4gxzJHbhJE7mg1G065wgCpEXzjiIPvR4vxl/tz44yihbX91cQ/GTv9Dc7NQZ92jwjZ7FKnpqNzsYBciaVuz7j7h2Rw2E9tUOASJbiJOQd9Z4fCPoNWEZ9eUlD3RyTQydXsUabG23cKjYAlvesteJejyqRxBDJu03qSK6gUD3Ksu0oeU1Qj+vt5nu 34HmWpRj vfFtxr3GgBVlzHTo94PBy9pbGZMUDZtgwrWMmhtPHVSunNHekqU8hUFrstyG5vE6vJ35HAVIX1F1eWpWaXn7Ld9Cqm654krkxpekxC2fl8KxK5ZL2WL9/MTdLszUAQVHIF5MiWNJMftRYlSEcFaBp2QUh8H/Anir7xYs6qNz/CIhA38IXuzwpX4f0A4Y85HRn1sa+VqfKQwW/Qs50XFxP18Pb7zZmjJaT2LPzSFhqjUdlcMZejX3b5VTf/qUPTeptj/03NSByk/DlFDGYJEfr+uJob6awHkWSrAJJL4JufMFCLXU9amqh8vGhZ3F+Kj5b0W28Gnuf9ap5XEYYqgONR8hDnWfG/DFyNRvGOL/gmJdvo7pDDqZgYEl/B6bMbgMjYohDl+a7XMFaXmQu/yVGxeyvEMWjJd+johNB1ApcHlfNu6L3Rh40DjoxaDfkQFGqMSEUBRhbIPCy/JwfYfElSTOlNxWGkDrVmXicDv42/cY+9ejGxb3uWzwGircikRzO68TvLwLoAiB0QxdHYp1o0rnMViwHlrgkAXRIKsDU1CIhTgw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove remaining _fn suffix from cpu_fbatches handlers, which are already self-explanatory. Signed-off-by: Yu Zhao --- mm/swap.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 774ae9eab1e6..4a66d2f87f26 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -160,7 +160,7 @@ EXPORT_SYMBOL(put_pages_list); typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); -static void lru_add_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_add(struct lruvec *lruvec, struct folio *folio) { int was_unevictable = folio_test_clear_unevictable(folio); long nr_pages = folio_nr_pages(folio); @@ -230,7 +230,7 @@ static void folio_batch_add_and_move(struct folio_batch *fbatch, folio_batch_move_lru(fbatch, move_fn); } -static void lru_move_tail_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_move_tail(struct lruvec *lruvec, struct folio *folio) { if (folio_test_unevictable(folio)) return; @@ -265,7 +265,7 @@ void folio_rotate_reclaimable(struct folio *folio) local_lock_irqsave(&cpu_fbatches.lock_irq, flags); fbatch = this_cpu_ptr(&cpu_fbatches.lru_move_tail); - folio_batch_add_and_move(fbatch, folio, lru_move_tail_fn); + folio_batch_add_and_move(fbatch, folio, lru_move_tail); local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } @@ -527,7 +527,7 @@ void folio_add_lru(struct folio *folio) folio_get(folio); local_lock(&cpu_fbatches.lock); fbatch = this_cpu_ptr(&cpu_fbatches.lru_add); - folio_batch_add_and_move(fbatch, folio, lru_add_fn); + folio_batch_add_and_move(fbatch, folio, lru_add); local_unlock(&cpu_fbatches.lock); } EXPORT_SYMBOL(folio_add_lru); @@ -571,7 +571,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma) * written out by flusher threads as this is much more efficient * than the single-page writeout from reclaim. */ -static void lru_deactivate_file_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_deactivate_file(struct lruvec *lruvec, struct folio *folio) { bool active = folio_test_active(folio); long nr_pages = folio_nr_pages(folio); @@ -612,7 +612,7 @@ static void lru_deactivate_file_fn(struct lruvec *lruvec, struct folio *folio) } } -static void lru_deactivate_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_deactivate(struct lruvec *lruvec, struct folio *folio) { long nr_pages = folio_nr_pages(folio); @@ -628,7 +628,7 @@ static void lru_deactivate_fn(struct lruvec *lruvec, struct folio *folio) __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, nr_pages); } -static void lru_lazyfree_fn(struct lruvec *lruvec, struct folio *folio) +static void lru_lazyfree(struct lruvec *lruvec, struct folio *folio) { long nr_pages = folio_nr_pages(folio); @@ -662,7 +662,7 @@ void lru_add_drain_cpu(int cpu) struct folio_batch *fbatch = &fbatches->lru_add; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_add_fn); + folio_batch_move_lru(fbatch, lru_add); fbatch = &fbatches->lru_move_tail; /* Disabling interrupts below acts as a compiler barrier. */ @@ -671,21 +671,21 @@ void lru_add_drain_cpu(int cpu) /* No harm done if a racing interrupt already did this */ local_lock_irqsave(&cpu_fbatches.lock_irq, flags); - folio_batch_move_lru(fbatch, lru_move_tail_fn); + folio_batch_move_lru(fbatch, lru_move_tail); local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); } fbatch = &fbatches->lru_deactivate_file; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_deactivate_file_fn); + folio_batch_move_lru(fbatch, lru_deactivate_file); fbatch = &fbatches->lru_deactivate; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_deactivate_fn); + folio_batch_move_lru(fbatch, lru_deactivate); fbatch = &fbatches->lru_lazyfree; if (folio_batch_count(fbatch)) - folio_batch_move_lru(fbatch, lru_lazyfree_fn); + folio_batch_move_lru(fbatch, lru_lazyfree); folio_activate_drain(cpu); } @@ -716,7 +716,7 @@ void deactivate_file_folio(struct folio *folio) local_lock(&cpu_fbatches.lock); fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate_file); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_file_fn); + folio_batch_add_and_move(fbatch, folio, lru_deactivate_file); local_unlock(&cpu_fbatches.lock); } @@ -743,7 +743,7 @@ void folio_deactivate(struct folio *folio) local_lock(&cpu_fbatches.lock); fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_fn); + folio_batch_add_and_move(fbatch, folio, lru_deactivate); local_unlock(&cpu_fbatches.lock); } @@ -770,7 +770,7 @@ void folio_mark_lazyfree(struct folio *folio) local_lock(&cpu_fbatches.lock); fbatch = this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree_fn); + folio_batch_add_and_move(fbatch, folio, lru_lazyfree); local_unlock(&cpu_fbatches.lock); } From patchwork Thu Jul 11 02:13:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13729934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DED1C3DA41 for ; Thu, 11 Jul 2024 02:13:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D62CF6B009C; Wed, 10 Jul 2024 22:13:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D11FA6B009D; Wed, 10 Jul 2024 22:13:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B651B6B009E; Wed, 10 Jul 2024 22:13:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9635D6B009C for ; Wed, 10 Jul 2024 22:13:38 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 132A0C02DE for ; Thu, 11 Jul 2024 02:13:38 +0000 (UTC) X-FDA: 82325850516.02.E9331AB Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf10.hostedemail.com (Postfix) with ESMTP id 5AA92C001B for ; Thu, 11 Jul 2024 02:13:36 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="sUnObo/h"; spf=pass (imf10.hostedemail.com: domain of 3zz-PZgYKCMYA6Btm0s00sxq.o0yxuz69-yyw7mow.03s@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3zz-PZgYKCMYA6Btm0s00sxq.o0yxuz69-yyw7mow.03s@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720663973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NwEn3rgyWV9MRh72z1BbFtM83NQgy5RTa5P2H131skI=; b=6/XP31qZdYxdKXbgXlGrg4PIPSTpGen5kAh/dsewSJT9KXAFUgJ2LeMWqMlzAfb8pDPiN/ HqWz04F99ZA3AVhgmgU2X6EHWrCt8qyoldwgVlbCw384tgHaOBiumIryzfEM2qDhsK9OSV qKmXB+nqOL0cAGQu7FCwh2xFOM7GVOw= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="sUnObo/h"; spf=pass (imf10.hostedemail.com: domain of 3zz-PZgYKCMYA6Btm0s00sxq.o0yxuz69-yyw7mow.03s@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3zz-PZgYKCMYA6Btm0s00sxq.o0yxuz69-yyw7mow.03s@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720663973; a=rsa-sha256; cv=none; b=lmciyV973sv0SeAruBknEPT/y+Kia1ow0V3sJMv98g/cu9RsTrEFDhaJEd+PN2JrCMdTT4 /ALTbUH3amX0rRFeA+bmzbAiJLXuXC6vQnTuuMt5qaVErkK5GJBQFJrdz7bjxt9JmfyBC2 Ft4BxuVyAjP53Xqhn4adFcVw6u32EQM= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e03623b24ddso736889276.1 for ; Wed, 10 Jul 2024 19:13:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1720664015; x=1721268815; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NwEn3rgyWV9MRh72z1BbFtM83NQgy5RTa5P2H131skI=; b=sUnObo/hpXTzBOy2ANdEeq7/YyRj016QVlYP2LLTeoCuSDJjSzNxI/fgBCsQRWidfz E4lTZafyc/aH1lGZiqNgRtCHEcyUuJGYP33YEqBzK2Pm+H0xhUjoejTRpYLkvFQWfyJB AA2fWhSF2j2BljLz3W6rYqYY/5bmhMpxveiXDNWDEIMobn4giyKT25J3HsUcasdNoZbS UmBdBl8lh1ymXiDtOhGDQbYzXLzcksD8Ij8YjHxjppGamZhIvOpLZYZO2Uidui0NZS2f u95XI3BTxguv0Ce43gT6bCZhpBbOB5GpX93Ilkt6JjiYMV7Gn5K/JLYdK+4NVVp0y0+5 FXGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720664015; x=1721268815; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NwEn3rgyWV9MRh72z1BbFtM83NQgy5RTa5P2H131skI=; b=iEhDzHbueUrefxDEWd4Vt8t+psgi+jx8BEN7I5PrPYgy+3IJEKddL5jtxA3/Jjyu9o Sr1zrcUsxrPw5L710cwohRNMdqVbH80FtSw9c7y9Wmx5JUzixNMfu80wJ9wVxT8sAKo4 PY60zoIr3ElAnGCy78dXpu/XUXRMKkPkrYpjKp7ofJsd/duxaY7wJczzKNjw1jpOp2Ut st7uyFe6lcXJjjODXeCv5nh9Jptbj3Cp3bl3R3cde3mq8fjkOD13LIdVQ7uaE/kjr5AT 6Xv4cqVv+mZequV1OXHdVCVkcZrRsROy3/9VGqqkT+xuUYVWWyZAKrPcvLRWq4tD+7sL tMfw== X-Gm-Message-State: AOJu0Yy63z6qz8G3Pg9F55F0k7Oq5D/gWBJ59mBoLMGvFUWhtaR+zoVv 6Y6hJFkDFaw4YWb06hIB6uzJ5JTn/J0P075z8U1HSNi45keiHJUMP5tt0OAoKfhGcDpABxDjgdW mAw== X-Google-Smtp-Source: AGHT+IEbVZmYW6DpRm49c6xD0RPJKOVNNtKwNJhKE+00HKB2nAyQq5npS5jud2Z6RbIqjV4/yQeeTz+gpoU= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:9b06:2f28:6675:a98b]) (user=yuzhao job=sendgmr) by 2002:a05:6902:158c:b0:e03:a0b2:f73 with SMTP id 3f1490d57ef6-e041b05c952mr151870276.6.1720664015346; Wed, 10 Jul 2024 19:13:35 -0700 (PDT) Date: Wed, 10 Jul 2024 20:13:17 -0600 In-Reply-To: <20240711021317.596178-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240711021317.596178-1-yuzhao@google.com> X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240711021317.596178-6-yuzhao@google.com> Subject: [PATCH mm-unstable v1 5/5] mm/swap: remove boilerplate From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5AA92C001B X-Stat-Signature: pjcuwc8acbjdixcxqcrjk3qf4trkwsek X-Rspam-User: X-HE-Tag: 1720664016-423998 X-HE-Meta: U2FsdGVkX19eF0Xnxr7qAe046CDgJ1wbc9KC6sVwR0j502ihPBK/p5FR4hVG2+nEYmygnzRtQgOsp/YHI/DXFZK2wf2Jnm2fLYYhcuou9D8WNsi9l/IN7Nm80GiFLCcMXFJIsg1y0lDrlOe+WvrdYUxmYQGCazJjmmTnwMeRIt8zxzIBPHhYQ6kEAILJe0cnPO2G/e+62qlRqlYcXe50pXxCnsYxPyGIr5PheWzK3WwFWbHllvk0frnmeFZSnDxlQ6cN5aKww4e4Ygwd7QOEymCDWl7TMVXMaU8OeOjBzt8/OzGHMWvkqnodzN4fDK+Xp8J8CXACc5pyr5A/EgqXEjEEXxQ1LHVYSa9Sm9PSL0ef/yRxWXTfcb5QqoDOzntl1wR70FWXgE0GgYPohEqj4DV0AB2p2S+omvA7Pb7Xi7ILA0kHDiuiwH5Hm1RSXpi68bNxSOSpiAXSyh/HAFHPB3BCm55It9S1hW2QGPDgdkPrhbWfbgkuI5IC6M+6qFsYN7AdCyqIBj5LOKAkjvNpvBPTYCc8i2o64uN6R5kP/isJ1tqNnTiz2LfKOp1vcgzGZrJkvSzs5T9677yBsZILfiPuEjGNGSUU3hTZ8hSwP6GTMg1Gr1a/CcrV0BRUIdwPKiBpvKVBcBK8eVYNTnKhf4t7u74289C2FE1m+3x9oSbDpEk5vzwZWv/vaIpYydgF34oHfHXk+9uPmD91kFABZTuQ4bFpOGoWPQZfEOn145UXy063szoMctT4bdTocvWJ5na6TUeo9gUvjLUUkmhUgBAQsOe7gKUf+AlfP0IylnZqEhH4efU31RFzb7lgDIlpR0M+ye8IGJMraNVdImnwz6L/36I4cDe5wZp20qHDr5ZzQ56gkkgw1KngFulXXQnTdQwwhsJj2TAHg5IfVantWP/zJ5CwOGXignT10+73x8/EOTPWbpmmfMRzWEZDjq8I2HIX4fnM71bOPZZtedu 5vhcxprg N/bp43P8RVD1h9XlLJQkcvyh9D2DVbzhDyJGdCq3BMmdnt4sf/PFk7L7xaSxKef9nQvamWfDhnO5YpqnSUa0ubVpx9aXV/rqZ5mE0pdVbl14k+g+KiCjLqof/NaK20Bqwt45NcMYhjo08aQ5Aq2Cl1uyEIUVhnCdqR5hYedthIVi6LI+Gckoo8co7fEf2d5Vo1QfV9YU1Dl+P8cj43kxSartPpbJylGNn0VPbKqJDjEh7l+xdUezQwO5FujksuYj5xwdUb09vCvqG/g6SBgwhv3EJ2zQpW6hOY3fb33MlbS3zYpPL2bXKywY8GmLGEGDGQliUl+2vCHikQD4HJCeGdOhrCs5cH/D4a1p3TBdAxFIp7l0rSHo06VTdsXl6XFsN+iePiERQH+LJEVoMH14XyQfM1/02DwwU1QtxQwY+p0m8Te/28MW0CwnmF5NiaQmtmaVhN1R1CpFJkU2EHV/4EMlaMiK54RYt9quWj56L6zDwF/i2ZdKlyG0YM9kVWe2lQANH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove boilerplate by using a macro to choose the corresponding lock and handler for each folio_batch in cpu_fbatches. Signed-off-by: Yu Zhao --- mm/swap.c | 107 +++++++++++++++++++----------------------------------- 1 file changed, 37 insertions(+), 70 deletions(-) diff --git a/mm/swap.c b/mm/swap.c index 4a66d2f87f26..342ff4e39ba4 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -220,16 +220,45 @@ static void folio_batch_move_lru(struct folio_batch *fbatch, move_fn_t move_fn) folios_put(fbatch); } -static void folio_batch_add_and_move(struct folio_batch *fbatch, - struct folio *folio, move_fn_t move_fn) +static void __folio_batch_add_and_move(struct folio_batch *fbatch, + struct folio *folio, move_fn_t move_fn, + bool on_lru, bool disable_irq) { + unsigned long flags; + + folio_get(folio); + + if (on_lru && !folio_test_clear_lru(folio)) { + folio_put(folio); + return; + } + if (folio_batch_add(fbatch, folio) && !folio_test_large(folio) && !lru_cache_disabled()) return; + if (disable_irq) + local_lock_irqsave(&cpu_fbatches.lock_irq, flags); + else + local_lock(&cpu_fbatches.lock); + folio_batch_move_lru(fbatch, move_fn); + + if (disable_irq) + local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); + else + local_unlock(&cpu_fbatches.lock); } +#define folio_batch_add_and_move(folio, op, on_lru) \ + __folio_batch_add_and_move( \ + this_cpu_ptr(&cpu_fbatches.op), \ + folio, \ + op, \ + on_lru, \ + offsetof(struct cpu_fbatches, op) > offsetof(struct cpu_fbatches, lock_irq) \ + ) + static void lru_move_tail(struct lruvec *lruvec, struct folio *folio) { if (folio_test_unevictable(folio)) @@ -250,23 +279,11 @@ static void lru_move_tail(struct lruvec *lruvec, struct folio *folio) */ void folio_rotate_reclaimable(struct folio *folio) { - struct folio_batch *fbatch; - unsigned long flags; - if (folio_test_locked(folio) || folio_test_dirty(folio) || folio_test_unevictable(folio)) return; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock_irqsave(&cpu_fbatches.lock_irq, flags); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_move_tail); - folio_batch_add_and_move(fbatch, folio, lru_move_tail); - local_unlock_irqrestore(&cpu_fbatches.lock_irq, flags); + folio_batch_add_and_move(folio, lru_move_tail, true); } void lru_note_cost(struct lruvec *lruvec, bool file, @@ -355,21 +372,10 @@ static void folio_activate_drain(int cpu) void folio_activate(struct folio *folio) { - struct folio_batch *fbatch; - if (folio_test_active(folio) || folio_test_unevictable(folio)) return; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_activate); - folio_batch_add_and_move(fbatch, folio, lru_activate); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_activate, true); } #else @@ -513,8 +519,6 @@ EXPORT_SYMBOL(folio_mark_accessed); */ void folio_add_lru(struct folio *folio) { - struct folio_batch *fbatch; - VM_BUG_ON_FOLIO(folio_test_active(folio) && folio_test_unevictable(folio), folio); VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); @@ -524,11 +528,7 @@ void folio_add_lru(struct folio *folio) lru_gen_in_fault() && !(current->flags & PF_MEMALLOC)) folio_set_active(folio); - folio_get(folio); - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_add); - folio_batch_add_and_move(fbatch, folio, lru_add); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_add, false); } EXPORT_SYMBOL(folio_add_lru); @@ -702,22 +702,11 @@ void lru_add_drain_cpu(int cpu) */ void deactivate_file_folio(struct folio *folio) { - struct folio_batch *fbatch; - /* Deactivating an unevictable folio will not accelerate reclaim */ if (folio_test_unevictable(folio)) return; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate_file); - folio_batch_add_and_move(fbatch, folio, lru_deactivate_file); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_deactivate_file, true); } /* @@ -730,21 +719,10 @@ void deactivate_file_folio(struct folio *folio) */ void folio_deactivate(struct folio *folio) { - struct folio_batch *fbatch; - if (folio_test_unevictable(folio) || !(folio_test_active(folio) || lru_gen_enabled())) return; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_deactivate); - folio_batch_add_and_move(fbatch, folio, lru_deactivate); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_deactivate, true); } /** @@ -756,22 +734,11 @@ void folio_deactivate(struct folio *folio) */ void folio_mark_lazyfree(struct folio *folio) { - struct folio_batch *fbatch; - if (!folio_test_anon(folio) || !folio_test_swapbacked(folio) || folio_test_swapcache(folio) || folio_test_unevictable(folio)) return; - folio_get(folio); - if (!folio_test_clear_lru(folio)) { - folio_put(folio); - return; - } - - local_lock(&cpu_fbatches.lock); - fbatch = this_cpu_ptr(&cpu_fbatches.lru_lazyfree); - folio_batch_add_and_move(fbatch, folio, lru_lazyfree); - local_unlock(&cpu_fbatches.lock); + folio_batch_add_and_move(folio, lru_lazyfree, true); } void lru_add_drain(void)