From patchwork Tue Jan 30 01:36:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D02C47DB3 for ; Tue, 30 Jan 2024 01:42:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F6086B00BA; Mon, 29 Jan 2024 20:42:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A43E6B00BC; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 144866B00BB; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F0A6C6B00B9 for ; Mon, 29 Jan 2024 20:42:35 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BEDFEA09B3 for ; Tue, 30 Jan 2024 01:42:35 +0000 (UTC) X-FDA: 81734277870.12.7FB1A35 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf02.hostedemail.com (Postfix) with ESMTP id 063B580022 for ; Tue, 30 Jan 2024 01:42:32 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lwBD+DaL; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=kN0lJuJ5reaGbSzqGY2pZJbI4FoP1O/R8+iDhfs1ePo1Q64tyeZom8E9zYJB5fo30LtoP1 g7jIgGkxDvLOpk1MR7LFB1t4MVJYOcJNQpg9E2n5F0r4grUlWh51+T9en2EfEVesji4Yly 5Cq6TefTl3CM+4wfmGd0ghmxHHasfIE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lwBD+DaL; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578953; a=rsa-sha256; cv=none; b=Cl9EoxFv8WwqevJkk+HQ/aez+jNMB7y8bMxD6smhV1hfPaTAJFyQq2HfJUhtecEWcSxTkE PKTH+brFoLt1ESlstNpESLn8Ew2dnyuO1LLUh00cqzqTSvYwR+CToU4WXgYnOExervI47u Gbh6TXG4EI1dIaLub8Glr0QXLlbufHY= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-783ced12f9bso236068285a.3 for ; Mon, 29 Jan 2024 17:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578952; x=1707183752; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=lwBD+DaLsNagwEQiD1x7nPZl9Z7uM8R+ctk0GVq437TjSsMD5K1rOpJo5QNudPYqxz FrJWWsrOS2T1WbhZTCKjWd0pToZ1wxY7PGfRTnTxDcCmqHcWq3Y0c1x7vcpKPLJucOSx FTpe+P3z7xV0c64fuE2TbT/dIq+X4MsXJdKxuODUbfL644wIbp7d+mOG6Hn1vWbk9Qeg De9NnK7qI/ZsOyMl1hpAzBfgzAsuGZYus6QtreaQblA58BiuwoFrUqarEsoVadDG4znT 83r9oTFZJbFB5uB+1+5wUXLod2NwPPbfpaNq8a+daQrkbUvj9Rr40B1H4HhHsFGkd9Ed vnIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578952; x=1707183752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=P7sI4UQymuUi/LocesBTafTKv0uyaNB/4AJew4ji0arsfQFJG/MLSGp4VGlEX+Kw2v Tvz5SgjJhK3JNGYHulY2FbiS7J7Q/zI9kulFXIBBv7mAHfCIpl1AoNjffJ1rkhVHT4QP NouhcfFvgY7PdBF1b1Wk7rlib7KN3ako3FBQX8ewVlBLsDv8d1yWGoq+uqlnbq94oDmQ +sgds5Myb4PJ1XsDU7jAu9vM7BZixUa+qVFwNZd7uo87iwPHiC01Px7AqCpESEPu51Nt jX6+jqlyRl+2eLUk4tsIxfezBONEGyqftCtH1SUAdAHoshlYIIV63lBY7+6ZOjYLB3Yq 9/iA== X-Gm-Message-State: AOJu0YzdGYDXBOS0spo74rKeRwdYywz48igSm4v3im3R1A4e02lEQ/8K NbW9jhciNCF3Zg17Vc8WWMjRUjNT6kwUxXuWcCQIXYnBDNvAy9zLzBIPS2VfL1M= X-Google-Smtp-Source: AGHT+IH0Ej2IVjNTByv8ncwfD8JySniOOvwcT3ZZLF3iZpkW+vzHER0j5wJzpzxzw9guVH1KYf0/nw== X-Received: by 2002:a05:6214:2681:b0:681:98bd:644 with SMTP id gm1-20020a056214268100b0068198bd0644mr8490769qvb.119.1706578952162; Mon, 29 Jan 2024 17:42:32 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ow9-20020a0562143f8900b0068692ea038esm3974062qvb.91.2024.01.29.17.42.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:31 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 15/20] mm: zswap: function ordering: move entry sections out of LRU section Date: Mon, 29 Jan 2024 20:36:51 -0500 Message-ID: <20240130014208.565554-16-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 063B580022 X-Rspam-User: X-Stat-Signature: f1hwp77qnocjyj53uowq3t93q1dyhxtn X-Rspamd-Server: rspam01 X-HE-Tag: 1706578952-113219 X-HE-Meta: U2FsdGVkX1/u5qULEJ7dk/bm2bu0JhCmuBJkANmSUmmhP06bEUGH1CvZOKrCgOleBH5S75+kcPNN98qDyIDKFKjelKHS9p1R8bTj5kvYa1eXubf962PGWAZQugbFFoKzopPJ59ys7HDW294F+Lll/gTIaj7u0YIR5ZIfK+aVMzLziVIjQe9ljfF403eeMEwpP+7oUZPtd+pU0kM2HdB6tj0gdJwZFL130ut7NqaUM29pWpArTcXuuXlq68JVazUbDSNsje2ljOqq/2nDNuypoTplphVCDvVTXyUksqwBFzILQpmiPmHY+mStKBST6g8G/h/M+OF6eulJXzEnGmSYSvXMko7oySneMTAu2/V3R7vuXTMfYBgRWD2AwpvLQwHSxoDETnbcI4uoqmBHf9tTKRsHdH5JzTnHt4SSTyrPINmbQ3heO5UmNMqeEF72VUOLYad6H8uhkcGpCqfzqJed5A8BCz2GqF7wkQEaIumwT2Il/070WNYBSVOx3CVj+BAFmy7L4wDZZx5vvYeXUT3XHWv6pZULAl2sG2qSiDQS6dM3T3YO3ogfVR4pr8+66fWEVtjZ/2ssSXkCBGaMu5NJ5N1K+CNRfQQN/M5MWHssCqS4mzA/sh63zVx+6536dlSCuHBXveuPCB0h7mvBd8FqzRj8qNhfV6SS9qh8ClOE1pmF4ihlsOyA0D7jOXbqSwf4ZrrTPOL5jurVjjZk+5OwdHR+9f79+vExi9H3qgBYuiNvs8rMAWtiPQB/3yRhe6MF8oR1XfBVIfx8UKtWhIDSSziKOmO24elvCk7vGIc9gBhLhQV5KlBr7ve1yjqLMJXNTl6HOsM1lzvkEz/lrs002xNW+D0MPF8xb3DxrHpqKw5v3pAedbhKBeuJ/x+ghWJz4L11/hcUyIRIaK0e86kFSWs1tfbfyj6z2ROIXCwGrFtnYKpTFc+e9ZsDbhTuwSLrIiz0RpxbdXd46zMKh7R TyGAPDAc 8DHcE6SJiM8YdjfH/YUoSkItKdvsleNzp3puJvmMZ6k09UEnrvTqzBUMuE3XytC6a/qCr2lg2dytem7N8SuXYWnv4kC3xihWu1/YOnGgB2sJrrlRo5EHxJq+TYWc0HrvMhTGjUyuciUE28YCybih+YYUf2gp7ZoJvgufxntO8pAVOd3BSeHU4PMnyVa+hjC5jTCCoO49vxW3tTX7IILEix7YANz9RWpd67y0BI2TAf7RIypzICPjMxlMjd6w4mj99dKD9KYEiRbu3cM7AmLCC8jFFqcpHcI/DsfVPUK5XgOgD0HONeP+WWkxKO4Ugzz6Hmty/fokqJ8Mab6hUDkGKsHDzr7P4TR8jHkpVllF7hFU2KZzZD6dChvAi1Yg2HvpCO7K836hf444PArZmdaTiTsnxeuCcRSRAZ8YiqUlVC6rpJ/HQSpkHt9NYYcxEHH2131kA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This completes consolidation of the LRU section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 101 ++++++++++++++++++++++++++--------------------------- 1 file changed, 49 insertions(+), 52 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 511bfafc1456..756d4d575efe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -768,58 +768,6 @@ static inline int entry_to_nid(struct zswap_entry *entry) return page_to_nid(virt_to_page(entry)); } -void zswap_lruvec_state_init(struct lruvec *lruvec) -{ - atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); -} - -void zswap_folio_swapin(struct folio *folio) -{ - struct lruvec *lruvec; - - if (folio) { - lruvec = folio_lruvec(folio); - atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); - } -} - -void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) -{ - struct zswap_pool *pool; - - /* lock out zswap pools list modification */ - spin_lock(&zswap_pools_lock); - list_for_each_entry(pool, &zswap_pools, list) { - if (pool->next_shrink == memcg) - pool->next_shrink = mem_cgroup_iter(NULL, pool->next_shrink, NULL); - } - spin_unlock(&zswap_pools_lock); -} - -/********************************* -* zswap entry functions -**********************************/ -static struct kmem_cache *zswap_entry_cache; - -static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) -{ - struct zswap_entry *entry; - entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); - if (!entry) - return NULL; - entry->refcount = 1; - RB_CLEAR_NODE(&entry->rbnode); - return entry; -} - -static void zswap_entry_cache_free(struct zswap_entry *entry) -{ - kmem_cache_free(zswap_entry_cache, entry); -} - -/********************************* -* lru functions -**********************************/ static void zswap_lru_add(struct list_lru *list_lru, struct zswap_entry *entry) { atomic_long_t *nr_zswap_protected; @@ -872,6 +820,55 @@ static void zswap_lru_del(struct list_lru *list_lru, struct zswap_entry *entry) rcu_read_unlock(); } +void zswap_lruvec_state_init(struct lruvec *lruvec) +{ + atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); +} + +void zswap_folio_swapin(struct folio *folio) +{ + struct lruvec *lruvec; + + if (folio) { + lruvec = folio_lruvec(folio); + atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); + } +} + +void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) +{ + struct zswap_pool *pool; + + /* lock out zswap pools list modification */ + spin_lock(&zswap_pools_lock); + list_for_each_entry(pool, &zswap_pools, list) { + if (pool->next_shrink == memcg) + pool->next_shrink = mem_cgroup_iter(NULL, pool->next_shrink, NULL); + } + spin_unlock(&zswap_pools_lock); +} + +/********************************* +* zswap entry functions +**********************************/ +static struct kmem_cache *zswap_entry_cache; + +static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) +{ + struct zswap_entry *entry; + entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); + if (!entry) + return NULL; + entry->refcount = 1; + RB_CLEAR_NODE(&entry->rbnode); + return entry; +} + +static void zswap_entry_cache_free(struct zswap_entry *entry) +{ + kmem_cache_free(zswap_entry_cache, entry); +} + /********************************* * rbtree functions **********************************/