From patchwork Tue Jan 30 01:36:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2165C47DB3 for ; Tue, 30 Jan 2024 01:42:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628876B00B3; Mon, 29 Jan 2024 20:42:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D6C26B00B4; Mon, 29 Jan 2024 20:42:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 402456B00B5; Mon, 29 Jan 2024 20:42:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2C9836B00B3 for ; Mon, 29 Jan 2024 20:42:30 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id ED0ADC0239 for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) X-FDA: 81734277618.19.F44AA39 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf07.hostedemail.com (Postfix) with ESMTP id 1907940017 for ; Tue, 30 Jan 2024 01:42:27 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=KrNTCHdW; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578948; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=hjTer1qfWXQp60OBywQNRGWmwgaqCVB++zywASWRCY5lYyFB0627ltHtNzbv/jpoZpmKna lm9OHkUtOcAS0kZ5C4g+3PvB5rLThNEz36EaFrL7Yu8y5wVKSsnnxUb6ZpMObSJe0nVCeK sXTtO3wqEEKjOfDPpS7mcZnjjMvyTmg= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=KrNTCHdW; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578948; a=rsa-sha256; cv=none; b=Ex1Rl8SwZ1jHUktniRGFWfRLjuQBnrG70J1V0UX3aMVD+RUeZweYzUqMVys6IeD+lGVFP3 MzmgfcHIqgmqYWXg8qHBR4eGShcG/mGzsoNr+5+KMK2fHuS+4iDLi9czuHEEz3JddIsMzP Hc/BW2TPmCv1/xfZ4vB7VOC+eZ9HGQ4= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-783da26489aso280796185a.3 for ; Mon, 29 Jan 2024 17:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578947; x=1707183747; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=KrNTCHdWI5yNYFfblX+EPD9Wt4htph+YbNYGgvbOn7f/HhJvBl/NbkPHZDwOONe5n4 cYPkXhLNAOntUgw7UkgZ4NztCBUfICrfILjmuQpQBl9sJyEhYejpdTpcexja7FyLuVTE SLSwCordhVxqwhjGK1kxPcRIDHVTPr2iJaZOOi7XaExAAnWWFuiH/rmVEKcV5qIRcFcS hOA3U4De7R0Gbxa/WMfM4ULp5U0Jhg7VbABdx+Ia50lFlVR7HrY43ofAYj605Qw9ifGV V/sw10JcwJHRfRUtYbOCFnllHH76WK3vleaVxv2u21zQTkQU4weZOFi7yets71uS2xqV LRgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578947; x=1707183747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=dlbAhqnSp2jRpx7bVuV3jO/LiMbwCQvpzWLpUXtIXCGPZ81L14BBGjXGmhPtW8byGy aQcTgDS/bD5scVu+9ARREy9u/fvn8wIuOS+dyu0OPsVXrMCPBe3iWopwH4rvAEOu4lsb nYmzxRAxpntf84Ij5kV173YJWUivQoBS9Gvvw/6izYmb+zfGwKnGM4Tb/AhPPVtsS6g1 5DFRym8P5S8QP4xYcPP2ogViHiBxorAaPpS/6/LZu1ugwCBKb0vN/zLlcR3Oe2xmGRve nMVxpVk6sZNwaEU5YaqD5Re2wyISAUMs+VW5NCKwdADL4cMqRYRWp/vyNEjj4ujPBhgU YhDg== X-Gm-Message-State: AOJu0YzFz1O+SO/zHcxB9bs2lAA4s0h+N4PQnECMiCaJYT4+CGqUhc3U RJaYMuZE0/JC+WvMsVdufGItyorv2QZbrjfYDEi1cxdb2MgmVqk+7B01hAgynno= X-Google-Smtp-Source: AGHT+IGbQQQZq+kfQrrdhSmxdR44XoO3pzuQK7Yh8fRiEenD92oHac9H3b2LTsWjU2hqhbKGsaWTag== X-Received: by 2002:a05:6214:cab:b0:686:9441:ac30 with SMTP id s11-20020a0562140cab00b006869441ac30mr8699046qvs.22.1706578947264; Mon, 29 Jan 2024 17:42:27 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id lr9-20020a0562145bc900b006819bb31533sm3993392qvb.99.2024.01.29.17.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:26 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/20] mm: zswap: function ordering: pool refcounting Date: Mon, 29 Jan 2024 20:36:47 -0500 Message-ID: <20240130014208.565554-12-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1907940017 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: hydy48a4oiyxsi9s4pabruh88maqd5fg X-HE-Tag: 1706578947-960855 X-HE-Meta: U2FsdGVkX1+vzuYM6I2ir6RBt18bwqlFatcr7bNb5YSOWCxCS8gJ98hoQ9nrAuhN1Ta5WQvQECZr3rhL7hfV1Oa888Fj4usfmj1QS/0Y9GWHa/hfVBh8mDal5lsp0v1umsPtxOCah5QjcvAmB+taz6TTST+heE+rMtJQYK1BA5AGh/DqAOzMSyvstzC7AG3AhTnbEuEyNeOB+UH14oZHuJUlMGZ6HuI0OGkl9Ii2/DY9mJl+l8Idqr/SKQTCGKsIIbz5VEgOcN/QV0dHnffMYiMqsD49BkeoRY0oCz09LvjLCcQRtkKaO+wCT+aIIw/pTGWziMDNFMWEywYIHzcQVWlnTgivtjgrW9N6W9QtPRTuc+c488bMq62NX5X7by8q3enM2O00+puqKUCBps0XYJVeJVRiRwLeV1lPv+8QuBSuFov3BvQk1iA2fLAMv3EOAuw0o2o18I3AOSi22hVWd12DoOiKLjqu1wT+SiQSUywCqnkuGF5l2vyRUCA9YKxoBQ13yHS75E8iSLUksKnOBi0xrDFbcdGbbF8AqyqjiCjHmpp8jCkoqCW7SGYhG0kYvKLG/tlY7C1vaR1nfntIntAf8UOD7y1jNr5fMzsKQ241G14qBGmyC4eH6lCRXyLB0E6nl6pIbqGQPUY6qNGJJ+/Th9dX/7Mld5U7iWXUVL1QSbX1HBvyPrs6gzWyZQPQUk3//nTAeSlZMonCR6Sbtt4RNUUJSZJe0BGEJZUR/5Uccd3+ZvLmEQKegmkWymtFbURgW0YE/AWhvj+PiGhehFSJmVksoHN0oaCf+31xSeqqxx+QNWPgNCfevYwb3kvZhfoVW3GzYkucMtDD3ppmd2oxwGPgSOqmL7KQAtaK+slw4kQxL1H+V0wpxb/4bdMHvWyxI6m9w4ac8OiM/H3uVRvV7mPqKT4u+MKiLSPMJIfNukGc3oKuxHMoX5mz0i3cECownjbinUMwEkmd/YO L3p6KhtD c9yqpnOr6MXfoTTcHJpPkLNIRgZwiJu/ynfYPAKdUQM0RuI/mvwQLGeQi/2UuIf5Ze21gosXoknESibDD7O9Lo6fwyexkpEgGc2PT5Bvis0TH1lDxcOvTggMj585/ewYF+R3cLQtAaO4gx5F0DPVLAUr1wumYyB/nLWjhvoIGnIkm/pfoo/BBU7vfX6yN1BwH+6oJbo94hZk6PFk+ZCWc0l4pR0zBf3jsywOk+9DJzraTn/VYNzhXDu0nsp68vL0tMgW81skDyf0tM4Kj7LBe3GxKGHF5p7uiWe9MeTWnHFNKBhFQsraV6s1c4UBoJsVbczvWapG05PmDl4YZccsvigRRsbkVsQkI4MBpHFKexsZtyuT8BQwnBYHr+UiPkqLQWqc7gZOZwWzV+nTpYGWPqzCtK2aHwO4Iw7FmwxLz2990wLb8AV68nwI+pA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move pool refcounting functions into the pool section. First the destroy functions, then the get and put which uses them. __zswap_pool_empty() has an upward reference to the global zswap_pools, to sanity check it's not the currently active pool that's being freed. That gets the forward decl for zswap_pool_cuyrrent(). This puts the get and put function above all callers, so kill the forward decls as well. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 94 +++++++++++++++++++++++++++--------------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 805d9a35f633..33775f2224b7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -278,8 +278,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) static int zswap_writeback_entry(struct zswap_entry *entry, swp_entry_t swpentry); -static int zswap_pool_get(struct zswap_pool *pool); -static void zswap_pool_put(struct zswap_pool *pool); static bool zswap_is_full(void) { @@ -472,6 +470,53 @@ static void zswap_pool_destroy(struct zswap_pool *pool) kfree(pool); } +static void __zswap_pool_release(struct work_struct *work) +{ + struct zswap_pool *pool = container_of(work, typeof(*pool), + release_work); + + synchronize_rcu(); + + /* nobody should have been able to get a kref... */ + WARN_ON(kref_get_unless_zero(&pool->kref)); + + /* pool is now off zswap_pools list and has no references. */ + zswap_pool_destroy(pool); +} + +static struct zswap_pool *zswap_pool_current(void); + +static void __zswap_pool_empty(struct kref *kref) +{ + struct zswap_pool *pool; + + pool = container_of(kref, typeof(*pool), kref); + + spin_lock(&zswap_pools_lock); + + WARN_ON(pool == zswap_pool_current()); + + list_del_rcu(&pool->list); + + INIT_WORK(&pool->release_work, __zswap_pool_release); + schedule_work(&pool->release_work); + + spin_unlock(&zswap_pools_lock); +} + +static int __must_check zswap_pool_get(struct zswap_pool *pool) +{ + if (!pool) + return 0; + + return kref_get_unless_zero(&pool->kref); +} + +static void zswap_pool_put(struct zswap_pool *pool) +{ + kref_put(&pool->kref, __zswap_pool_empty); +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1122,51 +1167,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static int __must_check zswap_pool_get(struct zswap_pool *pool) -{ - if (!pool) - return 0; - - return kref_get_unless_zero(&pool->kref); -} - -static void __zswap_pool_release(struct work_struct *work) -{ - struct zswap_pool *pool = container_of(work, typeof(*pool), - release_work); - - synchronize_rcu(); - - /* nobody should have been able to get a kref... */ - WARN_ON(kref_get_unless_zero(&pool->kref)); - - /* pool is now off zswap_pools list and has no references. */ - zswap_pool_destroy(pool); -} - -static void __zswap_pool_empty(struct kref *kref) -{ - struct zswap_pool *pool; - - pool = container_of(kref, typeof(*pool), kref); - - spin_lock(&zswap_pools_lock); - - WARN_ON(pool == zswap_pool_current()); - - list_del_rcu(&pool->list); - - INIT_WORK(&pool->release_work, __zswap_pool_release); - schedule_work(&pool->release_work); - - spin_unlock(&zswap_pools_lock); -} - -static void zswap_pool_put(struct zswap_pool *pool) -{ - kref_put(&pool->kref, __zswap_pool_empty); -} - /********************************* * param callbacks **********************************/