From patchwork Wed Mar 12 17:58:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 14013727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8A11C2BA1B for ; Wed, 12 Mar 2025 17:58:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8376B280003; Wed, 12 Mar 2025 13:58:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BE6B280001; Wed, 12 Mar 2025 13:58:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 639A6280003; Wed, 12 Mar 2025 13:58:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 43FD3280001 for ; Wed, 12 Mar 2025 13:58:30 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BE5681414ED for ; Wed, 12 Mar 2025 17:58:30 +0000 (UTC) X-FDA: 83213658780.29.690BDC5 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf07.hostedemail.com (Postfix) with ESMTP id E795F4000A for ; Wed, 12 Mar 2025 17:58:28 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=OXNtIV61; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3Q8vRZwUKCGYXEFFEKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3Q8vRZwUKCGYXEFFEKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--tabba.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741802309; a=rsa-sha256; cv=none; b=uVfd+nB1rc6R4Kl5zRvm7Qz8VkmAjwCfIjJRF/n0hxP4rkUnt7IKHu8+iwXEYUd7D+Xa+5 KqIsL82L8g+p5/fOvlJsb6sPx4S5MQC++UokDtJbe4dJ7+ezg7vgQnKqm+FGmd/m99E4N9 YCGiw8djwYCz6WgB/iVmZ8oAPMLQqvc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=OXNtIV61; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3Q8vRZwUKCGYXEFFEKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3Q8vRZwUKCGYXEFFEKSSKPI.GSQPMRYb-QQOZEGO.SVK@flex--tabba.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741802309; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a0XpxYSrLEkx+yIzSfmPSimPPvYP4gD4dCRRx7Mg8GE=; b=BfCfarisf9bCGQoDFyoBulaA9tVahGvvJdkdzEAUmLtIDx+BBchpjJ7/8Rm9Brgo/CgEYQ E9lzPGCNl++pyERGtU/rMROpuy+rS3dtKcWbfq+wf8E/HrlwTmxOo5B/8abnhifg3APgqZ s0sVGUdJAfYvKZlVVJUlC2gcUWeZUik= Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43cf172ff63so445545e9.3 for ; Wed, 12 Mar 2025 10:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1741802307; x=1742407107; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=a0XpxYSrLEkx+yIzSfmPSimPPvYP4gD4dCRRx7Mg8GE=; b=OXNtIV61/ihYDcObkAqzYZPttP6zpiIIve8hjOvXDfkFdRGYv9xP8+0LXpq9T0mv2h p4FIrSN6guWj3iPvAXPHu+zu9qssU2h78XNl/I6tAcFG3uG3oezucZSXW5nUDzZcpxHB ZRkq1O1sIPJTcjtLNL8Bm+sK8rWzQMp1Doxj1THIu1R0QDZCah1ABekiX7lKnmhhkKLg loZKTUJrKtqkAyWsCFDg0YLhfgrlezequv/uJ+LuUEzQXqARlBsuutsyF4i8CW3AYW/z f8Skak1zX+V8P0EJq4aGxzs9fg6BdshDPKfrQAMO4mNvZFHV1EEjwyp0JMzG8lGJEZAq NYIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741802307; x=1742407107; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=a0XpxYSrLEkx+yIzSfmPSimPPvYP4gD4dCRRx7Mg8GE=; b=YcfoZ+aYXdOVHvSvB6C+pCvLr0RBGdBCPubb4K0p94Vtdndn1kwpar/Zo7O755PmBY gHSjjymhwC71y2rc5zXVvPKUAS1cyGlb3tqQ09wo8jPVBln+a8+BNhEF1K3PZL6AR+lk psTkslcH8YAju3RUDwv6ksFh7h+A2pijN7wMgBHPU0641oO1hQEU0OnVHAbdSK6OBykP R1PB3wAOtyzIcQgbizd3/q7+5IS9W9EgLXPKc1A5yIh3CrWIlJaWTAfGMZoVkcjow/3B Uhj5fREdoz9Bixuyhd2WO5rriw0vOQP9mSbw7pT2tSKnnAOque2oUM2WVrWR0aMsAqIq wJqw== X-Forwarded-Encrypted: i=1; AJvYcCX8sfwsxFF8R9x/kpB+5JGy8ThzPfjD6V3zVhT15QeNU3hj2plTSp5NUjRnk9jqy3VmPOE0O919eQ==@kvack.org X-Gm-Message-State: AOJu0YzTEW7uYc54YQ5RwVjcDm/N3wv/DHFD2s0exKywCQAiARZZ7Fcf gBT68lHfJUgiyKkcN9COJx8qFbvLsZ/sZeY4EjiUVmLZMjPnvoNUlKnKF3BAeAwkR9eId/NFZA= = X-Google-Smtp-Source: AGHT+IE5UDnnh3yo17yLxT5VX1BO5KQAMs8ianrYZo0AfgH0k/k4UkL9Aw8NmMGZsi98ECarhiOo1EOQ4Q== X-Received: from wmbfp13.prod.google.com ([2002:a05:600c:698d:b0:43c:fb36:c419]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3512:b0:439:8c80:6af4 with SMTP id 5b1f17b1804b1-43d168d4625mr1074115e9.19.1741802307671; Wed, 12 Mar 2025 10:58:27 -0700 (PDT) Date: Wed, 12 Mar 2025 17:58:14 +0000 In-Reply-To: <20250312175824.1809636-1-tabba@google.com> Mime-Version: 1.0 References: <20250312175824.1809636-1-tabba@google.com> X-Mailer: git-send-email 2.49.0.rc0.332.g42c0ae87b1-goog Message-ID: <20250312175824.1809636-2-tabba@google.com> Subject: [PATCH v6 01/10] mm: Consolidate freeing of typed folios on final folio_put() From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, tabba@google.com X-Rspam-User: X-Rspamd-Queue-Id: E795F4000A X-Rspamd-Server: rspam05 X-Stat-Signature: b9f3mzff7u8j4rbxfwip34hth9nhyeba X-HE-Tag: 1741802308-935888 X-HE-Meta: U2FsdGVkX1+4dGVo4OoFfSjmRVMUNgQh2uiOpGemxpPHL9P5WLnlI3zGPknwe428YKGqYLxhAz1yKOIfmLHrP3cND03LNPb7sPSgpuwM9ys2/I3REiYRAhy02D+T0sAepOBKiQI8KfBIdDyNKsyTp/OjMfvRqhYLFYN3Mi/A0v2g+fsGbLX5j6cBbcWxnCkSQrg++A+Us8BT1sjfcCMTgV59l8Tp4ByUCRFWgYDlINGUjqhYEXO5u1PEj0WhXmrAvoY+VNPPo5RmhS8Sf3obpld4Ygn9H0eo8QMGYvERKufirV9RVIdG2Z7qyPS38xGEW40b0ynLHIv5Rjcg/6Sq2SYv9oGY+dvKgeOMOJtEPuhaYgGhJmOXt4GHvFMwfQ5rNXeTuPk+xhXL9HRWOv5c0joXn2YmyY7b82x2Z4NdFolyTo15RXBqVD0DCArBY4c2obzMCTgMrSzGaDjXXxuMvcTotAWFQDX66oCL0LUEu3n+4rhjOJ1lv3dujtpFnGHDyJuwl4erdCloGcz76gCrftjqvuL0khvp+QxFhlY/WOCq2HOkHXzEfkjkuclid3CdM8XCJ6TbV3PPpg7vyrljIOFstRlcGgKcRHq+LJGOzHd1j42Nl183RuXPxgNhg5i0xu/AvS1nFy1g0IL3IFmGCTvZeeR0ov8UuH7nOxV87C6MqDytChmuwkghpp1TXUZ6AhZJncww2lav2IKnM/p70JuSDmTXrBzT/VeMMXSsAwcixsTpghNdhKBGvXChDTb3GinO3Roqr+HsIsy9ze0g5rIA2TObCc1txiD5srcmnuZeRxEw0PkKeI5977OuPcvOqB7Doq3QZipo3rAy8kbGCF3sNH1G8obolRmoZcNGYNFNG8414gKNh29rNYvhXqVBcuIvOG5kTLAOqA+5JBskoRno752g/lWEk8FfebKOkkx+k3y/04FrO9rlCe0zuKnJOK9Jo8JFhtt2aK0bd+t 86a8IFSE QwiM28wENM4+HuxeqsHTrYUM7aaJT8maCdknk8N1bal3lddxlQ3bvIhydiNr6iuHaikCO/8hR731iZ3wPQnbG+BTIQovI0jALoO8u3FACIBWb4wCrk3v73Al0XS3+VqNdexMa6ueBTnXgGJSLjFUPHJm3WA9cirA37Hk/gxKzUPdB3f36M8ucNAerWrwulCZlKlwA2e9YNEPMxPZPNfQwB6D4dtyd42kwJ0UZ/DMRmzlxvNJy43BU1c2mEESleXuEM1dJxj3CEsQvY08IZnz8xdt+aE5t/p++d02avqmc7f3ur4UgG5+nvR3YI5Os8KUMSEH/PCJf3sBdsHpoi4rlDIe+nqBrY0JRcSWP/wp23Rok+2qskWhZ5cxKjNRjqfkUy62y+41yOJFLJWt20UoJvpCSrnXLNMoUeG2ZHRAbmEGrbJLm+tuh/jUt3ZRRYGhze0x+5P3UljwBNnOYtrbrVgNIZdAGycf4phpOQTHOvnfd1DHNl8vCB9fjGR5RkGh9gJ11UmhPHndlbvGy7u5hrXSgAePIvnMH0DPuO6V9ag4wAXR3qfyPyDjzJKgiI0l1tBwPtrkAOSGRk0VzAy3vWV2+1Uu4hljNYthlURW/9tG3obT8pfklkL9gteUTe0E360nfGz0j0b6AWG3wAOhBJ0LewHvVuYY6JRjd3ZtbRFXWti4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some folio types, such as hugetlb, handle freeing their own folios. Moreover, guest_memfd will require being notified once a folio's reference count reaches 0 to facilitate shared to private folio conversion, without the folio actually being freed at that point. As a first step towards that, this patch consolidates freeing folios that have a type. The first user is hugetlb folios. Later in this patch series, guest_memfd will become the second user of this. Suggested-by: David Hildenbrand Acked-by: Vlastimil Babka Acked-by: David Hildenbrand Signed-off-by: Fuad Tabba --- include/linux/page-flags.h | 15 +++++++++++++++ mm/swap.c | 23 ++++++++++++++++++----- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 36d283552f80..6dc2494bd002 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -953,6 +953,21 @@ static inline bool page_has_type(const struct page *page) return page_mapcount_is_type(data_race(page->page_type)); } +static inline int page_get_type(const struct page *page) +{ + return page->page_type >> 24; +} + +static inline bool folio_has_type(const struct folio *folio) +{ + return page_has_type(&folio->page); +} + +static inline int folio_get_type(const struct folio *folio) +{ + return page_get_type(&folio->page); +} + #define FOLIO_TYPE_OPS(lname, fname) \ static __always_inline bool folio_test_##fname(const struct folio *folio) \ { \ diff --git a/mm/swap.c b/mm/swap.c index fc8281ef4241..47bc1bb919cc 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,6 +94,19 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } +static void free_typed_folio(struct folio *folio) +{ + switch (folio_get_type(folio)) { +#ifdef CONFIG_HUGETLBFS + case PGTY_hugetlb: + free_huge_folio(folio); + return; +#endif + default: + WARN_ON_ONCE(1); + } +} + void __folio_put(struct folio *folio) { if (unlikely(folio_is_zone_device(folio))) { @@ -101,8 +114,8 @@ void __folio_put(struct folio *folio) return; } - if (folio_test_hugetlb(folio)) { - free_huge_folio(folio); + if (unlikely(folio_has_type(folio))) { + free_typed_folio(folio); return; } @@ -966,13 +979,13 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) if (!folio_ref_sub_and_test(folio, nr_refs)) continue; - /* hugetlb has its own memcg */ - if (folio_test_hugetlb(folio)) { + if (unlikely(folio_has_type(folio))) { + /* typed folios have their own memcg, if any */ if (lruvec) { unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - free_huge_folio(folio); + free_typed_folio(folio); continue; } folio_unqueue_deferred_split(folio);