From patchwork Fri Dec 13 16:47:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 13907459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7683E7717F for ; Fri, 13 Dec 2024 16:48:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7BF3F6B0095; Fri, 13 Dec 2024 11:48:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 797156B0098; Fri, 13 Dec 2024 11:48:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65F2B6B0099; Fri, 13 Dec 2024 11:48:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4831B6B0095 for ; Fri, 13 Dec 2024 11:48:19 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A1522121073 for ; Fri, 13 Dec 2024 16:48:18 +0000 (UTC) X-FDA: 82890518088.05.5797FF8 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf30.hostedemail.com (Postfix) with ESMTP id 8B7C78000C for ; Fri, 13 Dec 2024 16:47:27 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DFyO49oQ; spf=pass (imf30.hostedemail.com: domain of 3T2VcZwUKCMwBsttsy66y3w.u64305CF-442Dsu2.69y@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3T2VcZwUKCMwBsttsy66y3w.u64305CF-442Dsu2.69y@flex--tabba.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734108484; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zAuvJ3/1BpungxKC2V7q88+gZ9+3WIZOJ/4nQKuXfFs=; b=amdCHBegbsklooKdhs4hEn8Cr8wDR75Q4iBnEFu3xYbJrLB7FypQIhSYpuf2n3wx0zNdvl 2FQbp3s3cuRAG4XHChaI+qAMQKUsV3zYEqe3N9S82HhaGuHHCxVbX/+ci16MFPddGPf44t c8vqVIz1qAhe7brD4SKbuB8WL6po2W8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734108484; a=rsa-sha256; cv=none; b=S3TNhH//Xq+FNkAUHSpGa3UiE9MvbXx7x/jdw9X0HvBEa1hRep/qhi9RP1pYC5ZvHix/x5 iAeSs87AM6rkKHmM+ad2eis+aHCbIi42yzuHUFV/9blxNsFXDiTGkpE7Uqsl09MHVAYOOx KiCZd2n5dm/19Gj2oGgLfXyrxC0VOmw= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DFyO49oQ; spf=pass (imf30.hostedemail.com: domain of 3T2VcZwUKCMwBsttsy66y3w.u64305CF-442Dsu2.69y@flex--tabba.bounces.google.com designates 209.85.128.74 as permitted sender) smtp.mailfrom=3T2VcZwUKCMwBsttsy66y3w.u64305CF-442Dsu2.69y@flex--tabba.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43624b081f3so17361335e9.1 for ; Fri, 13 Dec 2024 08:48:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1734108495; x=1734713295; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=zAuvJ3/1BpungxKC2V7q88+gZ9+3WIZOJ/4nQKuXfFs=; b=DFyO49oQAg8AJNLIrnIBJFmeXjd+sohsaaM6XNqhskeXVIjR/Kn80MqEvwXL7hFKjC t2zjaqUeWZBP6CYX3QCQH8hydYuBaBt0gP0fztfHis+mu1Y7fWOm2tDFIX9ET2VW9/vu IKSkaeQQXgzxxBddWdx2lhZtTSLlpcUEbASEobmGtAk27+A0OBI76OF1ZjtfA+kFN/hW ahKppl1Xdx8GRAVdxY8q/W+D4ZCyjYVqNrjV2M1YJwEiHN4p5o0YFk4qrsE7YInP9Sss Pk/egiXAHvFW9A11SRmrMuFQUfogCpRDtxk+bISsynooWn0rXBWdmt4f1qhqDGEf493q 4Meg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734108495; x=1734713295; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zAuvJ3/1BpungxKC2V7q88+gZ9+3WIZOJ/4nQKuXfFs=; b=ksazMrdmbLw1xXo+zkkWuzXa+2qMMrB9FG9JPdZdpA3LPPkDwQ8ztsgre3mI5cbguj U+omSwpgpfhuSLTZFksIithNrWvsHoOptXL6clj9qjUZ/R+uXDWld/aEexZdRWShstx3 rjZVfkbVKi9TI/Gj43Q0N+Q4oPMxCHP/5SRffb0OmMHSgMj287DPmFa4jB87QLBEfcpo lj/TrL4lqYpLWllAdj97INQywFXSF30Pe+VmYZUgEy+HbB4Aee/sFZTL6u4N3MloDYX6 dwtwfZCLfWeuELYpPrJOn2DBBtvpc+KvuMd16fXc06AmSJmPeAhnBIzjFLmqCiPXrlMU OtZw== X-Forwarded-Encrypted: i=1; AJvYcCVzTfPVG3PEYjXFdbkgfNyxPtTEFXPCiHph/k+cjj4DnJzsqtM0nQm0rUBtfoGjF58ZOyW05xWTrQ==@kvack.org X-Gm-Message-State: AOJu0Yx0+EpmvjCP6hFC6nRfLvnIM3X4s690cxjNPoNhEQw0CnEqPHdr 8A9s4XG3p0A7A/5HUY/H5m/YnDU6O8H5/aiawfjNwfcKsynLml/lfk8tUXpo11lKL7lh+xQSgQ= = X-Google-Smtp-Source: AGHT+IHs8ZHuR71xrnh0Wig8GOEjO8jW22DFD8cUeagZ+0AiDaeSDaoT2cCNj9eWJgQpPRurrM0qMZgcmQ== X-Received: from wmbd13.prod.google.com ([2002:a05:600c:58cd:b0:434:9dec:7cc5]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:348f:b0:434:a802:43d with SMTP id 5b1f17b1804b1-4362aa9f682mr28943725e9.27.1734108495459; Fri, 13 Dec 2024 08:48:15 -0800 (PST) Date: Fri, 13 Dec 2024 16:47:57 +0000 In-Reply-To: <20241213164811.2006197-1-tabba@google.com> Mime-Version: 1.0 References: <20241213164811.2006197-1-tabba@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241213164811.2006197-2-tabba@google.com> Subject: [RFC PATCH v4 01/14] mm: Consolidate freeing of typed folios on final folio_put() From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, yu.c.zhang@linux.intel.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, tabba@google.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 8B7C78000C X-Stat-Signature: uppqhdsmtpdmc59jxbwa1qmckxbxg73d X-Rspam-User: X-HE-Tag: 1734108447-717745 X-HE-Meta: U2FsdGVkX1+3UMWyjLgqC5Ot+Rrdgw/i05beXAybuwYoW+p6t1nyuymFi1PbCbEHwrnNl3na2suFjeH4v7BLRmsBKe5ZxGKZlOVAjBqipEAjFN5ii4M7+vj5Fdh78BxyEyQSTIcIpzmGoEAyKRwiWHr6lmcs5k/1cl/m3MxwS1xIdiwBNGr3gxawsScBYq6uKX8fzFqN2+95RpMWMhO3laimwfyaB/1509nx10bL8c5aez7M1gdUeOBSFlSv6BiiuXif+Bc3OusLiqdKsOBrfzCwTS4VwzYhCXpBh91IZdyNSKCILU/0L+UPstJg92wlXADnQRaa2fPorXMIYxXdcS3AJC22eEEZ38gA5MhMjhWSnp0xSRnM6M9GmU6mhE1KEhg6BCfMZgBpwOnt5ElaqnxJP5F1pfGn2xNC+Nt5tW3+epHnbx2wS+nDNmpdz+nh1bnkK8811UvRZw1W+gkIGTxRvRxWJJ90gxf5eJt+HBS3H67bv9+ZNUbDSdZVEMo2nCYXRlTNKRUSmnD5UUtcMWpzTMJYpE1dGsIe8u7xhTUp+R1PrrP2myfnfoSMhl2xDGwfDgTo9bIA7zJTIwtM50eezSZazt8hhFPSmzHrOaFJgwa9awYxatNAeYgVKrVyXCrxhQZqThn8jll0baRZFWpqia+ST44+CDMsP7JWrEE0mb31O8iU1ARqRp2TswwcdVBcrGQkczB2dKWEYxmmfPMLUXlHdT1Snrl5WOF/uZEViNGyUw7MYtiNrhSR9goGta8z9qsTAKH371MxftUYMajHudUdNqT0M3qEo96sJcbLQCyZ0DNuldFGu9AyHCDXFtDsDC98zwCVebDJr2lgI82W0M/WtbvEgS+pwqkAjhcqXaxvWOtv8dPGwIATdyF0d+rxt+bfcr6L/Ah++UzVBHMz9YQF+cFymcdm3+V0ItybdgGYEsilEMf3JN9fAFDbpWEhs4cy3kWD8QJkC97 nIcTcJyc Chd4vivk/DHuCZ9IcxmYDRpqGTqx5lmoNbyLRhtUWj+2neWGUSZdiWlrWYOrhd0DdOQSvi/O+DmZ6E+qbWLtNefea1JD8STh91Bq7+4LKk7LCHyT/t+2r5z2QKoldJWZDbGVD/FiXixdoOKKukwsqwxVxIjfKLcB+/cQWw8sfzPCco4hlplUXNgYlef6Z3Wgy08FUlyqvQe49qg6I1Ua4Z/6y+Jeg4TUcPJ2vOqma+I+N/B2gaxUWFnaKAiHZNxrJACWPoUaqIMf5ghmaEVzVbSkpnRkW54cY26/D3MdwjYIq8YNm7HbsgjXgCLqc7L3bjDf1WNQ6hjOrDqHOMbxy2Fh8Mllj/tVnMpp6XmxJmS33BR3wyrHPBtDncw0UWkkN3L8xZvF1Jq2wCDh+eQ6EzCv4HPyH8pAabQg1lvBYrAJsrKJfwiuZc+ER6h2py0HKRW1Fsv3N2hxbMTteQxwbQavPE82Qh93EFG9E3M+MtJ1oXwFEujILnaFxVfXxShWkiyZpWLJdnPS0PZspNVQ1innqg042PdWGm7qXms9ZONspq5oT1K4PrBrYXSCM/BnkhyDP4GpoLe3bbFoWBn0WC7YWoYWq9h5XBkiEKHiOJNV7JnQDKC7/5q6oHWZ3SPkxMENZ9hUNu6ft3ElXE5ze8nd6UKZ6jIGg0lKVwBIUn9RY8vU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.020884, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some folio types, such as hugetlb, handle freeing their own folios. Moreover, guest_memfd will require being notified once a folio's reference count reaches 0 to facilitate shared to private folio conversion, without the folio actually being freed at that point. As a first step towards that, this patch consolidates freeing folios that have a type. The first user is hugetlb folios. Later in this patch series, guest_memfd will become the second user of this. Suggested-by: David Hildenbrand Signed-off-by: Fuad Tabba --- include/linux/page-flags.h | 15 +++++++++++++++ mm/swap.c | 24 +++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cf46ac720802..aca57802d7c7 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -970,6 +970,21 @@ static inline bool page_has_type(const struct page *page) return page_mapcount_is_type(data_race(page->page_type)); } +static inline int page_get_type(const struct page *page) +{ + return page->page_type >> 24; +} + +static inline bool folio_has_type(const struct folio *folio) +{ + return page_has_type(&folio->page); +} + +static inline int folio_get_type(const struct folio *folio) +{ + return page_get_type(&folio->page); +} + #define FOLIO_TYPE_OPS(lname, fname) \ static __always_inline bool folio_test_##fname(const struct folio *folio) \ { \ diff --git a/mm/swap.c b/mm/swap.c index 10decd9dffa1..6f01b56bce13 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,6 +94,20 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } +static void free_typed_folio(struct folio *folio) +{ + switch (folio_get_type(folio)) { + case PGTY_hugetlb: + free_huge_folio(folio); + return; + case PGTY_offline: + /* Nothing to do, it's offline. */ + return; + default: + WARN_ON_ONCE(1); + } +} + void __folio_put(struct folio *folio) { if (unlikely(folio_is_zone_device(folio))) { @@ -101,8 +115,8 @@ void __folio_put(struct folio *folio) return; } - if (folio_test_hugetlb(folio)) { - free_huge_folio(folio); + if (unlikely(folio_has_type(folio))) { + free_typed_folio(folio); return; } @@ -934,13 +948,13 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) if (!folio_ref_sub_and_test(folio, nr_refs)) continue; - /* hugetlb has its own memcg */ - if (folio_test_hugetlb(folio)) { + if (unlikely(folio_has_type(folio))) { + /* typed folios have their own memcg, if any */ if (lruvec) { unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - free_huge_folio(folio); + free_typed_folio(folio); continue; } folio_unqueue_deferred_split(folio);