From patchwork Thu Jun 22 20:57:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vishal Moola X-Patchwork-Id: 13289601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 643AEC001DC for ; Thu, 22 Jun 2023 20:58:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E28738D0007; Thu, 22 Jun 2023 16:58:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB1F08D0001; Thu, 22 Jun 2023 16:58:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2AD98D0007; Thu, 22 Jun 2023 16:58:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B607C8D0001 for ; Thu, 22 Jun 2023 16:58:07 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8617DC0269 for ; Thu, 22 Jun 2023 20:58:07 +0000 (UTC) X-FDA: 80931596214.01.19AC616 Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) by imf26.hostedemail.com (Postfix) with ESMTP id A9DCD140018 for ; Thu, 22 Jun 2023 20:58:05 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=ZxYyU7k2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687467485; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HtFJ7gYDDt6nFtJ3t4rf4iEtrxS/aE7Nt5tBsex6r9A=; b=t7yX2f4jncBYedhHf7/G4jWlyZzYvrDJt2Zgf35zjhkDwnuEnoFDlBEmIsrvhbxUY9IBGl UiXjlgXiU3ZHQaeZJ1iMG1oB89RYkDrx3HVz90mJZDzmzNLMfCYKtGzXOEdGf1YVsZy316 J1b+pshBf/xYK35YqSDl6GGfY2GREM4= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=ZxYyU7k2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687467485; a=rsa-sha256; cv=none; b=4M33lJP/7+gCYgIfZRg5e1u0iNasMXOvORtcS8tUK1IUii51hGu/uEABkk9Mc/ByTRWmXG j0ylBmJRtxVEyZ1iD1bVDaguMZ3QfZZQTdeco53SR4yzF6KUnsH9OEQUHTXRc0Q19opH1r mtxT6ZTHP9lZLpK1Qn4WyqzDCdVqFpQ= Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-bf5f41a87ceso4111578276.1 for ; Thu, 22 Jun 2023 13:58:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687467484; x=1690059484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HtFJ7gYDDt6nFtJ3t4rf4iEtrxS/aE7Nt5tBsex6r9A=; b=ZxYyU7k2vGalXaPVFbsAoA3LfcjeLDhIOlxkMUoxB7wD5DzRvEzQGJZMJxTOPdFAYE GF8H8J3v3pIjRre3uBFGoQhrehz40XLVdSwrbVpckO4S1lcUeENHrNTK6kfLjGoHYzvr nhXtSSc9VaEE/j5fCXHdi35T0qtxrEHQkX2kvNCXyGYBu1DiUkeGGXiy91rFgYuyJidG yhM8UfJ79m5utnGApQW/CB4uav8Cyl/ViwcY0GfzkowvqJFo3jCDUp7w3xYorcQhuQVB Ccqw/rMENg5s9Xp75lULxIZiDzj7mDSoiTj8XYlqSon7icuirfeluYeOq+6o64D11tV3 SMqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687467484; x=1690059484; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HtFJ7gYDDt6nFtJ3t4rf4iEtrxS/aE7Nt5tBsex6r9A=; b=W8OM6hIECoRnshLMHpNlHxkZE1UT95UtZfeIzMmT6rSRa8LHGW5nyLCzm/S1S7XdFY j0zOZ3AEeJsNtLi2Hk10uiLQ7/p/457CgRYmdLHQI/IG+eDMl6cqN782EM9aAowumIzu 6GQ5VfZfHrF4sfZgcK0nd/wM4msfu8LsYu66cGEBc9xQfR7jdU7NhWrU3f+O9PQAiUum iKh2m6A6MJj+c3HbPWMmvz/8o0QoSMizRl6X33ZrIVfPOpZ/IuQiw+xBmJebIm0InhK0 pYB2B8sn8CN6+cg+jquGieFMvN1vPJMo+L4Ayav1zSLUYC8tKZQqHifDQJAFMJ+Iu46B h2mQ== X-Gm-Message-State: AC+VfDw2jfwijfpGP78dC6YillPg9CnotQEm/iwTXGm9RcoVdbE6wtjJ PnWAcb72kae2pe8uOx4wnWA= X-Google-Smtp-Source: ACHHUZ5hnxQ163/uFnii7yUzhLCvlKkxDndNuqoPVJ1DEXuk1sFzA/lJHmZOgLxy0t1IuUHDUPN6qA== X-Received: by 2002:a25:df84:0:b0:bcc:f2cc:73e with SMTP id w126-20020a25df84000000b00bccf2cc073emr16743014ybg.22.1687467484442; Thu, 22 Jun 2023 13:58:04 -0700 (PDT) Received: from unknowna0e70b2ca394.attlocal.net ([2600:1700:2f7d:1800::36]) by smtp.googlemail.com with ESMTPSA id d18-20020a5b0c52000000b00bc501a1b062sm1684937ybr.42.2023.06.22.13.58.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Jun 2023 13:58:04 -0700 (PDT) From: "Vishal Moola (Oracle)" To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Hugh Dickins , "Vishal Moola (Oracle)" Subject: [PATCH v5 04/33] mm: add utility functions for ptdesc Date: Thu, 22 Jun 2023 13:57:16 -0700 Message-Id: <20230622205745.79707-5-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230622205745.79707-1-vishal.moola@gmail.com> References: <20230622205745.79707-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: z6d5krqcbjy5tcobwi44uena41mbshwk X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A9DCD140018 X-HE-Tag: 1687467485-746752 X-HE-Meta: U2FsdGVkX19AfAbIjlgth/361fR5ctoiUHI3KP8UepvhkIuHppsHAad0pC98DBlPF5Tz0bmsJpROEju+BCMRYPaho+GC6rJgL+NghjFpCmAYzYax2nbLPCHc5NUE89EryRt4ApQ24m/oh9m3vxe7vKGX+N3H8MOCXllWdmiuVUJuBGGIAZqUQ/g6seDxFZHSdzVPEBrBo82O1vwjvp5wCt5n1/T1n/X2Z/ydVkC6Q1IuiiLwh6psWJWcax0SUUoJ1bfLU5QvaVZAu8PBD24OakmQPaxUm93FsEYt72DQo67YznXEMSGN6q377clajI4yS0PMBd0KF7aGpppUwre3KK+yEDifTIwXcL9ax/GvL2eOtB122zB/UIiwQFYfWfxt9RY4R3klqX9/R2f/KZZSHoASZu22zInj+Ojlu6CaxjrtU0Arv/bJbFhOcHH45Cv+Rb2RMpMY5HP+gzVGWa7MkR589qbRQs84dOIBsaXtzZFMvD55Rq5VuK8KDe0yYHUVjJGMJen4ZCuypsWlpYLZI8b3FX712ag2vx5C3vgcgSs/CXuCK3UtQRK3tmp+a/nAoSmqY9seYOhgXFIwMxvVQlYqCg2F9dzoleTob5AOzQwGG5ZuvGupijJK8dpGvpioEwaWcrCnfLGUmxlEBvRZzb/Y5NBNpcDR+CWXr5zVML4ophVzYCPBykqZqRkaeNAUzO+eXSqFAmDHFYV1vDRoVW67X1Cxh9tlhnVmERJ678faik/evReLLarFF9wVT6RMEA5/+4i2KMqQ8DhBNBzKjQPKd9GOczxceQgxMiu4gPYIkI45um9uUfokTFGVUF1uYfa4Gobs3XcKh5fjZzfrbzFgkKCDfn4R+/pd9BqmnLRjArqDx8hU0+eaIFE91rqT1oj/phHNrn3uxGf3pY3HIno0qVBW08jMzS1uO6ai+R4m/M4zG3INYQDB4xRrfcqzS36FQ8W+1F7mDZOX35Q Ydz+gP0o p1ya6jf2du2MTEYDbQz27pOduJnQ28PX8lOGEFIJkWecUJ1V+6TGSIbn2NDXFzL+zbF91C+o3h9ffjYH8PpAPF7QN7Fwh+Lc9HPsvC0ilH+EBYB+57SZhzhgGQpSLJqjzDep79PQIr6w1b3TQADfi347xkD92Zasgdm0uCTUVdsKlh70GGoIO2Qi7njgHBbuFOgLAHRhcoUkdg6WvyTLhuQyS8QRs++ig9XwzVrsRqcIC1ulvQthp+2aS1kq8dyDoiwXQVpZrfh/jt4G5eb3oepXtcXOPmAY/fU5M59hf4VbtZG2Re307UqJHGCoEVMupqHFxbvFk0BR0TkiR7duUgwH3hI6v7ueve8Er66wOFju4P0H7/RcmOx+KxH12B1h0smoyBlQcQq/IVOo2fhn4oipmPoit6/DVMDUqZEPccHLLOxzcJx9UDEwuJBcLlDd7O2kpHe2eKk3W6uxpLiEYBjN/vtsG7ao3bN/7gBjpGNJxtW64DFcFR599IbFQfOGlADyuEcqb82DqIj2r0J2wgxwGDg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce utility functions setting the foundation for ptdescs. These will also assist in the splitting out of ptdesc from struct page. Functions that focus on the descriptor are prefixed with ptdesc_* while functions that focus on the pagetable are prefixed with pagetable_*. pagetable_alloc() is defined to allocate new ptdesc pages as compound pages. This is to standardize ptdescs by allowing for one allocation and one free function, in contrast to 2 allocation and 2 free functions. Signed-off-by: Vishal Moola (Oracle) --- include/asm-generic/tlb.h | 11 ++++++++ include/linux/mm.h | 56 +++++++++++++++++++++++++++++++++++++++ include/linux/pgtable.h | 12 +++++++++ 3 files changed, 79 insertions(+) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index b46617207c93..6bade9e0e799 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -481,6 +481,17 @@ static inline void tlb_remove_page(struct mmu_gather *tlb, struct page *page) return tlb_remove_page_size(tlb, page, PAGE_SIZE); } +static inline void tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) +{ + tlb_remove_table(tlb, pt); +} + +/* Like tlb_remove_ptdesc, but for page-like page directories. */ +static inline void tlb_remove_page_ptdesc(struct mmu_gather *tlb, struct ptdesc *pt) +{ + tlb_remove_page(tlb, ptdesc_page(pt)); +} + static inline void tlb_change_page_size(struct mmu_gather *tlb, unsigned int page_size) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 0dad5f40ef96..14d95d494958 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2744,6 +2744,57 @@ static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long a } #endif /* CONFIG_MMU */ +static inline struct ptdesc *virt_to_ptdesc(const void *x) +{ + return page_ptdesc(virt_to_page(x)); +} + +static inline void *ptdesc_to_virt(const struct ptdesc *pt) +{ + return page_to_virt(ptdesc_page(pt)); +} + +static inline void *ptdesc_address(const struct ptdesc *pt) +{ + return folio_address(ptdesc_folio(pt)); +} + +static inline bool pagetable_is_reserved(struct ptdesc *pt) +{ + return folio_test_reserved(ptdesc_folio(pt)); +} + +/** + * pagetable_alloc - Allocate pagetables + * @gfp: GFP flags + * @order: desired pagetable order + * + * pagetable_alloc allocates memory for page tables as well as a page table + * descriptor to describe that memory. + * + * Return: The ptdesc describing the allocated page tables. + */ +static inline struct ptdesc *pagetable_alloc(gfp_t gfp, unsigned int order) +{ + struct page *page = alloc_pages(gfp | __GFP_COMP, order); + + return page_ptdesc(page); +} + +/** + * pagetable_free - Free pagetables + * @pt: The page table descriptor + * + * pagetable_free frees the memory of all page tables described by a page + * table descriptor and the memory for the descriptor itself. + */ +static inline void pagetable_free(struct ptdesc *pt) +{ + struct page *page = ptdesc_page(pt); + + __free_pages(page, compound_order(page)); +} + #if USE_SPLIT_PTE_PTLOCKS #if ALLOC_SPLIT_PTLOCKS void __init ptlock_cache_init(void); @@ -2981,6 +3032,11 @@ static inline void mark_page_reserved(struct page *page) adjust_managed_page_count(page, -1); } +static inline void free_reserved_ptdesc(struct ptdesc *pt) +{ + free_reserved_page(ptdesc_page(pt)); +} + /* * Default method to free all the __init memory into the buddy system. * The freed pages will be poisoned with pattern "poison" if it's within diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index d46cb709ce08..e9bb5f18cade 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1055,6 +1055,18 @@ TABLE_MATCH(memcg_data, pt_memcg_data); #undef TABLE_MATCH static_assert(sizeof(struct ptdesc) <= sizeof(struct page)); +#define ptdesc_page(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct page *)(pt), \ + struct ptdesc *: (struct page *)(pt))) + +#define ptdesc_folio(pt) (_Generic((pt), \ + const struct ptdesc *: (const struct folio *)(pt), \ + struct ptdesc *: (struct folio *)(pt))) + +#define page_ptdesc(p) (_Generic((p), \ + const struct page *: (const struct ptdesc *)(p), \ + struct page *: (struct ptdesc *)(p))) + /* * No-op macros that just return the current protection value. Defined here * because these macros can be used even if CONFIG_MMU is not defined.