From patchwork Wed Oct 5 11:29:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qinglin Pan X-Patchwork-Id: 12999144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14393C433F5 for ; Wed, 5 Oct 2022 11:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FPqsmHKuW/LH3vx9dpsKkVlHBFuWgyUEQrcPpKyuz/8=; b=HTuRnERbCvr2lG eY1ocHae8T1f7gckSbzpt0JwpwTpq0/q2+JZprV8+KwMuaLuztZEc6Mt4OFqv9odZGWDGx7W5y8F3 rRnh62dxBHtMJRJ8DnCgkENVoJ5Nc0iNZRTtYBgku+4YOKOF0Sbse6uC3exJU61UVmlCD0WMfE6bI FyCgsSIQJ0tAW8SOU3rD9FDrYnb7KhzRm5D2bF4tRH9cSzLM0141zxLwIJedU9Zd6AavMmLyf7O9A 2s+ovDetMyhFORsA+iDPCnJEDsmhuRDaOrnnWp021EJcJXT8lag5ElT7knlieyVH6SC6JL4Gy1x1D qhHV3SaWjp8kbjDJmEgg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2b4-00DnMO-10; Wed, 05 Oct 2022 11:29:54 +0000 Received: from smtp84.cstnet.cn ([159.226.251.84] helo=cstnet.cn) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2ax-00DnK1-70 for linux-riscv@lists.infradead.org; Wed, 05 Oct 2022 11:29:51 +0000 Received: from localhost.localdomain (unknown [124.16.141.248]) by APP-05 (Coremail) with SMTP id zQCowADn13Sgaj1jGzAeAw--.15274S3; Wed, 05 Oct 2022 19:29:39 +0800 (CST) From: panqinglin2020@iscas.ac.cn To: palmer@dabbelt.com, linux-riscv@lists.infradead.org Cc: jeff@riscv.org, xuyinan@ict.ac.cn, conor@kernel.org, ajones@ventanamicro.com, Qinglin Pan Subject: [PATCH v6 1/4] riscv: mm: modify pte format for Svnapot Date: Wed, 5 Oct 2022 19:29:23 +0800 Message-Id: <20221005112926.3043280-2-panqinglin2020@iscas.ac.cn> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> References: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: zQCowADn13Sgaj1jGzAeAw--.15274S3 X-Coremail-Antispam: 1UD129KBjvJXoW3AryftFWUXrW5ZFyrCFy5XFb_yoW7KrWfpr 4kCrZa9Fs8Gw1fC3yIyrn8Ww18Xw4DWwsxKw1I9rW8Jay7AryxX34DAa43Gr1UJFWvq348 uFWrWF4rua17A3JanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPl14x267AKxVW8JVW5JwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r1I6r4UM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Jr0_JF4l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4kE6xkIj40Ew7xC0wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2js IE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZF pf9x0JUhdb8UUUUU= X-Originating-IP: [124.16.141.248] X-CM-SenderInfo: 5sdq1xpqjox0asqsiq5lvft2wodfhubq/1tbiCQAKDGM9K9Bb-wAAsO X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221005_042947_662312_D2F296DA X-CRM114-Status: GOOD ( 13.72 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Qinglin Pan Add one static key to enable/disable svnapot support, enable this static key when "svnapot" is in the "riscv,isa" field of fdt and SVNAPOT compile option is set. It will influence the behavior of has_svnapot. All code dependent on svnapot should make sure that has_svnapot return true firstly. Modify PTE definition for Svnapot, and creates some functions in pgtable.h to mark a PTE as napot and check if it is a Svnapot PTE. Until now, only 64KB napot size is supported in spec, so some macros has only 64KB version. Signed-off-by: Qinglin Pan Reviewed-by: Andrew Jones diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d557cc50295d..69e88ab8fafd 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -383,6 +383,20 @@ config RISCV_ISA_C If you don't know what to do here, say Y. +config RISCV_ISA_SVNAPOT + bool "SVNAPOT extension support" + depends on 64BIT && MMU + select RISCV_ALTERNATIVE + default y + help + Allow kernel to detect SVNAPOT ISA-extension dynamically in boot time + and enable its usage. + + SVNAPOT extension helps to mark contiguous PTEs as a range + of contiguous virtual-to-physical translations, with a naturally + aligned power-of-2 (NAPOT) granularity larger than the base 4KB page + size. + config RISCV_ISA_SVPBMT bool "SVPBMT extension support" depends on 64BIT && MMU diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 6f59ec64175e..2c45cc0d5d3c 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -54,6 +54,7 @@ extern unsigned long elf_hwcap; */ enum riscv_isa_ext_id { RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE, + RISCV_ISA_EXT_SVNAPOT, RISCV_ISA_EXT_SVPBMT, RISCV_ISA_EXT_ZICBOM, RISCV_ISA_EXT_ZIHINTPAUSE, @@ -68,6 +69,7 @@ enum riscv_isa_ext_id { */ enum riscv_isa_ext_key { RISCV_ISA_EXT_KEY_FPU, /* For 'F' and 'D' */ + RISCV_ISA_EXT_KEY_SVNAPOT, RISCV_ISA_EXT_KEY_ZIHINTPAUSE, RISCV_ISA_EXT_KEY_MAX, }; @@ -88,6 +90,8 @@ static __always_inline int riscv_isa_ext2key(int num) return RISCV_ISA_EXT_KEY_FPU; case RISCV_ISA_EXT_d: return RISCV_ISA_EXT_KEY_FPU; + case RISCV_ISA_EXT_SVNAPOT: + return RISCV_ISA_EXT_KEY_SVNAPOT; case RISCV_ISA_EXT_ZIHINTPAUSE: return RISCV_ISA_EXT_KEY_ZIHINTPAUSE; default: diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index dc42375c2357..1cd0ffbfbdaa 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -74,6 +74,19 @@ typedef struct { */ #define _PAGE_PFN_MASK GENMASK(53, 10) +/* + * [63] Svnapot definitions: + * 0 Svnapot disabled + * 1 Svnapot enabled + */ +#define _PAGE_NAPOT_SHIFT 63 +#define _PAGE_NAPOT BIT(_PAGE_NAPOT_SHIFT) +#define NAPOT_CONT64KB_ORDER 4UL +#define NAPOT_CONT64KB_SHIFT (NAPOT_CONT64KB_ORDER + PAGE_SHIFT) +#define NAPOT_CONT64KB_SIZE BIT(NAPOT_CONT64KB_SHIFT) +#define NAPOT_CONT64KB_MASK (NAPOT_CONT64KB_SIZE - 1UL) +#define NAPOT_64KB_PTE_NUM BIT(NAPOT_CONT64KB_ORDER) + /* * [62:61] Svpbmt Memory Type definitions: * diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 7ec936910a96..a8cab063fd05 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -6,10 +6,12 @@ #ifndef _ASM_RISCV_PGTABLE_H #define _ASM_RISCV_PGTABLE_H +#include #include #include #include +#include #ifndef CONFIG_MMU #define KERNEL_LINK_ADDR PAGE_OFFSET @@ -264,10 +266,38 @@ static inline pte_t pud_pte(pud_t pud) return __pte(pud_val(pud)); } +static __always_inline bool has_svnapot(void) +{ + return static_branch_likely(&riscv_isa_ext_keys[RISCV_ISA_EXT_KEY_SVNAPOT]); +} + +#ifdef CONFIG_RISCV_ISA_SVNAPOT + +static inline unsigned long pte_napot(pte_t pte) +{ + return pte_val(pte) & _PAGE_NAPOT; +} + +static inline pte_t pte_mknapot(pte_t pte, unsigned int order) +{ + int pos = order - 1 + _PAGE_PFN_SHIFT; + unsigned long napot_bit = BIT(pos); + unsigned long napot_mask = ~GENMASK(pos, _PAGE_PFN_SHIFT); + + return __pte((pte_val(pte) & napot_mask) | napot_bit | _PAGE_NAPOT); +} +#endif /* CONFIG_RISCV_ISA_SVNAPOT */ + /* Yields the page frame number (PFN) of a page table entry */ static inline unsigned long pte_pfn(pte_t pte) { - return __page_val_to_pfn(pte_val(pte)); + unsigned long val = pte_val(pte); + unsigned long res = __page_val_to_pfn(val); + + if (has_svnapot()) + res = res & (res - (val >> _PAGE_NAPOT_SHIFT)); + + return res; } #define pte_page(x) pfn_to_page(pte_pfn(x)) diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index 0be8a2403212..d2a61122c595 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -96,6 +96,7 @@ static struct riscv_isa_ext_data isa_ext_arr[] = { __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA(zihintpause, RISCV_ISA_EXT_ZIHINTPAUSE), __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), + __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT), __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), }; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 553d755483ed..bc247844e42d 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -204,6 +204,7 @@ void __init riscv_fill_hwcap(void) SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); SET_ISA_EXT_MAP("zihintpause", RISCV_ISA_EXT_ZIHINTPAUSE); SET_ISA_EXT_MAP("sstc", RISCV_ISA_EXT_SSTC); + SET_ISA_EXT_MAP("svnapot", RISCV_ISA_EXT_SVNAPOT); } #undef SET_ISA_EXT_MAP } From patchwork Wed Oct 5 11:29:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qinglin Pan X-Patchwork-Id: 12999145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 53278C4332F for ; Wed, 5 Oct 2022 11:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UklnQddks1j8CHbs1tnmjiRqW0FAKxkqNgQPnbZbvGo=; b=0JCivwNoKjTqyi Ub3wMAJEQqHBeiJFPOyL+3OWSrC3H2k8cz1aqkoC/2+NU3g/M4J3tqOv7NFLOsdAKcvn0kMqaS749 U16CSllnPmdShwg7AsbA5QyJ8qNfiSncXcq3RqDjEHN626tvDq7gO99rMn+kdrOeNkeYobTM1oYGF EuBpG85DF7jObxVNiF9BmqMXSXQ1UpQHhVaJElz+LrpzYZSkkeihe1U0Sn1IaNfQNjdOV3yMCkMLv JrDCpO5xkElg6JIYah1cLza6OTkSfe0qH7VLLoMfF/WCdpQQoNJjuCw1tgJKRtkddNAo5gd6Lwflq eVewd07FvzSDngoSpieQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2b2-00DnLt-Qa; Wed, 05 Oct 2022 11:29:52 +0000 Received: from smtp84.cstnet.cn ([159.226.251.84] helo=cstnet.cn) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2ax-00DnK2-73 for linux-riscv@lists.infradead.org; Wed, 05 Oct 2022 11:29:50 +0000 Received: from localhost.localdomain (unknown [124.16.141.248]) by APP-05 (Coremail) with SMTP id zQCowADn13Sgaj1jGzAeAw--.15274S4; Wed, 05 Oct 2022 19:29:40 +0800 (CST) From: panqinglin2020@iscas.ac.cn To: palmer@dabbelt.com, linux-riscv@lists.infradead.org Cc: jeff@riscv.org, xuyinan@ict.ac.cn, conor@kernel.org, ajones@ventanamicro.com, Qinglin Pan Subject: [PATCH v6 2/4] riscv: mm: support Svnapot in physical page linear-mapping Date: Wed, 5 Oct 2022 19:29:24 +0800 Message-Id: <20221005112926.3043280-3-panqinglin2020@iscas.ac.cn> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> References: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: zQCowADn13Sgaj1jGzAeAw--.15274S4 X-Coremail-Antispam: 1UD129KBjvJXoWxWFy8KFyDZFWxtr18JF18Krg_yoW5Aw48pr WxCrW8WF4DJF1UuFWkAr1UAry3Zas3G347Gay8Kas5A3WfGr1F9w45Ca4YgFyxXrsFgayx Zr4kZ3y5ua43t37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPl14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4kE6xkIj40Ew7xC0wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2js IE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZF pf9x0JUoWlkUUUUU= X-Originating-IP: [124.16.141.248] X-CM-SenderInfo: 5sdq1xpqjox0asqsiq5lvft2wodfhubq/1tbiBwAKDGM9Lg5W9AAAsd X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221005_042947_673570_75283335 X-CRM114-Status: GOOD ( 12.78 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Qinglin Pan Svnapot is powerful when a physical region is going to mapped to a virtual region. Kernel will do like this when mapping all allocable physical pages to kernel vm space. Modify the create_pte_mapping function used in linear-mapping procedure, so the kernel can be able to use Svnapot when both address and length of physical region are 64KB align. Code here will be executed only when other size huge page is not suitable, so it can be an addition of PMD_SIZE and PUD_SIZE mapping. Modify the best_map_size function to give map_size many times instead of only once, so a memory region can be mapped by both PMD_SIZE and 64KB napot size. It is tested by setting qemu's memory to a 262272k region, and the kernel can boot successfully. Currently, the modified create_pte_mapping will never take use of SVNAPOT, because this extension is detected and enabled in riscv_fill_hwcap(called from setup_arch) which is called after setup_vm_final. We will need to support function like riscv_fill_hwcap_early to fill hardware capabilities more earlier, and try to enable SVNAPOT more earlier in it, so that we can determine SVNAPOT's presence when doing linear-mapping. Signed-off-by: Qinglin Pan diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index b56a0a75533f..e6edc06b4543 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -373,9 +373,23 @@ static void __init create_pte_mapping(pte_t *ptep, phys_addr_t sz, pgprot_t prot) { uintptr_t pte_idx = pte_index(va); + pte_t pte; + + if (!IS_ENABLED(CONFIG_RISCV_ISA_SVNAPOT)) + goto normal_page; + + if (has_svnapot() && sz == NAPOT_CONT64KB_SIZE) { + do { + pte = pfn_pte(PFN_DOWN(pa), prot); + ptep[pte_idx] = pte_mknapot(pte, NAPOT_CONT64KB_ORDER); + pte_idx++; + sz -= PAGE_SIZE; + } while (sz > 0); + return; + } +normal_page: BUG_ON(sz != PAGE_SIZE); - if (pte_none(ptep[pte_idx])) ptep[pte_idx] = pfn_pte(PFN_DOWN(pa), prot); } @@ -673,10 +687,18 @@ void __init create_pgd_mapping(pgd_t *pgdp, static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) { /* Upgrade to PMD_SIZE mappings whenever possible */ - if ((base & (PMD_SIZE - 1)) || (size & (PMD_SIZE - 1))) + base &= PMD_SIZE - 1; + if (!base && size >= PMD_SIZE) + return PMD_SIZE; + + if (!has_svnapot()) return PAGE_SIZE; - return PMD_SIZE; + base &= NAPOT_CONT64KB_SIZE - 1; + if (!base && size >= NAPOT_CONT64KB_SIZE) + return NAPOT_CONT64KB_SIZE; + + return PAGE_SIZE; } #ifdef CONFIG_XIP_KERNEL @@ -1111,9 +1133,9 @@ static void __init setup_vm_final(void) if (end >= __pa(PAGE_OFFSET) + memory_limit) end = __pa(PAGE_OFFSET) + memory_limit; - map_size = best_map_size(start, end - start); for (pa = start; pa < end; pa += map_size) { va = (uintptr_t)__va(pa); + map_size = best_map_size(pa, end - pa); create_pgd_mapping(swapper_pg_dir, va, pa, map_size, pgprot_from_va(va)); From patchwork Wed Oct 5 11:29:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qinglin Pan X-Patchwork-Id: 12999147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70124C43217 for ; Wed, 5 Oct 2022 11:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kl2kuHkSD9zv1A/U3qj45Ko5j9i89omMohVXdTEGYds=; b=DrB4rxO3oxssk/ JdJPGQ4IOS8QgKC2rsWn4zPQr4xEdGywsrUiuR3MpPZP8ATsj9chEyMJe6QNTi8TT2e6luvdZYpeu UEvjd+sETJBwbBUtTRDCeb3cnXO29z0y9XvUxLRVx6CpJo4PBTgtm0/LKy25lDhy/8AW/vG2Pmfv3 X9UioegerLLLB6cG5AryeVsbKUVPbKoZWuhbLORRfYrBN6li9luwICZflm78ErslNqRYWhryq08lJ mQP3tLB0q5Qc88P0HC3UQfdgw6/wUta3M5V/htPVVHw10av8WJwBLgiJbDtBlz+obULX9tanMeQjS Mlpk/+X/FlUpTraAEMow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2b6-00DnN4-Bd; Wed, 05 Oct 2022 11:29:56 +0000 Received: from smtp84.cstnet.cn ([159.226.251.84] helo=cstnet.cn) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2ax-00DnK6-71 for linux-riscv@lists.infradead.org; Wed, 05 Oct 2022 11:29:51 +0000 Received: from localhost.localdomain (unknown [124.16.141.248]) by APP-05 (Coremail) with SMTP id zQCowADn13Sgaj1jGzAeAw--.15274S5; Wed, 05 Oct 2022 19:29:41 +0800 (CST) From: panqinglin2020@iscas.ac.cn To: palmer@dabbelt.com, linux-riscv@lists.infradead.org Cc: jeff@riscv.org, xuyinan@ict.ac.cn, conor@kernel.org, ajones@ventanamicro.com, Qinglin Pan Subject: [PATCH v6 3/4] riscv: mm: support Svnapot in hugetlb page Date: Wed, 5 Oct 2022 19:29:25 +0800 Message-Id: <20221005112926.3043280-4-panqinglin2020@iscas.ac.cn> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> References: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: zQCowADn13Sgaj1jGzAeAw--.15274S5 X-Coremail-Antispam: 1UD129KBjvJXoWfGFWUAr4xJw43KrW7tF1UWrg_yoWDCFWkpF W3Gw13ZrW3tw1xGw4Iqr15Jrn8twnxW34UJF97JFWFvr1Iv39xXas7GrWSyry7CrWkX3yx CrWrGFn8Ary7Xw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPl14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1l84 ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_Cr1U M2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64 kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm 72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYx C7M4kE6xkIj40Ew7xC0wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC2 0s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI 0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv2 0xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2js IE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZF pf9x0JU9Z23UUUUU= X-Originating-IP: [124.16.141.248] X-CM-SenderInfo: 5sdq1xpqjox0asqsiq5lvft2wodfhubq/1tbiAw0KDGM9I2pnjgAAs2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221005_042947_668201_B52EEDDB X-CRM114-Status: GOOD ( 14.63 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Qinglin Pan Svnapot can be used to support 64KB hugetlb page, so it can become a new option when using hugetlbfs. Add a basic implementation of hugetlb page, and support 64KB as a size in it by using Svnapot. For test, boot kernel with command line contains "default_hugepagesz=64K hugepagesz=64K hugepages=20" and run a simple test like this: ------------------------------------------------------------------------ int main() { void *addr; addr = mmap(NULL, 64 * 1024, PROT_WRITE | PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_HUGE_64KB, -1, 0); printf("back from mmap \n"); long *ptr = (long *)addr; unsigned int i = 0; for(; i < 8 * 1024;i += 512) { printf("%lp \n", ptr); *ptr = 0xdeafabcd12345678; ptr += 512; } ptr = (long *)addr; i = 0; for(; i < 8 * 1024;i += 512) { if (*ptr != 0xdeafabcd12345678) { printf("failed! 0x%lx \n", *ptr); break; } ptr += 512; } if(i == 8 * 1024) printf("simple test passed!\n"); } ------------------------------------------------------------------------ And it should be passed. Signed-off-by: Qinglin Pan diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 69e88ab8fafd..bc19786f25f5 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -43,7 +43,7 @@ config RISCV select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS - select ARCH_WANT_GENERAL_HUGETLB + select ARCH_WANT_GENERAL_HUGETLB if !RISCV_ISA_SVNAPOT select ARCH_WANT_HUGE_PMD_SHARE if 64BIT select BINFMT_FLAT_NO_DATA_START_OFFSET if !MMU select BUILDTIME_TABLE_SORT if MMU diff --git a/arch/riscv/include/asm/hugetlb.h b/arch/riscv/include/asm/hugetlb.h index a5c2ca1d1cd8..ebcfcce6d906 100644 --- a/arch/riscv/include/asm/hugetlb.h +++ b/arch/riscv/include/asm/hugetlb.h @@ -2,7 +2,42 @@ #ifndef _ASM_RISCV_HUGETLB_H #define _ASM_RISCV_HUGETLB_H -#include #include +#ifdef CONFIG_RISCV_ISA_SVNAPOT +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); +#define arch_make_huge_pte arch_make_huge_pte + +#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT +void set_huge_pte_at(struct mm_struct *mm, + unsigned long addr, pte_t *ptep, pte_t pte); + +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep); + +#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, pte_t *ptep, + pte_t pte, int dirty); + +#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, pte_t *ptep); + +#define __HAVE_ARCH_HUGE_PTE_CLEAR +void huge_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned long sz); + +#define set_huge_swap_pte_at riscv_set_huge_swap_pte_at +void riscv_set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz); +#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ + +#include + #endif /* _ASM_RISCV_HUGETLB_H */ diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index ac70b0fd9a9a..1ea06476902a 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -17,7 +17,7 @@ #define PAGE_MASK (~(PAGE_SIZE - 1)) #ifdef CONFIG_64BIT -#define HUGE_MAX_HSTATE 2 +#define HUGE_MAX_HSTATE 3 #else #define HUGE_MAX_HSTATE 1 #endif diff --git a/arch/riscv/mm/hugetlbpage.c b/arch/riscv/mm/hugetlbpage.c index 932dadfdca54..faa207826260 100644 --- a/arch/riscv/mm/hugetlbpage.c +++ b/arch/riscv/mm/hugetlbpage.c @@ -2,6 +2,239 @@ #include #include +#ifdef CONFIG_RISCV_ISA_SVNAPOT +pte_t *huge_pte_alloc(struct mm_struct *mm, + struct vm_area_struct *vma, + unsigned long addr, + unsigned long sz) +{ + pgd_t *pgdp = pgd_offset(mm, addr); + p4d_t *p4dp = p4d_alloc(mm, pgdp, addr); + pud_t *pudp = pud_alloc(mm, p4dp, addr); + pmd_t *pmdp = pmd_alloc(mm, pudp, addr); + + if (sz == NAPOT_CONT64KB_SIZE) { + if (!pmdp) + return NULL; + WARN_ON(addr & (sz - 1)); + return pte_alloc_map(mm, pmdp, addr); + } + + return NULL; +} + +pte_t *huge_pte_offset(struct mm_struct *mm, + unsigned long addr, + unsigned long sz) +{ + pgd_t *pgdp; + p4d_t *p4dp; + pud_t *pudp; + pmd_t *pmdp; + pte_t *ptep = NULL; + + pgdp = pgd_offset(mm, addr); + if (!pgd_present(READ_ONCE(*pgdp))) + return NULL; + + p4dp = p4d_offset(pgdp, addr); + if (!p4d_present(READ_ONCE(*p4dp))) + return NULL; + + pudp = pud_offset(p4dp, addr); + if (!pud_present(READ_ONCE(*pudp))) + return NULL; + + pmdp = pmd_offset(pudp, addr); + if (!pmd_present(READ_ONCE(*pmdp))) + return NULL; + + if (sz == NAPOT_CONT64KB_SIZE) + ptep = pte_offset_kernel(pmdp, (addr & ~NAPOT_CONT64KB_MASK)); + + return ptep; +} + +static int napot_pte_num(pte_t pte) +{ + if (pte_val(pte) & _PAGE_NAPOT) + return NAPOT_64KB_PTE_NUM; + + pr_warn("%s: unrecognized napot pte size 0x%lx\n", + __func__, pte_val(pte)); + return 1; +} + +static pte_t get_clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pte_num) +{ + pte_t orig_pte = huge_ptep_get(ptep); + bool valid = pte_val(orig_pte); + unsigned long i, saddr = addr; + + for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) { + pte_t pte = ptep_get_and_clear(mm, addr, ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + + if (valid) { + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + + flush_tlb_range(&vma, saddr, addr); + } + return orig_pte; +} + +static void clear_flush(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long pte_num) +{ + struct vm_area_struct vma = TLB_FLUSH_VMA(mm, 0); + unsigned long i, saddr = addr; + + for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_clear(mm, addr, ptep); + + flush_tlb_range(&vma, saddr, addr); +} + +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) +{ + if (shift == NAPOT_CONT64KB_SHIFT) + entry = pte_mknapot(entry, NAPOT_CONT64KB_SHIFT - PAGE_SHIFT); + + return entry; +} + +void set_huge_pte_at(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + pte_t pte) +{ + int i; + int pte_num; + + if (!pte_napot(pte)) { + set_pte_at(mm, addr, ptep, pte); + return; + } + + pte_num = napot_pte_num(pte); + for (i = 0; i < pte_num; i++, ptep++, addr += PAGE_SIZE) + set_pte_at(mm, addr, ptep, pte); +} + +int huge_ptep_set_access_flags(struct vm_area_struct *vma, + unsigned long addr, + pte_t *ptep, + pte_t pte, + int dirty) +{ + pte_t orig_pte; + int i; + int pte_num; + + if (!pte_napot(pte)) + return ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + pte_num = napot_pte_num(pte); + ptep = huge_pte_offset(vma->vm_mm, addr, NAPOT_CONT64KB_SIZE); + orig_pte = huge_ptep_get(ptep); + + if (pte_dirty(orig_pte)) + pte = pte_mkdirty(pte); + + if (pte_young(orig_pte)) + pte = pte_mkyoung(pte); + + for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + ptep_set_access_flags(vma, addr, ptep, pte, dirty); + + return true; +} + +pte_t huge_ptep_get_and_clear(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep) +{ + int pte_num; + pte_t orig_pte = huge_ptep_get(ptep); + + if (!pte_napot(orig_pte)) + return ptep_get_and_clear(mm, addr, ptep); + + pte_num = napot_pte_num(orig_pte); + return get_clear_flush(mm, addr, ptep, pte_num); +} + +void huge_ptep_set_wrprotect(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep) +{ + int i; + int pte_num; + pte_t pte = READ_ONCE(*ptep); + + if (!pte_napot(pte)) + return ptep_set_wrprotect(mm, addr, ptep); + + pte_num = napot_pte_num(pte); + ptep = huge_pte_offset(mm, addr, NAPOT_CONT64KB_SIZE); + + for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + ptep_set_wrprotect(mm, addr, ptep); +} + +pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, + unsigned long addr, + pte_t *ptep) +{ + int pte_num; + pte_t pte = READ_ONCE(*ptep); + + if (!pte_napot(pte)) { + ptep_clear_flush(vma, addr, ptep); + return pte; + } + + pte_num = napot_pte_num(pte); + clear_flush(vma->vm_mm, addr, ptep, pte_num); + + return pte; +} + +void huge_pte_clear(struct mm_struct *mm, + unsigned long addr, + pte_t *ptep, + unsigned long sz) +{ + int i, pte_num; + + pte_num = napot_pte_num(READ_ONCE(*ptep)); + for (i = 0; i < pte_num; i++, addr += PAGE_SIZE, ptep++) + pte_clear(mm, addr, ptep); +} + +void riscv_set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, pte_t pte, unsigned long sz) +{ + int i, pte_num; + + pte_num = napot_pte_num(READ_ONCE(*ptep)); + + for (i = 0; i < pte_num; i++, ptep++) + set_pte(ptep, pte); +} +#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ + int pud_huge(pud_t pud) { return pud_leaf(pud); @@ -18,17 +251,26 @@ bool __init arch_hugetlb_valid_size(unsigned long size) return true; else if (IS_ENABLED(CONFIG_64BIT) && size == PUD_SIZE) return true; +#ifdef CONFIG_RISCV_ISA_SVNAPOT + else if (has_svnapot() && size == NAPOT_CONT64KB_SIZE) + return true; +#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ else return false; } -#ifdef CONFIG_CONTIG_ALLOC -static __init int gigantic_pages_init(void) +static __init int hugetlbpage_init(void) { +#ifdef CONFIG_CONTIG_ALLOC /* With CONTIG_ALLOC, we can allocate gigantic pages at runtime */ if (IS_ENABLED(CONFIG_64BIT)) hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT); +#endif /*CONFIG_CONTIG_ALLOC*/ + hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT); +#ifdef CONFIG_RISCV_ISA_SVNAPOT + if (has_svnapot()) + hugetlb_add_hstate(NAPOT_CONT64KB_SHIFT - PAGE_SHIFT); +#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ return 0; } -arch_initcall(gigantic_pages_init); -#endif +arch_initcall(hugetlbpage_init); From patchwork Wed Oct 5 11:29:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qinglin Pan X-Patchwork-Id: 12999146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AFD5AC433FE for ; Wed, 5 Oct 2022 11:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kpUqDuecTNxvEtBRlMjMyXuSyCPs4oTncNzzni5QtDE=; b=jc/jx8uK7Z7xig d/k7o8IzQzDVILe85oMN83+g2B48eARP7B2nGU/sRn06828WGmnzlSK8VxuVXP+aIqLabdOGs/QPd 6Y5QTLe6OLcFjqBzCDYV0FEwSbJFSjur0Sc224kETN9eKYFSzrd8Nt9QL9yeZegxVBcunQ4VWJf8Z /IFzt4R/RwFbKQmb7wjiguirEG5lk/GAfZzJfk75j85FGbavdEpH7Gl9eUkj+aUFj7SOKw+aMukWN 91GLSCghvOjCXkFcAnWM3oRHRVgIjV5G2RG/3+ATqjiFL2d6V3+gfpP0RizUmoxhBFHCJ4wSmHIYQ 4uuxPw2+DTRV+370kB+A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2b5-00DnMj-6F; Wed, 05 Oct 2022 11:29:55 +0000 Received: from smtp84.cstnet.cn ([159.226.251.84] helo=cstnet.cn) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1og2ax-00DnK7-73 for linux-riscv@lists.infradead.org; Wed, 05 Oct 2022 11:29:51 +0000 Received: from localhost.localdomain (unknown [124.16.141.248]) by APP-05 (Coremail) with SMTP id zQCowADn13Sgaj1jGzAeAw--.15274S6; Wed, 05 Oct 2022 19:29:43 +0800 (CST) From: panqinglin2020@iscas.ac.cn To: palmer@dabbelt.com, linux-riscv@lists.infradead.org Cc: jeff@riscv.org, xuyinan@ict.ac.cn, conor@kernel.org, ajones@ventanamicro.com, Qinglin Pan Subject: [PATCH v6 4/4] riscv: mm: support Svnapot in huge vmap Date: Wed, 5 Oct 2022 19:29:26 +0800 Message-Id: <20221005112926.3043280-5-panqinglin2020@iscas.ac.cn> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> References: <20221005112926.3043280-1-panqinglin2020@iscas.ac.cn> MIME-Version: 1.0 X-CM-TRANSID: zQCowADn13Sgaj1jGzAeAw--.15274S6 X-Coremail-Antispam: 1UD129KBjvJXoWxGr13uFW5Gw4fXF1kur4UJwb_yoW5CFWxpr Z5CrnYyFZ8Ka4rCFWFyr109rW5ZanxW3yfK3s5GrWkAF47JrWkWr95t34Yqr18JFWv9rWx CFZ7WFW5C3yDJa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUPK14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_JFI_Gr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr 1l84ACjcxK6I8E87Iv67AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVCY1x0267AKxVW8Jr0_ Cr1UM2vYz4IE04k24VAvwVAKI4IrM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4 xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8 JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20V AGYxC7M4kE6xkIj40Ew7xC0wCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8 JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1V AFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xII jxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4 A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU 0xZFpf9x0JUvzuAUUUUU= X-Originating-IP: [124.16.141.248] X-CM-SenderInfo: 5sdq1xpqjox0asqsiq5lvft2wodfhubq/1tbiAwAKDGM9I2pnjQAAs4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221005_042947_685558_90A5194E X-CRM114-Status: UNSURE ( 9.26 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Qinglin Pan The HAVE_ARCH_HUGE_VMAP option can be used to help implement arch special huge vmap size. This commit selects this option by default and re-writes the arch_vmap_pte_range_map_size for Svnapot 64KB size. It can be tested when booting kernel in qemu with pci device, which will make the kernel to call pci driver using ioremap, and the re-written function will be called. Signed-off-by: Qinglin Pan diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index bc19786f25f5..4b6ca943168f 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -70,6 +70,7 @@ config RISCV select GENERIC_TIME_VSYSCALL if MMU && 64BIT select GENERIC_VDSO_TIME_NS if HAVE_GENERIC_VDSO select HAVE_ARCH_AUDITSYSCALL + select HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL if !XIP_KERNEL select HAVE_ARCH_JUMP_LABEL_RELATIVE if !XIP_KERNEL select HAVE_ARCH_KASAN if MMU && 64BIT diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index a8cab063fd05..aa7eda57054c 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -749,6 +749,43 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +static inline int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} + +static inline int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} + +static inline void p4d_clear_huge(p4d_t *p4d) { } + +static inline int pud_clear_huge(pud_t *pud) +{ + return 0; +} + +static inline int pmd_clear_huge(pmd_t *pmd) +{ + return 0; +} + +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + +static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) +{ + return 0; +} + +static inline int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) +{ + return 0; +} + /* * Encode and decode a swap entry * diff --git a/arch/riscv/include/asm/vmalloc.h b/arch/riscv/include/asm/vmalloc.h index ff9abc00d139..ecd1f784299b 100644 --- a/arch/riscv/include/asm/vmalloc.h +++ b/arch/riscv/include/asm/vmalloc.h @@ -1,4 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ #ifndef _ASM_RISCV_VMALLOC_H #define _ASM_RISCV_VMALLOC_H +#include + +#ifdef CONFIG_RISCV_ISA_SVNAPOT +#define arch_vmap_pte_range_map_size vmap_pte_range_map_size +static inline unsigned long +vmap_pte_range_map_size(unsigned long addr, unsigned long end, u64 pfn, + unsigned int max_page_shift) +{ + if (!has_svnapot()) + return PAGE_SIZE; + + if (addr & NAPOT_CONT64KB_MASK) + return PAGE_SIZE; + + if (pfn & (NAPOT_64KB_PTE_NUM - 1UL)) + return PAGE_SIZE; + + if ((end - addr) < NAPOT_CONT64KB_SIZE) + return PAGE_SIZE; + + if (max_page_shift < NAPOT_CONT64KB_SHIFT) + return PAGE_SIZE; + + return NAPOT_CONT64KB_SIZE; +} +#endif /*CONFIG_RISCV_ISA_SVNAPOT*/ + #endif /* _ASM_RISCV_VMALLOC_H */