From patchwork Wed Mar 1 08:25:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13155650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97EDAC7EE31 for ; Wed, 1 Mar 2023 08:27:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yUZo5nY4GYBFtTByqFxWOhTvObEAMCr++JtbsPbHTq0=; b=iqWq/lyR4WYVzq n+joGARxqBlPAVIsFcUb+0dCd2zSdzOsUXqcnZRutfxsVQhtGfgmMJkU6mbD1hdfiFNsZBUICaZrj MRugD7Z9rVp2rUC/LhBcMsXaa+kgSvvHhiFGugP+AsSoGzIgl3AVLIk+LOlF+kO0oHV4cK30om5/x LZgAQ8aVCFM/44Tmn8U/f2CdCgSf8vTaoTBNVZ/OObnCEjWc2uS7K2+srT8om758gYdXHBE8zZCaI r+7Ej8+y+4T0ZB6ozzO48anbYAw78vYhcr60aRE5mLWKJLBOPaGGqXopl5k6UXe9c+P8vS+b1DmKP /P253nrrtL6oB1WyTNnQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pXHo1-00FRsd-A1; Wed, 01 Mar 2023 08:27:21 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pXHny-00FRs1-Ca for linux-riscv@lists.infradead.org; Wed, 01 Mar 2023 08:27:19 +0000 Received: by mail-wr1-x430.google.com with SMTP id q16so12349235wrw.2 for ; Wed, 01 Mar 2023 00:27:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1677659237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V/vQkURrK2ciTBU+Pt35U7ABlbP4247yVAlnFnqItRY=; b=3wXq47ftSOAUsi4xd1gwHP/t2Qx2KSSDziZTN3nPLxzatHbZBgdoCKE6SAPoS7W1lu 1kuOgyEs4Tjc+RkZd93D/tViTECmkVthQO5CeRnm6i21/GbHLDryAaco6DrEB+OakKn+ 0nm0nkzgPiEJWD7GeblE24M92kfnQKUPRy7q6cGr6YX61df1jswOf9aBCWEmNb5TNmnA 6eE31v0qusiKaLJwSliWrxA/5MBcJYNkovnh+YXSKczWctIzFxeTmcgoJqDfCdNVVqgs W+7plCVvMuDDogVPvBOFfeTBth2+l5dy/CkjPMJXZ08frdc3+flmoNRZ4Ys8R/GaXzTv 12bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677659237; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V/vQkURrK2ciTBU+Pt35U7ABlbP4247yVAlnFnqItRY=; b=ve8DnqwsixQypJrkHo3i8+z3kq7LvXt8HqWqDFu3acT9/lpm0zNe7IlHnCxMzqR85N FbPojHycYuYF7IsiRNyrfR5GefWTfcCocdqwgZNt0Mu0EleqcHkiKYpFzYZ/4lMCBD3O sJFUe0TEP5/oxqpLfSl9kruh+1yHG7ShTpsQPz6EBPCce9Kih7BB1bGr1i8MX5qIPAUy dLjazrFvbsQ6nd1fJ6ThOUYGEr4igsPTNWOWYu4T9fCth1nAZsKaABJ8sgwlMfKjbSgU MKZIyfQjNomx2j12R9835aviIGuMWS9FEB/whB24MZ+kNrRINKO2+gn2yZUpywRm/w+o 3S2A== X-Gm-Message-State: AO0yUKWuIGGmqZ+e3LAFVwe3vsheXSz3g5N7DF9SgmQIESJkzYJMGssx 9yHZVshwK0cri+F8B9m6GApWAw== X-Google-Smtp-Source: AK7set9kiW+CiLCN+hsyoYIr1BHQff84nc5C3ZNm3+08O9K8LMZJnYtk8c6lYQ13uO/PhTaNmAd0uQ== X-Received: by 2002:a5d:6507:0:b0:2c5:a6d4:1398 with SMTP id x7-20020a5d6507000000b002c5a6d41398mr3821917wru.70.1677659237035; Wed, 01 Mar 2023 00:27:17 -0800 (PST) Received: from alex-rivos.home (lfbn-gre-1-235-32.w90-112.abo.wanadoo.fr. [90.112.194.32]) by smtp.gmail.com with ESMTPSA id n16-20020a5d4850000000b002c6e8cb612fsm11733252wrs.92.2023.03.01.00.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Mar 2023 00:27:16 -0800 (PST) From: Alexandre Ghiti To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Cc: Alexandre Ghiti , Andrew Jones Subject: [PATCH v6 1/2] riscv: Get rid of riscv_pfn_base variable Date: Wed, 1 Mar 2023 09:25:51 +0100 Message-Id: <20230301082552.274331-2-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230301082552.274331-1-alexghiti@rivosinc.com> References: <20230301082552.274331-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230301_002718_446016_D7338BCC X-CRM114-Status: GOOD ( 17.07 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Use directly phys_ram_base instead, riscv_pfn_base is just the pfn of the address contained in phys_ram_base. Even if there is no functional change intended in this patch, actually setting phys_ram_base that early changes the behaviour of kernel_mapping_pa_to_va during the early boot: phys_ram_base used to be zero before this patch and now it is set to the physical start address of the kernel. But it does not break the conversion of a kernel physical address into a virtual address since kernel_mapping_pa_to_va should only be used on kernel physical addresses, i.e. addresses greater than the physical start address of the kernel. Signed-off-by: Alexandre Ghiti Reviewed-by: Andrew Jones --- arch/riscv/include/asm/page.h | 3 +-- arch/riscv/mm/init.c | 6 +----- 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 9f432c1b5289..926af5a3d02e 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -91,8 +91,7 @@ typedef struct page *pgtable_t; #endif #ifdef CONFIG_MMU -extern unsigned long riscv_pfn_base; -#define ARCH_PFN_OFFSET (riscv_pfn_base) +#define ARCH_PFN_OFFSET (PFN_DOWN((unsigned long)phys_ram_base)) #else #define ARCH_PFN_OFFSET (PAGE_OFFSET >> PAGE_SHIFT) #endif /* CONFIG_MMU */ diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 04d20e41894e..bef639fa330b 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -271,9 +271,6 @@ static void __init setup_bootmem(void) #ifdef CONFIG_MMU struct pt_alloc_ops pt_ops __initdata; -unsigned long riscv_pfn_base __ro_after_init; -EXPORT_SYMBOL(riscv_pfn_base); - pgd_t swapper_pg_dir[PTRS_PER_PGD] __page_aligned_bss; pgd_t trampoline_pg_dir[PTRS_PER_PGD] __page_aligned_bss; static pte_t fixmap_pte[PTRS_PER_PTE] __page_aligned_bss; @@ -285,7 +282,6 @@ static pmd_t __maybe_unused early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAG #ifdef CONFIG_XIP_KERNEL #define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops)) -#define riscv_pfn_base (*(unsigned long *)XIP_FIXUP(&riscv_pfn_base)) #define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)) #define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) #define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) @@ -1009,7 +1005,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) kernel_map.va_pa_offset = PAGE_OFFSET - kernel_map.phys_addr; kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr; - riscv_pfn_base = PFN_DOWN(kernel_map.phys_addr); + phys_ram_base = kernel_map.phys_addr; /* * The default maximal physical memory size is KERN_VIRT_SIZE for 32-bit From patchwork Wed Mar 1 08:25:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13155651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4FB53C64ED6 for ; Wed, 1 Mar 2023 08:28:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=foIURHXNZdPxg3lZ/kheLCcydFj88qd4bRAzEIwdxL0=; b=VYszbVyNJIElko rfq4w5StyVnJsJOy4iZEX0qrHGRZEhgtGJ/o6QKN2wDoAVrz35U5FiRupVXeemXQ1mzzo/CstDpPU ZEDFqDCeDC+T0AoZZdcjGjZyxpwaipv4NP+tfK5D1WzLsFiKyyXg2m4Zrm5G8CozlpbdnYwL3PYLg zb67AnCvAZDfQefv9DI58ea9aBmE2HlFPsNpsvoyl7PxqBY0d71Up07/0Q+5VZ6hhDoqjS2D8RYdE RNWnpLk2XxbiXS6md33EHTETfCMRJVMhC69nArW3CsHThuRAO85Hk0SH/QfYx4AmEIeTStvGTy00k zAwY4arH/+zID6Kme4Bg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pXHp3-00FSEg-PC; Wed, 01 Mar 2023 08:28:25 +0000 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pXHoz-00FSAi-38 for linux-riscv@lists.infradead.org; Wed, 01 Mar 2023 08:28:23 +0000 Received: by mail-wm1-x32f.google.com with SMTP id j19-20020a05600c1c1300b003e9b564fae9so11234999wms.2 for ; Wed, 01 Mar 2023 00:28:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; t=1677659298; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pPUq9Py/DiQmbLbEFbMB4lOnk2+sQ/4m0uM/uyD9kiQ=; b=uQ/mg8fo8G4+dhkxzIraLbjS3VGc6m2sVWWXNLTRhu1/gbikacbxeQbQ/jkxTiu3Tz PAMmmil4+urirOZqR81EDwefR0UWVWz8IPz66fEWH/GUEmQ8q2vBhjQ1GXYgaEi433Ua 75Xeu4Z6nib5G5AZ6RBal0p2vhvknAywHw4Rb/7TfQUMb9XtPTz5nCE1hQYplNbHA4z1 +WZ7zrFN9+bte8gQoddjpdQ5dVHOMvDtNBdWsDAPp2EnjCISL6at370rmqAI+tXR1Cav gF9rKLHJBXayu3DnTGomhoViNifkFrOGhSt9SfKbjOPoPzHlDCHzxfySo7R6hjPGC91A iGLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677659298; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pPUq9Py/DiQmbLbEFbMB4lOnk2+sQ/4m0uM/uyD9kiQ=; b=PwiykZ4DrNZPRnX5QIpnmDnV8b/Si85laibhH036JtCd8e/VBli+IaFHLk7ksd7XZ2 fF2Cy1EFvXhPjpJhPTKBlHpHTub7Pk+vx2i2ndtfKEUe/i+3gylHPGsZ+LBwM7+k24w8 3F/BeHZF+9jlfQOM4lYAIvMGfFPlrYd3wmdqoE1+fpMDqrFO2dhpYslTMAqJ4Aa4it5b CF393mpRA+LxcCkK6HrXLIguqj5k5oUfvOrJLLJGxMG4Gl08iaG9ll3Y5vODEdiBtR75 6C2qLLCeaL3aD9afpVtl4rUC+9q5MIK6RTfYq57LloCzrjrvKqVtmMRUC55+vY06lg1V 5q0A== X-Gm-Message-State: AO0yUKURDxYO8IOXeTrap/GZVvcpC6jKFfK1jMw7TZut+tCd1x188Qzn KCAvXDRyvL9xOOO7Gj3JbVtUFQ== X-Google-Smtp-Source: AK7set+mp5A/BQkOBuVC5haYfRJP6w0XNA8s0a/84qPAqNMU4ov3Mpfhqv80h2l+Vmq3ulp//0rrHw== X-Received: by 2002:a05:600c:2ed3:b0:3eb:2de9:d01 with SMTP id q19-20020a05600c2ed300b003eb2de90d01mr4145025wmn.18.1677659298178; Wed, 01 Mar 2023 00:28:18 -0800 (PST) Received: from alex-rivos.home (lfbn-gre-1-235-32.w90-112.abo.wanadoo.fr. [90.112.194.32]) by smtp.gmail.com with ESMTPSA id t23-20020a1c7717000000b003daf672a616sm14661242wmi.22.2023.03.01.00.28.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Mar 2023 00:28:17 -0800 (PST) From: Alexandre Ghiti To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Rob Herring , Frank Rowand , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Cc: Alexandre Ghiti , Rob Herring , Andrew Jones Subject: [PATCH v6 2/2] riscv: Use PUD/P4D/PGD pages for the linear mapping Date: Wed, 1 Mar 2023 09:25:52 +0100 Message-Id: <20230301082552.274331-3-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230301082552.274331-1-alexghiti@rivosinc.com> References: <20230301082552.274331-1-alexghiti@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230301_002821_181253_C7783AE5 X-CRM114-Status: GOOD ( 28.00 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org During the early page table creation, we used to set the mapping for PAGE_OFFSET to the kernel load address: but the kernel load address is always offseted by PMD_SIZE which makes it impossible to use PUD/P4D/PGD pages as this physical address is not aligned on PUD/P4D/PGD size (whereas PAGE_OFFSET is). But actually we don't have to establish this mapping (ie set va_pa_offset) that early in the boot process because: - first, setup_vm installs a temporary kernel mapping and among other things, discovers the system memory, - then, setup_vm_final creates the final kernel mapping and takes advantage of the discovered system memory to create the linear mapping. During the first phase, we don't know the start of the system memory and then until the second phase is finished, we can't use the linear mapping at all and phys_to_virt/virt_to_phys translations must not be used because it would result in a different translation from the 'real' one once the final mapping is installed. So here we simply delay the initialization of va_pa_offset to after the system memory discovery. But to make sure noone uses the linear mapping before, we add some guard in the DEBUG_VIRTUAL config. Finally we can use PUD/P4D/PGD hugepages when possible, which will result in a better TLB utilization. Note that we rely on the firmware to protect itself using PMP. Signed-off-by: Alexandre Ghiti Acked-by: Rob Herring # DT bits Reviewed-by: Andrew Jones --- arch/riscv/include/asm/page.h | 16 ++++++++++++++++ arch/riscv/mm/init.c | 24 ++++++++++++++++++------ arch/riscv/mm/physaddr.c | 16 ++++++++++++++++ drivers/of/fdt.c | 11 ++++++----- 4 files changed, 56 insertions(+), 11 deletions(-) diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 926af5a3d02e..f670466b84a8 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -90,6 +90,14 @@ typedef struct page *pgtable_t; #define PTE_FMT "%08lx" #endif +#ifdef CONFIG_64BIT +/* + * We override this value as its generic definition uses __pa too early in + * the boot process (before kernel_map.va_pa_offset is set). + */ +#define MIN_MEMBLOCK_ADDR 0 +#endif + #ifdef CONFIG_MMU #define ARCH_PFN_OFFSET (PFN_DOWN((unsigned long)phys_ram_base)) #else @@ -121,7 +129,11 @@ extern phys_addr_t phys_ram_base; #define is_linear_mapping(x) \ ((x) >= PAGE_OFFSET && (!IS_ENABLED(CONFIG_64BIT) || (x) < PAGE_OFFSET + KERN_VIRT_SIZE)) +#ifndef CONFIG_DEBUG_VIRTUAL #define linear_mapping_pa_to_va(x) ((void *)((unsigned long)(x) + kernel_map.va_pa_offset)) +#else +void *linear_mapping_pa_to_va(unsigned long x); +#endif #define kernel_mapping_pa_to_va(y) ({ \ unsigned long _y = (unsigned long)(y); \ (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < phys_ram_base) ? \ @@ -130,7 +142,11 @@ extern phys_addr_t phys_ram_base; }) #define __pa_to_va_nodebug(x) linear_mapping_pa_to_va(x) +#ifndef CONFIG_DEBUG_VIRTUAL #define linear_mapping_va_to_pa(x) ((unsigned long)(x) - kernel_map.va_pa_offset) +#else +phys_addr_t linear_mapping_va_to_pa(unsigned long x); +#endif #define kernel_mapping_va_to_pa(y) ({ \ unsigned long _y = (unsigned long)(y); \ (IS_ENABLED(CONFIG_XIP_KERNEL) && _y < kernel_map.virt_addr + XIP_OFFSET) ? \ diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index bef639fa330b..fb19c80ac0c4 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -213,6 +213,13 @@ static void __init setup_bootmem(void) phys_ram_end = memblock_end_of_DRAM(); if (!IS_ENABLED(CONFIG_XIP_KERNEL)) phys_ram_base = memblock_start_of_DRAM(); + + /* + * Any use of __va/__pa before this point is wrong as we did not know the + * start of DRAM before. + */ + kernel_map.va_pa_offset = PAGE_OFFSET - phys_ram_base; + /* * memblock allocator is not aware of the fact that last 4K bytes of * the addressable memory can not be mapped because of IS_ERR_VALUE @@ -667,9 +674,16 @@ void __init create_pgd_mapping(pgd_t *pgdp, static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) { - /* Upgrade to PMD_SIZE mappings whenever possible */ - base &= PMD_SIZE - 1; - if (!base && size >= PMD_SIZE) + if (!(base & (PGDIR_SIZE - 1)) && size >= PGDIR_SIZE) + return PGDIR_SIZE; + + if (!(base & (P4D_SIZE - 1)) && size >= P4D_SIZE) + return P4D_SIZE; + + if (!(base & (PUD_SIZE - 1)) && size >= PUD_SIZE) + return PUD_SIZE; + + if (!(base & (PMD_SIZE - 1)) && size >= PMD_SIZE) return PMD_SIZE; return PAGE_SIZE; @@ -1002,11 +1016,9 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) set_satp_mode(dtb_pa); #endif - kernel_map.va_pa_offset = PAGE_OFFSET - kernel_map.phys_addr; + kernel_map.va_pa_offset = 0UL; kernel_map.va_kernel_pa_offset = kernel_map.virt_addr - kernel_map.phys_addr; - phys_ram_base = kernel_map.phys_addr; - /* * The default maximal physical memory size is KERN_VIRT_SIZE for 32-bit * kernel, whereas for 64-bit kernel, the end of the virtual address diff --git a/arch/riscv/mm/physaddr.c b/arch/riscv/mm/physaddr.c index 9b18bda74154..18706f457da7 100644 --- a/arch/riscv/mm/physaddr.c +++ b/arch/riscv/mm/physaddr.c @@ -33,3 +33,19 @@ phys_addr_t __phys_addr_symbol(unsigned long x) return __va_to_pa_nodebug(x); } EXPORT_SYMBOL(__phys_addr_symbol); + +phys_addr_t linear_mapping_va_to_pa(unsigned long x) +{ + BUG_ON(!kernel_map.va_pa_offset); + + return ((unsigned long)(x) - kernel_map.va_pa_offset); +} +EXPORT_SYMBOL(linear_mapping_va_to_pa); + +void *linear_mapping_pa_to_va(unsigned long x) +{ + BUG_ON(!kernel_map.va_pa_offset); + + return ((void *)((unsigned long)(x) + kernel_map.va_pa_offset)); +} +EXPORT_SYMBOL(linear_mapping_pa_to_va); diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c index d1a68b6d03b3..d14735a81301 100644 --- a/drivers/of/fdt.c +++ b/drivers/of/fdt.c @@ -887,12 +887,13 @@ const void * __init of_flat_dt_match_machine(const void *default_match, static void __early_init_dt_declare_initrd(unsigned long start, unsigned long end) { - /* ARM64 would cause a BUG to occur here when CONFIG_DEBUG_VM is - * enabled since __va() is called too early. ARM64 does make use - * of phys_initrd_start/phys_initrd_size so we can skip this - * conversion. + /* + * __va() is not yet available this early on some platforms. In that + * case, the platform uses phys_initrd_start/phys_initrd_size instead + * and does the VA conversion itself. */ - if (!IS_ENABLED(CONFIG_ARM64)) { + if (!IS_ENABLED(CONFIG_ARM64) && + !(IS_ENABLED(CONFIG_RISCV) && IS_ENABLED(CONFIG_64BIT))) { initrd_start = (unsigned long)__va(start); initrd_end = (unsigned long)__va(end); initrd_below_start_ok = 1;