| Message ID | 20211215064559.2843555-2-peng.fan@oss.nxp.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/2] arm64: mm: apply __ro_after_init to memory_limit | expand |
On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) <peng.fan@oss.nxp.com> wrote: > > From: Peng Fan <peng.fan@nxp.com> > > There is a "mem=[x]" boot parameter, but when there is a whole reserved > by secure TEE, the continuous DRAM area is split with two memblocks. > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > [0x50000000, 0x51000000), the memblock will be split into > [0x40000000, 0x50000000) and [0x51000000, 0xffffffff]. > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > However if need the max addr be 0x80000000, mem=1008MB should be used. > > There also might be multiple other holes that no visible to Linux, when > we wanna to limit the max addr usable by Linux, using "max_addr=[X]" is > much easier than "mem=[X]" > > Signed-off-by: Peng Fan <peng.fan@nxp.com> mem= is a hack already, please don't add another one. Limiting the memory like this is far too tricky, given that the kernel itself and the initrd could end up in memory that is excluded, and we have to go and fix things up if that happens. > --- > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index db63cc885771..3364b5e7a7fe 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > EXPORT_SYMBOL(pfn_is_map_memory); > > static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > /* > * Limit the memory size that was specified via FDT. > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) > } > early_param("mem", early_mem); > > +static int __init set_max_addr(char *p) > +{ > + if (!p) > + return 1; > + > + max_addr = memparse(p, &p) & PAGE_MASK; > + pr_notice("Memory max addr set to 0x%llx\n", max_addr); > + > + return 0; > +} > +early_param("max_addr", set_max_addr); > + > void __init arm64_memblock_init(void) > { > s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); > @@ -253,6 +266,9 @@ void __init arm64_memblock_init(void) > memblock_add(__pa_symbol(_text), (u64)(_end - _text)); > } > > + if (max_addr != PHYS_ADDR_MAX) > + memblock_cap_memory_range(0, max_addr); > + > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && phys_initrd_size) { > /* > * Add back the memory we just removed if it results in the > @@ -427,4 +443,9 @@ void dump_mem_limit(void) > } else { > pr_emerg("Memory Limit: none\n"); > } > + > + if (max_addr != PHYS_ADDR_MAX) > + pr_emerg("Max addr: 0x%llx\n", max_addr); > + else > + pr_emerg("Max addr: none\n"); > } > -- > 2.25.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) <peng.fan@oss.nxp.com> > wrote: > > > > From: Peng Fan <peng.fan@nxp.com> > > > > There is a "mem=[x]" boot parameter, but when there is a whole > > reserved by secure TEE, the continuous DRAM area is split with two > memblocks. > > > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > > [0x50000000, 0x51000000), the memblock will be split into [0x40000000, > > 0x50000000) and [0x51000000, 0xffffffff]. > > > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > > However if need the max addr be 0x80000000, mem=1008MB should be > used. > > > > There also might be multiple other holes that no visible to Linux, > > when we wanna to limit the max addr usable by Linux, using > > "max_addr=[X]" is much easier than "mem=[X]" > > > > Signed-off-by: Peng Fan <peng.fan@nxp.com> > > mem= is a hack already, please don't add another one. Limiting the memory > like this is far too tricky, given that the kernel itself and the initrd could end up > in memory that is excluded, and we have to go and fix things up if that > happens. We wanna to use the reserved memory with request_mem_region, but with commit 86588296acbfb1 ("fdt: Properly handle "no-map" field in the memory region ") request_mem_region will fail, because the reserved memory are now as kernel memory. So we use "mem=X" to work around the issue, but "mem=X" is not user friendly compared with "max_addr=" when there are multiple holes used by others. Thanks, Peng. > > > > --- > > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > > 1 file changed, 21 insertions(+) > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index > > db63cc885771..3364b5e7a7fe 100644 > > --- a/arch/arm64/mm/init.c > > +++ b/arch/arm64/mm/init.c > > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > > EXPORT_SYMBOL(pfn_is_map_memory); > > > > static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; > > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > > > /* > > * Limit the memory size that was specified via FDT. > > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } > > early_param("mem", early_mem); > > > > +static int __init set_max_addr(char *p) { > > + if (!p) > > + return 1; > > + > > + max_addr = memparse(p, &p) & PAGE_MASK; > > + pr_notice("Memory max addr set to 0x%llx\n", max_addr); > > + > > + return 0; > > +} > > +early_param("max_addr", set_max_addr); > > + > > void __init arm64_memblock_init(void) { > > s64 linear_region_size = PAGE_END - > > _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init > arm64_memblock_init(void) > > memblock_add(__pa_symbol(_text), (u64)(_end - > _text)); > > } > > > > + if (max_addr != PHYS_ADDR_MAX) > > + memblock_cap_memory_range(0, max_addr); > > + > > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && phys_initrd_size) > { > > /* > > * Add back the memory we just removed if it results > > in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) > > } else { > > pr_emerg("Memory Limit: none\n"); > > } > > + > > + if (max_addr != PHYS_ADDR_MAX) > > + pr_emerg("Max addr: 0x%llx\n", max_addr); > > + else > > + pr_emerg("Max addr: none\n"); > > } > > -- > > 2.25.1 > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists > > .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm-kernel&data=04% > 7C0 > > > 1%7Cpeng.fan%40nxp.com%7C3ad0ef697ad64542556208d9bf9d1e1f%7C68 > 6ea1d3bc > > > 2b4c6fa92cd99c5c301635%7C0%7C0%7C637751503805222488%7CUnknow > n%7CTWFpbG > > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 > Mn0% > > > 3D%7C3000&sdata=iKVO4PUPnaRr%2B5gHcXxaaRxBt%2BK%2Fjytg8eQ > dCqgqh5o% > > 3D&reserved=0
On Wed, Dec 15, 2021 at 07:59:45AM +0000, Peng Fan wrote: > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) <peng.fan@oss.nxp.com> > > wrote: > > > > > > From: Peng Fan <peng.fan@nxp.com> > > > > > > There is a "mem=[x]" boot parameter, but when there is a whole > > > reserved by secure TEE, the continuous DRAM area is split with two > > memblocks. > > > > > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > > > [0x50000000, 0x51000000), the memblock will be split into [0x40000000, > > > 0x50000000) and [0x51000000, 0xffffffff]. > > > > > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > > > However if need the max addr be 0x80000000, mem=1008MB should be > > used. > > > > > > There also might be multiple other holes that no visible to Linux, > > > when we wanna to limit the max addr usable by Linux, using > > > "max_addr=[X]" is much easier than "mem=[X]" > > > > > > Signed-off-by: Peng Fan <peng.fan@nxp.com> > > > > mem= is a hack already, please don't add another one. Limiting the memory > > like this is far too tricky, given that the kernel itself and the initrd could end up > > in memory that is excluded, and we have to go and fix things up if that > > happens. > > We wanna to use the reserved memory with request_mem_region, but with > commit 86588296acbfb1 ("fdt: Properly handle "no-map" field in the memory region ") > > request_mem_region will fail, because the reserved memory are now as > kernel memory. request_mem_region() is for MMIO. Why do you want to use it for RAM? > So we use "mem=X" to work around the issue, but "mem=X" is not user friendly > compared with "max_addr=" when there are multiple holes used by others. > > Thanks, > Peng. > > > > > > > > --- > > > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > > > 1 file changed, 21 insertions(+) > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index > > > db63cc885771..3364b5e7a7fe 100644 > > > --- a/arch/arm64/mm/init.c > > > +++ b/arch/arm64/mm/init.c > > > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > > > EXPORT_SYMBOL(pfn_is_map_memory); > > > > > > static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; > > > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > > > > > /* > > > * Limit the memory size that was specified via FDT. > > > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } > > > early_param("mem", early_mem); > > > > > > +static int __init set_max_addr(char *p) { > > > + if (!p) > > > + return 1; > > > + > > > + max_addr = memparse(p, &p) & PAGE_MASK; > > > + pr_notice("Memory max addr set to 0x%llx\n", max_addr); > > > + > > > + return 0; > > > +} > > > +early_param("max_addr", set_max_addr); > > > + > > > void __init arm64_memblock_init(void) { > > > s64 linear_region_size = PAGE_END - > > > _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init > > arm64_memblock_init(void) > > > memblock_add(__pa_symbol(_text), (u64)(_end - > > _text)); > > > } > > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > + memblock_cap_memory_range(0, max_addr); > > > + > > > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && phys_initrd_size) > > { > > > /* > > > * Add back the memory we just removed if it results > > > in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) > > > } else { > > > pr_emerg("Memory Limit: none\n"); > > > } > > > + > > > + if (max_addr != PHYS_ADDR_MAX) > > > + pr_emerg("Max addr: 0x%llx\n", max_addr); > > > + else > > > + pr_emerg("Max addr: none\n"); > > > } > > > -- > > > 2.25.1 > > > > > > > > > _______________________________________________ > > > linux-arm-kernel mailing list > > > linux-arm-kernel@lists.infradead.org > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists > > > .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm-kernel&data=04% > > 7C0 > > > > > 1%7Cpeng.fan%40nxp.com%7C3ad0ef697ad64542556208d9bf9d1e1f%7C68 > > 6ea1d3bc > > > > > 2b4c6fa92cd99c5c301635%7C0%7C0%7C637751503805222488%7CUnknow > > n%7CTWFpbG > > > > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 > > Mn0% > > > > > 3D%7C3000&sdata=iKVO4PUPnaRr%2B5gHcXxaaRxBt%2BK%2Fjytg8eQ > > dCqgqh5o% > > > 3D&reserved=0
> Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > On Wed, Dec 15, 2021 at 07:59:45AM +0000, Peng Fan wrote: > > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > > > On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) <peng.fan@oss.nxp.com> > > > wrote: > > > > > > > > From: Peng Fan <peng.fan@nxp.com> > > > > > > > > There is a "mem=[x]" boot parameter, but when there is a whole > > > > reserved by secure TEE, the continuous DRAM area is split with two > > > memblocks. > > > > > > > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > > > > [0x50000000, 0x51000000), the memblock will be split into > > > > [0x40000000, > > > > 0x50000000) and [0x51000000, 0xffffffff]. > > > > > > > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > > > > However if need the max addr be 0x80000000, mem=1008MB should be > > > used. > > > > > > > > There also might be multiple other holes that no visible to Linux, > > > > when we wanna to limit the max addr usable by Linux, using > > > > "max_addr=[X]" is much easier than "mem=[X]" > > > > > > > > Signed-off-by: Peng Fan <peng.fan@nxp.com> > > > > > > mem= is a hack already, please don't add another one. Limiting the > > > memory like this is far too tricky, given that the kernel itself and > > > the initrd could end up in memory that is excluded, and we have to > > > go and fix things up if that happens. > > > > We wanna to use the reserved memory with request_mem_region, but with > > commit 86588296acbfb1 ("fdt: Properly handle "no-map" field in the > > memory region ") > > > > request_mem_region will fail, because the reserved memory are now as > > kernel memory. > > request_mem_region() is for MMIO. Why do you want to use it for RAM? + Jan, the jailhouse hypervisor owner. There is an out of tree driver https://github.com/siemens/jailhouse/blob/master/driver/main.c#L466 The hypervisor jailhouse is loaded after linux boot up, and the hypervisor bin file needs to be loaded into DRAM that reserved in our device tree with node with no map property. And the hypervisor use virtual pci for communication between VMs, The virtual pci use part of the reserved DRAM area as PCI MMIO space. Maybe I should use /memreserve, but not node with no-map property. Thanks, Peng. > > > So we use "mem=X" to work around the issue, but "mem=X" is not user > > friendly compared with "max_addr=" when there are multiple holes used by > others. > > > > Thanks, > > Peng. > > > > > > > > > > > > --- > > > > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > > > > 1 file changed, 21 insertions(+) > > > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index > > > > db63cc885771..3364b5e7a7fe 100644 > > > > --- a/arch/arm64/mm/init.c > > > > +++ b/arch/arm64/mm/init.c > > > > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > > > > EXPORT_SYMBOL(pfn_is_map_memory); > > > > > > > > static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; > > > > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > > > > > > > /* > > > > * Limit the memory size that was specified via FDT. > > > > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } > > > > early_param("mem", early_mem); > > > > > > > > +static int __init set_max_addr(char *p) { > > > > + if (!p) > > > > + return 1; > > > > + > > > > + max_addr = memparse(p, &p) & PAGE_MASK; > > > > + pr_notice("Memory max addr set to 0x%llx\n", max_addr); > > > > + > > > > + return 0; > > > > +} > > > > +early_param("max_addr", set_max_addr); > > > > + > > > > void __init arm64_memblock_init(void) { > > > > s64 linear_region_size = PAGE_END - > > > > _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init > > > arm64_memblock_init(void) > > > > memblock_add(__pa_symbol(_text), (u64)(_end - > > > _text)); > > > > } > > > > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > + memblock_cap_memory_range(0, max_addr); > > > > + > > > > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > phys_initrd_size) > > > { > > > > /* > > > > * Add back the memory we just removed if it > > > > results in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) > > > > } else { > > > > pr_emerg("Memory Limit: none\n"); > > > > } > > > > + > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > + pr_emerg("Max addr: 0x%llx\n", max_addr); > > > > + else > > > > + pr_emerg("Max addr: none\n"); > > > > } > > > > -- > > > > 2.25.1 > > > > > > > > > > > > _______________________________________________ > > > > linux-arm-kernel mailing list > > > > linux-arm-kernel@lists.infradead.org > > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fl > > > > ists > > > > .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm-kernel&data=0 > 4 > > > > % > > > 7C0 > > > > > > > > 1%7Cpeng.fan%40nxp.com%7C3ad0ef697ad64542556208d9bf9d1e1f%7C68 > > > 6ea1d3bc > > > > > > > > 2b4c6fa92cd99c5c301635%7C0%7C0%7C637751503805222488%7CUnknow > > > n%7CTWFpbG > > > > > > > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 > > > Mn0% > > > > > > > > 3D%7C3000&sdata=iKVO4PUPnaRr%2B5gHcXxaaRxBt%2BK%2Fjytg8eQ > > > dCqgqh5o% > > > > 3D&reserved=0 > > -- > Sincerely yours, > Mike.
On Wed, Dec 15, 2021 at 09:30:36AM +0000, Peng Fan wrote: > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > On Wed, Dec 15, 2021 at 07:59:45AM +0000, Peng Fan wrote: > > > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > > > > > On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) <peng.fan@oss.nxp.com> > > > > wrote: > > > > > > > > > > From: Peng Fan <peng.fan@nxp.com> > > > > > > > > > > There is a "mem=[x]" boot parameter, but when there is a whole > > > > > reserved by secure TEE, the continuous DRAM area is split with two > > > > memblocks. > > > > > > > > > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > > > > > [0x50000000, 0x51000000), the memblock will be split into > > > > > [0x40000000, > > > > > 0x50000000) and [0x51000000, 0xffffffff]. > > > > > > > > > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > > > > > However if need the max addr be 0x80000000, mem=1008MB should be > > > > used. > > > > > > > > > > There also might be multiple other holes that no visible to Linux, > > > > > when we wanna to limit the max addr usable by Linux, using > > > > > "max_addr=[X]" is much easier than "mem=[X]" > > > > > > > > > > Signed-off-by: Peng Fan <peng.fan@nxp.com> > > > > > > > > mem= is a hack already, please don't add another one. Limiting the > > > > memory like this is far too tricky, given that the kernel itself and > > > > the initrd could end up in memory that is excluded, and we have to > > > > go and fix things up if that happens. > > > > > > We wanna to use the reserved memory with request_mem_region, but with > > > commit 86588296acbfb1 ("fdt: Properly handle "no-map" field in the > > > memory region ") > > > > > > request_mem_region will fail, because the reserved memory are now as > > > kernel memory. > > > > request_mem_region() is for MMIO. Why do you want to use it for RAM? > > + Jan, the jailhouse hypervisor owner. > > There is an out of tree driver > https://github.com/siemens/jailhouse/blob/master/driver/main.c#L466 > > The hypervisor jailhouse is loaded after linux boot up, and the hypervisor > bin file needs to be loaded into DRAM that reserved in our device > tree with node with no map property. > > And the hypervisor use virtual pci for communication between VMs, > The virtual pci use part of the reserved DRAM area as PCI MMIO space. > > Maybe I should use /memreserve, but not node with no-map property. So, my understanding is that you need a chunk of memory that Linux does not use and does not map into the kernel page tables. In that case /memreserve + nomap in the device tree could be a better solution than mem=X. > Thanks, > Peng. > > > > > > So we use "mem=X" to work around the issue, but "mem=X" is not user > > > friendly compared with "max_addr=" when there are multiple holes used by > > others. > > > > > > Thanks, > > > Peng. > > > > > > > > > > > > > > > > --- > > > > > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > > > > > 1 file changed, 21 insertions(+) > > > > > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index > > > > > db63cc885771..3364b5e7a7fe 100644 > > > > > --- a/arch/arm64/mm/init.c > > > > > +++ b/arch/arm64/mm/init.c > > > > > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > > > > > EXPORT_SYMBOL(pfn_is_map_memory); > > > > > > > > > > static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; > > > > > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > > > > > > > > > /* > > > > > * Limit the memory size that was specified via FDT. > > > > > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } > > > > > early_param("mem", early_mem); > > > > > > > > > > +static int __init set_max_addr(char *p) { > > > > > + if (!p) > > > > > + return 1; > > > > > + > > > > > + max_addr = memparse(p, &p) & PAGE_MASK; > > > > > + pr_notice("Memory max addr set to 0x%llx\n", max_addr); > > > > > + > > > > > + return 0; > > > > > +} > > > > > +early_param("max_addr", set_max_addr); > > > > > + > > > > > void __init arm64_memblock_init(void) { > > > > > s64 linear_region_size = PAGE_END - > > > > > _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init > > > > arm64_memblock_init(void) > > > > > memblock_add(__pa_symbol(_text), (u64)(_end - > > > > _text)); > > > > > } > > > > > > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > > + memblock_cap_memory_range(0, max_addr); > > > > > + > > > > > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > > phys_initrd_size) > > > > { > > > > > /* > > > > > * Add back the memory we just removed if it > > > > > results in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) > > > > > } else { > > > > > pr_emerg("Memory Limit: none\n"); > > > > > } > > > > > + > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > > + pr_emerg("Max addr: 0x%llx\n", max_addr); > > > > > + else > > > > > + pr_emerg("Max addr: none\n"); > > > > > } > > > > > -- > > > > > 2.25.1 > > > > > > > > > > > > > > > _______________________________________________ > > > > > linux-arm-kernel mailing list > > > > > linux-arm-kernel@lists.infradead.org > > > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fl > > > > > ists > > > > > .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm-kernel&data=0 > > 4 > > > > > % > > > > 7C0 > > > > > > > > > > > 1%7Cpeng.fan%40nxp.com%7C3ad0ef697ad64542556208d9bf9d1e1f%7C68 > > > > 6ea1d3bc > > > > > > > > > > > 2b4c6fa92cd99c5c301635%7C0%7C0%7C637751503805222488%7CUnknow > > > > n%7CTWFpbG > > > > > > > > > > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 > > > > Mn0% > > > > > > > > > > > 3D%7C3000&sdata=iKVO4PUPnaRr%2B5gHcXxaaRxBt%2BK%2Fjytg8eQ > > > > dCqgqh5o% > > > > > 3D&reserved=0 > > > > -- > > Sincerely yours, > > Mike.
> Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > On Wed, Dec 15, 2021 at 09:30:36AM +0000, Peng Fan wrote: > > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > > > On Wed, Dec 15, 2021 at 07:59:45AM +0000, Peng Fan wrote: > > > > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > > > > > > > On Wed, 15 Dec 2021 at 07:56, Peng Fan (OSS) > > > > > <peng.fan@oss.nxp.com> > > > > > wrote: > > > > > > > > > > > > From: Peng Fan <peng.fan@nxp.com> > > > > > > > > > > > > There is a "mem=[x]" boot parameter, but when there is a whole > > > > > > reserved by secure TEE, the continuous DRAM area is split with > > > > > > two > > > > > memblocks. > > > > > > > > > > > > For example, DRAM area [0x40000000, 0xffffffff], when TEE uses > > > > > > [0x50000000, 0x51000000), the memblock will be split into > > > > > > [0x40000000, > > > > > > 0x50000000) and [0x51000000, 0xffffffff]. > > > > > > > > > > > > If pass "mem=1024MB", the actually max addr will be 0x81000000. > > > > > > However if need the max addr be 0x80000000, mem=1008MB > should > > > > > > be > > > > > used. > > > > > > > > > > > > There also might be multiple other holes that no visible to > > > > > > Linux, when we wanna to limit the max addr usable by Linux, > > > > > > using "max_addr=[X]" is much easier than "mem=[X]" > > > > > > > > > > > > Signed-off-by: Peng Fan <peng.fan@nxp.com> > > > > > > > > > > mem= is a hack already, please don't add another one. Limiting > > > > > the memory like this is far too tricky, given that the kernel > > > > > itself and the initrd could end up in memory that is excluded, > > > > > and we have to go and fix things up if that happens. > > > > > > > > We wanna to use the reserved memory with request_mem_region, but > > > > with commit 86588296acbfb1 ("fdt: Properly handle "no-map" field > > > > in the memory region ") > > > > > > > > request_mem_region will fail, because the reserved memory are now > > > > as kernel memory. > > > > > > request_mem_region() is for MMIO. Why do you want to use it for > RAM? > > > > + Jan, the jailhouse hypervisor owner. > > > > There is an out of tree driver > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > > > ub.com%2Fsiemens%2Fjailhouse%2Fblob%2Fmaster%2Fdriver%2Fmain.c%23 > L466& > > > amp;data=04%7C01%7Cpeng.fan%40nxp.com%7C0540c0c55ed24688f9f308d > 9bfb0b7 > > > 7d%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6377515879788 > 01324%7CU > > > nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI > 6Ik1ha > > > WwiLCJXVCI6Mn0%3D%7C3000&sdata=owLVa3T4jnTPvohHwdEnP2%2F > uMiUJmYeTc > > 1TwM0i9iCE%3D&reserved=0 > > > > The hypervisor jailhouse is loaded after linux boot up, and the > > hypervisor bin file needs to be loaded into DRAM that reserved in our > > device tree with node with no map property. > > > > And the hypervisor use virtual pci for communication between VMs, The > > virtual pci use part of the reserved DRAM area as PCI MMIO space. > > > > Maybe I should use /memreserve, but not node with no-map property. > > So, my understanding is that you need a chunk of memory that Linux does not > use and does not map into the kernel page tables. > In that case /memreserve + nomap in the device tree could be a better > solution than mem=X. nomap not work now since commit 86588296acbfb1 ("fdt: Properly handle "no-map" field in the memory region ") I need try /memreserve BTW, do you think max_addr would be an option be added to memblock common code mm/memblock.c? Thanks, Peng. > > > Thanks, > > Peng. > > > > > > > > > So we use "mem=X" to work around the issue, but "mem=X" is not > > > > user friendly compared with "max_addr=" when there are multiple > > > > holes used by > > > others. > > > > > > > > Thanks, > > > > Peng. > > > > > > > > > > > > > > > > > > > > --- > > > > > > arch/arm64/mm/init.c | 21 +++++++++++++++++++++ > > > > > > 1 file changed, 21 insertions(+) > > > > > > > > > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index > > > > > > db63cc885771..3364b5e7a7fe 100644 > > > > > > --- a/arch/arm64/mm/init.c > > > > > > +++ b/arch/arm64/mm/init.c > > > > > > @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) > > > > > > EXPORT_SYMBOL(pfn_is_map_memory); > > > > > > > > > > > > static phys_addr_t memory_limit __ro_after_init = > > > > > > PHYS_ADDR_MAX; > > > > > > +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; > > > > > > > > > > > > /* > > > > > > * Limit the memory size that was specified via FDT. > > > > > > @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } > > > > > > early_param("mem", early_mem); > > > > > > > > > > > > +static int __init set_max_addr(char *p) { > > > > > > + if (!p) > > > > > > + return 1; > > > > > > + > > > > > > + max_addr = memparse(p, &p) & PAGE_MASK; > > > > > > + pr_notice("Memory max addr set to 0x%llx\n", > > > > > > + max_addr); > > > > > > + > > > > > > + return 0; > > > > > > +} > > > > > > +early_param("max_addr", set_max_addr); > > > > > > + > > > > > > void __init arm64_memblock_init(void) { > > > > > > s64 linear_region_size = PAGE_END - > > > > > > _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init > > > > > arm64_memblock_init(void) > > > > > > memblock_add(__pa_symbol(_text), (u64)(_end > - > > > > > _text)); > > > > > > } > > > > > > > > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > > > + memblock_cap_memory_range(0, max_addr); > > > > > > + > > > > > > if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && > > > phys_initrd_size) > > > > > { > > > > > > /* > > > > > > * Add back the memory we just removed if it > > > > > > results in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) > > > > > > } else { > > > > > > pr_emerg("Memory Limit: none\n"); > > > > > > } > > > > > > + > > > > > > + if (max_addr != PHYS_ADDR_MAX) > > > > > > + pr_emerg("Max addr: 0x%llx\n", max_addr); > > > > > > + else > > > > > > + pr_emerg("Max addr: none\n"); > > > > > > } > > > > > > -- > > > > > > 2.25.1 > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > > linux-arm-kernel mailing list > > > > > > linux-arm-kernel@lists.infradead.org > > > > > > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F > > > > > > %2Fl > > > > > > ists > > > > > > .infradead.org%2Fmailman%2Flistinfo%2Flinux-arm-kernel&dat > > > > > > a=0 > > > 4 > > > > > > % > > > > > 7C0 > > > > > > > > > > > > > > > 1%7Cpeng.fan%40nxp.com%7C3ad0ef697ad64542556208d9bf9d1e1f%7C68 > > > > > 6ea1d3bc > > > > > > > > > > > > > > > 2b4c6fa92cd99c5c301635%7C0%7C0%7C637751503805222488%7CUnknow > > > > > n%7CTWFpbG > > > > > > > > > > > > > > > Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6 > > > > > Mn0% > > > > > > > > > > > > > > > 3D%7C3000&sdata=iKVO4PUPnaRr%2B5gHcXxaaRxBt%2BK%2Fjytg8eQ > > > > > dCqgqh5o% > > > > > > 3D&reserved=0 > > > > > > -- > > > Sincerely yours, > > > Mike. > > -- > Sincerely yours, > Mike.
On Wed, Dec 15, 2021 at 12:05:36PM +0000, Peng Fan wrote: > > Subject: Re: [PATCH 2/2] arm64: mm: support bootparam max_addr > > > > > > > > > If pass "mem=1024MB", the actually max addr will be > > > > > > > 0x81000000. However if need the max addr be 0x80000000, > > > > > > > mem=1008MB should be used. > > > > > > > > > BTW, do you think max_addr would be an option be added to memblock > common code mm/memblock.c? You have a working solution with mem=1008MB, I don't see a need for additional kernel parameter.
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index db63cc885771..3364b5e7a7fe 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -173,6 +173,7 @@ int pfn_is_map_memory(unsigned long pfn) EXPORT_SYMBOL(pfn_is_map_memory); static phys_addr_t memory_limit __ro_after_init = PHYS_ADDR_MAX; +static phys_addr_t max_addr __ro_after_init = PHYS_ADDR_MAX; /* * Limit the memory size that was specified via FDT. @@ -189,6 +190,18 @@ static int __init early_mem(char *p) } early_param("mem", early_mem); +static int __init set_max_addr(char *p) +{ + if (!p) + return 1; + + max_addr = memparse(p, &p) & PAGE_MASK; + pr_notice("Memory max addr set to 0x%llx\n", max_addr); + + return 0; +} +early_param("max_addr", set_max_addr); + void __init arm64_memblock_init(void) { s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); @@ -253,6 +266,9 @@ void __init arm64_memblock_init(void) memblock_add(__pa_symbol(_text), (u64)(_end - _text)); } + if (max_addr != PHYS_ADDR_MAX) + memblock_cap_memory_range(0, max_addr); + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && phys_initrd_size) { /* * Add back the memory we just removed if it results in the @@ -427,4 +443,9 @@ void dump_mem_limit(void) } else { pr_emerg("Memory Limit: none\n"); } + + if (max_addr != PHYS_ADDR_MAX) + pr_emerg("Max addr: 0x%llx\n", max_addr); + else + pr_emerg("Max addr: none\n"); }