Message ID | 20231220230646.219816-1-prabhakar.mahadev-lad.rj@bp.renesas.com (mailing list archive) |
---|---|
Headers | show |
Series | Add support for Renesas RZ/Five RISC-V SoC | expand |
Hi! Ok, so I went through the series, and there's nothing too horrible. I'll have some comments sent to individual patches. I'm not a great fan of 5/, as it touches generic code, but. Testing did not reveal any breakage... which is not surprising as we don't test r-v at the moment. > Sending this series as an RFC mainly beacuse, as the kernel image is not > generic to all RISC-V platforms as the HW is noncoherent. I don't understand the "image is not generic". I see the hw is buggy and needs the workarounds provided in the series, but kernel configured for your hw should work elsewhere, too (slowly), no? > Complete defconfig file for RZ/Five can be found at [0]. Best regards, Pavel
Hi! > Sending this series as an RFC mainly beacuse, as the kernel image is not > generic to all RISC-V platforms as the HW is noncoherent. Does the hardware also have the "low memory does not work in userspace" problem? How is that handled? Best regards, Pavel
Hi Pavel, Thank you for the review. > > Hi! > > Ok, so I went through the series, and there's nothing too horrible. I'll have some comments sent to > individual patches. I'm not a great fan of 5/, as it touches generic code, but. > > Testing did not reveal any breakage... which is not surprising as we don't test r-v at the moment. > > > Sending this series as an RFC mainly beacuse, as the kernel image is > > not generic to all RISC-V platforms as the HW is noncoherent. > > I don't understand the "image is not generic". I see the hw is buggy and needs the workarounds > provided in the series, but kernel configured for your hw should work elsewhere, too (slowly), no? > For the noncoherent issue we carve out a chunk of memory from DDR (done by OpenSBI) as shared-dma-pool, below is the node passed to Linux reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; pma_resv0@58000000 { compatible = "shared-dma-pool"; reg = <0x0 0x58000000 0x0 0x08000000>; no-map; linux,dma-default; }; }; In Linux we enable DMA_GLOBAL_POOL config, so that all dma'ble memory requests happens only through this region. The DMA_GLOBAL_POOL config is compile time check rather than a runtime check [0] so for non-coherent platforms for example which want to use DMA_DIRECT_REMAP config the image built for RZ/Five wont work. [0] https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/tree/kernel/dma/direct.c?h=linux-6.1.y-cip#n239 Cheers, Prabhakar > > Complete defconfig file for RZ/Five can be found at [0]. > > Best regards, > Pavel > -- > DENX Software Engineering GmbH, Managing Director: Erika Unter > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Hi, > Hi! > > > Sending this series as an RFC mainly beacuse, as the kernel image is > > not generic to all RISC-V platforms as the HW is noncoherent. > > Does the hardware also have the "low memory does not work in userspace" problem? How is that handled? > For the local memory issues (ILM/DLM (Instruction/Data local memory)) we will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built with this change [0] can be used on other RISC-V platforms. https://github.com/prabhakarlad/meta-riscv/commit/da749379155980d1f906d8632432f35a01a9c9a1 Cheers, Prabhakar
On 27.12.23 17:18, Prabhakar Mahadev Lad wrote: > Hi, > >> Hi! >> >>> Sending this series as an RFC mainly beacuse, as the kernel image is >>> not generic to all RISC-V platforms as the HW is noncoherent. >> >> Does the hardware also have the "low memory does not work in userspace" problem? How is that handled? >> > For the local memory issues (ILM/DLM (Instruction/Data local memory)) we will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built with this change [0] can be used on other RISC-V platforms. > > https://github.com/prabhakarlad/meta-riscv/commit/da749379155980d1f906d8632432f35a01a9c9a1 > Just to emphasize this (again): Debian is not Yocto, buildroot or Gentoo, and we are not done by patching and rebuilding the toolchain. Unless you can also provide a reasonably short list of packages that require rebuilding after that patch is applied to binutils, it will not be practically feasible to address this hardware design issue via something like isar-cip-core. It would be better - not only having Debian but also other distros in mind - to look for a kernel/firmware-based workaround, even if ugly and potentially slower. Jan
Hello Jan, > From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On > Behalf Of Jan Kiszka via lists.cip-project.org > Sent: Tuesday, January 2, 2024 8:09 AM > > On 27.12.23 17:18, Prabhakar Mahadev Lad wrote: > > Hi, > > > >> Hi! > >> > >>> Sending this series as an RFC mainly beacuse, as the kernel image is > >>> not generic to all RISC-V platforms as the HW is noncoherent. > >> > >> Does the hardware also have the "low memory does not work in > userspace" problem? How is that handled? > >> > > For the local memory issues (ILM/DLM (Instruction/Data local memory)) we > will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built > with this change [0] can be used on other RISC-V platforms. > > > > > https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu > b.com%2Fprabhakarlad%2Fmeta- > riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05 > %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0 > b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975 > 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY > DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0 > > > > Just to emphasize this (again): > > Debian is not Yocto, buildroot or Gentoo, and we are not done by > patching and rebuilding the toolchain. Unless you can also provide a > reasonably short list of packages that require rebuilding after that > patch is applied to binutils, it will not be practically feasible to > address this hardware design issue via something like isar-cip-core. Perhaps we can come up with a small list of packages, at least to get CIP core working. If this is the case, would CIP want to compile these packages separately as part of isar-cip-core builds? Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core build? I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if this is still the case? So perhaps some options for now, in order: 1) Add support for RZ/Five to linux-cip and don't test at all 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal CI to test (similar to other non-official CIP reference platforms from Renesas) 3) Add support for RZ/Five to linux-cip and run limited testing with Renesas' Poky BSP 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas packages specifically for RZ/Five, or 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to compile binutils etc. specifically for RZ/Five Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform. Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform. I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there are no objections from the CIP maintainers. > > It would be better - not only having Debian but also other distros in > mind - to look for a kernel/firmware-based workaround, even if ugly and > potentially slower. It probably would be better, however sadly we haven't been able to come up with such a workaround. Kind regards, Chris > > Jan > > -- > Siemens AG, Technology > Linux Expert Center
On 02.01.24 15:14, Chris Paterson wrote: > Hello Jan, > >> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On >> Behalf Of Jan Kiszka via lists.cip-project.org >> Sent: Tuesday, January 2, 2024 8:09 AM >> >> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote: >>> Hi, >>> >>>> Hi! >>>> >>>>> Sending this series as an RFC mainly beacuse, as the kernel image is >>>>> not generic to all RISC-V platforms as the HW is noncoherent. >>>> >>>> Does the hardware also have the "low memory does not work in >> userspace" problem? How is that handled? >>>> >>> For the local memory issues (ILM/DLM (Instruction/Data local memory)) we >> will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built >> with this change [0] can be used on other RISC-V platforms. >>> >>> >> https://githu >> b.com%2Fprabhakarlad%2Fmeta- >> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05 >> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0 >> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975 >> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi >> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY >> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0 >>> >> >> Just to emphasize this (again): >> >> Debian is not Yocto, buildroot or Gentoo, and we are not done by >> patching and rebuilding the toolchain. Unless you can also provide a >> reasonably short list of packages that require rebuilding after that >> patch is applied to binutils, it will not be practically feasible to >> address this hardware design issue via something like isar-cip-core. > > Perhaps we can come up with a small list of packages, at least to get CIP core working. > If this is the case, would CIP want to compile these packages separately as part of isar-cip-core builds? > Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core build? > Depends on the length of the list and size (build-time) of the packages. > I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if this is still the case? I worked around the issue for now, waiting for review: https://groups.google.com/g/isar-users/c/0QBjhzcs3ac Once isar accepted it, isar-cip-core will pick it up quickly. > > So perhaps some options for now, in order: > 1) Add support for RZ/Five to linux-cip and don't test at all > 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal CI to test (similar to other non-official CIP reference platforms from Renesas) > 3) Add support for RZ/Five to linux-cip and run limited testing with Renesas' Poky BSP > 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas packages specifically for RZ/Five, or > 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to compile binutils etc. specifically for RZ/Five > > Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform. > Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform. > > I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there are no objections from the CIP maintainers. > >> >> It would be better - not only having Debian but also other distros in >> mind - to look for a kernel/firmware-based workaround, even if ugly and >> potentially slower. > > It probably would be better, however sadly we haven't been able to come up with such a workaround. Is there information missing when an access to the invalid virtual address happens, or is the implementation of a complete instruction emulator not realistic? Jan
Hi, > On 02.01.24 15:14, Chris Paterson wrote: > > Hello Jan, > > > >> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> > >> On Behalf Of Jan Kiszka via lists.cip-project.org > >> Sent: Tuesday, January 2, 2024 8:09 AM > >> > >> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote: > >>> Hi, > >>> > >>>> Hi! > >>>> > >>>>> Sending this series as an RFC mainly beacuse, as the kernel image > >>>>> is not generic to all RISC-V platforms as the HW is noncoherent. > >>>> > >>>> Does the hardware also have the "low memory does not work in > >> userspace" problem? How is that handled? > >>>> > >>> For the local memory issues (ILM/DLM (Instruction/Data local > >>> memory)) we > >> will need a similar patch [0] to cip-isar-core. Note the > >> cip-isar-core image built with this change [0] can be used on other RISC-V platforms. > >>> > >>> > >> https://githu > >> b.com%2Fprabhakarlad%2Fmeta- > >> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05 > >> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0 > >> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975 > >> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi > >> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY > >> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0 > >>> > >> > >> Just to emphasize this (again): > >> > >> Debian is not Yocto, buildroot or Gentoo, and we are not done by > >> patching and rebuilding the toolchain. Unless you can also provide a > >> reasonably short list of packages that require rebuilding after that > >> patch is applied to binutils, it will not be practically feasible to > >> address this hardware design issue via something like isar-cip-core. > > > > Perhaps we can come up with a small list of packages, at least to get CIP core working. > > If this is the case, would CIP want to compile these packages separately as part of isar-cip-core > builds? > > Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core > build? > > > > Depends on the length of the list and size (build-time) of the packages. > > > I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if > this is still the case? > > I worked around the issue for now, waiting for review: > https://groups.google.com/g/isar- > users%2Fc%2F0QBjhzcs3ac&data=05%7C02%7Cprabhakar.mahadev- > lad.rj%40bp.renesas.com%7C5eea955a6b3e43edc6f108dc0bb17ec1%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0% > 7C638398104168344805%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV > CI6Mn0%3D%7C3000%7C%7C%7C&sdata=3BOCfdyyKWJJtw1KF21pMIaZpuEoZoOSEG4JXKd7oLc%3D&reserved=0 > > Once isar accepted it, isar-cip-core will pick it up quickly. > > > > > So perhaps some options for now, in order: > > 1) Add support for RZ/Five to linux-cip and don't test at all > > 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal > > CI to test (similar to other non-official CIP reference platforms from > > Renesas) > > 3) Add support for RZ/Five to linux-cip and run limited testing with > > Renesas' Poky BSP > > 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils > > etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas > > packages specifically for RZ/Five, or > > 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to > > compile binutils etc. specifically for RZ/Five > > > > Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform. > > Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform. > > > > I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there > are no objections from the CIP maintainers. > > > >> > >> It would be better - not only having Debian but also other distros in > >> mind - to look for a kernel/firmware-based workaround, even if ugly > >> and potentially slower. > > > > It probably would be better, however sadly we haven't been able to come up with such a workaround. > > Is there information missing when an access to the invalid virtual address happens, or is the > implementation of a complete instruction emulator not realistic? > The virtual address here is valid, its just that it doesn't go through the cache/MMU therefore the virtual address matches physical address. (This happens for region 0x30000 - 0x4ffff hence we shifted to the start address to 0x50000 in binutils as pointed previously) Can you please elaborate on "implementation of a complete instruction emulator not realistic?" Cheers, Prabhakar
On 02.01.24 23:09, Prabhakar Mahadev Lad wrote: > Hi, > >> On 02.01.24 15:14, Chris Paterson wrote: >>> Hello Jan, >>> >>>> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> >>>> On Behalf Of Jan Kiszka via lists.cip-project.org >>>> Sent: Tuesday, January 2, 2024 8:09 AM >>>> >>>> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote: >>>>> Hi, >>>>> >>>>>> Hi! >>>>>> >>>>>>> Sending this series as an RFC mainly beacuse, as the kernel image >>>>>>> is not generic to all RISC-V platforms as the HW is noncoherent. >>>>>> >>>>>> Does the hardware also have the "low memory does not work in >>>> userspace" problem? How is that handled? >>>>>> >>>>> For the local memory issues (ILM/DLM (Instruction/Data local >>>>> memory)) we >>>> will need a similar patch [0] to cip-isar-core. Note the >>>> cip-isar-core image built with this change [0] can be used on other RISC-V platforms. >>>>> >>>>> >>>> https://githu >>>> b.com%2Fprabhakarlad%2Fmeta- >>>> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05 >>>> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0 >>>> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975 >>>> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi >>>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY >>>> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0 >>>>> >>>> >>>> Just to emphasize this (again): >>>> >>>> Debian is not Yocto, buildroot or Gentoo, and we are not done by >>>> patching and rebuilding the toolchain. Unless you can also provide a >>>> reasonably short list of packages that require rebuilding after that >>>> patch is applied to binutils, it will not be practically feasible to >>>> address this hardware design issue via something like isar-cip-core. >>> >>> Perhaps we can come up with a small list of packages, at least to get CIP core working. >>> If this is the case, would CIP want to compile these packages separately as part of isar-cip-core >> builds? >>> Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core >> build? >>> >> >> Depends on the length of the list and size (build-time) of the packages. >> >>> I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if >> this is still the case? >> >> I worked around the issue for now, waiting for review: >> https://groups.google.com/g/isar- >> users%2Fc%2F0QBjhzcs3ac&data=05%7C02%7Cprabhakar.mahadev- >> lad.rj%40bp.renesas.com%7C5eea955a6b3e43edc6f108dc0bb17ec1%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0% >> 7C638398104168344805%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV >> CI6Mn0%3D%7C3000%7C%7C%7C&sdata=3BOCfdyyKWJJtw1KF21pMIaZpuEoZoOSEG4JXKd7oLc%3D&reserved=0 >> >> Once isar accepted it, isar-cip-core will pick it up quickly. >> >>> >>> So perhaps some options for now, in order: >>> 1) Add support for RZ/Five to linux-cip and don't test at all >>> 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal >>> CI to test (similar to other non-official CIP reference platforms from >>> Renesas) >>> 3) Add support for RZ/Five to linux-cip and run limited testing with >>> Renesas' Poky BSP >>> 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils >>> etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas >>> packages specifically for RZ/Five, or >>> 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to >>> compile binutils etc. specifically for RZ/Five >>> >>> Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform. >>> Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform. >>> >>> I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there >> are no objections from the CIP maintainers. >>> >>>> >>>> It would be better - not only having Debian but also other distros in >>>> mind - to look for a kernel/firmware-based workaround, even if ugly >>>> and potentially slower. >>> >>> It probably would be better, however sadly we haven't been able to come up with such a workaround. >> >> Is there information missing when an access to the invalid virtual address happens, or is the >> implementation of a complete instruction emulator not realistic? >> > The virtual address here is valid, its just that it doesn't go through the cache/MMU therefore the virtual address matches physical address. (This happens for region 0x30000 - 0x4ffff hence we shifted to the start address to 0x50000 in binutils as pointed previously) So it's not valid for userspace because it can't be used in reality. > > Can you please elaborate on "implementation of a complete instruction emulator not realistic?" Given that the MMU is not respected for those magic virtual addresses, the only way to permit userspace to use them is trap-and-emulate: If userspace reads/writes from the virtual=physical address, emulate those accesses. Same when it places code into that region. That's similar to MMIO access emulation under KVM, rather more like early real mode emulation in KVM on x86 CPUs that didn't support real mode in VMs. Look at what linux/arch/x86/kvm/emulate.c has. Jan