mbox series

[RFC,00/30] Add support for Renesas RZ/Five RISC-V SoC

Message ID 20231220230646.219816-1-prabhakar.mahadev-lad.rj@bp.renesas.com (mailing list archive)
Headers show
Series Add support for Renesas RZ/Five RISC-V SoC | expand

Message

Lad Prabhakar Dec. 20, 2023, 11:06 p.m. UTC
Hi All,

The RZ/Five microprocessor includes a RISC-V CPU Core (AX45MP Single)
1.0 GHz, 16-bit DDR3L/DDR4 interface. And it also has many interfaces such
as Gbit-Ether, CAN, and USB 2.0, making it ideal for applications such as
entry-class social infrastructure gateway control and industrial gateway
control.

This patch series adds initial SoC DTSi support for Renesas RZ/Five
(R9A07G043) SoC and updates the bindings for the same. Below is the list
of IP blocks which will be supported by this series:
- ADC
- AX45MP CPU
- CANFD
- CPG
- DMAC
- Ethernet
- I2C
- OSTM
- PINCTRL
- PLIC
- RSPI
- SCIF
- SDHI/eMMC
- SSIF
- SYSC
- Thermal
- USB Host/Device
- WDT

Apart from first two patches all the patches have been cherry picked from
upstream kernel.

Sending this series as an RFC mainly beacuse, as the kernel image is not
generic to all RISC-V platforms as the HW is noncoherent.

Complete defconfig file for RZ/Five can be found at [0].

[0] https://gitlab.com/prabhakarlad/cip-kernel-config/-/raw/rzfive/6.1.y-cip/riscv/renesas_defconfig

Cheers,
Prabhakar

Arnd Bergmann (2):
  riscv: dma-mapping: only invalidate after DMA, not flush
  riscv: dma-mapping: skip invalidation before bidirectional DMA

Conor Dooley (1):
  riscv: dts: renesas: Clean up dtbs_check W=1 warning due to empty phy
    node

Jisheng Zhang (1):
  riscv: mm: mark noncoherent_supported as __ro_after_init

Lad Prabhakar (25):
  arm64: dts: renesas: rzg2ul-smarc: Drop including RZ/G2UL SoM DTSI
  arm64: dts: renesas: rzg2ul: Move PMOD_SCI0_EN macro to board DTS
  mmc: host: Kconfig: Make MMC_SDHI_INTERNAL_DMAC config option
    dependant on ARCH_RENESAS
  clocksource/drivers/riscv: Get rid of clocksource_arch_init() callback
  riscv: Kconfig: Enable cpufreq kconfig menu
  dt-bindings: riscv: Sort the CPU core list alphabetically
  dt-bindings: riscv: Add Andes AX45MP core to the list
  riscv: dma-mapping: switch over to generic implementation
  riscv: asm: vendorid_list: Add Andes Technology to the vendors list
  riscv: errata: Add Andes alternative ports
  riscv: mm: dma-noncoherent: nonstandard cache operations support
  dt-bindings: cache: andestech,ax45mp-cache: Add DT binding
    documentation for L2 cache controller
  cache: Add L2 cache management for Andes AX45MP RISC-V core
  soc: renesas: Kconfig: Select the required configs for RZ/Five SoC
  riscv: Kconfig.socs: Add ARCH_RENESAS kconfig option
  riscv: dts: renesas: Add initial devicetree for Renesas RZ/Five SoC
  riscv: dts: renesas: Add minimal DTS for Renesas RZ/Five SMARC EVK
  riscv: dts: renesas: r9a07g043f/rzfive-smarc-som: Enable
    ADC/OPP/Thermal Zones/TSU
  riscv: dts: renesas: rzfive-smarc: Enable CANFD/I2C
  riscv: dts: renesas: rzfive-smarc-som: Enable WDT
  riscv: dts: renesas: rzfive-smarc-som: Enable OSTM nodes
  riscv: dts: renesas: rzfive-smarc-som: Drop PHY interrupt support for
    ETH{0,1}
  riscv: dts: renesas: r9a07g043f: Add L2 cache node
  riscv: dts: renesas: r9a07g043f: Add dma-noncoherent property
  riscv: dts: renesas: rzfive-smarc: Enable the blocks which were
    explicitly disabled

Samuel Holland (1):
  clocksource/drivers/riscv: Increase the clock source rating

 .../cache/andestech,ax45mp-cache.yaml         |  81 +++++++
 .../devicetree/bindings/riscv/cpus.yaml       |  11 +-
 MAINTAINERS                                   |   7 +
 .../boot/dts/renesas/r9a07g043u11-smarc.dts   |   3 +
 arch/arm64/boot/dts/renesas/rzg2ul-smarc.dtsi |   4 -
 arch/riscv/Kconfig                            |  10 +-
 arch/riscv/Kconfig.erratas                    |  21 ++
 arch/riscv/Kconfig.socs                       |   5 +
 arch/riscv/boot/dts/Makefile                  |   1 +
 arch/riscv/boot/dts/renesas/Makefile          |   2 +
 arch/riscv/boot/dts/renesas/r9a07g043f.dtsi   |  72 ++++++
 .../boot/dts/renesas/r9a07g043f01-smarc.dts   |  27 +++
 .../boot/dts/renesas/rzfive-smarc-som.dtsi    |  24 ++
 arch/riscv/boot/dts/renesas/rzfive-smarc.dtsi |   8 +
 arch/riscv/errata/Makefile                    |   1 +
 arch/riscv/errata/andes/Makefile              |   1 +
 arch/riscv/errata/andes/errata.c              |  66 ++++++
 arch/riscv/include/asm/alternative.h          |   3 +
 arch/riscv/include/asm/dma-noncoherent.h      |  28 +++
 arch/riscv/include/asm/errata_list.h          |   5 +
 arch/riscv/include/asm/vendorid_list.h        |   1 +
 arch/riscv/kernel/alternative.c               |   5 +
 arch/riscv/kernel/time.c                      |   9 -
 arch/riscv/mm/dma-noncoherent.c               | 105 ++++++++-
 drivers/Kconfig                               |   2 +
 drivers/Makefile                              |   1 +
 drivers/cache/Kconfig                         |  11 +
 drivers/cache/Makefile                        |   3 +
 drivers/cache/ax45mp_cache.c                  | 213 ++++++++++++++++++
 drivers/clocksource/timer-riscv.c             |   7 +-
 drivers/mmc/host/Kconfig                      |   4 +-
 drivers/soc/renesas/Kconfig                   |   4 +
 32 files changed, 713 insertions(+), 32 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/cache/andestech,ax45mp-cache.yaml
 create mode 100644 arch/riscv/boot/dts/renesas/Makefile
 create mode 100644 arch/riscv/boot/dts/renesas/r9a07g043f.dtsi
 create mode 100644 arch/riscv/boot/dts/renesas/r9a07g043f01-smarc.dts
 create mode 100644 arch/riscv/boot/dts/renesas/rzfive-smarc-som.dtsi
 create mode 100644 arch/riscv/boot/dts/renesas/rzfive-smarc.dtsi
 create mode 100644 arch/riscv/errata/andes/Makefile
 create mode 100644 arch/riscv/errata/andes/errata.c
 create mode 100644 arch/riscv/include/asm/dma-noncoherent.h
 create mode 100644 drivers/cache/Kconfig
 create mode 100644 drivers/cache/Makefile
 create mode 100644 drivers/cache/ax45mp_cache.c

Comments

Pavel Machek Dec. 26, 2023, 6:30 p.m. UTC | #1
Hi!

Ok, so I went through the series, and there's nothing too
horrible. I'll have some comments sent to individual patches. I'm not
a great fan of 5/, as it touches generic code, but.

Testing did not reveal any breakage... which is not surprising as we
don't test r-v at the moment.

> Sending this series as an RFC mainly beacuse, as the kernel image is not
> generic to all RISC-V platforms as the HW is noncoherent.

I don't understand the "image is not generic". I see the hw is buggy
and needs the workarounds provided in the series, but kernel
configured for your hw should work elsewhere, too (slowly), no?

> Complete defconfig file for RZ/Five can be found at [0].

Best regards,
								Pavel
Pavel Machek Dec. 26, 2023, 6:38 p.m. UTC | #2
Hi!

> Sending this series as an RFC mainly beacuse, as the kernel image is not
> generic to all RISC-V platforms as the HW is noncoherent.

Does the hardware also have the "low memory does not work in
userspace" problem? How is that handled?

Best regards,
								Pavel
Lad Prabhakar Dec. 27, 2023, 3:40 p.m. UTC | #3
Hi Pavel,

Thank you for the review.

> 
> Hi!
> 
> Ok, so I went through the series, and there's nothing too horrible. I'll have some comments sent to
> individual patches. I'm not a great fan of 5/, as it touches generic code, but.
> 
> Testing did not reveal any breakage... which is not surprising as we don't test r-v at the moment.
> 
> > Sending this series as an RFC mainly beacuse, as the kernel image is
> > not generic to all RISC-V platforms as the HW is noncoherent.
> 
> I don't understand the "image is not generic". I see the hw is buggy and needs the workarounds
> provided in the series, but kernel configured for your hw should work elsewhere, too (slowly), no?
> 
For the noncoherent issue we carve out a chunk of memory from DDR (done by OpenSBI) as shared-dma-pool, below is the node passed to Linux
    reserved-memory {
        #address-cells = <2>;
        #size-cells = <2>;
        ranges;

        pma_resv0@58000000 {
            compatible = "shared-dma-pool";
            reg = <0x0 0x58000000 0x0 0x08000000>;
            no-map;
            linux,dma-default;
        };
    };

In Linux we enable DMA_GLOBAL_POOL config, so that all dma'ble memory requests happens only through this region. The DMA_GLOBAL_POOL config is compile time check rather than a runtime check [0] so for non-coherent platforms for example which want to use DMA_DIRECT_REMAP config the image built for RZ/Five wont work. 

[0] https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/tree/kernel/dma/direct.c?h=linux-6.1.y-cip#n239

Cheers,
Prabhakar

> > Complete defconfig file for RZ/Five can be found at [0].
> 
> Best regards,
> 								Pavel
> --
> DENX Software Engineering GmbH,        Managing Director: Erika Unter
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Lad Prabhakar Dec. 27, 2023, 4:18 p.m. UTC | #4
Hi,

> Hi!
> 
> > Sending this series as an RFC mainly beacuse, as the kernel image is
> > not generic to all RISC-V platforms as the HW is noncoherent.
> 
> Does the hardware also have the "low memory does not work in userspace" problem? How is that handled?
>
For the local memory issues (ILM/DLM (Instruction/Data local memory)) we will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built with this change [0] can be used on other RISC-V platforms.

https://github.com/prabhakarlad/meta-riscv/commit/da749379155980d1f906d8632432f35a01a9c9a1

Cheers,
Prabhakar
Jan Kiszka Jan. 2, 2024, 8:09 a.m. UTC | #5
On 27.12.23 17:18, Prabhakar Mahadev Lad wrote:
> Hi,
> 
>> Hi!
>>
>>> Sending this series as an RFC mainly beacuse, as the kernel image is
>>> not generic to all RISC-V platforms as the HW is noncoherent.
>>
>> Does the hardware also have the "low memory does not work in userspace" problem? How is that handled?
>>
> For the local memory issues (ILM/DLM (Instruction/Data local memory)) we will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built with this change [0] can be used on other RISC-V platforms.
> 
> https://github.com/prabhakarlad/meta-riscv/commit/da749379155980d1f906d8632432f35a01a9c9a1
> 

Just to emphasize this (again):

Debian is not Yocto, buildroot or Gentoo, and we are not done by
patching and rebuilding the toolchain. Unless you can also provide a
reasonably short list of packages that require rebuilding after that
patch is applied to binutils, it will not be practically feasible to
address this hardware design issue via something like isar-cip-core.

It would be better - not only having Debian but also other distros in
mind - to look for a kernel/firmware-based workaround, even if ugly and
potentially slower.

Jan
Chris Paterson Jan. 2, 2024, 2:14 p.m. UTC | #6
Hello Jan,

> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
> Behalf Of Jan Kiszka via lists.cip-project.org
> Sent: Tuesday, January 2, 2024 8:09 AM
> 
> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote:
> > Hi,
> >
> >> Hi!
> >>
> >>> Sending this series as an RFC mainly beacuse, as the kernel image is
> >>> not generic to all RISC-V platforms as the HW is noncoherent.
> >>
> >> Does the hardware also have the "low memory does not work in
> userspace" problem? How is that handled?
> >>
> > For the local memory issues (ILM/DLM (Instruction/Data local memory)) we
> will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built
> with this change [0] can be used on other RISC-V platforms.
> >
> >
> https://jpn01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithu
> b.com%2Fprabhakarlad%2Fmeta-
> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05
> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0
> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975
> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY
> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0
> >
> 
> Just to emphasize this (again):
> 
> Debian is not Yocto, buildroot or Gentoo, and we are not done by
> patching and rebuilding the toolchain. Unless you can also provide a
> reasonably short list of packages that require rebuilding after that
> patch is applied to binutils, it will not be practically feasible to
> address this hardware design issue via something like isar-cip-core.

Perhaps we can come up with a small list of packages, at least to get CIP core working.
If this is the case, would CIP want to compile these packages separately as part of isar-cip-core builds?
Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core build?

I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if this is still the case? 

So perhaps some options for now, in order:
1) Add support for RZ/Five to linux-cip and don't test at all 
2) Add support for RZ/Five to linux-cip and rely on Renesas' internal CI to test (similar to other non-official CIP reference platforms from Renesas)
3) Add support for RZ/Five to linux-cip and run limited testing with Renesas' Poky BSP
4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas packages specifically for RZ/Five, or
4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to compile binutils etc. specifically for RZ/Five

Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform.
Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform.

I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there are no objections from the CIP maintainers.

> 
> It would be better - not only having Debian but also other distros in
> mind - to look for a kernel/firmware-based workaround, even if ugly and
> potentially slower.

It probably would be better, however sadly we haven't been able to come up with such a workaround.

Kind regards, Chris

> 
> Jan
> 
> --
> Siemens AG, Technology
> Linux Expert Center
Jan Kiszka Jan. 2, 2024, 4:40 p.m. UTC | #7
On 02.01.24 15:14, Chris Paterson wrote:
> Hello Jan,
> 
>> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
>> Behalf Of Jan Kiszka via lists.cip-project.org
>> Sent: Tuesday, January 2, 2024 8:09 AM
>>
>> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote:
>>> Hi,
>>>
>>>> Hi!
>>>>
>>>>> Sending this series as an RFC mainly beacuse, as the kernel image is
>>>>> not generic to all RISC-V platforms as the HW is noncoherent.
>>>>
>>>> Does the hardware also have the "low memory does not work in
>> userspace" problem? How is that handled?
>>>>
>>> For the local memory issues (ILM/DLM (Instruction/Data local memory)) we
>> will need a similar patch [0] to cip-isar-core. Note the cip-isar-core image built
>> with this change [0] can be used on other RISC-V platforms.
>>>
>>>
>> https://githu
>> b.com%2Fprabhakarlad%2Fmeta-
>> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05
>> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0
>> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975
>> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY
>> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0
>>>
>>
>> Just to emphasize this (again):
>>
>> Debian is not Yocto, buildroot or Gentoo, and we are not done by
>> patching and rebuilding the toolchain. Unless you can also provide a
>> reasonably short list of packages that require rebuilding after that
>> patch is applied to binutils, it will not be practically feasible to
>> address this hardware design issue via something like isar-cip-core.
> 
> Perhaps we can come up with a small list of packages, at least to get CIP core working.
> If this is the case, would CIP want to compile these packages separately as part of isar-cip-core builds?
> Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core build?
> 

Depends on the length of the list and size (build-time) of the packages.

> I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if this is still the case? 

I worked around the issue for now, waiting for review:
https://groups.google.com/g/isar-users/c/0QBjhzcs3ac

Once isar accepted it, isar-cip-core will pick it up quickly.

> 
> So perhaps some options for now, in order:
> 1) Add support for RZ/Five to linux-cip and don't test at all 
> 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal CI to test (similar to other non-official CIP reference platforms from Renesas)
> 3) Add support for RZ/Five to linux-cip and run limited testing with Renesas' Poky BSP
> 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas packages specifically for RZ/Five, or
> 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to compile binutils etc. specifically for RZ/Five
> 
> Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform.
> Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform.
> 
> I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there are no objections from the CIP maintainers.
> 
>>
>> It would be better - not only having Debian but also other distros in
>> mind - to look for a kernel/firmware-based workaround, even if ugly and
>> potentially slower.
> 
> It probably would be better, however sadly we haven't been able to come up with such a workaround.

Is there information missing when an access to the invalid virtual
address happens, or is the implementation of a complete instruction
emulator not realistic?

Jan
Lad Prabhakar Jan. 2, 2024, 10:09 p.m. UTC | #8
Hi,

> On 02.01.24 15:14, Chris Paterson wrote:
> > Hello Jan,
> >
> >> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org>
> >> On Behalf Of Jan Kiszka via lists.cip-project.org
> >> Sent: Tuesday, January 2, 2024 8:09 AM
> >>
> >> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote:
> >>> Hi,
> >>>
> >>>> Hi!
> >>>>
> >>>>> Sending this series as an RFC mainly beacuse, as the kernel image
> >>>>> is not generic to all RISC-V platforms as the HW is noncoherent.
> >>>>
> >>>> Does the hardware also have the "low memory does not work in
> >> userspace" problem? How is that handled?
> >>>>
> >>> For the local memory issues (ILM/DLM (Instruction/Data local
> >>> memory)) we
> >> will need a similar patch [0] to cip-isar-core. Note the
> >> cip-isar-core image built with this change [0] can be used on other RISC-V platforms.
> >>>
> >>>
> >> https://githu
> >> b.com%2Fprabhakarlad%2Fmeta-
> >> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05
> >> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0
> >> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975
> >> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> >> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY
> >> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0
> >>>
> >>
> >> Just to emphasize this (again):
> >>
> >> Debian is not Yocto, buildroot or Gentoo, and we are not done by
> >> patching and rebuilding the toolchain. Unless you can also provide a
> >> reasonably short list of packages that require rebuilding after that
> >> patch is applied to binutils, it will not be practically feasible to
> >> address this hardware design issue via something like isar-cip-core.
> >
> > Perhaps we can come up with a small list of packages, at least to get CIP core working.
> > If this is the case, would CIP want to compile these packages separately as part of isar-cip-core
> builds?
> > Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core
> build?
> >
>
> Depends on the length of the list and size (build-time) of the packages.
>
> > I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if
> this is still the case?
>
> I worked around the issue for now, waiting for review:
> https://groups.google.com/g/isar-
> users%2Fc%2F0QBjhzcs3ac&data=05%7C02%7Cprabhakar.mahadev-
> lad.rj%40bp.renesas.com%7C5eea955a6b3e43edc6f108dc0bb17ec1%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%
> 7C638398104168344805%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
> CI6Mn0%3D%7C3000%7C%7C%7C&sdata=3BOCfdyyKWJJtw1KF21pMIaZpuEoZoOSEG4JXKd7oLc%3D&reserved=0
>
> Once isar accepted it, isar-cip-core will pick it up quickly.
>
> >
> > So perhaps some options for now, in order:
> > 1) Add support for RZ/Five to linux-cip and don't test at all
> > 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal
> > CI to test (similar to other non-official CIP reference platforms from
> > Renesas)
> > 3) Add support for RZ/Five to linux-cip and run limited testing with
> > Renesas' Poky BSP
> > 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils
> > etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas
> > packages specifically for RZ/Five, or
> > 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to
> > compile binutils etc. specifically for RZ/Five
> >
> > Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform.
> > Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform.
> >
> > I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there
> are no objections from the CIP maintainers.
> >
> >>
> >> It would be better - not only having Debian but also other distros in
> >> mind - to look for a kernel/firmware-based workaround, even if ugly
> >> and potentially slower.
> >
> > It probably would be better, however sadly we haven't been able to come up with such a workaround.
>
> Is there information missing when an access to the invalid virtual address happens, or is the
> implementation of a complete instruction emulator not realistic?
>
The virtual address here is valid, its just that it doesn't go through the cache/MMU therefore the virtual address matches physical address. (This happens for region 0x30000 - 0x4ffff hence we shifted to the start address to 0x50000 in binutils as pointed previously)

Can you please elaborate on "implementation of a complete instruction emulator not realistic?"

Cheers,
Prabhakar
Jan Kiszka Jan. 3, 2024, 10:20 a.m. UTC | #9
On 02.01.24 23:09, Prabhakar Mahadev Lad wrote:
> Hi,
> 
>> On 02.01.24 15:14, Chris Paterson wrote:
>>> Hello Jan,
>>>
>>>> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org>
>>>> On Behalf Of Jan Kiszka via lists.cip-project.org
>>>> Sent: Tuesday, January 2, 2024 8:09 AM
>>>>
>>>> On 27.12.23 17:18, Prabhakar Mahadev Lad wrote:
>>>>> Hi,
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>>> Sending this series as an RFC mainly beacuse, as the kernel image
>>>>>>> is not generic to all RISC-V platforms as the HW is noncoherent.
>>>>>>
>>>>>> Does the hardware also have the "low memory does not work in
>>>> userspace" problem? How is that handled?
>>>>>>
>>>>> For the local memory issues (ILM/DLM (Instruction/Data local
>>>>> memory)) we
>>>> will need a similar patch [0] to cip-isar-core. Note the
>>>> cip-isar-core image built with this change [0] can be used on other RISC-V platforms.
>>>>>
>>>>>
>>>> https://githu
>>>> b.com%2Fprabhakarlad%2Fmeta-
>>>> riscv%2Fcommit%2Fda749379155980d1f906d8632432f35a01a9c9a1&data=05
>>>> %7C02%7Cchris.paterson2%40renesas.com%7Ca5d360a701164419501808dc0
>>>> b6a1e2b%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%7C6383977975
>>>> 99811411%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
>>>> V2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dY
>>>> DnQGyt5JgcvbN2tCqYh7Ba6Tt8vEKooUjU0qBJ2do%3D&reserved=0
>>>>>
>>>>
>>>> Just to emphasize this (again):
>>>>
>>>> Debian is not Yocto, buildroot or Gentoo, and we are not done by
>>>> patching and rebuilding the toolchain. Unless you can also provide a
>>>> reasonably short list of packages that require rebuilding after that
>>>> patch is applied to binutils, it will not be practically feasible to
>>>> address this hardware design issue via something like isar-cip-core.
>>>
>>> Perhaps we can come up with a small list of packages, at least to get CIP core working.
>>> If this is the case, would CIP want to compile these packages separately as part of isar-cip-core
>> builds?
>>> Or would we (Renesas) need to compile them separately and pull them in during a normal isar-cip-core
>> build?
>>>
>>
>> Depends on the length of the list and size (build-time) of the packages.
>>
>>> I seem to remember you saying that RISC-V cross-compilation for Debian is broken. Do you know if
>> this is still the case?
>>
>> I worked around the issue for now, waiting for review:
>> https://groups.google.com/g/isar-
>> users%2Fc%2F0QBjhzcs3ac&data=05%7C02%7Cprabhakar.mahadev-
>> lad.rj%40bp.renesas.com%7C5eea955a6b3e43edc6f108dc0bb17ec1%7C53d82571da1947e49cb4625a166a4a2a%7C0%7C0%
>> 7C638398104168344805%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
>> CI6Mn0%3D%7C3000%7C%7C%7C&sdata=3BOCfdyyKWJJtw1KF21pMIaZpuEoZoOSEG4JXKd7oLc%3D&reserved=0
>>
>> Once isar accepted it, isar-cip-core will pick it up quickly.
>>
>>>
>>> So perhaps some options for now, in order:
>>> 1) Add support for RZ/Five to linux-cip and don't test at all
>>> 2) Add support for RZ/Five to linux-cip and rely on Renesas' internal
>>> CI to test (similar to other non-official CIP reference platforms from
>>> Renesas)
>>> 3) Add support for RZ/Five to linux-cip and run limited testing with
>>> Renesas' Poky BSP
>>> 4a) Add support for RZ/Five to linux-cip, Renesas compiles binutils
>>> etc. specifically for RZ/Five, isar-cip-core pulls in the Renesas
>>> packages specifically for RZ/Five, or
>>> 4b) Add support for RZ/Five to linux-cip, patch isar-cip-core to
>>> compile binutils etc. specifically for RZ/Five
>>>
>>> Options 1-3 would mean RZ/Five would _not_ become an official CIP reference platform.
>>> Option 4 _may_ open up the possibility for RZ/Five to become an official reference platform.
>>>
>>> I'd at least like to get 1-2, perhaps 3 working in the short term based on this RFC series, if there
>> are no objections from the CIP maintainers.
>>>
>>>>
>>>> It would be better - not only having Debian but also other distros in
>>>> mind - to look for a kernel/firmware-based workaround, even if ugly
>>>> and potentially slower.
>>>
>>> It probably would be better, however sadly we haven't been able to come up with such a workaround.
>>
>> Is there information missing when an access to the invalid virtual address happens, or is the
>> implementation of a complete instruction emulator not realistic?
>>
> The virtual address here is valid, its just that it doesn't go through the cache/MMU therefore the virtual address matches physical address. (This happens for region 0x30000 - 0x4ffff hence we shifted to the start address to 0x50000 in binutils as pointed previously)

So it's not valid for userspace because it can't be used in reality.

> 
> Can you please elaborate on "implementation of a complete instruction emulator not realistic?"

Given that the MMU is not respected for those magic virtual addresses,
the only way to permit userspace to use them is trap-and-emulate: If
userspace reads/writes from the virtual=physical address, emulate those
accesses. Same when it places code into that region. That's similar to
MMIO access emulation under KVM, rather more like early real mode
emulation in KVM on x86 CPUs that didn't support real mode in VMs. Look
at what linux/arch/x86/kvm/emulate.c has.

Jan