[v3,0/4] riscv: Separate vendor extensions from standard extensions

Message

Charlie Jenkins July 19, 2024, 4:15 p.m. UTC

All extensions, both standard and vendor, live in one struct
"riscv_isa_ext". There is currently one vendor extension, xandespmu, but
it is likely that more vendor extensions will be added to the kernel in
the future. As more vendor extensions (and standard extensions) are
added, riscv_isa_ext will become more bloated with a mix of vendor and
standard extensions.

This also allows each vendor to be conditionally enabled through
Kconfig.

---
This has been split out from the previous series that contained the
addition of xtheadvector due to lack of reviews. The xtheadvector
support will be posted again separately from this.

The alternative patching code from "riscv: Introduce vendor variants of
extension helpers" has been migrated to "riscv: Extend cpufeature.c to
detect vendor extensions" such that the andes patching still works in
that patch.

I also fix a bug in this patch from the previous series that the Andes
extensions were not being properly enabled due to the manual alternative
patching not incrementing the id to be greater than
RISCV_VENDOR_EXT_ALTERNATIVES_BASE. 

To: Paul Walmsley <[e-mail]>
To: Palmer Dabbelt <[e-mail]>
To: Albert Ou <[e-mail]>
To: Conor Dooley <[e-mail]>
To: Evan Green <[e-mail]>
To: Andy Chiu <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Signed-off-by: Charlie Jenkins <[e-mail]>

---
Changes in v3:
- Rebase onto for-next as there have been a lot of upstream changes!
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- Fixed issue in riscv_fill_vendor_ext_list() that initalizion was only
  happening properly for the first vendor. Add is_initialized field to
  riscv_isa_vendor_ext_data_list to allow intialization to be tracked on
  a per-vendor basis.
- Link to v1: https://lore.kernel.org/r/[e-mail]

---
Charlie Jenkins (4):
      riscv: Extend cpufeature.c to detect vendor extensions
      riscv: Add vendor extensions to /proc/cpuinfo
      riscv: Introduce vendor variants of extension helpers
      riscv: cpufeature: Extract common elements from extension checking

 arch/riscv/Kconfig                               |   2 +
 arch/riscv/Kconfig.vendor                        |  19 +++
 arch/riscv/errata/andes/errata.c                 |   3 +
 arch/riscv/errata/sifive/errata.c                |   3 +
 arch/riscv/errata/thead/errata.c                 |   3 +
 arch/riscv/include/asm/cpufeature.h              | 103 ++++++++++------
 arch/riscv/include/asm/hwcap.h                   |  25 ++--
 arch/riscv/include/asm/vendor_extensions.h       | 104 +++++++++++++++++
 arch/riscv/include/asm/vendor_extensions/andes.h |  19 +++
 arch/riscv/kernel/Makefile                       |   2 +
 arch/riscv/kernel/cpu.c                          |  35 +++++-
 arch/riscv/kernel/cpufeature.c                   | 143 ++++++++++++++++-------
 arch/riscv/kernel/vendor_extensions.c            |  56 +++++++++
 arch/riscv/kernel/vendor_extensions/Makefile     |   3 +
 arch/riscv/kernel/vendor_extensions/andes.c      |  18 +++
 drivers/perf/riscv_pmu_sbi.c                     |  11 +-
 16 files changed, 456 insertions(+), 93 deletions(-)
---
base-commit: 41c196e567fb1ea97f68a2ffb7faab451cd90854
change-id: 20240515-support_vendor_extensions-aa80120e4230

Comments

Andrew Jones July 22, 2024, 7:32 p.m. UTC | #1

On Fri, Jul 19, 2024 at 09:15:17AM GMT, Charlie Jenkins wrote:
> All extensions, both standard and vendor, live in one struct
> "riscv_isa_ext". There is currently one vendor extension, xandespmu, but
> it is likely that more vendor extensions will be added to the kernel in
> the future. As more vendor extensions (and standard extensions) are
> added, riscv_isa_ext will become more bloated with a mix of vendor and
> standard extensions.

But the mix doesn't hurt and with everything in one place it makes it easy
to know where to look.

> 
> This also allows each vendor to be conditionally enabled through
> Kconfig.

We can do that anyway by adding an extension menu for each vendor. If we
don't want a vendor's extensions bloating the array then we just need
some #ifdefs, e.g.

@@ -405,7 +405,9 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
        __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
        __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT),
        __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT),
+#ifdef RISCV_ISA_VENDOR_EXT_ANDES
        __RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_EXT_XANDESPMU),
+#endif
 };


So, I'm not convinced we want the additional complexity of vendor
extension arrays, but maybe I'm missing something.

Thanks,
drew

Charlie Jenkins July 22, 2024, 8:23 p.m. UTC | #2

On Mon, Jul 22, 2024 at 02:32:49PM -0500, Andrew Jones wrote:
> On Fri, Jul 19, 2024 at 09:15:17AM GMT, Charlie Jenkins wrote:
> > All extensions, both standard and vendor, live in one struct
> > "riscv_isa_ext". There is currently one vendor extension, xandespmu, but
> > it is likely that more vendor extensions will be added to the kernel in
> > the future. As more vendor extensions (and standard extensions) are
> > added, riscv_isa_ext will become more bloated with a mix of vendor and
> > standard extensions.
> 
> But the mix doesn't hurt and with everything in one place it makes it easy
> to know where to look.
> 
> > 
> > This also allows each vendor to be conditionally enabled through
> > Kconfig.
> 
> We can do that anyway by adding an extension menu for each vendor. If we
> don't want a vendor's extensions bloating the array then we just need
> some #ifdefs, e.g.
> 
> @@ -405,7 +405,9 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
>         __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
>         __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT),
>         __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT),
> +#ifdef RISCV_ISA_VENDOR_EXT_ANDES
>         __RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_EXT_XANDESPMU),
> +#endif
>  };
> 
> 
> So, I'm not convinced we want the additional complexity of vendor
> extension arrays, but maybe I'm missing something.
> 
> Thanks,
> drew

Vendor extensions are non-standard behavior, so this is largely an
organizational effort. A benefit this provides is to delegate
maintainership of vendor extensions to the vendor. Additionally, this
separation of vendor extensions prevents changes to vendor extensions
from affecting standard extensions and vice versa. 

Another motivation for this is that I expect vendor extensions to be
temporary fixes in a lot of cases. A good example is xtheadvector where
a previous version of the ratified vector spec was implemented. Another
case is vendors creating vendor extensions while waiting for RVI
ratification. This will cause these eventually-to-be-deprecated
extensions to pollute the namespace and require shuffling around of the
standard isa lists. When it's an internal structure it is less relevant,
but it is a bigger risk when it comes to hwprobe. Allowing these kinds
of vendor extensions into hwprobe runs the risk of causing a sparse key
space. We may end up with many dead bits for deprecated vendor
extensions that will never be able to be removed to maintain backward
compatibility. Having the vendor extensions split across vendors
streamlines the hwprobe implementation.

- Charlie

Palmer Dabbelt July 22, 2024, 11:45 p.m. UTC | #3

On Mon, 22 Jul 2024 13:23:48 PDT (-0700), Charlie Jenkins wrote:
> On Mon, Jul 22, 2024 at 02:32:49PM -0500, Andrew Jones wrote:
>> On Fri, Jul 19, 2024 at 09:15:17AM GMT, Charlie Jenkins wrote:
>> > All extensions, both standard and vendor, live in one struct
>> > "riscv_isa_ext". There is currently one vendor extension, xandespmu, but
>> > it is likely that more vendor extensions will be added to the kernel in
>> > the future. As more vendor extensions (and standard extensions) are
>> > added, riscv_isa_ext will become more bloated with a mix of vendor and
>> > standard extensions.
>>
>> But the mix doesn't hurt and with everything in one place it makes it easy
>> to know where to look.
>>
>> >
>> > This also allows each vendor to be conditionally enabled through
>> > Kconfig.
>>
>> We can do that anyway by adding an extension menu for each vendor. If we
>> don't want a vendor's extensions bloating the array then we just need
>> some #ifdefs, e.g.
>>
>> @@ -405,7 +405,9 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
>>         __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL),
>>         __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT),
>>         __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT),
>> +#ifdef RISCV_ISA_VENDOR_EXT_ANDES
>>         __RISCV_ISA_EXT_DATA(xandespmu, RISCV_ISA_EXT_XANDESPMU),
>> +#endif
>>  };
>>
>>
>> So, I'm not convinced we want the additional complexity of vendor
>> extension arrays, but maybe I'm missing something.
>>
>> Thanks,
>> drew
>
> Vendor extensions are non-standard behavior, so this is largely an
> organizational effort. A benefit this provides is to delegate
> maintainership of vendor extensions to the vendor. Additionally, this
> separation of vendor extensions prevents changes to vendor extensions
> from affecting standard extensions and vice versa.
>
> Another motivation for this is that I expect vendor extensions to be
> temporary fixes in a lot of cases. A good example is xtheadvector where
> a previous version of the ratified vector spec was implemented. Another
> case is vendors creating vendor extensions while waiting for RVI
> ratification. This will cause these eventually-to-be-deprecated
> extensions to pollute the namespace and require shuffling around of the
> standard isa lists. When it's an internal structure it is less relevant,
> but it is a bigger risk when it comes to hwprobe. Allowing these kinds
> of vendor extensions into hwprobe runs the risk of causing a sparse key
> space. We may end up with many dead bits for deprecated vendor
> extensions that will never be able to be removed to maintain backward
> compatibility. Having the vendor extensions split across vendors
> streamlines the hwprobe implementation.

IIRC this came up a few times.  I think it's hard to tell which will end 
up cleaner, but it's just an internal interface and it's already written 
this way.  So I'm fine going with this as-is, if it gets clunky we can 
always change later -- at a certain point trying to predict 
endor-specific stuff just leads to insanity, I'm sure some vendor will 
figure out something crazy enough regardless of what we do...

So I've got this on staging, assuming it builds it'll end up on for-next 
soon.

Thanks!

>
> - Charlie

[e-mail] July 23, 2024, 12:58 p.m. UTC | #4

Hello:

This series was applied to riscv/linux.git (for-next)
by Palmer Dabbelt <[e-mail]>:

On Fri, 19 Jul 2024 09:15:17 -0700 you wrote:
> All extensions, both standard and vendor, live in one struct
> "riscv_isa_ext". There is currently one vendor extension, xandespmu, but
> it is likely that more vendor extensions will be added to the kernel in
> the future. As more vendor extensions (and standard extensions) are
> added, riscv_isa_ext will become more bloated with a mix of vendor and
> standard extensions.
> 
> [...]

Here is the summary with links:
  - [v3,1/4] riscv: Extend cpufeature.c to detect vendor extensions
    https://git.kernel.org/riscv/c/23c996fc2bc1
  - [v3,2/4] riscv: Add vendor extensions to /proc/cpuinfo
    https://git.kernel.org/riscv/c/9448d9accdd8
  - [v3,3/4] riscv: Introduce vendor variants of extension helpers
    https://git.kernel.org/riscv/c/0f2425411101
  - [v3,4/4] riscv: cpufeature: Extract common elements from extension checking
    https://git.kernel.org/riscv/c/d4c8d79f5199

You are awesome, thank you!