[v4,7/9] riscv: vector: adjust minimum Vector requirement to ZVE32X

Commit Message

Andy Chiu April 12, 2024, 6:49 a.m. UTC

Make has_vector take one argument. This argument represents the minimum
Vector subextension that the following Vector actions assume.

Also, change riscv_v_first_use_handler(), and boot code that calls
riscv_v_setup_vsize() to accept the minimum Vector sub-extension,
ZVE32X.

Most kernel/user interfaces requires minimum of ZVE32X. Thus, programs
compiled and run with ZVE32X should be supported by the kernel on most
aspects. This includes context-switch, signal, ptrace, prctl, and
hwprobe.

One exception is that ELF_HWCAP returns 'V' only if full V is supported
on the platform. This means that the system without a full V must not
rely on ELF_HWCAP to tell whether it is allowable to execute Vector
without first invoking a prctl() check.

Signed-off-by: Andy Chiu <[e-mail]>
Acked-by: Joel Granados <[e-mail]>
---
Changelog v4:
- check static_assert for !CONFIG_RISCV_ISA_V case in has_vector.
Changelog v2:
 - update the comment in hwprobe.
---
 arch/riscv/include/asm/switch_to.h     |  2 +-
 arch/riscv/include/asm/vector.h        | 25 ++++++++++++++++---------
 arch/riscv/include/asm/xor.h           |  2 +-
 arch/riscv/kernel/cpufeature.c         |  5 ++++-
 arch/riscv/kernel/kernel_mode_vector.c |  4 ++--
 arch/riscv/kernel/process.c            |  4 ++--
 arch/riscv/kernel/signal.c             |  6 +++---
 arch/riscv/kernel/smpboot.c            |  2 +-
 arch/riscv/kernel/sys_hwprobe.c        |  8 ++++++--
 arch/riscv/kernel/vector.c             | 15 +++++++++------
 arch/riscv/lib/uaccess.S               |  2 +-
 11 files changed, 46 insertions(+), 29 deletions(-)

Comments

Conor Dooley April 18, 2024, 11:02 a.m. UTC | #1

+CC Eric, Jerry

On Fri, Apr 12, 2024 at 02:49:03PM +0800, Andy Chiu wrote:
> Make has_vector take one argument. This argument represents the minimum
> Vector subextension that the following Vector actions assume.
> 
> Also, change riscv_v_first_use_handler(), and boot code that calls
> riscv_v_setup_vsize() to accept the minimum Vector sub-extension,
> ZVE32X.
> 
> Most kernel/user interfaces requires minimum of ZVE32X. Thus, programs
> compiled and run with ZVE32X should be supported by the kernel on most
> aspects. This includes context-switch, signal, ptrace, prctl, and
> hwprobe.
> 
> One exception is that ELF_HWCAP returns 'V' only if full V is supported
> on the platform. This means that the system without a full V must not
> rely on ELF_HWCAP to tell whether it is allowable to execute Vector
> without first invoking a prctl() check.
> 
> Signed-off-by: Andy Chiu <[e-mail]>
> Acked-by: Joel Granados <[e-mail]>

I'm not sure that I like this patch to be honest. As far as I can tell,
every user here of has_vector(ext) is ZVE32X, so why bother actually
having an argument?

Could we just document that has_vector() is just a tyre kick of "is
there a vector unit and are we allowed to use it", and anything
requiring more than the bare-minimum (so zve32x?)must explicitly check
for that form of vector using riscv_has_extension_[un]likely()?

Finally, the in-kernel crypto stuff or other things that use
can_use_simd() to check for vector support - do they all function correctly
with all of the vector flavours? I don't understand the vector
extensions well enough to evaluate that - I know that they do check for
the individual extensions like Zvkb during probe but don't have anything
for the vector version (at least in the chacha20 and sha256 glue code).
If they don't, then we need to make sure those drivers do not probe with
the cut-down variants.

Eric/Jerry (although read the previous paragraph too):
I noticed that the sha256 glue code calls crypto_simd_usable(), and in
turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
does not call either, which seems to violate the edict in
kernel_vector_begin()'s kerneldoc:
"Must not be called unless may_use_simd() returns true."

What am I missing there?

Cheers,
Conor.


> diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c
> index c8219b82fbfc..e7c3fcac62a1 100644
> --- a/arch/riscv/kernel/sys_hwprobe.c
> +++ b/arch/riscv/kernel/sys_hwprobe.c
> @@ -69,7 +69,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
>  	if (riscv_isa_extension_available(NULL, c))
>  		pair->value |= RISCV_HWPROBE_IMA_C;
>  
> -	if (has_vector())
> +	if (has_vector(v))
>  		pair->value |= RISCV_HWPROBE_IMA_V;
>  
>  	/*
> @@ -112,7 +112,11 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
>  		EXT_KEY(ZACAS);
>  		EXT_KEY(ZICOND);
>  
> -		if (has_vector()) {
> +		/*
> +		 *  Vector crypto and ZVE* extensions are supported only if
> +		 *  kernel has minimum V support of ZVE32X.
> +		 */
> +		if (has_vector(ZVE32X)) {
>  			EXT_KEY(ZVE32X);
>  			EXT_KEY(ZVE32F);
>  			EXT_KEY(ZVE64X);

I find this to be an indicate of the new has_vector() being a poor API,
as it is confusing that a check
> diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c
> index 6727d1d3b8f2..e8a47fa72351 100644
> --- a/arch/riscv/kernel/vector.c
> +++ b/arch/riscv/kernel/vector.c
> @@ -53,7 +53,7 @@ int riscv_v_setup_vsize(void)
>  
>  void __init riscv_v_setup_ctx_cache(void)
>  {
> -	if (!has_vector())
> +	if (!has_vector(ZVE32X))
>  		return;
>  
>  	riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx",
> @@ -173,8 +173,11 @@ bool riscv_v_first_use_handler(struct pt_regs *regs)
>  	u32 __user *epc = (u32 __user *)regs->epc;
>  	u32 insn = (u32)regs->badaddr;
>  
> +	if (!has_vector(ZVE32X))
> +		return false;
> +
>  	/* Do not handle if V is not supported, or disabled */
> -	if (!(ELF_HWCAP & COMPAT_HWCAP_ISA_V))
> +	if (!riscv_v_vstate_ctrl_user_allowed())
>  		return false;
>  
>  	/* If V has been enabled then it is not the first-use trap */
> @@ -213,7 +216,7 @@ void riscv_v_vstate_ctrl_init(struct task_struct *tsk)
>  	bool inherit;
>  	int cur, next;
>  
> -	if (!has_vector())
> +	if (!has_vector(ZVE32X))
>  		return;
>  
>  	next = riscv_v_ctrl_get_next(tsk);
> @@ -235,7 +238,7 @@ void riscv_v_vstate_ctrl_init(struct task_struct *tsk)
>  
>  long riscv_v_vstate_ctrl_get_current(void)
>  {
> -	if (!has_vector())
> +	if (!has_vector(ZVE32X))
>  		return -EINVAL;
>  
>  	return current->thread.vstate_ctrl & PR_RISCV_V_VSTATE_CTRL_MASK;
> @@ -246,7 +249,7 @@ long riscv_v_vstate_ctrl_set_current(unsigned long arg)
>  	bool inherit;
>  	int cur, next;
>  
> -	if (!has_vector())
> +	if (!has_vector(ZVE32X))
>  		return -EINVAL;
>  
>  	if (arg & ~PR_RISCV_V_VSTATE_CTRL_MASK)
> @@ -296,7 +299,7 @@ static struct ctl_table riscv_v_default_vstate_table[] = {
>  
>  static int __init riscv_v_sysctl_init(void)
>  {
> -	if (has_vector())
> +	if (has_vector(ZVE32X))
>  		if (!register_sysctl("abi", riscv_v_default_vstate_table))
>  			return -EINVAL;
>  	return 0;
> diff --git a/arch/riscv/lib/uaccess.S b/arch/riscv/lib/uaccess.S
> index bc22c078aba8..bbe143bb32a0 100644
> --- a/arch/riscv/lib/uaccess.S
> +++ b/arch/riscv/lib/uaccess.S
> @@ -14,7 +14,7 @@
>  
>  SYM_FUNC_START(__asm_copy_to_user)
>  #ifdef CONFIG_RISCV_ISA_V
> -	ALTERNATIVE("j fallback_scalar_usercopy", "nop", 0, RISCV_ISA_EXT_v, CONFIG_RISCV_ISA_V)
> +	ALTERNATIVE("j fallback_scalar_usercopy", "nop", 0, RISCV_ISA_EXT_ZVE32X, CONFIG_RISCV_ISA_V)
>  	REG_L	t0, riscv_v_usercopy_threshold
>  	bltu	a2, t0, fallback_scalar_usercopy
>  	tail enter_vector_usercopy
> 
> -- 
> 2.44.0.rc2
>

Eric Biggers April 18, 2024, 3:52 p.m. UTC | #2

Hi Conor,

On Thu, Apr 18, 2024 at 12:02:10PM +0100, Conor Dooley wrote:
> +CC Eric, Jerry
> 
> On Fri, Apr 12, 2024 at 02:49:03PM +0800, Andy Chiu wrote:
> > Make has_vector take one argument. This argument represents the minimum
> > Vector subextension that the following Vector actions assume.
> > 
> > Also, change riscv_v_first_use_handler(), and boot code that calls
> > riscv_v_setup_vsize() to accept the minimum Vector sub-extension,
> > ZVE32X.
> > 
> > Most kernel/user interfaces requires minimum of ZVE32X. Thus, programs
> > compiled and run with ZVE32X should be supported by the kernel on most
> > aspects. This includes context-switch, signal, ptrace, prctl, and
> > hwprobe.
> > 
> > One exception is that ELF_HWCAP returns 'V' only if full V is supported
> > on the platform. This means that the system without a full V must not
> > rely on ELF_HWCAP to tell whether it is allowable to execute Vector
> > without first invoking a prctl() check.
> > 
> > Signed-off-by: Andy Chiu <[e-mail]>
> > Acked-by: Joel Granados <[e-mail]>
> 
> I'm not sure that I like this patch to be honest. As far as I can tell,
> every user here of has_vector(ext) is ZVE32X, so why bother actually
> having an argument?
> 
> Could we just document that has_vector() is just a tyre kick of "is
> there a vector unit and are we allowed to use it", and anything
> requiring more than the bare-minimum (so zve32x?)must explicitly check
> for that form of vector using riscv_has_extension_[un]likely()?
> 
> Finally, the in-kernel crypto stuff or other things that use
> can_use_simd() to check for vector support - do they all function correctly
> with all of the vector flavours? I don't understand the vector
> extensions well enough to evaluate that - I know that they do check for
> the individual extensions like Zvkb during probe but don't have anything
> for the vector version (at least in the chacha20 and sha256 glue code).
> If they don't, then we need to make sure those drivers do not probe with
> the cut-down variants.

As far as I know, none of the RISC-V vector crypto code has been tested with
Zve* yet.  Currently it always checks for VLEN >= 128, which should exclude most
Zve* implementations.

Currently it doesn't check for EEW >= 64, even though it sometimes assumes that.
It looks like a check for EEW >= 64 needs to be added in order to exclude Zve32x
and Zve32f implementations that don't support EEW == 64.

If it would be useful to do so, we should be able to enable some of the code
with a smaller VLEN and/or EEW once it has been tested in those configurations.
Some of it should work, but some of it won't be able to work.  (For example, the
SHA512 instructions require EEW==64.)

Also note that currently all the RISC-V vector crypto code only supports riscv64
(XLEN=64).  Similarly, that could be relaxed in the future if people really need
the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
need to be revised and tested in that configuration.

> Eric/Jerry (although read the previous paragraph too):
> I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> does not call either, which seems to violate the edict in
> kernel_vector_begin()'s kerneldoc:
> "Must not be called unless may_use_simd() returns true."

skcipher algorithms can only be invoked in process and softirq context.  This
differs from shash algorithms which can be invoked in any context.

My understanding is that, like arm64, RISC-V always allows non-nested
kernel-mode vector to be used in process and softirq context -- and in fact,
this was intentionally done in order to support use cases like this.  So that's
why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
kernel_vector_begin().

Has that changed?  If so, why?

Some architectures like x86 do provide no-SIMD fallbacks for all skcipher
algorithms, but it's very annoying to do.  We were hoping to avoid that in
RISC-V.

- Eric

Conor Dooley April 18, 2024, 4:53 p.m. UTC | #3

On Thu, Apr 18, 2024 at 08:52:56AM -0700, Eric Biggers wrote:
> Hi Conor,
> 
> On Thu, Apr 18, 2024 at 12:02:10PM +0100, Conor Dooley wrote:
> > +CC Eric, Jerry
> > 
> > On Fri, Apr 12, 2024 at 02:49:03PM +0800, Andy Chiu wrote:
> > > Make has_vector take one argument. This argument represents the minimum
> > > Vector subextension that the following Vector actions assume.
> > > 
> > > Also, change riscv_v_first_use_handler(), and boot code that calls
> > > riscv_v_setup_vsize() to accept the minimum Vector sub-extension,
> > > ZVE32X.
> > > 
> > > Most kernel/user interfaces requires minimum of ZVE32X. Thus, programs
> > > compiled and run with ZVE32X should be supported by the kernel on most
> > > aspects. This includes context-switch, signal, ptrace, prctl, and
> > > hwprobe.
> > > 
> > > One exception is that ELF_HWCAP returns 'V' only if full V is supported
> > > on the platform. This means that the system without a full V must not
> > > rely on ELF_HWCAP to tell whether it is allowable to execute Vector
> > > without first invoking a prctl() check.
> > > 
> > > Signed-off-by: Andy Chiu <[e-mail]>
> > > Acked-by: Joel Granados <[e-mail]>
> > 
> > I'm not sure that I like this patch to be honest. As far as I can tell,
> > every user here of has_vector(ext) is ZVE32X, so why bother actually
> > having an argument?
> > 
> > Could we just document that has_vector() is just a tyre kick of "is
> > there a vector unit and are we allowed to use it", and anything
> > requiring more than the bare-minimum (so zve32x?)must explicitly check
> > for that form of vector using riscv_has_extension_[un]likely()?
> > 
> > Finally, the in-kernel crypto stuff or other things that use
> > can_use_simd() to check for vector support - do they all function correctly
> > with all of the vector flavours? I don't understand the vector
> > extensions well enough to evaluate that - I know that they do check for
> > the individual extensions like Zvkb during probe but don't have anything
> > for the vector version (at least in the chacha20 and sha256 glue code).
> > If they don't, then we need to make sure those drivers do not probe with
> > the cut-down variants.
> 
> As far as I know, none of the RISC-V vector crypto code has been tested with
> Zve* yet.  Currently it always checks for VLEN >= 128, which should exclude most
> Zve* implementations.
> 
> Currently it doesn't check for EEW >= 64, even though it sometimes assumes that.
> It looks like a check for EEW >= 64 needs to be added in order to exclude Zve32x
> and Zve32f implementations that don't support EEW == 64.

Cool, glad I asked then :)

> If it would be useful to do so, we should be able to enable some of the code
> with a smaller VLEN and/or EEW once it has been tested in those configurations.
> Some of it should work, but some of it won't be able to work.  (For example, the
> SHA512 instructions require EEW==64.)
> 
> Also note that currently all the RISC-V vector crypto code only supports riscv64
> (XLEN=64).  Similarly, that could be relaxed in the future if people really need
> the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
> need to be revised and tested in that configuration.
> 
> > Eric/Jerry (although read the previous paragraph too):
> > I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> > turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> > does not call either, which seems to violate the edict in
> > kernel_vector_begin()'s kerneldoc:
> > "Must not be called unless may_use_simd() returns true."
> 
> skcipher algorithms can only be invoked in process and softirq context.  This
> differs from shash algorithms which can be invoked in any context.
> 
> My understanding is that, like arm64, RISC-V always allows non-nested
> kernel-mode vector to be used in process and softirq context -- and in fact,
> this was intentionally done in order to support use cases like this.  So that's
> why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
> kernel_vector_begin().

I see, thanks for explaining that. I think you should probably check
somewhere if has_vector() returns true in that driver though before
using vector instructions. Only checking vlen seems to me like relying on
an implementation detail and if we set vlen for the T-Head/0.7.1 vector
it'd be fooled. That said, I don't think that any of the 0.7.1 vector
systems actually support Zvkb, but I hope you get my drift.

Thanks,
Conor.

Eric Biggers April 18, 2024, 5:32 p.m. UTC | #4

On Thu, Apr 18, 2024 at 05:53:55PM +0100, Conor Dooley wrote:
> > If it would be useful to do so, we should be able to enable some of the code
> > with a smaller VLEN and/or EEW once it has been tested in those configurations.
> > Some of it should work, but some of it won't be able to work.  (For example, the
> > SHA512 instructions require EEW==64.)
> > 
> > Also note that currently all the RISC-V vector crypto code only supports riscv64
> > (XLEN=64).  Similarly, that could be relaxed in the future if people really need
> > the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
> > need to be revised and tested in that configuration.
> > 
> > > Eric/Jerry (although read the previous paragraph too):
> > > I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> > > turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> > > does not call either, which seems to violate the edict in
> > > kernel_vector_begin()'s kerneldoc:
> > > "Must not be called unless may_use_simd() returns true."
> > 
> > skcipher algorithms can only be invoked in process and softirq context.  This
> > differs from shash algorithms which can be invoked in any context.
> > 
> > My understanding is that, like arm64, RISC-V always allows non-nested
> > kernel-mode vector to be used in process and softirq context -- and in fact,
> > this was intentionally done in order to support use cases like this.  So that's
> > why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
> > kernel_vector_begin().
> 
> I see, thanks for explaining that. I think you should probably check
> somewhere if has_vector() returns true in that driver though before
> using vector instructions. Only checking vlen seems to me like relying on
> an implementation detail and if we set vlen for the T-Head/0.7.1 vector
> it'd be fooled. That said, I don't think that any of the 0.7.1 vector
> systems actually support Zvkb, but I hope you get my drift.

All the algorithms check for at least one of the vector crypto extensions being
supported, for example Zvkb.  'if (riscv_isa_extension_available(NULL, ZVKB))'
should return whether the ratified version of Zvkb is supported, and likewise
for the other vector crypto extensions.  The ratified version of the vector
crypto extensions depends on the ratified version of the vector extension.  So
there should be no issue.  If there is, the RISC-V core architecture code needs
to be fixed to not declare that extensions are supported when they are actually
incompatible non-standard versions of those extensions.  Incompatible
non-standard extensions should be represented as separate extensions.

- Eric

Eric Biggers April 18, 2024, 5:39 p.m. UTC | #5

On Thu, Apr 18, 2024 at 10:32:03AM -0700, Eric Biggers wrote:
> On Thu, Apr 18, 2024 at 05:53:55PM +0100, Conor Dooley wrote:
> > > If it would be useful to do so, we should be able to enable some of the code
> > > with a smaller VLEN and/or EEW once it has been tested in those configurations.
> > > Some of it should work, but some of it won't be able to work.  (For example, the
> > > SHA512 instructions require EEW==64.)
> > > 
> > > Also note that currently all the RISC-V vector crypto code only supports riscv64
> > > (XLEN=64).  Similarly, that could be relaxed in the future if people really need
> > > the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
> > > need to be revised and tested in that configuration.
> > > 
> > > > Eric/Jerry (although read the previous paragraph too):
> > > > I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> > > > turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> > > > does not call either, which seems to violate the edict in
> > > > kernel_vector_begin()'s kerneldoc:
> > > > "Must not be called unless may_use_simd() returns true."
> > > 
> > > skcipher algorithms can only be invoked in process and softirq context.  This
> > > differs from shash algorithms which can be invoked in any context.
> > > 
> > > My understanding is that, like arm64, RISC-V always allows non-nested
> > > kernel-mode vector to be used in process and softirq context -- and in fact,
> > > this was intentionally done in order to support use cases like this.  So that's
> > > why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
> > > kernel_vector_begin().
> > 
> > I see, thanks for explaining that. I think you should probably check
> > somewhere if has_vector() returns true in that driver though before
> > using vector instructions. Only checking vlen seems to me like relying on
> > an implementation detail and if we set vlen for the T-Head/0.7.1 vector
> > it'd be fooled. That said, I don't think that any of the 0.7.1 vector
> > systems actually support Zvkb, but I hope you get my drift.
> 
> All the algorithms check for at least one of the vector crypto extensions being
> supported, for example Zvkb.  'if (riscv_isa_extension_available(NULL, ZVKB))'
> should return whether the ratified version of Zvkb is supported, and likewise
> for the other vector crypto extensions.  The ratified version of the vector
> crypto extensions depends on the ratified version of the vector extension.  So
> there should be no issue.  If there is, the RISC-V core architecture code needs
> to be fixed to not declare that extensions are supported when they are actually
> incompatible non-standard versions of those extensions.  Incompatible
> non-standard extensions should be represented as separate extensions.
> 

It probably makes sense to check has_vector() to exclude Zve* for now, though.

I am just concerned about how you're suggesting that non-standard extensions
might be pretending to be standard ones and individual users of kernel-mode
vector would need to work around that.  I think that neither has_vector() nor
'if (riscv_isa_extension_available(NULL, ZVKB))' should return true if the CPU's
vector extension is non-standard.

- Eric

Conor Dooley April 18, 2024, 6:26 p.m. UTC | #6

On Thu, Apr 18, 2024 at 10:39:46AM -0700, Eric Biggers wrote:
> On Thu, Apr 18, 2024 at 10:32:03AM -0700, Eric Biggers wrote:
> > On Thu, Apr 18, 2024 at 05:53:55PM +0100, Conor Dooley wrote:
> > > > If it would be useful to do so, we should be able to enable some of the code
> > > > with a smaller VLEN and/or EEW once it has been tested in those configurations.
> > > > Some of it should work, but some of it won't be able to work.  (For example, the
> > > > SHA512 instructions require EEW==64.)
> > > > 
> > > > Also note that currently all the RISC-V vector crypto code only supports riscv64
> > > > (XLEN=64).  Similarly, that could be relaxed in the future if people really need
> > > > the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
> > > > need to be revised and tested in that configuration.
> > > > 
> > > > > Eric/Jerry (although read the previous paragraph too):
> > > > > I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> > > > > turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> > > > > does not call either, which seems to violate the edict in
> > > > > kernel_vector_begin()'s kerneldoc:
> > > > > "Must not be called unless may_use_simd() returns true."
> > > > 
> > > > skcipher algorithms can only be invoked in process and softirq context.  This
> > > > differs from shash algorithms which can be invoked in any context.
> > > > 
> > > > My understanding is that, like arm64, RISC-V always allows non-nested
> > > > kernel-mode vector to be used in process and softirq context -- and in fact,
> > > > this was intentionally done in order to support use cases like this.  So that's
> > > > why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
> > > > kernel_vector_begin().
> > > 
> > > I see, thanks for explaining that. I think you should probably check
> > > somewhere if has_vector() returns true in that driver though before
> > > using vector instructions. Only checking vlen seems to me like relying on
> > > an implementation detail and if we set vlen for the T-Head/0.7.1 vector
> > > it'd be fooled. That said, I don't think that any of the 0.7.1 vector
> > > systems actually support Zvkb, but I hope you get my drift.
> > 
> > All the algorithms check for at least one of the vector crypto extensions being
> > supported, for example Zvkb.  'if (riscv_isa_extension_available(NULL, ZVKB))'
> > should return whether the ratified version of Zvkb is supported, and likewise
> > for the other vector crypto extensions.  The ratified version of the vector
> > crypto extensions depends on the ratified version of the vector extension.

That's great if it does require that the version of the vector extension
must be standard. Higher quality spec than most if it does. But
"supported" in the context of riscv_isa_extension_available() means that
the hardware supports it (or set of harts), not that the currently
running kernel does. The Kconfig deps that must be met for the code to be
built at least mean the kernel is built with vector support, leaving only
"the kernel was built with vector support and the hardware supports vector
but for $reason the kernel refused to enable it".

I'm not sure if that final condition is actually possible with the system
ending up in a broken state, however - I'm not sure that we ever do turn
off access to the VPU at present (after we mark it usable), and if we do
it doesn't get reflected in has_vector() so the kernel and userspace would
both break, with what a crypto driver does probably being the least of
your worries.

> > So
> > there should be no issue.  If there is, the RISC-V core architecture code needs
> > to be fixed to not declare that extensions are supported when they are actually
> > incompatible non-standard versions of those extensions.  Incompatible
> > non-standard extensions should be represented as separate extensions.
> > 
> 
> It probably makes sense to check has_vector() to exclude Zve* for now, though.

I think you might actually be better served at present, given the code can
only be built if the core vector code is, by using
riscv_isa_extension_available(NULL, v). That way you know for sure that
you're getting the ratified extension and nothing else.

Prior to this conversation I thought that has_vector() should return true
if there's a standard compliant vector unit available - given all users
Andy added only need Zve32x.

> I am just concerned about how you're suggesting that non-standard extensions
> might be pretending to be standard ones and individual users of kernel-mode
> vector would need to work around that.

I am absolutely not suggesting that non-standard extensions should
masquerade as standard ones, I don't know where you got that from. What
I said was that a non-standard vector extension could reuse riscv_v_vlen
(and should IMO for simplicity reasons), not that any of the APIs we have
for checking extension availability would lie and say it was standard.
riscv_v_vlen having a value greater than 128 is not one of those APIs ;)

> I think that neither has_vector() nor
> 'if (riscv_isa_extension_available(NULL, ZVKB))' should return true if the CPU's
> vector extension is non-standard.

riscv_isa_extension_available(NULL, ZVKB) only checks whether the extension
was present in DT or ACPI for all harts. It doesn't check whether or not
the required config option for vector has been set or anything related
to dependencies. has_vector() at least checks that the vector core has
been enabled (and uses the alternative-patched version of the check
given it is used in some hotter paths). That's kinda moot for code
that's only built if the vector core stuff is enabled as I said above
though.

We could of course make riscv_isa_extension_available() check
extension dependencies, but I'd rather leave dt validation to the dt
tooling (apparently ACPI tables are never wrong...). Either would allow
you to rely on the crypto extensions present only when the standard vector
extensions unless someone's DT/ACPI stuff is shite, but then they keep the
pieces IMO :)

Hope that makes sense?
Conor.

Conor Dooley April 18, 2024, 6:28 p.m. UTC | #7

On Thu, Apr 18, 2024 at 07:26:00PM +0100, Conor Dooley wrote:
> On Thu, Apr 18, 2024 at 10:39:46AM -0700, Eric Biggers wrote:
> > On Thu, Apr 18, 2024 at 10:32:03AM -0700, Eric Biggers wrote:
> > > On Thu, Apr 18, 2024 at 05:53:55PM +0100, Conor Dooley wrote:
> > > > > If it would be useful to do so, we should be able to enable some of the code
> > > > > with a smaller VLEN and/or EEW once it has been tested in those configurations.
> > > > > Some of it should work, but some of it won't be able to work.  (For example, the
> > > > > SHA512 instructions require EEW==64.)
> > > > > 
> > > > > Also note that currently all the RISC-V vector crypto code only supports riscv64
> > > > > (XLEN=64).  Similarly, that could be relaxed in the future if people really need
> > > > > the vector crypto acceleration on 32-bit CPUs...  But similarly, the code would
> > > > > need to be revised and tested in that configuration.
> > > > > 
> > > > > > Eric/Jerry (although read the previous paragraph too):
> > > > > > I noticed that the sha256 glue code calls crypto_simd_usable(), and in
> > > > > > turn may_use_simd() before kernel_vector_begin(). The chacha20 glue code
> > > > > > does not call either, which seems to violate the edict in
> > > > > > kernel_vector_begin()'s kerneldoc:
> > > > > > "Must not be called unless may_use_simd() returns true."
> > > > > 
> > > > > skcipher algorithms can only be invoked in process and softirq context.  This
> > > > > differs from shash algorithms which can be invoked in any context.
> > > > > 
> > > > > My understanding is that, like arm64, RISC-V always allows non-nested
> > > > > kernel-mode vector to be used in process and softirq context -- and in fact,
> > > > > this was intentionally done in order to support use cases like this.  So that's
> > > > > why the RISC-V skcipher algorithms don't check for may_use_simd() before calling
> > > > > kernel_vector_begin().
> > > > 
> > > > I see, thanks for explaining that. I think you should probably check
> > > > somewhere if has_vector() returns true in that driver though before
> > > > using vector instructions. Only checking vlen seems to me like relying on
> > > > an implementation detail and if we set vlen for the T-Head/0.7.1 vector
> > > > it'd be fooled. That said, I don't think that any of the 0.7.1 vector
> > > > systems actually support Zvkb, but I hope you get my drift.
> > > 
> > > All the algorithms check for at least one of the vector crypto extensions being
> > > supported, for example Zvkb.  'if (riscv_isa_extension_available(NULL, ZVKB))'
> > > should return whether the ratified version of Zvkb is supported, and likewise
> > > for the other vector crypto extensions.  The ratified version of the vector
> > > crypto extensions depends on the ratified version of the vector extension.
> 
> That's great if it does require that the version of the vector extension
> must be standard. Higher quality spec than most if it does. But
> "supported" in the context of riscv_isa_extension_available() means that
> the hardware supports it (or set of harts), not that the currently
> running kernel does. The Kconfig deps that must be met for the code to be
> built at least mean the kernel is built with vector support, leaving only
> "the kernel was built with vector support and the hardware supports vector
> but for $reason the kernel refused to enable it".
> 
> I'm not sure if that final condition is actually possible with the system
> ending up in a broken state, however - I'm not sure that we ever do turn
> off access to the VPU at present (after we mark it usable), and if we do
> it doesn't get reflected in has_vector() so the kernel and userspace would
> both break, with what a crypto driver does probably being the least of
> your worries.
> 
> > > So
> > > there should be no issue.  If there is, the RISC-V core architecture code needs
> > > to be fixed to not declare that extensions are supported when they are actually
> > > incompatible non-standard versions of those extensions.  Incompatible
> > > non-standard extensions should be represented as separate extensions.
> > > 
> > 
> > It probably makes sense to check has_vector() to exclude Zve* for now, though.
> 
> I think you might actually be better served at present, given the code can
> only be built if the core vector code is, by using
> riscv_isa_extension_available(NULL, v). That way you know for sure that
> you're getting the ratified extension and nothing else.

Poor choice of wording here - I meant, of course, the "main" vector
extension, rather than the Zve* variants.

> Prior to this conversation I thought that has_vector() should return true
> if there's a standard compliant vector unit available - given all users
> Andy added only need Zve32x.
> 
> > I am just concerned about how you're suggesting that non-standard extensions
> > might be pretending to be standard ones and individual users of kernel-mode
> > vector would need to work around that.
> 
> I am absolutely not suggesting that non-standard extensions should
> masquerade as standard ones, I don't know where you got that from. What
> I said was that a non-standard vector extension could reuse riscv_v_vlen
> (and should IMO for simplicity reasons), not that any of the APIs we have
> for checking extension availability would lie and say it was standard.
> riscv_v_vlen having a value greater than 128 is not one of those APIs ;)
> 
> > I think that neither has_vector() nor
> > 'if (riscv_isa_extension_available(NULL, ZVKB))' should return true if the CPU's
> > vector extension is non-standard.
> 
> riscv_isa_extension_available(NULL, ZVKB) only checks whether the extension
> was present in DT or ACPI for all harts. It doesn't check whether or not
> the required config option for vector has been set or anything related
> to dependencies. has_vector() at least checks that the vector core has
> been enabled (and uses the alternative-patched version of the check
> given it is used in some hotter paths). That's kinda moot for code
> that's only built if the vector core stuff is enabled as I said above
> though.
> 
> We could of course make riscv_isa_extension_available() check
> extension dependencies, but I'd rather leave dt validation to the dt
> tooling (apparently ACPI tables are never wrong...). Either would allow
> you to rely on the crypto extensions present only when the standard vector
> extensions unless someone's DT/ACPI stuff is shite, but then they keep the
> pieces IMO :)
> 
> Hope that makes sense?
> Conor.

Eric Biggers April 18, 2024, 6:41 p.m. UTC | #8

On Thu, Apr 18, 2024 at 07:26:00PM +0100, Conor Dooley wrote:
> That's great if it does require that the version of the vector extension
> must be standard. Higher quality spec than most if it does. But
> "supported" in the context of riscv_isa_extension_available() means that
> the hardware supports it (or set of harts), not that the currently
> running kernel does. The Kconfig deps that must be met for the code to be
> built at least mean the kernel is built with vector support, leaving only
> "the kernel was built with vector support and the hardware supports vector
> but for $reason the kernel refused to enable it".
> 
> I'm not sure if that final condition is actually possible with the system
> ending up in a broken state, however - I'm not sure that we ever do turn
> off access to the VPU at present (after we mark it usable), and if we do
> it doesn't get reflected in has_vector() so the kernel and userspace would
> both break, with what a crypto driver does probably being the least of
> your worries.
>
> > I am just concerned about how you're suggesting that non-standard extensions
> > might be pretending to be standard ones and individual users of kernel-mode
> > vector would need to work around that.
> 
> I am absolutely not suggesting that non-standard extensions should
> masquerade as standard ones, I don't know where you got that from. What
> I said was that a non-standard vector extension could reuse riscv_v_vlen
> (and should IMO for simplicity reasons), not that any of the APIs we have
> for checking extension availability would lie and say it was standard.
> riscv_v_vlen having a value greater than 128 is not one of those APIs ;)

It sounded like you were suggesting that a CPU could plausibly have a
pre-standard version of the vector extension but also have standard Zvkb.  I
don't think this makes sense, due to the dependency.

> > I think that neither has_vector() nor
> > 'if (riscv_isa_extension_available(NULL, ZVKB))' should return true if the CPU's
> > vector extension is non-standard.
> 
> riscv_isa_extension_available(NULL, ZVKB) only checks whether the extension
> was present in DT or ACPI for all harts. It doesn't check whether or not
> the required config option for vector has been set or anything related
> to dependencies. has_vector() at least checks that the vector core has
> been enabled (and uses the alternative-patched version of the check
> given it is used in some hotter paths). That's kinda moot for code
> that's only built if the vector core stuff is enabled as I said above
> though.
> 
> We could of course make riscv_isa_extension_available() check
> extension dependencies, but I'd rather leave dt validation to the dt
> tooling (apparently ACPI tables are never wrong...). Either would allow
> you to rely on the crypto extensions present only when the standard vector
> extensions unless someone's DT/ACPI stuff is shite, but then they keep the
> pieces IMO :)
> 
> Hope that makes sense?

If the RISC-V kernel ever disables V, then it should also disable everything
that depends on V.

This would be similar to how on x86, if the kernel decides to disable AVX to
mitigate the Gather Data Sampling vulnerability, it also disables AVX2, AVX512,
VAES, VPCLMULQDQ, etc.  See cpuid_deps[] in arch/x86/kernel/cpu/cpuid-deps.c.

Sometimes CPU features depend on other ones.  That's just the way things work.
Whenever possible that should be handled centrally, not pushed down to every
user both in-kernel and userspace.

- Eric

Conor Dooley April 18, 2024, 8 p.m. UTC | #9

On Thu, Apr 18, 2024 at 11:41:29AM -0700, Eric Biggers wrote:

> If the RISC-V kernel ever disables V, then it should also disable everything
> that depends on V.
> 
> This would be similar to how on x86, if the kernel decides to disable AVX to
> mitigate the Gather Data Sampling vulnerability, it also disables AVX2, AVX512,
> VAES, VPCLMULQDQ, etc.  See cpuid_deps[] in arch/x86/kernel/cpu/cpuid-deps.c.
> 
> Sometimes CPU features depend on other ones.  That's just the way things work.

Patch

diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h
index 7efdb0584d47..df1adf196c4f 100644
--- a/arch/riscv/include/asm/switch_to.h
+++ b/arch/riscv/include/asm/switch_to.h
@@ -78,7 +78,7 @@  do {							\
 	struct task_struct *__next = (next);		\
 	if (has_fpu())					\
 		__switch_to_fpu(__prev, __next);	\
-	if (has_vector())					\
+	if (has_vector(ZVE32X))			\
 		__switch_to_vector(__prev, __next);	\
 	((last) = __switch_to(__prev, __next));		\
 } while (0)
diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h
index 731dcd0ed4de..ed5fb6515d54 100644
--- a/arch/riscv/include/asm/vector.h
+++ b/arch/riscv/include/asm/vector.h
@@ -8,6 +8,19 @@ 
 
 #include <linux/types.h>
 #include <uapi/asm-generic/errno.h>
+#include <asm/cpufeature.h>
+#include <asm/hwcap.h>
+
+#define has_vector(VEXT)									\
+({												\
+	static_assert(RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_ZVE32X ||				\
+		      RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_ZVE32F ||				\
+		      RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_ZVE64X ||				\
+		      RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_ZVE64F ||				\
+		      RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_ZVE64D ||				\
+		      RISCV_ISA_EXT_##VEXT == RISCV_ISA_EXT_v);					\
+	IS_ENABLED(CONFIG_RISCV_ISA_V) && riscv_has_extension_unlikely(RISCV_ISA_EXT_##VEXT);	\
+})
 
 #ifdef CONFIG_RISCV_ISA_V
 
@@ -15,9 +28,9 @@ 
 #include <linux/sched.h>
 #include <linux/sched/task_stack.h>
 #include <asm/ptrace.h>
-#include <asm/cpufeature.h>
 #include <asm/csr.h>
 #include <asm/asm.h>
+#include <asm/bug.h>
 
 extern unsigned long riscv_v_vsize;
 int riscv_v_setup_vsize(void);
@@ -35,11 +48,6 @@  static inline u32 riscv_v_flags(void)
 	return READ_ONCE(current->thread.riscv_v_flags);
 }
 
-static __always_inline bool has_vector(void)
-{
-	return riscv_has_extension_unlikely(RISCV_ISA_EXT_v);
-}
-
 static inline void __riscv_v_vstate_clean(struct pt_regs *regs)
 {
 	regs->status = (regs->status & ~SR_VS) | SR_VS_CLEAN;
@@ -131,7 +139,7 @@  static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_
 	riscv_v_enable();
 	asm volatile (
 		".option push\n\t"
-		".option arch, +v\n\t"
+		".option arch, +zve32x\n\t"
 		"vsetvli	%0, x0, e8, m8, ta, ma\n\t"
 		"vle8.v		v0, (%1)\n\t"
 		"add		%1, %1, %0\n\t"
@@ -153,7 +161,7 @@  static inline void __riscv_v_vstate_discard(void)
 	riscv_v_enable();
 	asm volatile (
 		".option push\n\t"
-		".option arch, +v\n\t"
+		".option arch, +zve32x\n\t"
 		"vsetvli	%0, x0, e8, m8, ta, ma\n\t"
 		"vmv.v.i	v0, -1\n\t"
 		"vmv.v.i	v8, -1\n\t"
@@ -267,7 +275,6 @@  bool riscv_v_vstate_ctrl_user_allowed(void);
 struct pt_regs;
 
 static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; }
-static __always_inline bool has_vector(void) { return false; }
 static inline bool riscv_v_first_use_handler(struct pt_regs *regs) { return false; }
 static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return false; }
 static inline bool riscv_v_vstate_ctrl_user_allowed(void) { return false; }
diff --git a/arch/riscv/include/asm/xor.h b/arch/riscv/include/asm/xor.h
index 96011861e46b..46042ef5a2f7 100644
--- a/arch/riscv/include/asm/xor.h
+++ b/arch/riscv/include/asm/xor.h
@@ -61,7 +61,7 @@  static struct xor_block_template xor_block_rvv = {
 	do {        \
 		xor_speed(&xor_block_8regs);    \
 		xor_speed(&xor_block_32regs);    \
-		if (has_vector()) { \
+		if (has_vector(ZVE32X)) { \
 			xor_speed(&xor_block_rvv);\
 		} \
 	} while (0)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 38d09de518b1..8b52060649d2 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -715,12 +715,15 @@  void __init riscv_fill_hwcap(void)
 		elf_hwcap &= ~COMPAT_HWCAP_ISA_F;
 	}
 
-	if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
+	if (__riscv_isa_extension_available(NULL, RISCV_ISA_EXT_ZVE32X)) {
 		/*
 		 * This callsite can't fail here. It cannot fail when called on
 		 * the boot hart.
 		 */
 		riscv_v_setup_vsize();
+	}
+
+	if (elf_hwcap & COMPAT_HWCAP_ISA_V) {
 		/*
 		 * ISA string in device tree might have 'v' flag, but
 		 * CONFIG_RISCV_ISA_V is disabled in kernel.
diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c
index 6afe80c7f03a..0d4d1a03d1c7 100644
--- a/arch/riscv/kernel/kernel_mode_vector.c
+++ b/arch/riscv/kernel/kernel_mode_vector.c
@@ -208,7 +208,7 @@  void kernel_vector_begin(void)
 {
 	bool nested = false;
 
-	if (WARN_ON(!has_vector()))
+	if (WARN_ON(!has_vector(ZVE32X)))
 		return;
 
 	BUG_ON(!may_use_simd());
@@ -236,7 +236,7 @@  EXPORT_SYMBOL_GPL(kernel_vector_begin);
  */
 void kernel_vector_end(void)
 {
-	if (WARN_ON(!has_vector()))
+	if (WARN_ON(!has_vector(ZVE32X)))
 		return;
 
 	riscv_v_disable();
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 92922dbd5b5c..919e72f9fff6 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -178,7 +178,7 @@  void flush_thread(void)
 void arch_release_task_struct(struct task_struct *tsk)
 {
 	/* Free the vector context of datap. */
-	if (has_vector())
+	if (has_vector(ZVE32X))
 		riscv_v_thread_free(tsk);
 }
 
@@ -225,7 +225,7 @@  int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
 		p->thread.s[0] = 0;
 	}
 	p->thread.riscv_v_flags = 0;
-	if (has_vector())
+	if (has_vector(ZVE32X))
 		riscv_v_thread_alloc(p);
 	p->thread.ra = (unsigned long)ret_from_fork;
 	p->thread.sp = (unsigned long)childregs; /* kernel sp */
diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c
index 501e66debf69..a96e6e969a3f 100644
--- a/arch/riscv/kernel/signal.c
+++ b/arch/riscv/kernel/signal.c
@@ -188,7 +188,7 @@  static long restore_sigcontext(struct pt_regs *regs,
 
 			return 0;
 		case RISCV_V_MAGIC:
-			if (!has_vector() || !riscv_v_vstate_query(regs) ||
+			if (!has_vector(ZVE32X) || !riscv_v_vstate_query(regs) ||
 			    size != riscv_v_sc_size)
 				return -EINVAL;
 
@@ -210,7 +210,7 @@  static size_t get_rt_frame_size(bool cal_all)
 
 	frame_size = sizeof(*frame);
 
-	if (has_vector()) {
+	if (has_vector(ZVE32X)) {
 		if (cal_all || riscv_v_vstate_query(task_pt_regs(current)))
 			total_context_size += riscv_v_sc_size;
 	}
@@ -283,7 +283,7 @@  static long setup_sigcontext(struct rt_sigframe __user *frame,
 	if (has_fpu())
 		err |= save_fp_state(regs, &sc->sc_fpregs);
 	/* Save the vector state. */
-	if (has_vector() && riscv_v_vstate_query(regs))
+	if (has_vector(ZVE32X) && riscv_v_vstate_query(regs))
 		err |= save_v_state(regs, (void __user **)&sc_ext_ptr);
 	/* Write zero to fp-reserved space and check it on restore_sigcontext */
 	err |= __put_user(0, &sc->sc_extdesc.reserved);
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index 673437ccc13d..7252666ce0da 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -214,7 +214,7 @@  asmlinkage __visible void smp_callin(void)
 	struct mm_struct *mm = &init_mm;
 	unsigned int curr_cpuid = smp_processor_id();
 
-	if (has_vector()) {
+	if (has_vector(ZVE32X)) {
 		/*
 		 * Return as early as possible so the hart with a mismatching
 		 * vlen won't boot.
diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c
index c8219b82fbfc..e7c3fcac62a1 100644
--- a/arch/riscv/kernel/sys_hwprobe.c
+++ b/arch/riscv/kernel/sys_hwprobe.c
@@ -69,7 +69,7 @@  static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
 	if (riscv_isa_extension_available(NULL, c))
 		pair->value |= RISCV_HWPROBE_IMA_C;
 
-	if (has_vector())
+	if (has_vector(v))
 		pair->value |= RISCV_HWPROBE_IMA_V;
 
 	/*
@@ -112,7 +112,11 @@  static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
 		EXT_KEY(ZACAS);
 		EXT_KEY(ZICOND);
 
-		if (has_vector()) {
+		/*
+		 *  Vector crypto and ZVE* extensions are supported only if
+		 *  kernel has minimum V support of ZVE32X.
+		 */
+		if (has_vector(ZVE32X)) {
 			EXT_KEY(ZVE32X);
 			EXT_KEY(ZVE32F);
 			EXT_KEY(ZVE64X);
diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c
index 6727d1d3b8f2..e8a47fa72351 100644
--- a/arch/riscv/kernel/vector.c
+++ b/arch/riscv/kernel/vector.c
@@ -53,7 +53,7 @@  int riscv_v_setup_vsize(void)
 
 void __init riscv_v_setup_ctx_cache(void)
 {
-	if (!has_vector())
+	if (!has_vector(ZVE32X))
 		return;
 
 	riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx",
@@ -173,8 +173,11 @@  bool riscv_v_first_use_handler(struct pt_regs *regs)
 	u32 __user *epc = (u32 __user *)regs->epc;
 	u32 insn = (u32)regs->badaddr;
 
+	if (!has_vector(ZVE32X))
+		return false;
+
 	/* Do not handle if V is not supported, or disabled */
-	if (!(ELF_HWCAP & COMPAT_HWCAP_ISA_V))
+	if (!riscv_v_vstate_ctrl_user_allowed())
 		return false;
 
 	/* If V has been enabled then it is not the first-use trap */
@@ -213,7 +216,7 @@  void riscv_v_vstate_ctrl_init(struct task_struct *tsk)
 	bool inherit;
 	int cur, next;
 
-	if (!has_vector())
+	if (!has_vector(ZVE32X))
 		return;
 
 	next = riscv_v_ctrl_get_next(tsk);
@@ -235,7 +238,7 @@  void riscv_v_vstate_ctrl_init(struct task_struct *tsk)
 
 long riscv_v_vstate_ctrl_get_current(void)
 {
-	if (!has_vector())
+	if (!has_vector(ZVE32X))
 		return -EINVAL;
 
 	return current->thread.vstate_ctrl & PR_RISCV_V_VSTATE_CTRL_MASK;
@@ -246,7 +249,7 @@  long riscv_v_vstate_ctrl_set_current(unsigned long arg)
 	bool inherit;
 	int cur, next;
 
-	if (!has_vector())
+	if (!has_vector(ZVE32X))
 		return -EINVAL;
 
 	if (arg & ~PR_RISCV_V_VSTATE_CTRL_MASK)
@@ -296,7 +299,7 @@  static struct ctl_table riscv_v_default_vstate_table[] = {
 
 static int __init riscv_v_sysctl_init(void)
 {
-	if (has_vector())
+	if (has_vector(ZVE32X))
 		if (!register_sysctl("abi", riscv_v_default_vstate_table))
 			return -EINVAL;
 	return 0;
diff --git a/arch/riscv/lib/uaccess.S b/arch/riscv/lib/uaccess.S
index bc22c078aba8..bbe143bb32a0 100644
--- a/arch/riscv/lib/uaccess.S
+++ b/arch/riscv/lib/uaccess.S
@@ -14,7 +14,7 @@ 
 
 SYM_FUNC_START(__asm_copy_to_user)
 #ifdef CONFIG_RISCV_ISA_V
-	ALTERNATIVE("j fallback_scalar_usercopy", "nop", 0, RISCV_ISA_EXT_v, CONFIG_RISCV_ISA_V)
+	ALTERNATIVE("j fallback_scalar_usercopy", "nop", 0, RISCV_ISA_EXT_ZVE32X, CONFIG_RISCV_ISA_V)
 	REG_L	t0, riscv_v_usercopy_threshold
 	bltu	a2, t0, fallback_scalar_usercopy
 	tail enter_vector_usercopy