| Message ID | 20231122164700.127954-9-ajones@ventanamicro.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | e178bf146e4b8c774a7b00aa2419e400f4f7894f |
| Headers | show |
| Series | RISC-V: hwprobe: Introduce which-cpus | expand |
On Wed, Nov 22, 2023 at 8:47 AM Andrew Jones <ajones@ventanamicro.com> wrote: > > Introduce the first flag for the hwprobe syscall. The flag basically > reverses its behavior, i.e. instead of populating the values of keys > for a given set of cpus, the set of cpus after the call is the result > of finding a set which supports the values of the keys. In order to > do this, we implement a pair compare function which takes the type of > value (a single value vs. a bitmask of booleans) into consideration. > We also implement vdso support for the new flag. > > Signed-off-by: Andrew Jones <ajones@ventanamicro.com> I spent a little time pondering whether there was a nicer way of handling unknown keys (rather than rejecting the whole call), but I can't think of a way that doesn't cause danger and/or extra work for the happy path. So this looks good to me! Reviewed-by: Evan Green <evan@rivosinc.com> > --- > Documentation/arch/riscv/hwprobe.rst | 17 ++++- > arch/riscv/include/asm/hwprobe.h | 24 +++++++ > arch/riscv/include/uapi/asm/hwprobe.h | 3 + > arch/riscv/kernel/sys_hwprobe.c | 94 +++++++++++++++++++++++++-- > arch/riscv/kernel/vdso/hwprobe.c | 80 ++++++++++++++++++++--- > 5 files changed, 204 insertions(+), 14 deletions(-) > > diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst > index 132e9acaa8f4..12f9b0a043ac 100644 > --- a/Documentation/arch/riscv/hwprobe.rst > +++ b/Documentation/arch/riscv/hwprobe.rst > @@ -25,8 +25,21 @@ arch, impl), the returned value will only be valid if all CPUs in the given set > have the same value. Otherwise -1 will be returned. For boolean-like keys, the > value returned will be a logical AND of the values for the specified CPUs. > Usermode can supply NULL for ``cpus`` and 0 for ``cpusetsize`` as a shortcut for > -all online CPUs. There are currently no flags, this value must be zero for > -future compatibility. > +all online CPUs. The currently supported flags are: > + > +* :c:macro:`RISCV_HWPROBE_WHICH_CPUS`: This flag basically reverses the behavior > + of sys_riscv_hwprobe(). Instead of populating the values of keys for a given > + set of CPUs, the values of each key are given and the set of CPUs is reduced > + by sys_riscv_hwprobe() to only those which match each of the key-value pairs. > + How matching is done depends on the key type. For value-like keys, matching > + means to be the exact same as the value. For boolean-like keys, matching > + means the result of a logical AND of the pair's value with the CPU's value is > + exactly the same as the pair's value. Additionally, when ``cpus`` is an empty > + set, then it is initialized to all online CPUs which fit within it, i.e. the > + CPU set returned is the reduction of all the online CPUs which can be > + represented with a CPU set of size ``cpusetsize``. > + > +All other flags are reserved for future compatibility and must be zero. > > On success 0 is returned, on failure a negative error code is returned. > > diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h > index 5c48f48e79a6..630507dff5ea 100644 > --- a/arch/riscv/include/asm/hwprobe.h > +++ b/arch/riscv/include/asm/hwprobe.h > @@ -15,4 +15,28 @@ static inline bool riscv_hwprobe_key_is_valid(__s64 key) > return key >= 0 && key <= RISCV_HWPROBE_MAX_KEY; > } > > +static inline bool hwprobe_key_is_bitmask(__s64 key) > +{ > + switch (key) { > + case RISCV_HWPROBE_KEY_BASE_BEHAVIOR: > + case RISCV_HWPROBE_KEY_IMA_EXT_0: > + case RISCV_HWPROBE_KEY_CPUPERF_0: > + return true; > + } > + > + return false; > +} > + > +static inline bool riscv_hwprobe_pair_cmp(struct riscv_hwprobe *pair, > + struct riscv_hwprobe *other_pair) > +{ > + if (pair->key != other_pair->key) > + return false; > + > + if (hwprobe_key_is_bitmask(pair->key)) > + return (pair->value & other_pair->value) == other_pair->value; > + > + return pair->value == other_pair->value; > +} > + > #endif > diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h > index b659ffcfcdb4..7aa7d5c71e79 100644 > --- a/arch/riscv/include/uapi/asm/hwprobe.h > +++ b/arch/riscv/include/uapi/asm/hwprobe.h > @@ -40,4 +40,7 @@ struct riscv_hwprobe { > #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 > /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ > > +/* Flags */ > +#define RISCV_HWPROBE_WHICH_CPUS (1 << 0) > + > #endif > diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c > index c0ccf1f45aaa..b7cfb26ce31c 100644 > --- a/arch/riscv/kernel/sys_hwprobe.c > +++ b/arch/riscv/kernel/sys_hwprobe.c > @@ -179,10 +179,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, > } > } > > -static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, > - size_t pair_count, size_t cpusetsize, > - unsigned long __user *cpus_user, > - unsigned int flags) > +static int hwprobe_get_values(struct riscv_hwprobe __user *pairs, > + size_t pair_count, size_t cpusetsize, > + unsigned long __user *cpus_user, > + unsigned int flags) > { > size_t out; > int ret; > @@ -236,6 +236,92 @@ static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, > return 0; > } > > +static int hwprobe_get_cpus(struct riscv_hwprobe __user *pairs, > + size_t pair_count, size_t cpusetsize, > + unsigned long __user *cpus_user, > + unsigned int flags) > +{ > + cpumask_t cpus, one_cpu; > + bool clear_all = false; > + size_t i; > + int ret; > + > + if (flags != RISCV_HWPROBE_WHICH_CPUS) > + return -EINVAL; > + > + if (!cpusetsize || !cpus_user) > + return -EINVAL; > + > + if (cpusetsize > cpumask_size()) > + cpusetsize = cpumask_size(); > + > + ret = copy_from_user(&cpus, cpus_user, cpusetsize); > + if (ret) > + return -EFAULT; > + > + if (cpumask_empty(&cpus)) > + cpumask_copy(&cpus, cpu_online_mask); > + > + cpumask_and(&cpus, &cpus, cpu_online_mask); > + > + cpumask_clear(&one_cpu); > + > + for (i = 0; i < pair_count; i++) { > + struct riscv_hwprobe pair, tmp; > + int cpu; > + > + ret = copy_from_user(&pair, &pairs[i], sizeof(pair)); > + if (ret) > + return -EFAULT; > + > + if (!riscv_hwprobe_key_is_valid(pair.key)) { > + clear_all = true; > + pair = (struct riscv_hwprobe){ .key = -1, }; > + ret = copy_to_user(&pairs[i], &pair, sizeof(pair)); > + if (ret) > + return -EFAULT; > + } > + > + if (clear_all) > + continue; > + > + tmp = (struct riscv_hwprobe){ .key = pair.key, }; > + > + for_each_cpu(cpu, &cpus) { > + cpumask_set_cpu(cpu, &one_cpu); > + > + hwprobe_one_pair(&tmp, &one_cpu); > + > + if (!riscv_hwprobe_pair_cmp(&tmp, &pair)) > + cpumask_clear_cpu(cpu, &cpus); > + > + cpumask_clear_cpu(cpu, &one_cpu); > + } > + } > + > + if (clear_all) > + cpumask_clear(&cpus); > + > + ret = copy_to_user(cpus_user, &cpus, cpusetsize); > + if (ret) > + return -EFAULT; > + > + return 0; > +} > + > +static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, > + size_t pair_count, size_t cpusetsize, > + unsigned long __user *cpus_user, > + unsigned int flags) > +{ > + if (flags & RISCV_HWPROBE_WHICH_CPUS) > + return hwprobe_get_cpus(pairs, pair_count, cpusetsize, > + cpus_user, flags); > + > + return hwprobe_get_values(pairs, pair_count, cpusetsize, > + cpus_user, flags); > +} > + > #ifdef CONFIG_MMU > > static int __init init_hwprobe_vdso_data(void) > diff --git a/arch/riscv/kernel/vdso/hwprobe.c b/arch/riscv/kernel/vdso/hwprobe.c > index 026b7645c5ab..1e926e4b5881 100644 > --- a/arch/riscv/kernel/vdso/hwprobe.c > +++ b/arch/riscv/kernel/vdso/hwprobe.c > @@ -3,6 +3,7 @@ > * Copyright 2023 Rivos, Inc > */ > > +#include <linux/string.h> > #include <linux/types.h> > #include <vdso/datapage.h> > #include <vdso/helpers.h> > @@ -11,14 +12,9 @@ extern int riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > size_t cpusetsize, unsigned long *cpus, > unsigned int flags); > > -/* Add a prototype to avoid -Wmissing-prototypes warning. */ > -int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > - size_t cpusetsize, unsigned long *cpus, > - unsigned int flags); > - > -int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > - size_t cpusetsize, unsigned long *cpus, > - unsigned int flags) > +static int riscv_vdso_get_values(struct riscv_hwprobe *pairs, size_t pair_count, > + size_t cpusetsize, unsigned long *cpus, > + unsigned int flags) > { > const struct vdso_data *vd = __arch_get_vdso_data(); > const struct arch_vdso_data *avd = &vd->arch_data; > @@ -50,3 +46,71 @@ int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > > return 0; > } > + > +static int riscv_vdso_get_cpus(struct riscv_hwprobe *pairs, size_t pair_count, > + size_t cpusetsize, unsigned long *cpus, > + unsigned int flags) > +{ > + const struct vdso_data *vd = __arch_get_vdso_data(); > + const struct arch_vdso_data *avd = &vd->arch_data; > + struct riscv_hwprobe *p = pairs; > + struct riscv_hwprobe *end = pairs + pair_count; > + unsigned char *c = (unsigned char *)cpus; > + bool empty_cpus = true; > + bool clear_all = false; > + int i; > + > + if (!cpusetsize || !cpus) > + return -EINVAL; > + > + for (i = 0; i < cpusetsize; i++) { > + if (c[i]) { > + empty_cpus = false; > + break; > + } > + } > + > + if (empty_cpus || flags != RISCV_HWPROBE_WHICH_CPUS || !avd->homogeneous_cpus) > + return riscv_hwprobe(pairs, pair_count, cpusetsize, cpus, flags); > + > + while (p < end) { > + if (riscv_hwprobe_key_is_valid(p->key)) { > + struct riscv_hwprobe t = { > + .key = p->key, > + .value = avd->all_cpu_hwprobe_values[p->key], > + }; > + > + if (!riscv_hwprobe_pair_cmp(&t, p)) > + clear_all = true; > + } else { > + clear_all = true; > + p->key = -1; > + p->value = 0; > + } > + p++; > + } > + > + if (clear_all) { > + for (i = 0; i < cpusetsize; i++) > + c[i] = 0; > + } > + > + return 0; > +} > + > +/* Add a prototype to avoid -Wmissing-prototypes warning. */ > +int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > + size_t cpusetsize, unsigned long *cpus, > + unsigned int flags); > + > +int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, > + size_t cpusetsize, unsigned long *cpus, > + unsigned int flags) > +{ > + if (flags & RISCV_HWPROBE_WHICH_CPUS) > + return riscv_vdso_get_cpus(pairs, pair_count, cpusetsize, > + cpus, flags); > + > + return riscv_vdso_get_values(pairs, pair_count, cpusetsize, > + cpus, flags); > +} > -- > 2.41.0 >
diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index 132e9acaa8f4..12f9b0a043ac 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -25,8 +25,21 @@ arch, impl), the returned value will only be valid if all CPUs in the given set have the same value. Otherwise -1 will be returned. For boolean-like keys, the value returned will be a logical AND of the values for the specified CPUs. Usermode can supply NULL for ``cpus`` and 0 for ``cpusetsize`` as a shortcut for -all online CPUs. There are currently no flags, this value must be zero for -future compatibility. +all online CPUs. The currently supported flags are: + +* :c:macro:`RISCV_HWPROBE_WHICH_CPUS`: This flag basically reverses the behavior + of sys_riscv_hwprobe(). Instead of populating the values of keys for a given + set of CPUs, the values of each key are given and the set of CPUs is reduced + by sys_riscv_hwprobe() to only those which match each of the key-value pairs. + How matching is done depends on the key type. For value-like keys, matching + means to be the exact same as the value. For boolean-like keys, matching + means the result of a logical AND of the pair's value with the CPU's value is + exactly the same as the pair's value. Additionally, when ``cpus`` is an empty + set, then it is initialized to all online CPUs which fit within it, i.e. the + CPU set returned is the reduction of all the online CPUs which can be + represented with a CPU set of size ``cpusetsize``. + +All other flags are reserved for future compatibility and must be zero. On success 0 is returned, on failure a negative error code is returned. diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h index 5c48f48e79a6..630507dff5ea 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -15,4 +15,28 @@ static inline bool riscv_hwprobe_key_is_valid(__s64 key) return key >= 0 && key <= RISCV_HWPROBE_MAX_KEY; } +static inline bool hwprobe_key_is_bitmask(__s64 key) +{ + switch (key) { + case RISCV_HWPROBE_KEY_BASE_BEHAVIOR: + case RISCV_HWPROBE_KEY_IMA_EXT_0: + case RISCV_HWPROBE_KEY_CPUPERF_0: + return true; + } + + return false; +} + +static inline bool riscv_hwprobe_pair_cmp(struct riscv_hwprobe *pair, + struct riscv_hwprobe *other_pair) +{ + if (pair->key != other_pair->key) + return false; + + if (hwprobe_key_is_bitmask(pair->key)) + return (pair->value & other_pair->value) == other_pair->value; + + return pair->value == other_pair->value; +} + #endif diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index b659ffcfcdb4..7aa7d5c71e79 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -40,4 +40,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ +/* Flags */ +#define RISCV_HWPROBE_WHICH_CPUS (1 << 0) + #endif diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index c0ccf1f45aaa..b7cfb26ce31c 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -179,10 +179,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, } } -static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, - size_t pair_count, size_t cpusetsize, - unsigned long __user *cpus_user, - unsigned int flags) +static int hwprobe_get_values(struct riscv_hwprobe __user *pairs, + size_t pair_count, size_t cpusetsize, + unsigned long __user *cpus_user, + unsigned int flags) { size_t out; int ret; @@ -236,6 +236,92 @@ static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, return 0; } +static int hwprobe_get_cpus(struct riscv_hwprobe __user *pairs, + size_t pair_count, size_t cpusetsize, + unsigned long __user *cpus_user, + unsigned int flags) +{ + cpumask_t cpus, one_cpu; + bool clear_all = false; + size_t i; + int ret; + + if (flags != RISCV_HWPROBE_WHICH_CPUS) + return -EINVAL; + + if (!cpusetsize || !cpus_user) + return -EINVAL; + + if (cpusetsize > cpumask_size()) + cpusetsize = cpumask_size(); + + ret = copy_from_user(&cpus, cpus_user, cpusetsize); + if (ret) + return -EFAULT; + + if (cpumask_empty(&cpus)) + cpumask_copy(&cpus, cpu_online_mask); + + cpumask_and(&cpus, &cpus, cpu_online_mask); + + cpumask_clear(&one_cpu); + + for (i = 0; i < pair_count; i++) { + struct riscv_hwprobe pair, tmp; + int cpu; + + ret = copy_from_user(&pair, &pairs[i], sizeof(pair)); + if (ret) + return -EFAULT; + + if (!riscv_hwprobe_key_is_valid(pair.key)) { + clear_all = true; + pair = (struct riscv_hwprobe){ .key = -1, }; + ret = copy_to_user(&pairs[i], &pair, sizeof(pair)); + if (ret) + return -EFAULT; + } + + if (clear_all) + continue; + + tmp = (struct riscv_hwprobe){ .key = pair.key, }; + + for_each_cpu(cpu, &cpus) { + cpumask_set_cpu(cpu, &one_cpu); + + hwprobe_one_pair(&tmp, &one_cpu); + + if (!riscv_hwprobe_pair_cmp(&tmp, &pair)) + cpumask_clear_cpu(cpu, &cpus); + + cpumask_clear_cpu(cpu, &one_cpu); + } + } + + if (clear_all) + cpumask_clear(&cpus); + + ret = copy_to_user(cpus_user, &cpus, cpusetsize); + if (ret) + return -EFAULT; + + return 0; +} + +static int do_riscv_hwprobe(struct riscv_hwprobe __user *pairs, + size_t pair_count, size_t cpusetsize, + unsigned long __user *cpus_user, + unsigned int flags) +{ + if (flags & RISCV_HWPROBE_WHICH_CPUS) + return hwprobe_get_cpus(pairs, pair_count, cpusetsize, + cpus_user, flags); + + return hwprobe_get_values(pairs, pair_count, cpusetsize, + cpus_user, flags); +} + #ifdef CONFIG_MMU static int __init init_hwprobe_vdso_data(void) diff --git a/arch/riscv/kernel/vdso/hwprobe.c b/arch/riscv/kernel/vdso/hwprobe.c index 026b7645c5ab..1e926e4b5881 100644 --- a/arch/riscv/kernel/vdso/hwprobe.c +++ b/arch/riscv/kernel/vdso/hwprobe.c @@ -3,6 +3,7 @@ * Copyright 2023 Rivos, Inc */ +#include <linux/string.h> #include <linux/types.h> #include <vdso/datapage.h> #include <vdso/helpers.h> @@ -11,14 +12,9 @@ extern int riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, size_t cpusetsize, unsigned long *cpus, unsigned int flags); -/* Add a prototype to avoid -Wmissing-prototypes warning. */ -int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, - size_t cpusetsize, unsigned long *cpus, - unsigned int flags); - -int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, - size_t cpusetsize, unsigned long *cpus, - unsigned int flags) +static int riscv_vdso_get_values(struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpusetsize, unsigned long *cpus, + unsigned int flags) { const struct vdso_data *vd = __arch_get_vdso_data(); const struct arch_vdso_data *avd = &vd->arch_data; @@ -50,3 +46,71 @@ int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, return 0; } + +static int riscv_vdso_get_cpus(struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpusetsize, unsigned long *cpus, + unsigned int flags) +{ + const struct vdso_data *vd = __arch_get_vdso_data(); + const struct arch_vdso_data *avd = &vd->arch_data; + struct riscv_hwprobe *p = pairs; + struct riscv_hwprobe *end = pairs + pair_count; + unsigned char *c = (unsigned char *)cpus; + bool empty_cpus = true; + bool clear_all = false; + int i; + + if (!cpusetsize || !cpus) + return -EINVAL; + + for (i = 0; i < cpusetsize; i++) { + if (c[i]) { + empty_cpus = false; + break; + } + } + + if (empty_cpus || flags != RISCV_HWPROBE_WHICH_CPUS || !avd->homogeneous_cpus) + return riscv_hwprobe(pairs, pair_count, cpusetsize, cpus, flags); + + while (p < end) { + if (riscv_hwprobe_key_is_valid(p->key)) { + struct riscv_hwprobe t = { + .key = p->key, + .value = avd->all_cpu_hwprobe_values[p->key], + }; + + if (!riscv_hwprobe_pair_cmp(&t, p)) + clear_all = true; + } else { + clear_all = true; + p->key = -1; + p->value = 0; + } + p++; + } + + if (clear_all) { + for (i = 0; i < cpusetsize; i++) + c[i] = 0; + } + + return 0; +} + +/* Add a prototype to avoid -Wmissing-prototypes warning. */ +int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpusetsize, unsigned long *cpus, + unsigned int flags); + +int __vdso_riscv_hwprobe(struct riscv_hwprobe *pairs, size_t pair_count, + size_t cpusetsize, unsigned long *cpus, + unsigned int flags) +{ + if (flags & RISCV_HWPROBE_WHICH_CPUS) + return riscv_vdso_get_cpus(pairs, pair_count, cpusetsize, + cpus, flags); + + return riscv_vdso_get_values(pairs, pair_count, cpusetsize, + cpus, flags); +}
Introduce the first flag for the hwprobe syscall. The flag basically reverses its behavior, i.e. instead of populating the values of keys for a given set of cpus, the set of cpus after the call is the result of finding a set which supports the values of the keys. In order to do this, we implement a pair compare function which takes the type of value (a single value vs. a bitmask of booleans) into consideration. We also implement vdso support for the new flag. Signed-off-by: Andrew Jones <ajones@ventanamicro.com> --- Documentation/arch/riscv/hwprobe.rst | 17 ++++- arch/riscv/include/asm/hwprobe.h | 24 +++++++ arch/riscv/include/uapi/asm/hwprobe.h | 3 + arch/riscv/kernel/sys_hwprobe.c | 94 +++++++++++++++++++++++++-- arch/riscv/kernel/vdso/hwprobe.c | 80 ++++++++++++++++++++--- 5 files changed, 204 insertions(+), 14 deletions(-)