diff mbox series

[06/16] target/riscv: use KVM scratch CPUs to init KVM properties

Message ID 20230530194623.272652-7-dbarboza@ventanamicro.com (mailing list archive)
State New, archived
Headers show
Series target/riscv, KVM: fixes and enhancements | expand

Commit Message

Daniel Henrique Barboza May 30, 2023, 7:46 p.m. UTC 
Certain validations, such as the validations done for the machine IDs
(mvendorid/marchid/mimpid), are done before starting the CPU.
Non-dynamic (named) CPUs tries to match user input with a preset
default. As it is today we can't prefetch a KVM default for these cases
because we're only able to read/write KVM regs after the vcpu is
spinning.

Our target/arm friends use a concept called "scratch CPU", which
consists of creating a vcpu for doing queries and validations and so on,
which is discarded shortly after use [1]. This is a suitable solution
for what we need so let's implement it in target/riscv as well.

kvm_riscv_init_machine_ids() will be used to do any pre-launch setup for
KVM CPUs, via riscv_cpu_add_user_properties(). The function will create
a KVM scratch CPU, fetch KVM regs that work as default values for user
properties, and then discard the scratch CPU afterwards.

We're starting by initializing 'mvendorid'. This concept will be used to
init other KVM specific properties in the next patches as well.

[1] target/arm/kvm.c, kvm_arm_create_scratch_host_vcpu()

Suggested-by: Andrew Jones <ajones@ventanamicro.com>
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>
---
 target/riscv/cpu.c       |  4 ++
 target/riscv/kvm.c       | 85 ++++++++++++++++++++++++++++++++++++++++
 target/riscv/kvm_riscv.h |  1 +
 3 files changed, 90 insertions(+)

Comments

Andrew Jones June 6, 2023, 3:46 p.m. UTC | #1 
On Tue, May 30, 2023 at 04:46:13PM -0300, Daniel Henrique Barboza wrote:
> Certain validations, such as the validations done for the machine IDs
> (mvendorid/marchid/mimpid), are done before starting the CPU.
> Non-dynamic (named) CPUs tries to match user input with a preset
> default. As it is today we can't prefetch a KVM default for these cases
> because we're only able to read/write KVM regs after the vcpu is
> spinning.
> 
> Our target/arm friends use a concept called "scratch CPU", which
> consists of creating a vcpu for doing queries and validations and so on,
> which is discarded shortly after use [1]. This is a suitable solution
> for what we need so let's implement it in target/riscv as well.
> 
> kvm_riscv_init_machine_ids() will be used to do any pre-launch setup for
> KVM CPUs, via riscv_cpu_add_user_properties(). The function will create
> a KVM scratch CPU, fetch KVM regs that work as default values for user
> properties, and then discard the scratch CPU afterwards.
> 
> We're starting by initializing 'mvendorid'. This concept will be used to
> init other KVM specific properties in the next patches as well.
> 
> [1] target/arm/kvm.c, kvm_arm_create_scratch_host_vcpu()
> 
> Suggested-by: Andrew Jones <ajones@ventanamicro.com>
> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>
> ---
>  target/riscv/cpu.c       |  4 ++
>  target/riscv/kvm.c       | 85 ++++++++++++++++++++++++++++++++++++++++
>  target/riscv/kvm_riscv.h |  1 +
>  3 files changed, 90 insertions(+)
>

Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Alistair Francis June 12, 2023, 3:59 a.m. UTC | #2 
On Wed, May 31, 2023 at 5:50 AM Daniel Henrique Barboza
<dbarboza@ventanamicro.com> wrote:
>
> Certain validations, such as the validations done for the machine IDs
> (mvendorid/marchid/mimpid), are done before starting the CPU.
> Non-dynamic (named) CPUs tries to match user input with a preset
> default. As it is today we can't prefetch a KVM default for these cases
> because we're only able to read/write KVM regs after the vcpu is
> spinning.
>
> Our target/arm friends use a concept called "scratch CPU", which
> consists of creating a vcpu for doing queries and validations and so on,
> which is discarded shortly after use [1]. This is a suitable solution
> for what we need so let's implement it in target/riscv as well.
>
> kvm_riscv_init_machine_ids() will be used to do any pre-launch setup for
> KVM CPUs, via riscv_cpu_add_user_properties(). The function will create
> a KVM scratch CPU, fetch KVM regs that work as default values for user
> properties, and then discard the scratch CPU afterwards.
>
> We're starting by initializing 'mvendorid'. This concept will be used to
> init other KVM specific properties in the next patches as well.
>
> [1] target/arm/kvm.c, kvm_arm_create_scratch_host_vcpu()
>
> Suggested-by: Andrew Jones <ajones@ventanamicro.com>
> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>

Acked-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
>  target/riscv/cpu.c       |  4 ++
>  target/riscv/kvm.c       | 85 ++++++++++++++++++++++++++++++++++++++++
>  target/riscv/kvm_riscv.h |  1 +
>  3 files changed, 90 insertions(+)
>
> diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
> index d6e23bfd83..749d8bf5eb 100644
> --- a/target/riscv/cpu.c
> +++ b/target/riscv/cpu.c
> @@ -1703,6 +1703,10 @@ static void riscv_cpu_add_user_properties(Object *obj)
>      Property *prop;
>      DeviceState *dev = DEVICE(obj);
>
> +    if (riscv_running_KVM()) {
> +        kvm_riscv_init_user_properties(obj);
> +    }
> +
>      riscv_cpu_add_misa_properties(obj);
>
>      for (prop = riscv_cpu_extensions; prop && prop->name; prop++) {
> diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
> index 0f932a5b96..37f0f70794 100644
> --- a/target/riscv/kvm.c
> +++ b/target/riscv/kvm.c
> @@ -309,6 +309,91 @@ static void kvm_riscv_put_regs_timer(CPUState *cs)
>      env->kvm_timer_dirty = false;
>  }
>
> +typedef struct KVMScratchCPU {
> +    int kvmfd;
> +    int vmfd;
> +    int cpufd;
> +} KVMScratchCPU;
> +
> +/*
> + * Heavily inspired by kvm_arm_create_scratch_host_vcpu()
> + * from target/arm/kvm.c.
> + */
> +static bool kvm_riscv_create_scratch_vcpu(KVMScratchCPU *scratch)
> +{
> +    int kvmfd = -1, vmfd = -1, cpufd = -1;
> +
> +    kvmfd = qemu_open_old("/dev/kvm", O_RDWR);
> +    if (kvmfd < 0) {
> +        goto err;
> +    }
> +    do {
> +        vmfd = ioctl(kvmfd, KVM_CREATE_VM, 0);
> +    } while (vmfd == -1 && errno == EINTR);
> +    if (vmfd < 0) {
> +        goto err;
> +    }
> +    cpufd = ioctl(vmfd, KVM_CREATE_VCPU, 0);
> +    if (cpufd < 0) {
> +        goto err;
> +    }
> +
> +    scratch->kvmfd =  kvmfd;
> +    scratch->vmfd = vmfd;
> +    scratch->cpufd = cpufd;
> +
> +    return true;
> +
> + err:
> +    if (cpufd >= 0) {
> +        close(cpufd);
> +    }
> +    if (vmfd >= 0) {
> +        close(vmfd);
> +    }
> +    if (kvmfd >= 0) {
> +        close(kvmfd);
> +    }
> +
> +    return false;
> +}
> +
> +static void kvm_riscv_destroy_scratch_vcpu(KVMScratchCPU *scratch)
> +{
> +    close(scratch->cpufd);
> +    close(scratch->vmfd);
> +    close(scratch->kvmfd);
> +}
> +
> +static void kvm_riscv_init_machine_ids(RISCVCPU *cpu, KVMScratchCPU *kvmcpu)
> +{
> +    CPURISCVState *env = &cpu->env;
> +    struct kvm_one_reg reg;
> +    int ret;
> +
> +    reg.id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG,
> +                              KVM_REG_RISCV_CONFIG_REG(mvendorid));
> +    reg.addr = (uint64_t)&cpu->cfg.mvendorid;
> +    ret = ioctl(kvmcpu->cpufd, KVM_GET_ONE_REG, &reg);
> +    if (ret != 0) {
> +        error_report("Unable to retrieve mvendorid from host, error %d", ret);
> +    }
> +}
> +
> +void kvm_riscv_init_user_properties(Object *cpu_obj)
> +{
> +    RISCVCPU *cpu = RISCV_CPU(cpu_obj);
> +    KVMScratchCPU kvmcpu;
> +
> +    if (!kvm_riscv_create_scratch_vcpu(&kvmcpu)) {
> +        return;
> +    }
> +
> +    kvm_riscv_init_machine_ids(cpu, &kvmcpu);
> +
> +    kvm_riscv_destroy_scratch_vcpu(&kvmcpu);
> +}
> +
>  const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
>      KVM_CAP_LAST_INFO
>  };
> diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
> index ed281bdce0..e3ba935808 100644
> --- a/target/riscv/kvm_riscv.h
> +++ b/target/riscv/kvm_riscv.h
> @@ -19,6 +19,7 @@
>  #ifndef QEMU_KVM_RISCV_H
>  #define QEMU_KVM_RISCV_H
>
> +void kvm_riscv_init_user_properties(Object *cpu_obj);
>  void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
>  void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);
>
> --
> 2.40.1
>
>
diff mbox series

Patch

diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index d6e23bfd83..749d8bf5eb 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -1703,6 +1703,10 @@  static void riscv_cpu_add_user_properties(Object *obj)
     Property *prop;
     DeviceState *dev = DEVICE(obj);
 
+    if (riscv_running_KVM()) {
+        kvm_riscv_init_user_properties(obj);
+    }
+
     riscv_cpu_add_misa_properties(obj);
 
     for (prop = riscv_cpu_extensions; prop && prop->name; prop++) {
diff --git a/target/riscv/kvm.c b/target/riscv/kvm.c
index 0f932a5b96..37f0f70794 100644
--- a/target/riscv/kvm.c
+++ b/target/riscv/kvm.c
@@ -309,6 +309,91 @@  static void kvm_riscv_put_regs_timer(CPUState *cs)
     env->kvm_timer_dirty = false;
 }
 
+typedef struct KVMScratchCPU {
+    int kvmfd;
+    int vmfd;
+    int cpufd;
+} KVMScratchCPU;
+
+/*
+ * Heavily inspired by kvm_arm_create_scratch_host_vcpu()
+ * from target/arm/kvm.c.
+ */
+static bool kvm_riscv_create_scratch_vcpu(KVMScratchCPU *scratch)
+{
+    int kvmfd = -1, vmfd = -1, cpufd = -1;
+
+    kvmfd = qemu_open_old("/dev/kvm", O_RDWR);
+    if (kvmfd < 0) {
+        goto err;
+    }
+    do {
+        vmfd = ioctl(kvmfd, KVM_CREATE_VM, 0);
+    } while (vmfd == -1 && errno == EINTR);
+    if (vmfd < 0) {
+        goto err;
+    }
+    cpufd = ioctl(vmfd, KVM_CREATE_VCPU, 0);
+    if (cpufd < 0) {
+        goto err;
+    }
+
+    scratch->kvmfd =  kvmfd;
+    scratch->vmfd = vmfd;
+    scratch->cpufd = cpufd;
+
+    return true;
+
+ err:
+    if (cpufd >= 0) {
+        close(cpufd);
+    }
+    if (vmfd >= 0) {
+        close(vmfd);
+    }
+    if (kvmfd >= 0) {
+        close(kvmfd);
+    }
+
+    return false;
+}
+
+static void kvm_riscv_destroy_scratch_vcpu(KVMScratchCPU *scratch)
+{
+    close(scratch->cpufd);
+    close(scratch->vmfd);
+    close(scratch->kvmfd);
+}
+
+static void kvm_riscv_init_machine_ids(RISCVCPU *cpu, KVMScratchCPU *kvmcpu)
+{
+    CPURISCVState *env = &cpu->env;
+    struct kvm_one_reg reg;
+    int ret;
+
+    reg.id = kvm_riscv_reg_id(env, KVM_REG_RISCV_CONFIG,
+                              KVM_REG_RISCV_CONFIG_REG(mvendorid));
+    reg.addr = (uint64_t)&cpu->cfg.mvendorid;
+    ret = ioctl(kvmcpu->cpufd, KVM_GET_ONE_REG, &reg);
+    if (ret != 0) {
+        error_report("Unable to retrieve mvendorid from host, error %d", ret);
+    }
+}
+
+void kvm_riscv_init_user_properties(Object *cpu_obj)
+{
+    RISCVCPU *cpu = RISCV_CPU(cpu_obj);
+    KVMScratchCPU kvmcpu;
+
+    if (!kvm_riscv_create_scratch_vcpu(&kvmcpu)) {
+        return;
+    }
+
+    kvm_riscv_init_machine_ids(cpu, &kvmcpu);
+
+    kvm_riscv_destroy_scratch_vcpu(&kvmcpu);
+}
+
 const KVMCapabilityInfo kvm_arch_required_capabilities[] = {
     KVM_CAP_LAST_INFO
 };
diff --git a/target/riscv/kvm_riscv.h b/target/riscv/kvm_riscv.h
index ed281bdce0..e3ba935808 100644
--- a/target/riscv/kvm_riscv.h
+++ b/target/riscv/kvm_riscv.h
@@ -19,6 +19,7 @@ 
 #ifndef QEMU_KVM_RISCV_H
 #define QEMU_KVM_RISCV_H
 
+void kvm_riscv_init_user_properties(Object *cpu_obj);
 void kvm_riscv_reset_vcpu(RISCVCPU *cpu);
 void kvm_riscv_set_irq(RISCVCPU *cpu, int irq, int level);