Message ID | 20240430122808.72025-1-philmd@linaro.org (mailing list archive) |
---|---|
Headers | show |
Series | exec: Rework around CPUState user fields (part 2) | expand |
On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote: > Missing WASM testing by Ilya (branch available at > https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) Hmm, it dies very early now: # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 217 cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); (gdb) bt #0 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284 #2 0x000055555563f76b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510 #3 0x000055555564363d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 #4 0x0000555555646b9b in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:1471 #5 0x000055555564a43f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 #6 0x0000555555647204 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:1541 #7 0x000055555564025c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291 #8 0x000055555559bbb4 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61 #9 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811 (gdb) p cpu $1 = (CPUState *) 0x5555557c28c0 (gdb) p cpu->accel $2 = (AccelCPUState *) 0x0 Configured with: '/home/iii/myrepos/qemu/configure' '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' '--disable-fdt' '--disable-capstone' '--disable-docs' If you don't see what can be wrong here right away, I can debug this. > Since v2: > - Move cpu_loop_exit_requested() to "exec/cpu-loop.h" > - Added R-b tags > > Since v1: > - First 13 patches queued > - Restrict qemu_plugin_vcpu_exit_hook() to (TCG) plugins > - Restrict cpu_plugin_mem_cbs_enabled() to TCG (plugins) > - Addressed Richard review comments on the others: > - Move cpu_plugin_mem_cbs_enabled() > - Do not move mem_io_pc, waiting for [*] > - Mention can_do_io restricted > > Finish extracting TCG fields from CPUState: > - Extract tcg_cpu_exit() from cpu_exit() > - Introduce AccelOpsClass::exit_vcpu_thread() > - cpu_exit() calls exit_vcpu_thread=tcg_cpu_exit for TCG > - Forward declare TaskState and more uses of get_task_state() > - Introduce TCG AccelCPUState > - Move TCG specific fields from CPUState to AccelCPUState > - Restrict "exec/tlb-common.h" to TCG > - Restrict iommu_notifiers, icount to system emulation > > [*] https://lore.kernel.org/qemu-devel/20240416040609.1313605-3-richard.henderson@linaro.org/ > > Based-on: https://gitlab.com/philmd/qemu/-/commits/accel-next > > Philippe Mathieu-Daudé (13): > accel/tcg: Restrict qemu_plugin_vcpu_exit_hook() to TCG plugins > accel/tcg: Restrict cpu_plugin_mem_cbs_enabled() to TCG > accel/tcg: Move @plugin_mem_cbs from CPUState to > CPUNegativeOffsetState > accel/tcg: Move @plugin_state from CPUState to TCG AccelCPUState > accel/tcg: Restrict cpu_loop_exit_requested() to TCG > accel/tcg: Restrict IcountDecr / can_do_io / CPUTLB to TCG > accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState > accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState > accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState > accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState > accel/tcg: Restrict icount to system emulation > accel/tcg: Move icount fields from CPUState to TCG AccelCPUState > accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState > > accel/tcg/internal-common.h | 18 ++++++++++ > accel/tcg/tb-jmp-cache.h | 4 +-- > accel/tcg/tcg-accel-ops.h | 1 + > accel/tcg/vcpu-state.h | 20 +++++++++++ > include/exec/cpu-loop.h | 35 +++++++++++++++++++ > include/exec/exec-all.h | 17 ---------- > include/exec/tlb-common.h | 4 +++ > include/hw/core/cpu.h | 58 ++++++++------------------------ > include/qemu/plugin.h | 2 +- > include/qemu/typedefs.h | 1 - > accel/tcg/cpu-exec-common.c | 2 +- > accel/tcg/cpu-exec.c | 52 +++++++++++++++------------- > accel/tcg/cputlb.c | 2 +- > accel/tcg/icount-common.c | 7 ++-- > accel/tcg/plugin-gen.c | 9 +++-- > accel/tcg/tb-maint.c | 6 ++-- > accel/tcg/tcg-accel-ops-icount.c | 14 ++++---- > accel/tcg/tcg-accel-ops.c | 2 ++ > accel/tcg/translate-all.c | 9 ++--- > accel/tcg/watchpoint.c | 5 +-- > hw/core/cpu-common.c | 9 +++-- > linux-user/main.c | 2 +- > plugins/core.c | 9 ++--- > system/physmem.c | 37 +++++++++++++++----- > target/arm/tcg/helper-a64.c | 1 + > target/s390x/tcg/mem_helper.c | 1 + > 26 files changed, 195 insertions(+), 132 deletions(-) > create mode 100644 include/exec/cpu-loop.h > > -- > 2.41.0 >
Hi Ilya, On 30/4/24 19:55, Ilya Leoshkevich wrote: > On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote: >> Missing WASM testing by Ilya (branch available at >> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) > > Hmm, it dies very early now: > > # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 > > Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. > 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 > 217 cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); > > (gdb) bt > #0 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 > #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284 > #2 0x000055555563f76b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510 > #3 0x000055555564363d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 > #4 0x0000555555646b9b in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/object.c:1471 > #5 0x000055555564a43f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 > #6 0x0000555555647204 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/object.c:1541 > #7 0x000055555564025c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291 > #8 0x000055555559bbb4 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61 > #9 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811 > > (gdb) p cpu > $1 = (CPUState *) 0x5555557c28c0 > (gdb) p cpu->accel > $2 = (AccelCPUState *) 0x0 > > Configured with: '/home/iii/myrepos/qemu/configure' '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' '--disable-fdt' '--disable-capstone' '--disable-docs' > > If you don't see what can be wrong here right away, I can debug this. Useful enough I guess, but I'll ask you to test again later. Does it work without the last patch? Is it possible to share component_fuzz_util-d10a3a6b4ad8af47? Thanks for the testing, Phil.
On 30/4/24 20:45, Philippe Mathieu-Daudé wrote: > Hi Ilya, > > On 30/4/24 19:55, Ilya Leoshkevich wrote: >> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote: >>> Missing WASM testing by Ilya (branch available at >>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) >> >> Hmm, it dies very early now: >> >> # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu >> /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 >> >> Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. >> 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, >> errp=<optimized out>) at >> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >> 217 cpu->accel->plugin_state = >> qemu_plugin_create_vcpu_state(); >> >> (gdb) bt >> #0 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, >> errp=<optimized out>) at >> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >> #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, >> errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284 >> #2 0x000055555563f76b in device_set_realized (obj=<optimized out>, >> value=<optimized out>, errp=0x7fffffffe2e0) at >> ../home/iii/myrepos/qemu/hw/core/qdev.c:510 >> #3 0x000055555564363d in property_set_bool (obj=0x5555557c28c0, >> v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, >> errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 >> #4 0x0000555555646b9b in object_property_set >> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 >> "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0) >> at ../home/iii/myrepos/qemu/qom/object.c:1471 >> #5 0x000055555564a43f in object_property_set_qobject >> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 >> "realized", value=value@entry=0x5555557a7a90, >> errp=errp@entry=0x7fffffffe2e0) >> at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 >> #6 0x0000555555647204 in object_property_set_bool >> (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", >> value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) >> at ../home/iii/myrepos/qemu/qom/object.c:1541 >> #7 0x000055555564025c in qdev_realize (dev=<optimized out>, >> bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at >> ../home/iii/myrepos/qemu/hw/core/qdev.c:291 >> #8 0x000055555559bbb4 in cpu_create (typename=<optimized out>) at >> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61 >> #9 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, >> envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811 >> >> (gdb) p cpu >> $1 = (CPUState *) 0x5555557c28c0 >> (gdb) p cpu->accel >> $2 = (AccelCPUState *) 0x0 >> >> Configured with: '/home/iii/myrepos/qemu/configure' >> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' >> '--disable-fdt' '--disable-capstone' '--disable-docs' >> >> If you don't see what can be wrong here right away, I can debug this. I added this commit in the same branch: -- >8 -- Author: Philippe Mathieu-Daudé <philmd@linaro.org> Date: Tue Apr 30 20:57:15 2024 +0200 accel/tcg: Initialize TCG plugins in cpu-target.c Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> diff --git a/cpu-target.c b/cpu-target.c index 5af120e8aa..585533cfa3 100644 --- a/cpu-target.c +++ b/cpu-target.c @@ -46,6 +46,10 @@ #include "hw/core/accel-cpu.h" #include "trace/trace-root.h" #include "qemu/accel.h" +#ifdef CONFIG_PLUGIN +#include "accel/tcg/vcpu-state.h" +#include "qemu/plugin.h" +#endif #ifndef CONFIG_USER_ONLY static int cpu_common_post_load(void *opaque, int version_id) @@ -131,6 +135,13 @@ const VMStateDescription vmstate_cpu_common = { }; #endif +#ifdef CONFIG_PLUGIN +static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data unused) +{ + qemu_plugin_vcpu_init_hook(cpu); +} +#endif + bool cpu_exec_realizefn(CPUState *cpu, Error **errp) { /* cache the cpu class for the hotpath */ @@ -143,6 +154,15 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp) /* Wait until cpu initialization complete before exposing cpu. */ cpu_list_add(cpu); +#ifdef CONFIG_PLUGIN + assert(cpu->accel); + /* Plugin initialization must wait until the cpu start executing code */ + if (tcg_enabled()) { + cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); + async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, RUN_ON_CPU_NULL); + } +#endif + #ifdef CONFIG_USER_ONLY assert(qdev_get_vmsd(DEVICE(cpu)) == NULL || qdev_get_vmsd(DEVICE(cpu))->unmigratable); @@ -171,6 +191,13 @@ void cpu_exec_unrealizefn(CPUState *cpu) } #endif +#ifdef CONFIG_PLUGIN + /* Call the plugin hook before clearing the cpu is fully unrealized */ + if (tcg_enabled()) { + qemu_plugin_vcpu_exit_hook(cpu); + } +#endif + cpu_list_remove(cpu); /* * Now that the vCPU has been removed from the RCU list, we can call diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c index e03d31876f..cd8bd99131 100644 --- a/hw/core/cpu-common.c +++ b/hw/core/cpu-common.c @@ -30,10 +30,6 @@ #include "hw/boards.h" #include "hw/qdev-properties.h" #include "trace.h" -#ifdef CONFIG_PLUGIN -#include "accel/tcg/vcpu-state.h" -#include "qemu/plugin.h" -#endif CPUState *cpu_by_arch_id(int64_t id) { @@ -181,13 +177,6 @@ static void cpu_common_parse_features(const char *typename, char *features, } } -#ifdef CONFIG_PLUGIN -static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data unused) -{ - qemu_plugin_vcpu_init_hook(cpu); -} -#endif - static void cpu_common_realizefn(DeviceState *dev, Error **errp) { CPUState *cpu = CPU(dev); @@ -211,14 +200,6 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp) cpu_resume(cpu); } - /* Plugin initialization must wait until the cpu start executing code */ -#ifdef CONFIG_PLUGIN - if (tcg_enabled()) { - cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); - async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, RUN_ON_CPU_NULL); - } -#endif - /* NOTE: latest generic point where the cpu is fully realized */ } @@ -226,13 +207,6 @@ static void cpu_common_unrealizefn(DeviceState *dev) { CPUState *cpu = CPU(dev); - /* Call the plugin hook before clearing the cpu is fully unrealized */ -#ifdef CONFIG_PLUGIN - if (tcg_enabled()) { - qemu_plugin_vcpu_exit_hook(cpu); - } -#endif - /* NOTE: latest generic point before the cpu is fully unrealized */ cpu_exec_unrealizefn(cpu); } --- Totally untested here because it is late (only built...).
On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote: > On 30/4/24 20:45, Philippe Mathieu-Daudé wrote: > > Hi Ilya, > > > > On 30/4/24 19:55, Ilya Leoshkevich wrote: > > > On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote: > > > > Missing WASM testing by Ilya (branch available at > > > > https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) > > > > > > Hmm, it dies very early now: > > > > > > # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 > > > > > > Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. > > > 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, > > > errp=<optimized out>) at > > > ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 > > > 217 cpu->accel->plugin_state = > > > qemu_plugin_create_vcpu_state(); > > > > > > (gdb) bt > > > #0 0x000055555559b718 in cpu_common_realizefn > > > (dev=0x5555557c28c0, errp=<optimized out>) at > > > ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 > > > #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, > > > errp=0x7fffffffe1a0) at > > > ../home/iii/myrepos/qemu/target/s390x/cpu.c:284 > > > #2 0x000055555563f76b in device_set_realized (obj=<optimized > > > out>, value=<optimized out>, errp=0x7fffffffe2e0) at > > > ../home/iii/myrepos/qemu/hw/core/qdev.c:510 > > > #3 0x000055555564363d in property_set_bool (obj=0x5555557c28c0, > > > v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, > > > errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 > > > #4 0x0000555555646b9b in object_property_set > > > (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 > > > "realized", v=v@entry=0x5555557c6650, > > > errp=errp@entry=0x7fffffffe2e0) > > > at ../home/iii/myrepos/qemu/qom/object.c:1471 > > > #5 0x000055555564a43f in object_property_set_qobject > > > (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 > > > "realized", value=value@entry=0x5555557a7a90, > > > errp=errp@entry=0x7fffffffe2e0) > > > at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 > > > #6 0x0000555555647204 in object_property_set_bool > > > (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", > > > value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) > > > at ../home/iii/myrepos/qemu/qom/object.c:1541 > > > #7 0x000055555564025c in qdev_realize (dev=<optimized out>, > > > bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at > > > ../home/iii/myrepos/qemu/hw/core/qdev.c:291 > > > #8 0x000055555559bbb4 in cpu_create (typename=<optimized out>) > > > at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61 > > > #9 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, > > > envp=<optimized out>) at > > > ../home/iii/myrepos/qemu/linux-user/main.c:811 > > > > > > (gdb) p cpu > > > $1 = (CPUState *) 0x5555557c28c0 > > > (gdb) p cpu->accel > > > $2 = (AccelCPUState *) 0x0 > > > > > > Configured with: '/home/iii/myrepos/qemu/configure' > > > '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' > > > '--disable-fdt' '--disable-capstone' '--disable-docs' > > > > > > If you don't see what can be wrong here right away, I can debug this. > > I added this commit in the same branch: > > -- >8 -- > Author: Philippe Mathieu-Daudé <philmd@linaro.org> > Date: Tue Apr 30 20:57:15 2024 +0200 > > accel/tcg: Initialize TCG plugins in cpu-target.c > > Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> > > diff --git a/cpu-target.c b/cpu-target.c > index 5af120e8aa..585533cfa3 100644 > --- a/cpu-target.c > +++ b/cpu-target.c > @@ -46,6 +46,10 @@ > #include "hw/core/accel-cpu.h" > #include "trace/trace-root.h" > #include "qemu/accel.h" > +#ifdef CONFIG_PLUGIN > +#include "accel/tcg/vcpu-state.h" > +#include "qemu/plugin.h" > +#endif > > #ifndef CONFIG_USER_ONLY > static int cpu_common_post_load(void *opaque, int version_id) > @@ -131,6 +135,13 @@ const VMStateDescription vmstate_cpu_common = { > }; > #endif > > +#ifdef CONFIG_PLUGIN > +static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data > unused) > +{ > + qemu_plugin_vcpu_init_hook(cpu); > +} > +#endif > + > bool cpu_exec_realizefn(CPUState *cpu, Error **errp) > { > /* cache the cpu class for the hotpath */ > @@ -143,6 +154,15 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp) > /* Wait until cpu initialization complete before exposing cpu. */ > cpu_list_add(cpu); > > +#ifdef CONFIG_PLUGIN > + assert(cpu->accel); > + /* Plugin initialization must wait until the cpu start executing code > */ > + if (tcg_enabled()) { > + cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); > + async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, > RUN_ON_CPU_NULL); > + } > +#endif > + > #ifdef CONFIG_USER_ONLY > assert(qdev_get_vmsd(DEVICE(cpu)) == NULL || > qdev_get_vmsd(DEVICE(cpu))->unmigratable); > @@ -171,6 +191,13 @@ void cpu_exec_unrealizefn(CPUState *cpu) > } > #endif > > +#ifdef CONFIG_PLUGIN > + /* Call the plugin hook before clearing the cpu is fully unrealized */ > + if (tcg_enabled()) { > + qemu_plugin_vcpu_exit_hook(cpu); > + } > +#endif > + > cpu_list_remove(cpu); > /* > * Now that the vCPU has been removed from the RCU list, we can call > diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c > index e03d31876f..cd8bd99131 100644 > --- a/hw/core/cpu-common.c > +++ b/hw/core/cpu-common.c > @@ -30,10 +30,6 @@ > #include "hw/boards.h" > #include "hw/qdev-properties.h" > #include "trace.h" > -#ifdef CONFIG_PLUGIN > -#include "accel/tcg/vcpu-state.h" > -#include "qemu/plugin.h" > -#endif > > CPUState *cpu_by_arch_id(int64_t id) > { > @@ -181,13 +177,6 @@ static void cpu_common_parse_features(const char > *typename, char *features, > } > } > > -#ifdef CONFIG_PLUGIN > -static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data > unused) > -{ > - qemu_plugin_vcpu_init_hook(cpu); > -} > -#endif > - > static void cpu_common_realizefn(DeviceState *dev, Error **errp) > { > CPUState *cpu = CPU(dev); > @@ -211,14 +200,6 @@ static void cpu_common_realizefn(DeviceState *dev, > Error **errp) > cpu_resume(cpu); > } > > - /* Plugin initialization must wait until the cpu start executing code > */ > -#ifdef CONFIG_PLUGIN > - if (tcg_enabled()) { > - cpu->accel->plugin_state = qemu_plugin_create_vcpu_state(); > - async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, > RUN_ON_CPU_NULL); > - } > -#endif > - > /* NOTE: latest generic point where the cpu is fully realized */ > } > > @@ -226,13 +207,6 @@ static void cpu_common_unrealizefn(DeviceState *dev) > { > CPUState *cpu = CPU(dev); > > - /* Call the plugin hook before clearing the cpu is fully unrealized */ > -#ifdef CONFIG_PLUGIN > - if (tcg_enabled()) { > - qemu_plugin_vcpu_exit_hook(cpu); > - } > -#endif > - > /* NOTE: latest generic point before the cpu is fully unrealized */ > cpu_exec_unrealizefn(cpu); > } > --- > > Totally untested here because it is late (only built...). Now I get: Thread 1 "qemu-s390x" received signal SIGABRT, Aborted. __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 44 ./nptl/pthread_kill.c: No such file or directory. (gdb) bt #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 #1 0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 #2 0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #3 0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79 #4 0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555556d71b8 "cpu->accel", file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:92 #5 0x00007ffff7bebeb2 in __GI___assert_fail (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:101 #6 0x00005555555d44ca in cpu_exec_realizefn (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at ../home/iii/myrepos/qemu/cpu-target.c:158 #7 0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261 #8 0x000055555563f78b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510 #9 0x000055555564365d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 #10 0x0000555555646bbb in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:1471 #11 0x000055555564a45f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 #12 0x0000555555647224 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:1541 #13 0x000055555564027c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291 #14 0x000055555559bb54 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:57 #15 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811 Here is the executable file: http://0x0.st/XXHp.gz sha256sum: 58eb8d2a90c08f772ae94e20a7a8c7567bd886fe022a6b9e117912cc13acbd82 Best regards, Ilya
On 30/4/24 23:42, Ilya Leoshkevich wrote: > On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote: >> On 30/4/24 20:45, Philippe Mathieu-Daudé wrote: >>> Hi Ilya, >>> >>> On 30/4/24 19:55, Ilya Leoshkevich wrote: >>>> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote: >>>>> Missing WASM testing by Ilya (branch available at >>>>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) >>>> >>>> Hmm, it dies very early now: >>>> >>>> # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 >>>> >>>> Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. >>>> 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, >>>> errp=<optimized out>) at >>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >>>> 217 cpu->accel->plugin_state = >>>> qemu_plugin_create_vcpu_state(); >>>> >>>> (gdb) bt >>>> #0 0x000055555559b718 in cpu_common_realizefn >>>> (dev=0x5555557c28c0, errp=<optimized out>) at >>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >>>> #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, >>>> errp=0x7fffffffe1a0) at >>>> ../home/iii/myrepos/qemu/target/s390x/cpu.c:284 >>>> #2 0x000055555563f76b in device_set_realized (obj=<optimized >>>> out>, value=<optimized out>, errp=0x7fffffffe2e0) at >>>> ../home/iii/myrepos/qemu/hw/core/qdev.c:510 >>>> #3 0x000055555564363d in property_set_bool (obj=0x5555557c28c0, >>>> v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, >>>> errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 >>>> #4 0x0000555555646b9b in object_property_set >>>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 >>>> "realized", v=v@entry=0x5555557c6650, >>>> errp=errp@entry=0x7fffffffe2e0) >>>> at ../home/iii/myrepos/qemu/qom/object.c:1471 >>>> #5 0x000055555564a43f in object_property_set_qobject >>>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 >>>> "realized", value=value@entry=0x5555557a7a90, >>>> errp=errp@entry=0x7fffffffe2e0) >>>> at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 >>>> #6 0x0000555555647204 in object_property_set_bool >>>> (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", >>>> value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) >>>> at ../home/iii/myrepos/qemu/qom/object.c:1541 >>>> #7 0x000055555564025c in qdev_realize (dev=<optimized out>, >>>> bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at >>>> ../home/iii/myrepos/qemu/hw/core/qdev.c:291 >>>> #8 0x000055555559bbb4 in cpu_create (typename=<optimized out>) >>>> at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61 >>>> #9 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, >>>> envp=<optimized out>) at >>>> ../home/iii/myrepos/qemu/linux-user/main.c:811 >>>> >>>> (gdb) p cpu >>>> $1 = (CPUState *) 0x5555557c28c0 >>>> (gdb) p cpu->accel >>>> $2 = (AccelCPUState *) 0x0 >>>> >>>> Configured with: '/home/iii/myrepos/qemu/configure' >>>> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' >>>> '--disable-fdt' '--disable-capstone' '--disable-docs' >>>> > Now I get: > > Thread 1 "qemu-s390x" received signal SIGABRT, Aborted. > __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 > 44 ./nptl/pthread_kill.c: No such file or directory. > (gdb) bt > #0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 > #1 0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78 > #2 0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 > #3 0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79 > #4 0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555556d71b8 "cpu->accel", > file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:92 > #5 0x00007ffff7bebeb2 in __GI___assert_fail (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, > function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:101 > #6 0x00005555555d44ca in cpu_exec_realizefn (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at ../home/iii/myrepos/qemu/cpu-target.c:158 > #7 0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261 > #8 0x000055555563f78b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510 > #9 0x000055555564365d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362 > #10 0x0000555555646bbb in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/object.c:1471 > #11 0x000055555564a45f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28 > #12 0x0000555555647224 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0) > at ../home/iii/myrepos/qemu/qom/object.c:1541 > #13 0x000055555564027c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291 > #14 0x000055555559bb54 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:57 > #15 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811 From code review I think the problem is my commit bb6cf6f016 ("accel/tcg: Factor tcg_cpu_reset_hold() out") which wanted to restrict tlb_flush() to system emulation, but inadvertently also restricted tcg_flush_jmp_cache(), which was before called via Realize -> Reset -> cpu_common_reset_hold(). Apparently now this code can't happen on user emulation.
On 2/5/24 12:27, Philippe Mathieu-Daudé wrote: > On 30/4/24 23:42, Ilya Leoshkevich wrote: >> On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote: >>> On 30/4/24 20:45, Philippe Mathieu-Daudé wrote: >>>> Hi Ilya, >>>> >>>> On 30/4/24 19:55, Ilya Leoshkevich wrote: >>>>> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé >>>>> wrote: >>>>>> Missing WASM testing by Ilya (branch available at >>>>>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache) >>>>> >>>>> Hmm, it dies very early now: >>>>> >>>>> # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu >>>>> /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47 >>>>> >>>>> Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault. >>>>> 0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, >>>>> errp=<optimized out>) at >>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >>>>> 217 cpu->accel->plugin_state = >>>>> qemu_plugin_create_vcpu_state(); >>>>> >>>>> (gdb) bt >>>>> #0 0x000055555559b718 in cpu_common_realizefn >>>>> (dev=0x5555557c28c0, errp=<optimized out>) at >>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217 >>>>> #1 0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, >>>>> errp=0x7fffffffe1a0) at >>>>> (gdb) p cpu >>>>> $1 = (CPUState *) 0x5555557c28c0 >>>>> (gdb) p cpu->accel >>>>> $2 = (AccelCPUState *) 0x0 >>>>> >>>>> Configured with: '/home/iii/myrepos/qemu/configure' >>>>> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' >>>>> '--disable-fdt' '--disable-capstone' '--disable-docs' >>>>> > > >> Now I get: >> >> Thread 1 "qemu-s390x" received signal SIGABRT, Aborted. >> __pthread_kill_implementation (threadid=<optimized out>, >> signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 >> 44 ./nptl/pthread_kill.c: No such file or directory. >> (gdb) bt >> #0 __pthread_kill_implementation (threadid=<optimized out>, >> signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44 >> #1 0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, >> threadid=<optimized out>) at ./nptl/pthread_kill.c:78 >> #2 0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at >> ../sysdeps/posix/raise.c:26 >> #3 0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79 >> #4 0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 >> "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", >> assertion=assertion@entry=0x5555556d71b8 "cpu->accel", >> file=file@entry=0x5555556d70e0 >> "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, >> function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> >> "cpu_exec_realizefn") at ./assert/assert.c:92 >> #5 0x00007ffff7bebeb2 in __GI___assert_fail >> (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", >> file=file@entry=0x5555556d70e0 >> "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, >> function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> >> "cpu_exec_realizefn") at ./assert/assert.c:101 >> #6 0x00005555555d44ca in cpu_exec_realizefn >> (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at >> ../home/iii/myrepos/qemu/cpu-target.c:158 >> #7 0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, >> errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261 > From code review I think the problem is my commit bb6cf6f016 > ("accel/tcg: Factor tcg_cpu_reset_hold() out") which wanted > to restrict tlb_flush() to system emulation, but inadvertently > also restricted tcg_flush_jmp_cache(), which was before called > via Realize -> Reset -> cpu_common_reset_hold(). Apparently > now this code can't happen on user emulation. This is indeed the root cause, I'll post a series fixing it.