mbox series

[v3,00/13] exec: Rework around CPUState user fields (part 2)

Message ID 20240430122808.72025-1-philmd@linaro.org (mailing list archive)
Headers show
Series exec: Rework around CPUState user fields (part 2) | expand

Message

Philippe Mathieu-Daudé April 30, 2024, 12:27 p.m. UTC
Missing WASM testing by Ilya (branch available at
https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)

Since v2:
- Move cpu_loop_exit_requested() to "exec/cpu-loop.h"
- Added R-b tags

Since v1:
- First 13 patches queued
- Restrict qemu_plugin_vcpu_exit_hook() to (TCG) plugins
- Restrict cpu_plugin_mem_cbs_enabled() to TCG (plugins)
- Addressed Richard review comments on the others:
  - Move cpu_plugin_mem_cbs_enabled()
  - Do not move mem_io_pc, waiting for [*]
  - Mention can_do_io restricted

Finish extracting TCG fields from CPUState:
- Extract tcg_cpu_exit() from cpu_exit()
- Introduce AccelOpsClass::exit_vcpu_thread()
- cpu_exit() calls exit_vcpu_thread=tcg_cpu_exit for TCG
- Forward declare TaskState and more uses of get_task_state()
- Introduce TCG AccelCPUState
- Move TCG specific fields from CPUState to AccelCPUState
- Restrict "exec/tlb-common.h" to TCG
- Restrict iommu_notifiers, icount to system emulation

[*] https://lore.kernel.org/qemu-devel/20240416040609.1313605-3-richard.henderson@linaro.org/

Based-on: https://gitlab.com/philmd/qemu/-/commits/accel-next

Philippe Mathieu-Daudé (13):
  accel/tcg: Restrict qemu_plugin_vcpu_exit_hook() to TCG plugins
  accel/tcg: Restrict cpu_plugin_mem_cbs_enabled() to TCG
  accel/tcg: Move @plugin_mem_cbs from CPUState to
    CPUNegativeOffsetState
  accel/tcg: Move @plugin_state from CPUState to TCG AccelCPUState
  accel/tcg: Restrict cpu_loop_exit_requested() to TCG
  accel/tcg: Restrict IcountDecr / can_do_io / CPUTLB to TCG
  accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState
  accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState
  accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState
  accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState
  accel/tcg: Restrict icount to system emulation
  accel/tcg: Move icount fields from CPUState to TCG AccelCPUState
  accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState

 accel/tcg/internal-common.h      | 18 ++++++++++
 accel/tcg/tb-jmp-cache.h         |  4 +--
 accel/tcg/tcg-accel-ops.h        |  1 +
 accel/tcg/vcpu-state.h           | 20 +++++++++++
 include/exec/cpu-loop.h          | 35 +++++++++++++++++++
 include/exec/exec-all.h          | 17 ----------
 include/exec/tlb-common.h        |  4 +++
 include/hw/core/cpu.h            | 58 ++++++++------------------------
 include/qemu/plugin.h            |  2 +-
 include/qemu/typedefs.h          |  1 -
 accel/tcg/cpu-exec-common.c      |  2 +-
 accel/tcg/cpu-exec.c             | 52 +++++++++++++++-------------
 accel/tcg/cputlb.c               |  2 +-
 accel/tcg/icount-common.c        |  7 ++--
 accel/tcg/plugin-gen.c           |  9 +++--
 accel/tcg/tb-maint.c             |  6 ++--
 accel/tcg/tcg-accel-ops-icount.c | 14 ++++----
 accel/tcg/tcg-accel-ops.c        |  2 ++
 accel/tcg/translate-all.c        |  9 ++---
 accel/tcg/watchpoint.c           |  5 +--
 hw/core/cpu-common.c             |  9 +++--
 linux-user/main.c                |  2 +-
 plugins/core.c                   |  9 ++---
 system/physmem.c                 | 37 +++++++++++++++-----
 target/arm/tcg/helper-a64.c      |  1 +
 target/s390x/tcg/mem_helper.c    |  1 +
 26 files changed, 195 insertions(+), 132 deletions(-)
 create mode 100644 include/exec/cpu-loop.h

Comments

Ilya Leoshkevich April 30, 2024, 5:55 p.m. UTC | #1
On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote:
> Missing WASM testing by Ilya (branch available at
> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)

Hmm, it dies very early now:

  # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47

  Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
  0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
  217             cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();

  (gdb) bt
  #0  0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
  #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284
  #2  0x000055555563f76b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510
  #3  0x000055555564363d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
  #4  0x0000555555646b9b in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/object.c:1471
  #5  0x000055555564a43f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
  #6  0x0000555555647204 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/object.c:1541
  #7  0x000055555564025c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291
  #8  0x000055555559bbb4 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61
  #9  0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811

  (gdb) p cpu
  $1 = (CPUState *) 0x5555557c28c0
  (gdb) p cpu->accel
  $2 = (AccelCPUState *) 0x0

Configured with: '/home/iii/myrepos/qemu/configure' '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' '--disable-fdt' '--disable-capstone' '--disable-docs'

If you don't see what can be wrong here right away, I can debug this.

> Since v2:
> - Move cpu_loop_exit_requested() to "exec/cpu-loop.h"
> - Added R-b tags
> 
> Since v1:
> - First 13 patches queued
> - Restrict qemu_plugin_vcpu_exit_hook() to (TCG) plugins
> - Restrict cpu_plugin_mem_cbs_enabled() to TCG (plugins)
> - Addressed Richard review comments on the others:
>   - Move cpu_plugin_mem_cbs_enabled()
>   - Do not move mem_io_pc, waiting for [*]
>   - Mention can_do_io restricted
> 
> Finish extracting TCG fields from CPUState:
> - Extract tcg_cpu_exit() from cpu_exit()
> - Introduce AccelOpsClass::exit_vcpu_thread()
> - cpu_exit() calls exit_vcpu_thread=tcg_cpu_exit for TCG
> - Forward declare TaskState and more uses of get_task_state()
> - Introduce TCG AccelCPUState
> - Move TCG specific fields from CPUState to AccelCPUState
> - Restrict "exec/tlb-common.h" to TCG
> - Restrict iommu_notifiers, icount to system emulation
> 
> [*] https://lore.kernel.org/qemu-devel/20240416040609.1313605-3-richard.henderson@linaro.org/
> 
> Based-on: https://gitlab.com/philmd/qemu/-/commits/accel-next
> 
> Philippe Mathieu-Daudé (13):
>   accel/tcg: Restrict qemu_plugin_vcpu_exit_hook() to TCG plugins
>   accel/tcg: Restrict cpu_plugin_mem_cbs_enabled() to TCG
>   accel/tcg: Move @plugin_mem_cbs from CPUState to
>     CPUNegativeOffsetState
>   accel/tcg: Move @plugin_state from CPUState to TCG AccelCPUState
>   accel/tcg: Restrict cpu_loop_exit_requested() to TCG
>   accel/tcg: Restrict IcountDecr / can_do_io / CPUTLB to TCG
>   accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState
>   accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState
>   accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState
>   accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState
>   accel/tcg: Restrict icount to system emulation
>   accel/tcg: Move icount fields from CPUState to TCG AccelCPUState
>   accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState
> 
>  accel/tcg/internal-common.h      | 18 ++++++++++
>  accel/tcg/tb-jmp-cache.h         |  4 +--
>  accel/tcg/tcg-accel-ops.h        |  1 +
>  accel/tcg/vcpu-state.h           | 20 +++++++++++
>  include/exec/cpu-loop.h          | 35 +++++++++++++++++++
>  include/exec/exec-all.h          | 17 ----------
>  include/exec/tlb-common.h        |  4 +++
>  include/hw/core/cpu.h            | 58 ++++++++------------------------
>  include/qemu/plugin.h            |  2 +-
>  include/qemu/typedefs.h          |  1 -
>  accel/tcg/cpu-exec-common.c      |  2 +-
>  accel/tcg/cpu-exec.c             | 52 +++++++++++++++-------------
>  accel/tcg/cputlb.c               |  2 +-
>  accel/tcg/icount-common.c        |  7 ++--
>  accel/tcg/plugin-gen.c           |  9 +++--
>  accel/tcg/tb-maint.c             |  6 ++--
>  accel/tcg/tcg-accel-ops-icount.c | 14 ++++----
>  accel/tcg/tcg-accel-ops.c        |  2 ++
>  accel/tcg/translate-all.c        |  9 ++---
>  accel/tcg/watchpoint.c           |  5 +--
>  hw/core/cpu-common.c             |  9 +++--
>  linux-user/main.c                |  2 +-
>  plugins/core.c                   |  9 ++---
>  system/physmem.c                 | 37 +++++++++++++++-----
>  target/arm/tcg/helper-a64.c      |  1 +
>  target/s390x/tcg/mem_helper.c    |  1 +
>  26 files changed, 195 insertions(+), 132 deletions(-)
>  create mode 100644 include/exec/cpu-loop.h
> 
> -- 
> 2.41.0
>
Philippe Mathieu-Daudé April 30, 2024, 6:45 p.m. UTC | #2
Hi Ilya,

On 30/4/24 19:55, Ilya Leoshkevich wrote:
> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote:
>> Missing WASM testing by Ilya (branch available at
>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)
> 
> Hmm, it dies very early now:
> 
>    # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47
> 
>    Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
>    0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>    217             cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();
> 
>    (gdb) bt
>    #0  0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, errp=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>    #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284
>    #2  0x000055555563f76b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510
>    #3  0x000055555564363d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
>    #4  0x0000555555646b9b in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/object.c:1471
>    #5  0x000055555564a43f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
>    #6  0x0000555555647204 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/object.c:1541
>    #7  0x000055555564025c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291
>    #8  0x000055555559bbb4 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61
>    #9  0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811
> 
>    (gdb) p cpu
>    $1 = (CPUState *) 0x5555557c28c0
>    (gdb) p cpu->accel
>    $2 = (AccelCPUState *) 0x0
> 
> Configured with: '/home/iii/myrepos/qemu/configure' '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' '--disable-fdt' '--disable-capstone' '--disable-docs'
> 
> If you don't see what can be wrong here right away, I can debug this.

Useful enough I guess, but I'll ask you to test again later.

Does it work without the last patch?

Is it possible to share component_fuzz_util-d10a3a6b4ad8af47?

Thanks for the testing,

Phil.
Philippe Mathieu-Daudé April 30, 2024, 7 p.m. UTC | #3
On 30/4/24 20:45, Philippe Mathieu-Daudé wrote:
> Hi Ilya,
> 
> On 30/4/24 19:55, Ilya Leoshkevich wrote:
>> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote:
>>> Missing WASM testing by Ilya (branch available at
>>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)
>>
>> Hmm, it dies very early now:
>>
>>    # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu 
>> /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47
>>
>>    Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
>>    0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, 
>> errp=<optimized out>) at 
>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>    217             cpu->accel->plugin_state = 
>> qemu_plugin_create_vcpu_state();
>>
>>    (gdb) bt
>>    #0  0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0, 
>> errp=<optimized out>) at 
>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>    #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0, 
>> errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:284
>>    #2  0x000055555563f76b in device_set_realized (obj=<optimized out>, 
>> value=<optimized out>, errp=0x7fffffffe2e0) at 
>> ../home/iii/myrepos/qemu/hw/core/qdev.c:510
>>    #3  0x000055555564363d in property_set_bool (obj=0x5555557c28c0, 
>> v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, 
>> errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
>>    #4  0x0000555555646b9b in object_property_set 
>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 
>> "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0)
>>        at ../home/iii/myrepos/qemu/qom/object.c:1471
>>    #5  0x000055555564a43f in object_property_set_qobject 
>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 
>> "realized", value=value@entry=0x5555557a7a90, 
>> errp=errp@entry=0x7fffffffe2e0)
>>        at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
>>    #6  0x0000555555647204 in object_property_set_bool 
>> (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", 
>> value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
>>        at ../home/iii/myrepos/qemu/qom/object.c:1541
>>    #7  0x000055555564025c in qdev_realize (dev=<optimized out>, 
>> bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at 
>> ../home/iii/myrepos/qemu/hw/core/qdev.c:291
>>    #8  0x000055555559bbb4 in cpu_create (typename=<optimized out>) at 
>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61
>>    #9  0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, 
>> envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811
>>
>>    (gdb) p cpu
>>    $1 = (CPUState *) 0x5555557c28c0
>>    (gdb) p cpu->accel
>>    $2 = (AccelCPUState *) 0x0
>>
>> Configured with: '/home/iii/myrepos/qemu/configure' 
>> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp' 
>> '--disable-fdt' '--disable-capstone' '--disable-docs'
>>
>> If you don't see what can be wrong here right away, I can debug this.

I added this commit in the same branch:

-- >8 --
Author: Philippe Mathieu-Daudé <philmd@linaro.org>
Date:   Tue Apr 30 20:57:15 2024 +0200

     accel/tcg: Initialize TCG plugins in cpu-target.c

     Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

diff --git a/cpu-target.c b/cpu-target.c
index 5af120e8aa..585533cfa3 100644
--- a/cpu-target.c
+++ b/cpu-target.c
@@ -46,6 +46,10 @@
  #include "hw/core/accel-cpu.h"
  #include "trace/trace-root.h"
  #include "qemu/accel.h"
+#ifdef CONFIG_PLUGIN
+#include "accel/tcg/vcpu-state.h"
+#include "qemu/plugin.h"
+#endif

  #ifndef CONFIG_USER_ONLY
  static int cpu_common_post_load(void *opaque, int version_id)
@@ -131,6 +135,13 @@ const VMStateDescription vmstate_cpu_common = {
  };
  #endif

+#ifdef CONFIG_PLUGIN
+static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data 
unused)
+{
+    qemu_plugin_vcpu_init_hook(cpu);
+}
+#endif
+
  bool cpu_exec_realizefn(CPUState *cpu, Error **errp)
  {
      /* cache the cpu class for the hotpath */
@@ -143,6 +154,15 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp)
      /* Wait until cpu initialization complete before exposing cpu. */
      cpu_list_add(cpu);

+#ifdef CONFIG_PLUGIN
+    assert(cpu->accel);
+    /* Plugin initialization must wait until the cpu start executing 
code */
+    if (tcg_enabled()) {
+        cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();
+        async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, 
RUN_ON_CPU_NULL);
+    }
+#endif
+
  #ifdef CONFIG_USER_ONLY
      assert(qdev_get_vmsd(DEVICE(cpu)) == NULL ||
             qdev_get_vmsd(DEVICE(cpu))->unmigratable);
@@ -171,6 +191,13 @@ void cpu_exec_unrealizefn(CPUState *cpu)
      }
  #endif

+#ifdef CONFIG_PLUGIN
+    /* Call the plugin hook before clearing the cpu is fully unrealized */
+    if (tcg_enabled()) {
+        qemu_plugin_vcpu_exit_hook(cpu);
+    }
+#endif
+
      cpu_list_remove(cpu);
      /*
       * Now that the vCPU has been removed from the RCU list, we can call
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index e03d31876f..cd8bd99131 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -30,10 +30,6 @@
  #include "hw/boards.h"
  #include "hw/qdev-properties.h"
  #include "trace.h"
-#ifdef CONFIG_PLUGIN
-#include "accel/tcg/vcpu-state.h"
-#include "qemu/plugin.h"
-#endif

  CPUState *cpu_by_arch_id(int64_t id)
  {
@@ -181,13 +177,6 @@ static void cpu_common_parse_features(const char 
*typename, char *features,
      }
  }

-#ifdef CONFIG_PLUGIN
-static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data 
unused)
-{
-    qemu_plugin_vcpu_init_hook(cpu);
-}
-#endif
-
  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
  {
      CPUState *cpu = CPU(dev);
@@ -211,14 +200,6 @@ static void cpu_common_realizefn(DeviceState *dev, 
Error **errp)
          cpu_resume(cpu);
      }

-    /* Plugin initialization must wait until the cpu start executing 
code */
-#ifdef CONFIG_PLUGIN
-    if (tcg_enabled()) {
-        cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();
-        async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, 
RUN_ON_CPU_NULL);
-    }
-#endif
-
      /* NOTE: latest generic point where the cpu is fully realized */
  }

@@ -226,13 +207,6 @@ static void cpu_common_unrealizefn(DeviceState *dev)
  {
      CPUState *cpu = CPU(dev);

-    /* Call the plugin hook before clearing the cpu is fully unrealized */
-#ifdef CONFIG_PLUGIN
-    if (tcg_enabled()) {
-        qemu_plugin_vcpu_exit_hook(cpu);
-    }
-#endif
-
      /* NOTE: latest generic point before the cpu is fully unrealized */
      cpu_exec_unrealizefn(cpu);
  }
---

Totally untested here because it is late (only built...).
Ilya Leoshkevich April 30, 2024, 9:42 p.m. UTC | #4
On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote:
> On 30/4/24 20:45, Philippe Mathieu-Daudé wrote:
> > Hi Ilya,
> > 
> > On 30/4/24 19:55, Ilya Leoshkevich wrote:
> > > On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote:
> > > > Missing WASM testing by Ilya (branch available at
> > > > https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)
> > > 
> > > Hmm, it dies very early now:
> > > 
> > >    # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47
> > > 
> > >    Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
> > >    0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0,
> > > errp=<optimized out>) at
> > > ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
> > >    217             cpu->accel->plugin_state =
> > > qemu_plugin_create_vcpu_state();
> > > 
> > >    (gdb) bt
> > >    #0  0x000055555559b718 in cpu_common_realizefn
> > > (dev=0x5555557c28c0, errp=<optimized out>) at
> > > ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
> > >    #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0,
> > > errp=0x7fffffffe1a0) at
> > > ../home/iii/myrepos/qemu/target/s390x/cpu.c:284
> > >    #2  0x000055555563f76b in device_set_realized (obj=<optimized
> > > out>, value=<optimized out>, errp=0x7fffffffe2e0) at
> > > ../home/iii/myrepos/qemu/hw/core/qdev.c:510
> > >    #3  0x000055555564363d in property_set_bool (obj=0x5555557c28c0,
> > > v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140,
> > > errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
> > >    #4  0x0000555555646b9b in object_property_set
> > > (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2
> > > "realized", v=v@entry=0x5555557c6650,
> > > errp=errp@entry=0x7fffffffe2e0)
> > >        at ../home/iii/myrepos/qemu/qom/object.c:1471
> > >    #5  0x000055555564a43f in object_property_set_qobject
> > > (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2
> > > "realized", value=value@entry=0x5555557a7a90,
> > > errp=errp@entry=0x7fffffffe2e0)
> > >        at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
> > >    #6  0x0000555555647204 in object_property_set_bool
> > > (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized",
> > > value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
> > >        at ../home/iii/myrepos/qemu/qom/object.c:1541
> > >    #7  0x000055555564025c in qdev_realize (dev=<optimized out>,
> > > bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at
> > > ../home/iii/myrepos/qemu/hw/core/qdev.c:291
> > >    #8  0x000055555559bbb4 in cpu_create (typename=<optimized out>)
> > > at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61
> > >    #9  0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8,
> > > envp=<optimized out>) at
> > > ../home/iii/myrepos/qemu/linux-user/main.c:811
> > > 
> > >    (gdb) p cpu
> > >    $1 = (CPUState *) 0x5555557c28c0
> > >    (gdb) p cpu->accel
> > >    $2 = (AccelCPUState *) 0x0
> > > 
> > > Configured with: '/home/iii/myrepos/qemu/configure'
> > > '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp'
> > > '--disable-fdt' '--disable-capstone' '--disable-docs'
> > > 
> > > If you don't see what can be wrong here right away, I can debug this.
> 
> I added this commit in the same branch:
> 
> -- >8 --
> Author: Philippe Mathieu-Daudé <philmd@linaro.org>
> Date:   Tue Apr 30 20:57:15 2024 +0200
> 
>     accel/tcg: Initialize TCG plugins in cpu-target.c
> 
>     Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> 
> diff --git a/cpu-target.c b/cpu-target.c
> index 5af120e8aa..585533cfa3 100644
> --- a/cpu-target.c
> +++ b/cpu-target.c
> @@ -46,6 +46,10 @@
>  #include "hw/core/accel-cpu.h"
>  #include "trace/trace-root.h"
>  #include "qemu/accel.h"
> +#ifdef CONFIG_PLUGIN
> +#include "accel/tcg/vcpu-state.h"
> +#include "qemu/plugin.h"
> +#endif
> 
>  #ifndef CONFIG_USER_ONLY
>  static int cpu_common_post_load(void *opaque, int version_id)
> @@ -131,6 +135,13 @@ const VMStateDescription vmstate_cpu_common = {
>  };
>  #endif
> 
> +#ifdef CONFIG_PLUGIN
> +static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data
> unused)
> +{
> +    qemu_plugin_vcpu_init_hook(cpu);
> +}
> +#endif
> +
>  bool cpu_exec_realizefn(CPUState *cpu, Error **errp)
>  {
>      /* cache the cpu class for the hotpath */
> @@ -143,6 +154,15 @@ bool cpu_exec_realizefn(CPUState *cpu, Error **errp)
>      /* Wait until cpu initialization complete before exposing cpu. */
>      cpu_list_add(cpu);
> 
> +#ifdef CONFIG_PLUGIN
> +    assert(cpu->accel);
> +    /* Plugin initialization must wait until the cpu start executing code
> */
> +    if (tcg_enabled()) {
> +        cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();
> +        async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async,
> RUN_ON_CPU_NULL);
> +    }
> +#endif
> +
>  #ifdef CONFIG_USER_ONLY
>      assert(qdev_get_vmsd(DEVICE(cpu)) == NULL ||
>             qdev_get_vmsd(DEVICE(cpu))->unmigratable);
> @@ -171,6 +191,13 @@ void cpu_exec_unrealizefn(CPUState *cpu)
>      }
>  #endif
> 
> +#ifdef CONFIG_PLUGIN
> +    /* Call the plugin hook before clearing the cpu is fully unrealized */
> +    if (tcg_enabled()) {
> +        qemu_plugin_vcpu_exit_hook(cpu);
> +    }
> +#endif
> +
>      cpu_list_remove(cpu);
>      /*
>       * Now that the vCPU has been removed from the RCU list, we can call
> diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
> index e03d31876f..cd8bd99131 100644
> --- a/hw/core/cpu-common.c
> +++ b/hw/core/cpu-common.c
> @@ -30,10 +30,6 @@
>  #include "hw/boards.h"
>  #include "hw/qdev-properties.h"
>  #include "trace.h"
> -#ifdef CONFIG_PLUGIN
> -#include "accel/tcg/vcpu-state.h"
> -#include "qemu/plugin.h"
> -#endif
> 
>  CPUState *cpu_by_arch_id(int64_t id)
>  {
> @@ -181,13 +177,6 @@ static void cpu_common_parse_features(const char
> *typename, char *features,
>      }
>  }
> 
> -#ifdef CONFIG_PLUGIN
> -static void qemu_plugin_vcpu_init__async(CPUState *cpu, run_on_cpu_data
> unused)
> -{
> -    qemu_plugin_vcpu_init_hook(cpu);
> -}
> -#endif
> -
>  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
>      CPUState *cpu = CPU(dev);
> @@ -211,14 +200,6 @@ static void cpu_common_realizefn(DeviceState *dev,
> Error **errp)
>          cpu_resume(cpu);
>      }
> 
> -    /* Plugin initialization must wait until the cpu start executing code
> */
> -#ifdef CONFIG_PLUGIN
> -    if (tcg_enabled()) {
> -        cpu->accel->plugin_state = qemu_plugin_create_vcpu_state();
> -        async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async,
> RUN_ON_CPU_NULL);
> -    }
> -#endif
> -
>      /* NOTE: latest generic point where the cpu is fully realized */
>  }
> 
> @@ -226,13 +207,6 @@ static void cpu_common_unrealizefn(DeviceState *dev)
>  {
>      CPUState *cpu = CPU(dev);
> 
> -    /* Call the plugin hook before clearing the cpu is fully unrealized */
> -#ifdef CONFIG_PLUGIN
> -    if (tcg_enabled()) {
> -        qemu_plugin_vcpu_exit_hook(cpu);
> -    }
> -#endif
> -
>      /* NOTE: latest generic point before the cpu is fully unrealized */
>      cpu_exec_unrealizefn(cpu);
>  }
> ---
> 
> Totally untested here because it is late (only built...).

Now I get:

  Thread 1 "qemu-s390x" received signal SIGABRT, Aborted.
  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
  44      ./nptl/pthread_kill.c: No such file or directory.
  (gdb) bt
  #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
  #1  0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
  #2  0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
  #3  0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79
  #4  0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555556d71b8 "cpu->accel", 
      file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:92
  #5  0x00007ffff7bebeb2 in __GI___assert_fail (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, 
      function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:101
  #6  0x00005555555d44ca in cpu_exec_realizefn (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at ../home/iii/myrepos/qemu/cpu-target.c:158
  #7  0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261
  #8  0x000055555563f78b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510
  #9  0x000055555564365d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
  #10 0x0000555555646bbb in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/object.c:1471
  #11 0x000055555564a45f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
  #12 0x0000555555647224 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
      at ../home/iii/myrepos/qemu/qom/object.c:1541
  #13 0x000055555564027c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291
  #14 0x000055555559bb54 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:57
  #15 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811

Here is the executable file: http://0x0.st/XXHp.gz
sha256sum: 58eb8d2a90c08f772ae94e20a7a8c7567bd886fe022a6b9e117912cc13acbd82

Best regards,
Ilya
Philippe Mathieu-Daudé May 2, 2024, 10:27 a.m. UTC | #5
On 30/4/24 23:42, Ilya Leoshkevich wrote:
> On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote:
>> On 30/4/24 20:45, Philippe Mathieu-Daudé wrote:
>>> Hi Ilya,
>>>
>>> On 30/4/24 19:55, Ilya Leoshkevich wrote:
>>>> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé wrote:
>>>>> Missing WASM testing by Ilya (branch available at
>>>>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)
>>>>
>>>> Hmm, it dies very early now:
>>>>
>>>>     # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47
>>>>
>>>>     Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
>>>>     0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0,
>>>> errp=<optimized out>) at
>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>>>     217             cpu->accel->plugin_state =
>>>> qemu_plugin_create_vcpu_state();
>>>>
>>>>     (gdb) bt
>>>>     #0  0x000055555559b718 in cpu_common_realizefn
>>>> (dev=0x5555557c28c0, errp=<optimized out>) at
>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>>>     #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0,
>>>> errp=0x7fffffffe1a0) at
>>>> ../home/iii/myrepos/qemu/target/s390x/cpu.c:284
>>>>     #2  0x000055555563f76b in device_set_realized (obj=<optimized
>>>> out>, value=<optimized out>, errp=0x7fffffffe2e0) at
>>>> ../home/iii/myrepos/qemu/hw/core/qdev.c:510
>>>>     #3  0x000055555564363d in property_set_bool (obj=0x5555557c28c0,
>>>> v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140,
>>>> errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
>>>>     #4  0x0000555555646b9b in object_property_set
>>>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2
>>>> "realized", v=v@entry=0x5555557c6650,
>>>> errp=errp@entry=0x7fffffffe2e0)
>>>>         at ../home/iii/myrepos/qemu/qom/object.c:1471
>>>>     #5  0x000055555564a43f in object_property_set_qobject
>>>> (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2
>>>> "realized", value=value@entry=0x5555557a7a90,
>>>> errp=errp@entry=0x7fffffffe2e0)
>>>>         at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
>>>>     #6  0x0000555555647204 in object_property_set_bool
>>>> (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized",
>>>> value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
>>>>         at ../home/iii/myrepos/qemu/qom/object.c:1541
>>>>     #7  0x000055555564025c in qdev_realize (dev=<optimized out>,
>>>> bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at
>>>> ../home/iii/myrepos/qemu/hw/core/qdev.c:291
>>>>     #8  0x000055555559bbb4 in cpu_create (typename=<optimized out>)
>>>> at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:61
>>>>     #9  0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8,
>>>> envp=<optimized out>) at
>>>> ../home/iii/myrepos/qemu/linux-user/main.c:811
>>>>
>>>>     (gdb) p cpu
>>>>     $1 = (CPUState *) 0x5555557c28c0
>>>>     (gdb) p cpu->accel
>>>>     $2 = (AccelCPUState *) 0x0
>>>>
>>>> Configured with: '/home/iii/myrepos/qemu/configure'
>>>> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp'
>>>> '--disable-fdt' '--disable-capstone' '--disable-docs'
>>>>


> Now I get:
> 
>    Thread 1 "qemu-s390x" received signal SIGABRT, Aborted.
>    __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
>    44      ./nptl/pthread_kill.c: No such file or directory.
>    (gdb) bt
>    #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
>    #1  0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
>    #2  0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
>    #3  0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79
>    #4  0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555556d71b8 "cpu->accel",
>        file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:92
>    #5  0x00007ffff7bebeb2 in __GI___assert_fail (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", file=file@entry=0x5555556d70e0 "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158,
>        function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> "cpu_exec_realizefn") at ./assert/assert.c:101
>    #6  0x00005555555d44ca in cpu_exec_realizefn (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at ../home/iii/myrepos/qemu/cpu-target.c:158
>    #7  0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261
>    #8  0x000055555563f78b in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:510
>    #9  0x000055555564365d in property_set_bool (obj=0x5555557c28c0, v=<optimized out>, name=<optimized out>, opaque=0x5555557a9140, errp=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/qom/object.c:2362
>    #10 0x0000555555646bbb in object_property_set (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", v=v@entry=0x5555557c6650, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/object.c:1471
>    #11 0x000055555564a45f in object_property_set_qobject (obj=obj@entry=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=0x5555557a7a90, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/qom-qobject.c:28
>    #12 0x0000555555647224 in object_property_set_bool (obj=0x5555557c28c0, name=name@entry=0x5555556e8ae2 "realized", value=value@entry=true, errp=errp@entry=0x7fffffffe2e0)
>        at ../home/iii/myrepos/qemu/qom/object.c:1541
>    #13 0x000055555564027c in qdev_realize (dev=<optimized out>, bus=bus@entry=0x0, errp=errp@entry=0x7fffffffe2e0) at ../home/iii/myrepos/qemu/hw/core/qdev.c:291
>    #14 0x000055555559bb54 in cpu_create (typename=<optimized out>) at ../home/iii/myrepos/qemu/hw/core/cpu-common.c:57
>    #15 0x000055555559a467 in main (argc=4, argv=0x7fffffffeaa8, envp=<optimized out>) at ../home/iii/myrepos/qemu/linux-user/main.c:811

 From code review I think the problem is my commit bb6cf6f016
("accel/tcg: Factor tcg_cpu_reset_hold() out") which wanted
to restrict tlb_flush() to system emulation, but inadvertently
also restricted tcg_flush_jmp_cache(), which was before called
via Realize -> Reset -> cpu_common_reset_hold(). Apparently
now this code can't happen on user emulation.
Philippe Mathieu-Daudé May 2, 2024, 1:35 p.m. UTC | #6
On 2/5/24 12:27, Philippe Mathieu-Daudé wrote:
> On 30/4/24 23:42, Ilya Leoshkevich wrote:
>> On Tue, Apr 30, 2024 at 09:00:17PM +0200, Philippe Mathieu-Daudé wrote:
>>> On 30/4/24 20:45, Philippe Mathieu-Daudé wrote:
>>>> Hi Ilya,
>>>>
>>>> On 30/4/24 19:55, Ilya Leoshkevich wrote:
>>>>> On Tue, Apr 30, 2024 at 02:27:54PM +0200, Philippe Mathieu-Daudé 
>>>>> wrote:
>>>>>> Missing WASM testing by Ilya (branch available at
>>>>>> https://gitlab.com/philmd/qemu/-/commits/tcg_flush_jmp_cache)
>>>>>
>>>>> Hmm, it dies very early now:
>>>>>
>>>>>     # gdb --args ./qemu-s390x -L /usr/s390x-linux-gnu 
>>>>> /build/wasmtime/target/s390x-unknown-linux-gnu/debug/deps/component_fuzz_util-d10a3a6b4ad8af47
>>>>>
>>>>>     Thread 1 "qemu-s390x" received signal SIGSEGV, Segmentation fault.
>>>>>     0x000055555559b718 in cpu_common_realizefn (dev=0x5555557c28c0,
>>>>> errp=<optimized out>) at
>>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>>>>     217             cpu->accel->plugin_state =
>>>>> qemu_plugin_create_vcpu_state();
>>>>>
>>>>>     (gdb) bt
>>>>>     #0  0x000055555559b718 in cpu_common_realizefn
>>>>> (dev=0x5555557c28c0, errp=<optimized out>) at
>>>>> ../home/iii/myrepos/qemu/hw/core/cpu-common.c:217
>>>>>     #1  0x000055555559f59a in s390_cpu_realizefn (dev=0x5555557c28c0,
>>>>> errp=0x7fffffffe1a0) at


>>>>>     (gdb) p cpu
>>>>>     $1 = (CPUState *) 0x5555557c28c0
>>>>>     (gdb) p cpu->accel
>>>>>     $2 = (AccelCPUState *) 0x0
>>>>>
>>>>> Configured with: '/home/iii/myrepos/qemu/configure'
>>>>> '--target-list=s390x-linux-user' '--disable-tools' '--disable-slirp'
>>>>> '--disable-fdt' '--disable-capstone' '--disable-docs'
>>>>>
> 
> 
>> Now I get:
>>
>>    Thread 1 "qemu-s390x" received signal SIGABRT, Aborted.
>>    __pthread_kill_implementation (threadid=<optimized out>, 
>> signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
>>    44      ./nptl/pthread_kill.c: No such file or directory.
>>    (gdb) bt
>>    #0  __pthread_kill_implementation (threadid=<optimized out>, 
>> signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
>>    #1  0x00007ffff7c41e8f in __pthread_kill_internal (signo=6, 
>> threadid=<optimized out>) at ./nptl/pthread_kill.c:78
>>    #2  0x00007ffff7bf2fb2 in __GI_raise (sig=sig@entry=6) at 
>> ../sysdeps/posix/raise.c:26
>>    #3  0x00007ffff7bdd472 in __GI_abort () at ./stdlib/abort.c:79
>>    #4  0x00007ffff7bdd395 in __assert_fail_base (fmt=0x7ffff7d51a90 
>> "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
>> assertion=assertion@entry=0x5555556d71b8 "cpu->accel",
>>        file=file@entry=0x5555556d70e0 
>> "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158, 
>> function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> 
>> "cpu_exec_realizefn") at ./assert/assert.c:92
>>    #5  0x00007ffff7bebeb2 in __GI___assert_fail 
>> (assertion=assertion@entry=0x5555556d71b8 "cpu->accel", 
>> file=file@entry=0x5555556d70e0 
>> "../home/iii/myrepos/qemu/cpu-target.c", line=line@entry=158,
>>        function=function@entry=0x5555556d7260 <__PRETTY_FUNCTION__.3> 
>> "cpu_exec_realizefn") at ./assert/assert.c:101
>>    #6  0x00005555555d44ca in cpu_exec_realizefn 
>> (cpu=cpu@entry=0x5555557c28c0, errp=errp@entry=0x7fffffffe140) at 
>> ../home/iii/myrepos/qemu/cpu-target.c:158
>>    #7  0x000055555559f50b in s390_cpu_realizefn (dev=0x5555557c28c0, 
>> errp=0x7fffffffe1a0) at ../home/iii/myrepos/qemu/target/s390x/cpu.c:261

>  From code review I think the problem is my commit bb6cf6f016
> ("accel/tcg: Factor tcg_cpu_reset_hold() out") which wanted
> to restrict tlb_flush() to system emulation, but inadvertently
> also restricted tcg_flush_jmp_cache(), which was before called
> via Realize -> Reset -> cpu_common_reset_hold(). Apparently
> now this code can't happen on user emulation.

This is indeed the root cause, I'll post a series fixing it.