| Message ID | 20250114180611.353-1-alireza.sanaee@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Specifying cache topology on ARM | expand |
Apologies for the duplication. I needed to resend this patch set because I inadvertently included an extra patch in the version I sent a few minutes ago. I have now removed the additional patch and resent the corrected set.
Hi everyone,
There is a bug in this patch-set on loongarch (function signature of build_pptt), that I will fix in the next version. Just wanted to point this out. Anyhow I will allow time for hopefully some reviews first.
Thanks,
Alireza
-----Original Message-----
From: qemu-arm-bounces+alireza.sanaee=huawei.com@nongnu.org <qemu-arm-bounces+alireza.sanaee=huawei.com@nongnu.org> On Behalf Of Alireza Sanaee via
Sent: Tuesday, January 14, 2025 6:06 PM
To: qemu-devel@nongnu.org; qemu-arm@nongnu.org
Cc: zhao1.liu@intel.com; dapeng1.mi@linux.intel.com; armbru@redhat.com; farman@linux.ibm.com; peter.maydell@linaro.org; mst@redhat.com; anisinha@redhat.com; shannon.zhaosl@gmail.com; imammedo@redhat.com; mtosatti@redhat.com; berrange@redhat.com; richard.henderson@linaro.org; Linuxarm <linuxarm@huawei.com>; Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>; Jonathan Cameron <jonathan.cameron@huawei.com>; jiangkunkun <jiangkunkun@huawei.com>; yangyicong <yangyicong@huawei.com>; sarsanaee@gmail.com
Subject: [RFC PATCH v6 0/6] Specifying cache topology on ARM
Specifying the cache layout in virtual machines is useful for applications and operating systems to fetch accurate information about the cache structure and make appropriate adjustments. Enforcing correct sharing information can lead to better optimizations. This patch enables the specification of cache layout through a command line parameter, building on a patch set by Intel [1,2,3]. It uses this set as a foundation. The device tree and ACPI/PPTT table, and device tree are populated based on user-provided information and CPU topology.
Example:
+----------------+ +----------------+
| Socket 0 | | Socket 1 |
| (L3 Cache) | | (L3 Cache) |
+--------+-------+ +--------+-------+
| |
+--------+--------+ +--------+--------+
| Cluster 0 | | Cluster 0 |
| (L2 Cache) | | (L2 Cache) |
+--------+--------+ +--------+--------+
| |
+--------+--------+ +--------+--------+ +--------+--------+ +--------+----+
| Core 0 | | Core 1 | | Core 0 | | Core 1 |
| (L1i, L1d) | | (L1i, L1d) | | (L1i, L1d) | | (L1i, L1d)|
+--------+--------+ +--------+--------+ +--------+--------+ +--------+----+
| | | |
+--------+ +--------+ +--------+ +--------+
|Thread 0| |Thread 1| |Thread 1| |Thread 0|
+--------+ +--------+ +--------+ +--------+
|Thread 1| |Thread 0| |Thread 0| |Thread 1|
+--------+ +--------+ +--------+ +--------+
The following command will represent the system relying on **ACPI PPTT tables**.
./qemu-system-aarch64 \
-machine virt,smp-cache.0.cache=l1i,smp-cache.0.topology=core,smp-cache.1.cache=l1d,smp-cache.1.topology=core,smp-cache.2.cache=l2,smp-cache.2.topology=cluseter,smp-cache.3.cache=l3,smp-cache.3.topology=socket \ -cpu max \ -m 2048 \ -smp sockets=2,clusters=1,cores=2,threads=2 \ -kernel ./Image.gz \ -append "console=ttyAMA0 root=/dev/ram rdinit=/init acpi=force" \ -initrd rootfs.cpio.gz \ -bios ./edk2-aarch64-code.fd \ -nographic
The following command will represent the system relying on **the device tree**.
./qemu-system-aarch64 \
-machine virt,smp-cache.0.cache=l1i,smp-cache.0.topology=core,smp-cache.1.cache=l1d,smp-cache.1.topology=core,smp-cache.2.cache=l2,smp-cache.2.topology=cluseter,smp-cache.3.cache=l3,smp-cache.3.topology=socket \ -cpu max \ -m 2048 \ -smp sockets=2,clusters=1,cores=2,threads=2 \ -kernel ./Image.gz \ -append "console=ttyAMA0 root=/dev/ram rdinit=/init" \ -initrd rootfs.cpio.gz \ -nographic
Failure cases:
1) There are scenarios where caches exist in systems' registers but
not left unspecified by users. In this case qemu returns failure.
2) At the moment, the device tree is not able to describe caches
shared at core level. In another word, SMT threads cannot share
caches. This will need adjustments in the SPEC. It is worth noting
that this particular case is completely OK in ACPI PPTT tables.
Currently only three levels of caches are supported to be specified from the command line. However, increasing the value does not require significant changes. Further, this patch assumes l2 and l3 unified caches and does not allow l(2/3)(i/d). The level terminology is thread/core/cluster/socket right now.
Here is the hierarchy assumed in this patch:
Socket level = Cluster level + 1 = Core level + 2 = Thread level + 3;
TODO:
1) Making the code to work with arbitrary levels
2) Separated data and instruction cache at L2 and L3.
3) Additional cache controls. e.g. size of L3 may not want to just
match the underlying system, because only some of the associated host
CPUs may be bound to this VM.
Depends-on: Building PPTT with root node and identical implementation flag
Depends-on: Msg-id: 20240926113323.55991-1-yangyicong@huawei.com
Depends-on: i386: Support SMP Cache Topology
Depends-on: Msg-id: 20250110145115.1574345-1-zhao1.liu@intel.com
[1] https://lore.kernel.org/kvm/20240908125920.1160236-1-zhao1.liu@intel.com/
[2] https://lore.kernel.org/qemu-devel/20241101083331.340178-1-zhao1.liu@intel.com/
[3] https://lore.kernel.org/qemu-devel/20250110145115.1574345-1-zhao1.liu@intel.com/
Change Log:
v5->v6:
* Minor bug fix.
* rebase based on new Intel patchset.
- https://lore.kernel.org/qemu-devel/20250110145115.1574345-1-zhao1.liu@intel.com/
v4->v5:
* Added Reviewed-by tags.
* Applied some comments.
v3->v4:
* Device tree added.
Alireza Sanaee (6):
target/arm/tcg: increase cache level for cpu=max
arm/virt.c: add cache hierarchy to device tree
bios-tables-test: prepare to change ARM ACPI virt PPTT
hw/acpi/aml-build.c: add cache hierarchy to pptt table
tests/qtest/bios-table-test: testing new ARM ACPI PPTT topology
Update the ACPI tables according to the acpi aml_build change, also
empty bios-tables-test-allowed-diff.h.
hw/acpi/aml-build.c | 205 +++++++++++-
hw/arm/virt-acpi-build.c | 8 +-
hw/arm/virt.c | 349 +++++++++++++++++++++
hw/cpu/core.c | 92 ++++++
include/hw/acpi/aml-build.h | 4 +-
include/hw/arm/virt.h | 4 +
include/hw/cpu/core.h | 27 ++
target/arm/tcg/cpu64.c | 13 +
tests/data/acpi/aarch64/virt/PPTT.topology | Bin 356 -> 540 bytes
tests/qtest/bios-tables-test.c | 4 +
10 files changed, 699 insertions(+), 7 deletions(-)
--
2.34.1
Specifying the cache layout in virtual machines is useful for applications and operating systems to fetch accurate information about the cache structure and make appropriate adjustments. Enforcing correct sharing information can lead to better optimizations. This patch enables the specification of cache layout through a command line parameter, building on a patch set by Intel [1,2,3]. It uses this set as a foundation. The device tree and ACPI/PPTT table, and device tree are populated based on user-provided information and CPU topology. Example: +----------------+ +----------------+ | Socket 0 | | Socket 1 | | (L3 Cache) | | (L3 Cache) | +--------+-------+ +--------+-------+ | | +--------+--------+ +--------+--------+ | Cluster 0 | | Cluster 0 | | (L2 Cache) | | (L2 Cache) | +--------+--------+ +--------+--------+ | | +--------+--------+ +--------+--------+ +--------+--------+ +--------+----+ | Core 0 | | Core 1 | | Core 0 | | Core 1 | | (L1i, L1d) | | (L1i, L1d) | | (L1i, L1d) | | (L1i, L1d)| +--------+--------+ +--------+--------+ +--------+--------+ +--------+----+ | | | | +--------+ +--------+ +--------+ +--------+ |Thread 0| |Thread 1| |Thread 1| |Thread 0| +--------+ +--------+ +--------+ +--------+ |Thread 1| |Thread 0| |Thread 0| |Thread 1| +--------+ +--------+ +--------+ +--------+ The following command will represent the system relying on **ACPI PPTT tables**. ./qemu-system-aarch64 \ -machine virt,smp-cache.0.cache=l1i,smp-cache.0.topology=core,smp-cache.1.cache=l1d,smp-cache.1.topology=core,smp-cache.2.cache=l2,smp-cache.2.topology=cluseter,smp-cache.3.cache=l3,smp-cache.3.topology=socket \ -cpu max \ -m 2048 \ -smp sockets=2,clusters=1,cores=2,threads=2 \ -kernel ./Image.gz \ -append "console=ttyAMA0 root=/dev/ram rdinit=/init acpi=force" \ -initrd rootfs.cpio.gz \ -bios ./edk2-aarch64-code.fd \ -nographic The following command will represent the system relying on **the device tree**. ./qemu-system-aarch64 \ -machine virt,smp-cache.0.cache=l1i,smp-cache.0.topology=core,smp-cache.1.cache=l1d,smp-cache.1.topology=core,smp-cache.2.cache=l2,smp-cache.2.topology=cluseter,smp-cache.3.cache=l3,smp-cache.3.topology=socket \ -cpu max \ -m 2048 \ -smp sockets=2,clusters=1,cores=2,threads=2 \ -kernel ./Image.gz \ -append "console=ttyAMA0 root=/dev/ram rdinit=/init" \ -initrd rootfs.cpio.gz \ -nographic Failure cases: 1) There are scenarios where caches exist in systems' registers but not left unspecified by users. In this case qemu returns failure. 2) At the moment, the device tree is not able to describe caches shared at core level. In another word, SMT threads cannot share caches. This will need adjustments in the SPEC. It is worth noting that this particular case is completely OK in ACPI PPTT tables. Currently only three levels of caches are supported to be specified from the command line. However, increasing the value does not require significant changes. Further, this patch assumes l2 and l3 unified caches and does not allow l(2/3)(i/d). The level terminology is thread/core/cluster/socket right now. Here is the hierarchy assumed in this patch: Socket level = Cluster level + 1 = Core level + 2 = Thread level + 3; TODO: 1) Making the code to work with arbitrary levels 2) Separated data and instruction cache at L2 and L3. 3) Additional cache controls. e.g. size of L3 may not want to just match the underlying system, because only some of the associated host CPUs may be bound to this VM. Depends-on: Building PPTT with root node and identical implementation flag Depends-on: Msg-id: 20240926113323.55991-1-yangyicong@huawei.com Depends-on: i386: Support SMP Cache Topology Depends-on: Msg-id: 20250110145115.1574345-1-zhao1.liu@intel.com [1] https://lore.kernel.org/kvm/20240908125920.1160236-1-zhao1.liu@intel.com/ [2] https://lore.kernel.org/qemu-devel/20241101083331.340178-1-zhao1.liu@intel.com/ [3] https://lore.kernel.org/qemu-devel/20250110145115.1574345-1-zhao1.liu@intel.com/ Change Log: v5->v6: * Minor bug fix. * rebase based on new Intel patchset. - https://lore.kernel.org/qemu-devel/20250110145115.1574345-1-zhao1.liu@intel.com/ v4->v5: * Added Reviewed-by tags. * Applied some comments. v3->v4: * Device tree added. Alireza Sanaee (6): target/arm/tcg: increase cache level for cpu=max arm/virt.c: add cache hierarchy to device tree bios-tables-test: prepare to change ARM ACPI virt PPTT hw/acpi/aml-build.c: add cache hierarchy to pptt table tests/qtest/bios-table-test: testing new ARM ACPI PPTT topology Update the ACPI tables according to the acpi aml_build change, also empty bios-tables-test-allowed-diff.h. hw/acpi/aml-build.c | 205 +++++++++++- hw/arm/virt-acpi-build.c | 8 +- hw/arm/virt.c | 349 +++++++++++++++++++++ hw/cpu/core.c | 92 ++++++ include/hw/acpi/aml-build.h | 4 +- include/hw/arm/virt.h | 4 + include/hw/cpu/core.h | 27 ++ target/arm/tcg/cpu64.c | 13 + tests/data/acpi/aarch64/virt/PPTT.topology | Bin 356 -> 540 bytes tests/qtest/bios-tables-test.c | 4 + 10 files changed, 699 insertions(+), 7 deletions(-)