From patchwork Fri Jun 30 09:17:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13297842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 743F1C001B0 for ; Fri, 30 Jun 2023 09:18:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qFAGe-0002CQ-Ra; Fri, 30 Jun 2023 05:18:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFAGd-0002C5-9t; Fri, 30 Jun 2023 05:18:15 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qFAGZ-0001dv-RJ; Fri, 30 Jun 2023 05:18:15 -0400 Received: from pps.filterd (m0353726.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35U9HRPc029579; Fri, 30 Jun 2023 09:18:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=3NWofwlNKtvMmm1/f2CrB5ZDrskg7USBUXVrU2Fmngo=; b=iifoVmZFoq/fLuzrh5LxPDpdvbunvtLxV/Wg4Vfb+ng5sMirh46SHBllZSBOXkua6ZVO 6xZM9r+COZkJAimjLzOUu33T83lrypJWwX1xuUamEMuwLqk1/8yREdCEYvKDrGcsoETR f9IrNZhiufxrmu20zEBbmWHkEnGetbthnaLnumHpeWq9x7EKQSnPQOdk2qzNBRq+JISH mlIgh7g/OK6L19I52BnZ+YztxyTwpnxUIT0M9OB+MAj+aNamy53h8OBM/Xjq6W7z8HzQ 30nyqpkwghyC1QK0csuG1Y1mOf4D8S9N0s7peBZQZjtbZMqghqx6gex96xYaurR5AYqy iA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rhv9jr0hw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 09:18:03 +0000 Received: from m0353726.ppops.net (m0353726.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35U9I3pQ031884; Fri, 30 Jun 2023 09:18:03 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3rhv9jr0g9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 09:18:03 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 35U3gTFX028493; Fri, 30 Jun 2023 09:18:00 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma06fra.de.ibm.com (PPS) with ESMTPS id 3rdqre3092-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 30 Jun 2023 09:18:00 +0000 Received: from smtpav05.fra02v.mail.ibm.com (smtpav05.fra02v.mail.ibm.com [10.20.54.104]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 35U9HsCx42271456 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 30 Jun 2023 09:17:54 GMT Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96A542004D; Fri, 30 Jun 2023 09:17:54 +0000 (GMT) Received: from smtpav05.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3D1F62004B; Fri, 30 Jun 2023 09:17:53 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.38.86]) by smtpav05.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 30 Jun 2023 09:17:53 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v21 00/20] s390x: CPU Topology Date: Fri, 30 Jun 2023 11:17:32 +0200 Message-Id: <20230630091752.67190-1-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: etLHSOj8Efn5U2TupPQPju_djdf0AiKh X-Proofpoint-GUID: WwHjY5sjzmIdf1B9YTcsexU3Dn2xY4J2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-30_05,2023-06-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 phishscore=0 mlxscore=0 clxscore=1015 impostorscore=0 malwarescore=0 bulkscore=0 lowpriorityscore=0 adultscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306300076 Received-SPF: pass client-ip=148.163.156.1; envelope-from=pmorel@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Hi, What is new: - changed name of target/s390/kvm/cpu_topology to target/s390/kvm/stsi-topology - moved the TLE list head from a global to a local in insert_stsi_15_1_x() - cleaning and merging some of the avocado tests - modification patch "query-cpu-fast" due to upstream modifications I removed the RB but the changes are minimal. Implementation discussions ========================== CPU models ---------- Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already in the CPU model for old QEMU we could not activate it as usual from KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY. Checking and enabling this capability enables facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY. It is the responsibility of the admin to ensure the same CPU model for source and target host in a migration. Migration --------- When the target guest is started, the Multi-processor Topology Change Report (MTCR) bit is set during the creation of the vCPU by KVM. We do not need to migrate its state, in the worst case, the target guest will see the MTCR and actualize its view of the topology without necessity, but this will be done only one time. Reset ----- Reseting the topology is done during subsystem reset, the polarization is reset to horizontal polarization. Topology attributes ------------------- The topology attributes are carried by the CPU object and defined on object creation. In the case the new attributes, socket, book, drawer, dedicated, entitlement are not provided QEMU provides defaults values. - Geometry defaults The geometry default are based on the core-id of the core to fill the geometry in a monotone way starting with drawer 0, book 0, and filling socket 0 with the number of cores per socket, then filling socket 1, socket 2 ... etc until the book is complete and all books until the first drawer is complete before starting with the next drawer. This allows to keep existing start scripts and Libvirt existing interface until it is extended. - Modifiers defaults Default entitlement is medium Default dedication is not dedicated. - Machine polarization default to horizontal Dynamic topology modification ----------------------------- QAPI interface is extended with: - a command: 'set-cpu-topology' - a query: 'query-cpu-polarization' - a query: extension of qmp 'query-cpus-fast' - a query: extension of hmp 'hotpluggable-cpus' - an event: 'CPU_POLARITY_CHANGE' New command and interface are specified as unstable. The admin may use query-cpus-fast to verify the topology provided to the guest and set-cpu-topology to modify it. The event CPU_POLARITY_CHANGE is sent when the guest successfuly uses the PTF(2) instruction to request a polarization change. In that case, the admin is supposed to modify the CPU provisioning accordingly. Testing ======= To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches: f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac.. Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series. This series provide 12 avocado tests using Fedora-35 kernel and initrd image. Documentation ============= To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch of this series. The admin will want to match the host and the guest topology, taking into account that the guest does not recognize multithreading. Consequently, two vCPU assigned to threads of the same real CPU should preferably be assigned to the same socket of the guest machine. Regards, Pierre Pierre Morel (20): s390x/cpu topology: add s390 specifics to CPU topology s390x/cpu topology: add topology entries on CPU hotplug target/s390x/cpu topology: handle STSI(15) and build the SYSIB s390x/sclp: reporting the maximum nested topology entries s390x/cpu topology: resetting the Topology-Change-Report s390x/cpu topology: interception of PTF instruction target/s390x/cpu topology: activate CPU topology qapi/s390x/cpu topology: set-cpu-topology qmp command machine: adding s390 topology to query-cpu-fast machine: adding s390 topology to info hotpluggable-cpus qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event qapi/s390x/cpu topology: query-cpu-polarization qmp command docs/s390x/cpu topology: document s390x cpu topology tests/avocado: s390x cpu topology core tests/avocado: s390x cpu topology polarisation tests/avocado: s390x cpu topology entitlement tests tests/avocado: s390x cpu topology test dedicated CPU tests/avocado: s390x cpu topology test socket full tests/avocado: s390x cpu topology dedicated errors tests/avocado: s390x cpu topology bad move MAINTAINERS | 10 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 170 ++++++++++ docs/system/s390x/cpu-topology.rst | 240 ++++++++++++++ docs/system/target-s390x.rst | 1 + qapi/machine-common.json | 22 ++ qapi/machine-target.json | 113 +++++++ qapi/machine.json | 30 +- include/hw/boards.h | 10 +- include/hw/qdev-properties-system.h | 4 + include/hw/s390x/cpu-topology.h | 80 +++++ include/hw/s390x/s390-virtio-ccw.h | 6 + include/hw/s390x/sclp.h | 4 +- target/s390x/cpu.h | 83 +++++ target/s390x/kvm/kvm_s390x.h | 1 + hw/core/machine-hmp-cmds.c | 6 + hw/core/machine-smp.c | 48 ++- hw/core/machine.c | 4 + hw/core/qdev-properties-system.c | 13 + hw/s390x/cpu-topology.c | 475 ++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 27 +- hw/s390x/sclp.c | 5 + softmmu/vl.c | 6 + target/s390x/cpu-sysemu.c | 13 + target/s390x/cpu.c | 11 + target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 42 ++- target/s390x/kvm/stsi-topology.c | 310 ++++++++++++++++++ hw/s390x/meson.build | 1 + qapi/meson.build | 1 + qemu-options.hx | 7 +- target/s390x/kvm/meson.build | 3 +- tests/avocado/s390_topology.py | 415 ++++++++++++++++++++++++ 33 files changed, 2143 insertions(+), 20 deletions(-) create mode 100644 docs/devel/s390-cpu-topology.rst create mode 100644 docs/system/s390x/cpu-topology.rst create mode 100644 qapi/machine-common.json create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c create mode 100644 target/s390x/kvm/stsi-topology.c create mode 100644 tests/avocado/s390_topology.py