From patchwork Thu Jan 5 14:53:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090058 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA1D0C54EBD for ; Thu, 5 Jan 2023 14:54:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234250AbjAEOyB (ORCPT ); Thu, 5 Jan 2023 09:54:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234212AbjAEOxi (ORCPT ); Thu, 5 Jan 2023 09:53:38 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D36F1104 for ; Thu, 5 Jan 2023 06:53:36 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EBSoq027923; Thu, 5 Jan 2023 14:53:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=lUuo1ktLOHgFbctzGbWL3ikxi4/x+NRurfOwn7SqwbE=; b=MLGcA2fU/abp0Jqt+JKPOgoV44xzNXApTpg8xBgWY4fV8sUHUx94cN7uO8s/kZ4WTIME 4y4hKEYv93IRYdaBZkhXe2zNJXqq/rpZgKTrXOs/o3lnNLH+N9Why7PbdnUxC0+ByhGb MKADUsYy/QZx53KctS01go7iKLLn9ezpqkg6amofdyyw6XcExX8MzOjCd1ak/e/9e60/ G692lznkkWTN4ZMJ+OKxVlKDuq/5sb/2ubDHURhW5M+Ek2gK9W0u8/RUuVQHX5XMcdBn u8KK6/cfRQMtLXGodCkE5kjYqQiZ9MTArnGD7t0RAomYwv5nbXuBmLvH/QtPaPxiEuBy xg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1j4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:23 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EfssD029475; Thu, 5 Jan 2023 14:53:22 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1ha-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:22 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3054W6U0001927; Thu, 5 Jan 2023 14:53:20 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma03ams.nl.ibm.com (PPS) with ESMTPS id 3mtcq6ewj0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:19 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErFwI46072256 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:15 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B6DAB20043; Thu, 5 Jan 2023 14:53:15 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B6E7920040; Thu, 5 Jan 2023 14:53:14 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:14 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 00/11] s390x: CPU Topology Date: Thu, 5 Jan 2023 15:53:02 +0100 Message-Id: <20230105145313.168489-1-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: oVJi6vV7pK5SzTxPQNDtI14WNtTBjEdg X-Proofpoint-GUID: aypnJFwsN2Gy9N7VM4K9UJ9xNkqe6iWg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 clxscore=1011 adultscore=0 priorityscore=1501 malwarescore=0 phishscore=0 spamscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi, This new series adds three features: 1) the implementation of drawers and books in the topology geometry and adds polarization and dedication as topology modifiers. 2) The Maximum Nested topology level (MNEST) can be read with SCLP read info. 3) Dynamic modification of the CPU topology using the QEMU monitor. Implementation discussions ========================== CPU models ---------- Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already in the CPU model for old QEMU we could not activate it as usual from KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY. Checking and enabling this capability enables facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY. It is the responsibility of the admin to ensure the same CPU model for source and target host in a migration. Migration --------- When the target guest is started, the Multi-processor Topology Change Report (MTCR) bit is set during the creation of the vCPU by KVM. We do not need to migrate its state, in the worst case, the target guest will see the MTCR and actualize its view of the topology without necessity, but this will be done only one time. Reset ----- Reseting the topology is done during subsystem reset, the polarization is reset to horizontal polarization. Topology attributes ------------------- The topology attributes are carried by the CPU object and defined on object creation. In the case the new attributes, socket, book, drawer, dedicated, polarity are not provided QEMU provides defaults values. - Geometry defaults The geometry default are based on the core-id of the core to fill the geometry in a monotone way starting with drawer 0, book 0, and filling socket 0 with the number of cores per socket, then filling socket 1, socket 2 ... etc until the book is complete and all books until the first drawer is complete before starting with the next drawer. This allows to keep existing start scripts and Libvirt existing interface until it is extended. - Modifiers defaults Default polarization is horizontal Default dedication is not dedicated. Dynamic topology modification ----------------------------- QAPI interface is extended with: - a command: 'change-topology' - a query: 'query-topology' - a event: 'POLARITY_CHANGE' The admin may use query-topology to verify the topology provided to the guest and change-topology to modify it. The event POLARITY_CHANGE is sent when the guest uses the PTF instruction to request a polarization change. In that case, the admin is supposed to modify the CPU provisioning accordingly. Testing ======= To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches: f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac.. Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series. Documentation ============= To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch of this series. The admin will want to match the host and the guest topology, taking into account that the guest does not recognize multithreading. Consequently, two vCPU assigned to threads of the same real CPU should preferably be assigned to the same socket of the guest machine. Regards, Pierre Pierre Morel (11): s390x/cpu topology: adding s390 specificities to CPU topology s390x/cpu topology: add topology entries on CPU hotplug target/s390x/cpu topology: handle STSI(15) and build the SYSIB s390x/sclp: reporting the maximum nested topology entries s390x/cpu topology: resetting the Topology-Change-Report s390x/cpu topology: interception of PTF instruction target/s390x/cpu topology: activating CPU topology qapi/s390/cpu topology: change-topology monitor command qapi/s390/cpu topology: monitor query topology information qapi/s390/cpu topology: POLARITY_CHANGE qapi event docs/s390x/cpu topology: document s390x cpu topology docs/system/s390x/cpu-topology.rst | 292 ++++++++++++++ docs/system/target-s390x.rst | 1 + qapi/machine-target.json | 116 ++++++ qapi/machine.json | 14 +- include/hw/boards.h | 10 +- include/hw/s390x/cpu-topology.h | 77 ++++ include/hw/s390x/s390-virtio-ccw.h | 6 + include/hw/s390x/sclp.h | 4 +- include/monitor/hmp.h | 2 + target/s390x/cpu.h | 86 ++++ target/s390x/kvm/kvm_s390x.h | 1 + hw/core/machine-smp.c | 48 ++- hw/core/machine.c | 4 + hw/s390x/cpu-topology.c | 604 +++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 13 + hw/s390x/sclp.c | 4 + softmmu/vl.c | 6 + target/s390x/cpu-sysemu.c | 29 ++ target/s390x/cpu.c | 10 + target/s390x/cpu_models.c | 1 + target/s390x/kvm/cpu_topology.c | 136 +++++++ target/s390x/kvm/kvm.c | 46 ++- hmp-commands-info.hx | 16 + hmp-commands.hx | 16 + hw/s390x/meson.build | 1 + qemu-options.hx | 6 +- target/s390x/kvm/meson.build | 3 +- 27 files changed, 1537 insertions(+), 15 deletions(-) create mode 100644 docs/system/s390x/cpu-topology.rst create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c create mode 100644 target/s390x/kvm/cpu_topology.c