From patchwork Tue Nov 29 17:42:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058916 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6C8AC46467 for ; Tue, 29 Nov 2022 17:42:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236669AbiK2Rmh (ORCPT ); Tue, 29 Nov 2022 12:42:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236420AbiK2Rm3 (ORCPT ); Tue, 29 Nov 2022 12:42:29 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 076B24A9FA for ; Tue, 29 Nov 2022 09:42:28 -0800 (PST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGx6xN007810; Tue, 29 Nov 2022 17:42:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=NZma6QDvoIn7Ehk6HFkS1Dkzhlaf/5zDgGiklZB8KN0=; b=PAjPxC35HKnXzPoXJJOnW+/0Bm4cheulRUgfPrVzpNTTTk1EdZVXiEJ6OxQiinwrX4rJ cae9F6AKgfIMWkznvi3QOdkgG5WqX8SxOcGSwFu1RREfn+6hCXV99fgfiWyh3sjajxaT UvND1QYAndlLm8jIw3BZ+9l2+OcsmmJrcaGfBwnKL2zUq/E30s1vKmpDNfpYgRBvxH+A HOczuQsRXRxUnkjMNxGgSxN68aUtOHnytS/0aNqDKEwAB6oFOXAg4R0hykIXhifg6gPJ qxW5iM+PfpgRKnZ82qEagNlKl46Tm/Wkmd/MfQ/BzmBxyDvatKQfO9LmZ9xIhw0VW5b1 Ug== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5nb3aswa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:15 +0000 Received: from m0098396.ppops.net (m0098396.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATGx5mR007637; Tue, 29 Nov 2022 17:42:15 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5nb3asvh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:15 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHaEsx012308; Tue, 29 Nov 2022 17:42:12 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma06ams.nl.ibm.com with ESMTP id 3m3a2hvfv3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:12 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgqEs2491034 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:52 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB4B8A404D; Tue, 29 Nov 2022 17:42:09 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A73D6A4040; Tue, 29 Nov 2022 17:42:08 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:08 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 1/7] s390x/cpu topology: Creating CPU topology device Date: Tue, 29 Nov 2022 18:42:00 +0100 Message-Id: <20221129174206.84882-2-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 876uLa8-CuKmeE2pyKbZ-YbwrMtzbFDm X-Proofpoint-ORIG-GUID: 7IB82V2HkOTp2H3pDFcw-PLtIVUKl35B X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_10,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 lowpriorityscore=0 mlxscore=0 clxscore=1011 malwarescore=0 adultscore=0 phishscore=0 priorityscore=1501 impostorscore=0 spamscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org We will need a Topology device to transfer the topology during migration and to implement machine reset. The device creation is fenced by s390_has_topology(). Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 44 +++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 1 + hw/s390x/cpu-topology.c | 87 ++++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 25 +++++++++ hw/s390x/meson.build | 1 + 5 files changed, 158 insertions(+) create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h new file mode 100644 index 0000000000..e88059ccec --- /dev/null +++ b/include/hw/s390x/cpu-topology.h @@ -0,0 +1,44 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H + +#include "hw/qdev-core.h" +#include "qom/object.h" + +#define S390_TOPOLOGY_CPU_IFL 0x03 +#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64) + +#define S390_TOPOLOGY_POLARITY_HORIZONTAL 0x00 +#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW 0x01 +#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02 +#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH 0x03 + +typedef struct S390TopoSocket { + int active_count; + uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN]; +} S390TopoSocket; + +struct S390Topology { + SysBusDevice parent_obj; + uint32_t num_cores; + uint32_t num_sockets; + S390TopoSocket *socket; +}; + +#define TYPE_S390_CPU_TOPOLOGY "s390-topology" +OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) + +static inline bool s390_has_topology(void) +{ + return false; +} + +#endif diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h index 9bba21a916..47ce0aa6fa 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -28,6 +28,7 @@ struct S390CcwMachineState { bool dea_key_wrap; bool pv; uint8_t loadparm[8]; + DeviceState *topology; }; struct S390CcwMachineClass { diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c new file mode 100644 index 0000000000..bbf97cd66a --- /dev/null +++ b/hw/s390x/cpu-topology.c @@ -0,0 +1,87 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/sysbus.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/** + * s390_topology_realize: + * @dev: the device state + * + * We free the socket array allocated in realize. + */ +static void s390_topology_unrealize(DeviceState *dev) +{ + S390Topology *topo = S390_CPU_TOPOLOGY(dev); + + g_free(topo->socket); +} + +/** + * s390_topology_realize: + * @dev: the device state + * @errp: the error pointer (not used) + * + * During realize the machine CPU topology is initialized with the + * QEMU -smp parameters. + * The maximum count of CPU TLE in the all Topology can not be greater + * than the maximum CPUs. + */ +static void s390_topology_realize(DeviceState *dev, Error **errp) +{ + S390Topology *topo = S390_CPU_TOPOLOGY(dev); + + topo->socket = g_new0(S390TopoSocket, topo->num_sockets); +} + +static Property s390_topology_properties[] = { + DEFINE_PROP_UINT32("num-cores", S390Topology, num_cores, 1), + DEFINE_PROP_UINT32("num-sockets", S390Topology, num_sockets, 1), + DEFINE_PROP_END_OF_LIST(), +}; + +/** + * topology_class_init: + * @oc: Object class + * @data: (not used) + * + * A very simple object we will need for reset and migration. + */ +static void topology_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + + dc->realize = s390_topology_realize; + dc->unrealize = s390_topology_unrealize; + device_class_set_props(dc, s390_topology_properties); + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +} + +static const TypeInfo cpu_topology_info = { + .name = TYPE_S390_CPU_TOPOLOGY, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(S390Topology), + .class_init = topology_class_init, +}; + +static void topology_register(void) +{ + type_register_static(&cpu_topology_info); +} +type_init(topology_register); diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 2e64ffab45..973bbdd36e 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -44,6 +44,7 @@ #include "hw/s390x/pv.h" #include "migration/blocker.h" #include "qapi/visitor.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker; @@ -102,6 +103,24 @@ static void s390_init_cpus(MachineState *machine) } } +static DeviceState *s390_init_topology(MachineState *machine, Error **errp) +{ + DeviceState *dev; + + dev = qdev_new(TYPE_S390_CPU_TOPOLOGY); + + object_property_add_child(&machine->parent_obj, + TYPE_S390_CPU_TOPOLOGY, OBJECT(dev)); + object_property_set_int(OBJECT(dev), "num-cores", + machine->smp.cores * machine->smp.threads, errp); + object_property_set_int(OBJECT(dev), "num-sockets", + machine->smp.sockets, errp); + + sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), errp); + + return dev; +} + static const char *const reset_dev_types[] = { TYPE_VIRTUAL_CSS_BRIDGE, "s390-sclp-event-facility", @@ -255,6 +274,12 @@ static void ccw_init(MachineState *machine) /* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus(machine); + /* Need CPU model to be determined before we can set up topology */ + if (s390_has_topology()) { + S390_CCW_MACHINE(machine)->topology = s390_init_topology(machine, + &error_fatal); + } + /* Need CPU model to be determined before we can set up PV */ s390_pv_init(machine->cgs, &error_fatal); diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build index f291016fee..653f6ab488 100644 --- a/hw/s390x/meson.build +++ b/hw/s390x/meson.build @@ -2,6 +2,7 @@ s390x_ss = ss.source_set() s390x_ss.add(files( 'ap-bridge.c', 'ap-device.c', + 'cpu-topology.c', 'ccw-device.c', 'css-bridge.c', 'css.c', From patchwork Tue Nov 29 17:42:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058917 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90047C4167B for ; Tue, 29 Nov 2022 17:42:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236671AbiK2Rmk (ORCPT ); Tue, 29 Nov 2022 12:42:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236661AbiK2Rmd (ORCPT ); Tue, 29 Nov 2022 12:42:33 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D5C4B2 for ; Tue, 29 Nov 2022 09:42:31 -0800 (PST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATHfR3T023517; Tue, 29 Nov 2022 17:42:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Da86DJOn7z51dfl0noV58t91RTXpk78j+f5Y0MgPIjA=; b=iTCMtm/PHWsw6P4t0/uaO9Drp5HWv3SPBZPmZ6wAYu/O403eK6FtjTXkKeYqX+xjGmjv pjMqtFpC7nFPtbtEI2U7cZJ2cLLFWIlbzR+yC+zPpBlhdWjeB0K7lFRqkjVmQZyM9yrk KvttZKu10cRx2o4tJMaaiHS7meAtjzR4VBsh5Q2OySBBwwkUZ30nOqNGdp8xB36v4I7D tq8HPVedfdlGU49QrEm7HrZFyWACF4rY311BsGfDCNBckBZT+rbigeF8a6f2F+zSVchj HwyxKYZlvQ2Ziq/NDZLE6hTfGhconQ27rpEmp9VtWwbj4YVN1IvoZpYYpim1Bjee3/WQ xw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5ppyr068-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:17 +0000 Received: from m0098399.ppops.net (m0098399.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATHgGDA026016; Tue, 29 Nov 2022 17:42:16 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5ppyr05n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:16 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHaERH012313; Tue, 29 Nov 2022 17:42:14 GMT Received: from b06avi18626390.portsmouth.uk.ibm.com (b06avi18626390.portsmouth.uk.ibm.com [9.149.26.192]) by ppma06ams.nl.ibm.com with ESMTP id 3m3a2hvfv4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:13 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHZhv411666032 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:35:43 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CEF87A4051; Tue, 29 Nov 2022 17:42:10 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C0227A4040; Tue, 29 Nov 2022 17:42:09 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:09 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 2/7] s390x/cpu topology: reporting the CPU topology to the guest Date: Tue, 29 Nov 2022 18:42:01 +0100 Message-Id: <20221129174206.84882-3-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: DySFIHWd4PjIMYEsxhsd-pUIkyt2Q4CG X-Proofpoint-GUID: NoqZy_tkUGGMo9nHGaSbPrbmGPGfxro2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 impostorscore=0 spamscore=0 phishscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 malwarescore=0 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The guest uses the STSI instruction to get information on the CPU topology. Let us implement the STSI instruction for the basis CPU topology level, level 2. Signed-off-by: Pierre Morel --- target/s390x/cpu.h | 77 +++++++++++++++ hw/s390x/s390-virtio-ccw.c | 12 +-- target/s390x/cpu_topology.c | 186 ++++++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 5 files changed, 274 insertions(+), 8 deletions(-) create mode 100644 target/s390x/cpu_topology.c diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 7d6d01325b..dd878ac916 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -175,6 +175,7 @@ struct ArchCPU { /* needed for live migration */ void *irqstate; uint32_t irqstate_saved_size; + void *machine_data; }; @@ -565,6 +566,80 @@ typedef union SysIB { } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096); +/* + * CPU Topology List provided by STSI with fc=15 provides a list + * of two different Topology List Entries (TLE) types to specify + * the topology hierarchy. + * + * - Container Topology List Entry + * Defines a container to contain other Topology List Entries + * of any type, nested containers or CPU. + * - CPU Topology List Entry + * Specifies the CPUs position, type, entitlement and polarization + * of the CPUs contained in the last Container TLE. + * + * There can be theoretically up to five levels of containers, QEMU + * uses only one level, the socket level. + * + * A container of with a nesting level (NL) greater than 1 can only + * contain another container of nesting level NL-1. + * + * A container of nesting level 1 (socket), contains as many CPU TLE + * as needed to describe the position and qualities of all CPUs inside + * the container. + * The qualities of a CPU are polarization, entitlement and type. + * + * The CPU TLE defines the position of the CPUs of identical qualities + * using a 64bits mask which first bit has its offset defined by + * the CPU address orgin field of the CPU TLE like in: + * CPU address = origin * 64 + bit position within the mask + * + */ +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8); + +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; + uint8_t reserved1:5; + uint8_t dedicated:1; + uint8_t polarity:2; + uint8_t type; + uint16_t origin; + uint64_t mask; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16); + +#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[0]; +} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x; +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16); + +/* Max size of a SYSIB structure is when all CPU are alone in a container */ +#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x) + \ + S390_MAX_CPUS * (sizeof(SysIBTl_container) + \ + sizeof(SysIBTl_cpu))) + /* MMU defines */ #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ #define ASCE_SUBSPACE 0x200 /* subspace group control */ @@ -843,4 +918,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr); #include "exec/cpu-all.h" +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar); + #endif diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 973bbdd36e..4be07959fd 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -64,11 +64,10 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr) return S390_CPU(ms->possible_cpus->cpus[cpu_addr].cpu); } -static S390CPU *s390x_new_cpu(const char *typename, uint32_t core_id, - Error **errp) +static void s390x_new_cpu(MachineState *ms, uint32_t core_id, Error **errp) { - S390CPU *cpu = S390_CPU(object_new(typename)); - S390CPU *ret = NULL; + S390CcwMachineState *s390ms = S390_CCW_MACHINE(ms); + S390CPU *cpu = S390_CPU(object_new(ms->cpu_type)); if (!object_property_set_int(OBJECT(cpu), "core-id", core_id, errp)) { goto out; @@ -76,11 +75,10 @@ static S390CPU *s390x_new_cpu(const char *typename, uint32_t core_id, if (!qdev_realize(DEVICE(cpu), NULL, errp)) { goto out; } - ret = cpu; + cpu->machine_data = s390ms; out: object_unref(OBJECT(cpu)); - return ret; } static void s390_init_cpus(MachineState *machine) @@ -99,7 +97,7 @@ static void s390_init_cpus(MachineState *machine) mc->possible_cpu_arch_ids(machine); for (i = 0; i < machine->smp.cpus; i++) { - s390x_new_cpu(machine->cpu_type, i, &error_fatal); + s390x_new_cpu(machine, i, &error_fatal); } } diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c new file mode 100644 index 0000000000..b81f016ba1 --- /dev/null +++ b/target/s390x/cpu_topology.c @@ -0,0 +1,186 @@ +/* + * QEMU S390x CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/cpu-topology.h" +#include "hw/s390x/sclp.h" + +/* + * s390_topology_add_cpu: + * @topo: pointer to the topology + * @cpu : pointer to the new CPU + * + * The topology pointed by S390CPU, gives us the CPU topology + * established by the -smp QEMU aruments. + * The core-id is used to calculate the position of the CPU inside + * the topology: + * - the socket, container TLE, containing the CPU, we have one socket + * for every num_cores cores. + * - the CPU TLE inside the socket, we have potentionly up to 4 CPU TLE + * in a container TLE with the assumption that all CPU are identical + * with the same polarity and entitlement because we have maximum 256 + * CPUs and each TLE can hold up to 64 identical CPUs. + * - the bit in the 64 bit CPU TLE core mask + */ +static void s390_topology_add_cpu(S390Topology *topo, S390CPU *cpu) +{ + int core_id = cpu->env.core_id; + int bit, origin; + int socket_id; + + cpu->machine_data = topo; + socket_id = core_id / topo->num_cores; + /* + * At the core level, each CPU is represented by a bit in a 64bit + * uint64_t which represent the presence of a CPU. + * The firmware assume that all CPU in a CPU TLE have the same + * type, polarization and are all dedicated or shared. + * In that case the origin variable represents the offset of the first + * CPU in the CPU container. + * More than 64 CPUs per socket are represented in several CPU containers + * inside the socket container. + * The only reason to have several S390TopologyCores inside a socket is + * to have more than 64 CPUs. + * In that case the origin variable represents the offset of the first CPU + * in the CPU container. More than 64 CPUs per socket are represented in + * several CPU containers inside the socket container. + */ + bit = core_id; + origin = bit / 64; + bit %= 64; + bit = 63 - bit; + + topo->socket[socket_id].active_count++; + set_bit(bit, &topo->socket[socket_id].mask[origin]); +} + +/* + * s390_prepare_topology: + * @s390ms : pointer to the S390CcwMachite State + * + * Calls s390_topology_add_cpu to organize the topology + * inside the topology device before writing the SYSIB. + * + * The topology is currently fixed on boot and do not change + * even on migration. + */ +static void s390_prepare_topology(S390CcwMachineState *s390ms) +{ + const MachineState *ms = MACHINE(s390ms); + static bool done; + int i; + + if (done) { + return; + } + + for (i = 0; i < ms->possible_cpus->len; i++) { + if (ms->possible_cpus->cpus[i].cpu) { + s390_topology_add_cpu(S390_CPU_TOPOLOGY(s390ms->topology), + S390_CPU(ms->possible_cpus->cpus[i].cpu)); + } + } + + done = true; +} + +static char *fill_container(char *p, int level, int id) +{ + SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; + return p + sizeof(*tle); +} + +static char *fill_tle_cpu(char *p, uint64_t mask, int origin) +{ + SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl = 0; + tle->dedicated = 1; + tle->polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL; + tle->type = S390_TOPOLOGY_CPU_IFL; + tle->origin = cpu_to_be64(origin * 64); + tle->mask = cpu_to_be64(mask); + return p + sizeof(*tle); +} + +static char *s390_top_set_level2(S390Topology *topo, char *p) +{ + int i, origin; + + for (i = 0; i < topo->num_sockets; i++) { + if (!topo->socket[i].active_count) { + continue; + } + p = fill_container(p, 1, i); + for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) { + uint64_t mask = 0L; + + mask = topo->socket[i].mask[origin]; + if (mask) { + p = fill_tle_cpu(p, mask, origin); + } + } + } + return p; +} + +static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level) +{ + S390Topology *topo = (S390Topology *)cpu->machine_data; + char *p = sysib->tle; + + sysib->mnest = level; + switch (level) { + case 2: + sysib->mag[S390_TOPOLOGY_MAG2] = topo->num_sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = topo->num_cores; + p = s390_top_set_level2(topo, p); + break; + } + + return p - (char *)sysib; +} + +#define S390_TOPOLOGY_MAX_MNEST 2 + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) +{ + union { + char place_holder[S390_TOPOLOGY_SYSIB_SIZE]; + SysIB_151x sysib; + } buffer QEMU_ALIGNED(8); + int len; + + if (s390_is_pv() || !s390_has_topology() || + sel2 < 2 || sel2 > S390_TOPOLOGY_MAX_MNEST) { + setcc(cpu, 3); + return; + } + + s390_prepare_topology(S390_CCW_MACHINE(cpu->machine_data)); + + len = setup_stsi(cpu, &buffer.sysib, sel2); + + if (len > 4096) { + setcc(cpu, 3); + return; + } + + buffer.sysib.length = cpu_to_be16(len); + s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len); + setcc(cpu, 0); +} + diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 3ac7ec9acf..7dc96f3663 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -51,6 +51,7 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/s390-virtio-hcall.h" #include "hw/s390x/pv.h" +#include "hw/s390x/cpu-topology.h" #ifndef DEBUG_KVM #define DEBUG_KVM 0 @@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu) if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) { return 0; } - /* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar); return 0; + case 15: + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); + return 0; default: return 0; } diff --git a/target/s390x/meson.build b/target/s390x/meson.build index 84c1402a6a..890ccfa789 100644 --- a/target/s390x/meson.build +++ b/target/s390x/meson.build @@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files( 'sigp.c', 'cpu-sysemu.c', 'cpu_models_sysemu.c', + 'cpu_topology.c', )) s390x_user_ss = ss.source_set() From patchwork Tue Nov 29 17:42:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 542BBC433FE for ; Tue, 29 Nov 2022 17:42:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236659AbiK2Rmc (ORCPT ); Tue, 29 Nov 2022 12:42:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38358 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236661AbiK2Rm1 (ORCPT ); Tue, 29 Nov 2022 12:42:27 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51D724C259 for ; Tue, 29 Nov 2022 09:42:26 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGxEcD006304; Tue, 29 Nov 2022 17:42:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=LL2J36UY3mJeGg2Fa/gGwxTjR1gkmCl9uO7odaGmziI=; b=iClaMJl0kbJy8y7voYHKq7V6Q+15G4ikCGaH2WEiV4p/fo5s0Rb6P9+LpvULoq+vcBIs 8F1UHL33oRYQeOu0h9J1130y5qUykFyJqgdZj5+LWbUE3rB6unfECOYYVFenq54z+RHE DLt9xbiKYkYzxlbwxav1oxmg8NprI0vwtbyFL3YTEO2ZKMW6rnbxMZ7B8rpduTG5dLtB j5GBG1nOIXwORrS0Tdvg8YWUAjuc94vBPadknOHCpU8JOxIMm7J1sk/Lv72VsWr6QdVt cEE2jJF0WcbqiJ+MSVwTZQsXQEWSvy2aZhkMI8cK2lcips7uycBJV9QZb9vzUTPIzq97 xQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5k6q7qvf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:18 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATH0Pn5013204; Tue, 29 Nov 2022 17:42:17 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5k6q7quk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:17 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHZuKM027968; Tue, 29 Nov 2022 17:42:15 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma04ams.nl.ibm.com with ESMTP id 3m3ae9cf51-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:15 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgCjt63766990 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:12 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E9BABA404D; Tue, 29 Nov 2022 17:42:11 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E2284A4040; Tue, 29 Nov 2022 17:42:10 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:10 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 3/7] s390x/cpu_topology: resetting the Topology-Change-Report Date: Tue, 29 Nov 2022 18:42:02 +0100 Message-Id: <20221129174206.84882-4-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: JPPPnQ9bKvQ6ZvComxAZ-m456y50-oxj X-Proofpoint-GUID: 4vL3Slv-XgOv5U66UVIhxsEf21iGsE_Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 mlxlogscore=999 spamscore=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 phishscore=0 malwarescore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org During a subsystem reset the Topology-Change-Report is cleared by the machine. Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset. Signed-off-by: Pierre Morel Reviewed-by: Nico Boehr Reviewed-by: Janis Schoetterl-Glausch --- target/s390x/cpu.h | 1 + target/s390x/kvm/kvm_s390x.h | 1 + hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 13 +++++++++++++ target/s390x/kvm/kvm.c | 17 +++++++++++++++++ 6 files changed, 45 insertions(+) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index dd878ac916..2bd3d417e4 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -854,6 +854,7 @@ void s390_enable_css_support(S390CPU *cpu); void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); +void s390_cpu_topology_reset(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h index f9785564d0..649dae5948 100644 --- a/target/s390x/kvm/kvm_s390x.h +++ b/target/s390x/kvm/kvm_s390x.h @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void); void kvm_s390_restart_interrupt(S390CPU *cpu); void kvm_s390_stop_interrupt(S390CPU *cpu); void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info); +int kvm_s390_topology_set_mtcr(uint64_t attr); #endif /* KVM_S390X_H */ diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index bbf97cd66a..b1082a4b88 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -56,6 +56,17 @@ static Property s390_topology_properties[] = { DEFINE_PROP_END_OF_LIST(), }; +/** + * s390_topology_reset: + * @dev: the device + * + * Calls the sysemu topology reset + */ +static void s390_topology_reset(DeviceState *dev) +{ + s390_cpu_topology_reset(); +} + /** * topology_class_init: * @oc: Object class @@ -71,6 +82,7 @@ static void topology_class_init(ObjectClass *oc, void *data) dc->unrealize = s390_topology_unrealize; device_class_set_props(dc, s390_topology_properties); set_bit(DEVICE_CATEGORY_MISC, dc->categories); + dc->reset = s390_topology_reset; } static const TypeInfo cpu_topology_info = { diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 4be07959fd..f29e383d23 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -125,6 +125,7 @@ static const char *const reset_dev_types[] = { "s390-flic", "diag288", TYPE_S390_PCI_HOST_BRIDGE, + TYPE_S390_CPU_TOPOLOGY, }; static void subsystem_reset(void) diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index 948e4bd3e0..e27864c5f5 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg) kvm_s390_set_diag318(cs, arg.host_ulong); } } + +void s390_cpu_topology_reset(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(0); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 7dc96f3663..5b6383eab0 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2593,6 +2593,23 @@ int kvm_s390_get_zpci_op(void) return cap_zpci_op; } +int kvm_s390_topology_set_mtcr(uint64_t attr) +{ + struct kvm_device_attr attribute = { + .group = KVM_S390_VM_CPU_TOPOLOGY, + .attr = attr, + }; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + return 0; + } + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) { + return -ENOTSUP; + } + + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute); +} + void kvm_arch_accel_class_init(ObjectClass *oc) { } From patchwork Tue Nov 29 17:42:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 025C5C4332F for ; Tue, 29 Nov 2022 17:42:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236665AbiK2Rmd (ORCPT ); Tue, 29 Nov 2022 12:42:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236662AbiK2Rm2 (ORCPT ); Tue, 29 Nov 2022 12:42:28 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5E134E413 for ; Tue, 29 Nov 2022 09:42:26 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGxkri007859; Tue, 29 Nov 2022 17:42:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=kfgr0rXvVSTOANq5NxUO25W1tzTZTRSJCcaTh6oT8RI=; b=cqw8Ynhuraal27hm3y4SNavTxQpYkXnIjzm1yNJE93cHjyfe5qeml8bKFeKCltQpgic8 Sq7F2AhRXOFqNkcXYfnqsN6lF6Gt2pXdGMl6BcAH4Qir9nZR9uqaTi4rSGWlvAbX1HLd 25vkyxEGp4CR1bOBZbQIUxQldvZhozM+Q+ipnEEAc2BUKpRCC83SgyBgAC84UUJ3gFA+ wQ7S+U2VWKXG4kDN/iSADJ8H6N+QqP5m6t+qt4HF13hjVAQoSBNt80G14auTQ+xzasO7 rVijvYm6gIOaqSc1MsT99cgyPcWt6a4NaBrqUvrE0gI7ZkWfiG8TlA+dPziVWENIR6Xb MA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5nbejqmr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:18 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATGxjR4007809; Tue, 29 Nov 2022 17:42:18 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5nbejqky-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:18 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHZaR4019178; Tue, 29 Nov 2022 17:42:16 GMT Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by ppma03ams.nl.ibm.com with ESMTP id 3m3ae9cfev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:16 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgDHx36634890 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:13 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 329B0A4051; Tue, 29 Nov 2022 17:42:13 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 09878A4040; Tue, 29 Nov 2022 17:42:12 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:11 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 4/7] s390x/cpu_topology: CPU topology migration Date: Tue, 29 Nov 2022 18:42:03 +0100 Message-Id: <20221129174206.84882-5-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: oojZI9IF-vm3JIz51pfmyPeUegiYVnpA X-Proofpoint-GUID: HSI7C7qP_aIruFsgSEiEXyX4v4RvnXg- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 impostorscore=0 phishscore=0 adultscore=0 malwarescore=0 suspectscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 priorityscore=1501 spamscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility. We indicate a change in topology during migration postload for the case the topology changed between source and destination. Signed-off-by: Pierre Morel --- target/s390x/cpu.h | 1 + hw/s390x/cpu-topology.c | 49 +++++++++++++++++++++++++++++++++++++++ target/s390x/cpu-sysemu.c | 8 +++++++ 3 files changed, 58 insertions(+) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 2bd3d417e4..d8c00de980 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -855,6 +855,7 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); void s390_cpu_topology_reset(void); +int s390_cpu_topology_mtcr_set(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index b1082a4b88..32908d13bb 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -19,6 +19,7 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" /** * s390_topology_realize: @@ -67,6 +68,53 @@ static void s390_topology_reset(DeviceState *dev) s390_cpu_topology_reset(); } +/** + * cpu_topology_postload + * @opaque: a pointer to the S390Topology + * @version_id: version identifier + * + * We check that the topology is used or is not used + * on both side identically. + * + * If the topology is in use we set the Modified Topology Change Report + * on the destination host. + */ +static int cpu_topology_postload(void *opaque, int version_id) +{ + int ret; + + /* We do not support CPU Topology, all is good */ + if (!s390_has_topology()) { + return 0; + } + + /* We support CPU Topology, set the MTCR unconditionally */ + ret = s390_cpu_topology_mtcr_set(); + if (ret) { + error_report("Failed to set MTCR: %s", strerror(-ret)); + } + return ret; +} + +/** + * cpu_topology_needed: + * @opaque: The pointer to the S390Topology + * + * We always need to know if source and destination use the topology. + */ +static bool cpu_topology_needed(void *opaque) +{ + return s390_has_topology(); +} + +const VMStateDescription vmstate_cpu_topology = { + .name = "cpu_topology", + .version_id = 1, + .post_load = cpu_topology_postload, + .minimum_version_id = 1, + .needed = cpu_topology_needed, +}; + /** * topology_class_init: * @oc: Object class @@ -83,6 +131,7 @@ static void topology_class_init(ObjectClass *oc, void *data) device_class_set_props(dc, s390_topology_properties); set_bit(DEVICE_CATEGORY_MISC, dc->categories); dc->reset = s390_topology_reset; + dc->vmsd = &vmstate_cpu_topology; } static const TypeInfo cpu_topology_info = { diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index e27864c5f5..a8e3e6219d 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -319,3 +319,11 @@ void s390_cpu_topology_reset(void) } } } + +int s390_cpu_topology_mtcr_set(void) +{ + if (kvm_enabled()) { + return kvm_s390_topology_set_mtcr(1); + } + return -ENOENT; +} From patchwork Tue Nov 29 17:42:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19BCEC4332F for ; Tue, 29 Nov 2022 17:42:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236666AbiK2Rmi (ORCPT ); Tue, 29 Nov 2022 12:42:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236663AbiK2Rmd (ORCPT ); Tue, 29 Nov 2022 12:42:33 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EDDA33BD for ; Tue, 29 Nov 2022 09:42:31 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGxrpV020335; Tue, 29 Nov 2022 17:42:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=d0znzUKB7SWj3dtsRsDaoxqyP1TM5ryXxZv4tTznqVU=; b=QSFo7JefNd3Ph+c61y8bqKVoqj5hN9okFXGoqDSvBPUL0batX82lIfQ5YZ+CGlNnfyCJ eRjkYEcm/e+vDeByU5Z2IjM96ndtn9qu1Y+axw+pNrntetCJylxqG3y3DSp1P3Ko8hg9 u4kfS3xZCC+Nyk73Aqz8omyGvRNz4SCow7ysEL69TdfrFT0nG+FZ9QZS06UQ4gz7lYJ3 Df1409f8Zw8G5SF1DYNcRHvGOcp5zk03WdpddBnlXHe5+KC/M7rONE1OdOWvjudCSg0E 6hok8nunNyMcJz+u1WMCLY5nh27rmXPt1b1PfPPnA7Zjf/NcKlzPhJNIRwU9S5o3MgBp kg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5p16hafg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:20 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATHKRci033668; Tue, 29 Nov 2022 17:42:20 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5p16haef-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:19 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHZcGg027924; Tue, 29 Nov 2022 17:42:17 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma04ams.nl.ibm.com with ESMTP id 3m3ae9cf52-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:17 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgEhM26477240 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:14 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 48754A4051; Tue, 29 Nov 2022 17:42:14 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4704BA4040; Tue, 29 Nov 2022 17:42:13 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:13 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 5/7] s390x/cpu_topology: interception of PTF instruction Date: Tue, 29 Nov 2022 18:42:04 +0100 Message-Id: <20221129174206.84882-6-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: dXxVJnN44W32JqZAZWCNq_MRj0xz8rnQ X-Proofpoint-ORIG-GUID: 2ThKAd3Lu8rYhL5s2P0HKXHx70jHVc-y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 priorityscore=1501 spamscore=0 impostorscore=0 mlxlogscore=999 phishscore=0 adultscore=0 bulkscore=0 lowpriorityscore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension. The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor. Signed-off-by: Pierre Morel Reviewed-by: Janis Schoetterl-Glausch --- include/hw/s390x/s390-virtio-ccw.h | 6 ++++ hw/s390x/cpu-topology.c | 52 ++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 11 +++++++ 3 files changed, 69 insertions(+) diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h index 47ce0aa6fa..d78ea8843c 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -31,6 +31,12 @@ struct S390CcwMachineState { DeviceState *topology; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra); + struct S390CcwMachineClass { /*< private >*/ MachineClass parent_class; diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 32908d13bb..12fcd041a3 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -20,6 +20,58 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "migration/vmstate.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" + +/* + * s390_handle_ptf: + * + * @register 1: contains the function code + * + * Function codes 0 and 1 handle the CPU polarization. + * We assume an horizontal topology, the only one supported currently + * by Linux, consequently we answer to function code 0, requesting + * horizontal polarization that it is already the current polarization + * and reject vertical polarization request without further explanation. + * + * Function code 2 is handling topology changes and is interpreted + * by the SIE. + */ +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) +{ + CPUS390XState *env = &cpu->env; + uint64_t reg = env->regs[r1]; + uint8_t fc = reg & S390_TOPO_FC_MASK; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + s390_program_interrupt(env, PGM_OPERATION, ra); + return; + } + + if (env->psw.mask & PSW_MASK_PSTATE) { + s390_program_interrupt(env, PGM_PRIVILEGED, ra); + return; + } + + if (reg & ~S390_TOPO_FC_MASK) { + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + return; + } + + switch (fc) { + case 0: /* Horizontal polarization is already set */ + env->regs[r1] |= S390_PTF_REASON_DONE; + setcc(cpu, 2); + break; + case 1: /* Vertical polarization is not supported */ + env->regs[r1] |= S390_PTF_REASON_NONE; + setcc(cpu, 2); + break; + default: + /* Note that fc == 2 is interpreted by the SIE */ + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + } +} /** * s390_topology_realize: diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 5b6383eab0..a79fdf1c79 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -97,6 +97,7 @@ #define PRIV_B9_EQBS 0x9c #define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2 #define PRIV_B9_PCISTG 0xd0 #define PRIV_B9_PCILG 0xd2 #define PRIV_B9_RPCIT 0xd3 @@ -1465,6 +1466,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run) +{ + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f; + + s390_handle_ptf(cpu, r1, RA_IGNORED); +} + static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) { int r = 0; @@ -1482,6 +1490,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) case PRIV_B9_RPCIT: r = kvm_rpcit_service_call(cpu, run); break; + case PRIV_B9_PTF: + kvm_handle_ptf(cpu, run); + break; case PRIV_B9_EQBS: /* just inject exception */ r = -1; From patchwork Tue Nov 29 17:42:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058918 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4E08C4332F for ; Tue, 29 Nov 2022 17:42:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236677AbiK2Rml (ORCPT ); Tue, 29 Nov 2022 12:42:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236670AbiK2Rme (ORCPT ); Tue, 29 Nov 2022 12:42:34 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B12AB1025 for ; Tue, 29 Nov 2022 09:42:33 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGxFFI006316; Tue, 29 Nov 2022 17:42:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=x7y4rxkj5YRS87iryWPdRAEHHgsbdeZ1JK44KtLWH7I=; b=NaBWBQC0TVveHbmCrFvoFvmmZMgXXlcDCxWbNJS+H501PXTU/Ey4Vh4Npkg6nATZIacQ xNMvHQu3Gf/e3XeK9utINJt//cLcf92YautJuBromzTNfi/ZGDcPuPQFlUrut9Q8y2H9 18C9ad4j9ch71eZX6kwAg0MRnSXPApzGwQD7T6AC2obp4wGw+Yi8IX1Au9q/Jjs8zzwb jSdQ+z12NdUbszI4dhTtgLh96Cw4nu6ZBNxito1SMcmw+gpxCSNSZskyT2tw8Fxmdvm+ STh5aRuLIBcYCt5VlDgSp9iKonsAOdLnYyVcxb/8tKqhPM+oTaTYfVqbZ1hjT8GozXmF Dg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5k6q7qxh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:21 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATGxOZI007111; Tue, 29 Nov 2022 17:42:21 GMT Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com [149.81.74.108]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5k6q7qwc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:21 +0000 Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1]) by ppma05fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHZicE018462; Tue, 29 Nov 2022 17:42:18 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma05fra.de.ibm.com with ESMTP id 3m3ae9b830-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:18 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgFe960883298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:15 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5BDFEA4051; Tue, 29 Nov 2022 17:42:15 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 60E97A4040; Tue, 29 Nov 2022 17:42:14 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:14 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 6/7] s390x/cpu_topology: activating CPU topology Date: Tue, 29 Nov 2022 18:42:05 +0100 Message-Id: <20221129174206.84882-7-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: aLHspoLo-lSy_wUwyPamrdf9kWNYkbuk X-Proofpoint-GUID: CTQjm1wELMKG4Ilg9m8xKiOaJSU80iSX X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 clxscore=1015 mlxlogscore=996 spamscore=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 phishscore=0 malwarescore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The KVM capability, KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility for the guest in the case the topology is available in QEMU and in KVM. The feature is fenced for SE (secure execution). To allow smooth migration with old QEMU the feature is disabled by default using the CPU flag -disable-topology. Making the S390_FEAT_CONFIGURATION_TOPOLOGY belonging to the default features makes the -ctop CPU flag is no more necessary, turning the topology feature on is done with -disable-topology only. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 5 +---- target/s390x/cpu_features_def.h.inc | 1 + target/s390x/cpu_models.c | 17 +++++++++++++++++ target/s390x/cpu_topology.c | 5 +++++ target/s390x/gen-features.c | 3 +++ target/s390x/kvm/kvm.c | 14 ++++++++++++++ 6 files changed, 41 insertions(+), 4 deletions(-) diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index e88059ccec..b2fa24ba93 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -36,9 +36,6 @@ struct S390Topology { #define TYPE_S390_CPU_TOPOLOGY "s390-topology" OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) -static inline bool s390_has_topology(void) -{ - return false; -} +bool s390_has_topology(void); #endif diff --git a/target/s390x/cpu_features_def.h.inc b/target/s390x/cpu_features_def.h.inc index e3cfe63735..016a720e38 100644 --- a/target/s390x/cpu_features_def.h.inc +++ b/target/s390x/cpu_features_def.h.inc @@ -147,6 +147,7 @@ DEF_FEAT(SIE_CEI, "cei", SCLP_CPU, 43, "SIE: Conditional-external-interception f DEF_FEAT(DAT_ENH_2, "dateh2", MISC, 0, "DAT-enhancement facility 2") DEF_FEAT(CMM, "cmm", MISC, 0, "Collaborative-memory-management facility") DEF_FEAT(AP, "ap", MISC, 0, "AP instructions installed") +DEF_FEAT(DISABLE_CPU_TOPOLOGY, "disable-topology", MISC, 0, "Disable CPU Topology") /* Features exposed via the PLO instruction. */ DEF_FEAT(PLO_CL, "plo-cl", PLO, 0, "PLO Compare and load (32 bit in general registers)") diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index c3a4f80633..1f5348d6a3 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -253,6 +253,7 @@ bool s390_has_feat(S390Feat feat) case S390_FEAT_SIE_CMMA: case S390_FEAT_SIE_PFMFI: case S390_FEAT_SIE_IBS: + case S390_FEAT_CONFIGURATION_TOPOLOGY: return false; break; default: @@ -422,6 +423,21 @@ void s390_cpu_list(void) } } +static void check_incompatibility(S390CPUModel *model, Error **errp) +{ + static int dep[][2] = { + { S390_FEAT_CONFIGURATION_TOPOLOGY, S390_FEAT_DISABLE_CPU_TOPOLOGY }, + }; + int i; + + for (i = 0; i < ARRAY_SIZE(dep); i++) { + if (test_bit(dep[i][0], model->features) && + test_bit(dep[i][1], model->features)) { + clear_bit(dep[i][0], model->features); + } + } +} + static void check_consistency(const S390CPUModel *model) { static int dep[][2] = { @@ -592,6 +608,7 @@ void s390_realize_cpu_model(CPUState *cs, Error **errp) cpu->model->cpu_id_format = max_model->cpu_id_format; cpu->model->cpu_ver = max_model->cpu_ver; + check_incompatibility(cpu->model, &err); check_consistency(cpu->model); check_compatibility(max_model, cpu->model, &err); if (err) { diff --git a/target/s390x/cpu_topology.c b/target/s390x/cpu_topology.c index b81f016ba1..8123e6ddf0 100644 --- a/target/s390x/cpu_topology.c +++ b/target/s390x/cpu_topology.c @@ -15,6 +15,11 @@ #include "hw/s390x/cpu-topology.h" #include "hw/s390x/sclp.h" +bool s390_has_topology(void) +{ + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY); +} + /* * s390_topology_add_cpu: * @topo: pointer to the topology diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c index 1e3b7c0dc9..f3acfdd9a5 100644 --- a/target/s390x/gen-features.c +++ b/target/s390x/gen-features.c @@ -488,6 +488,7 @@ static uint16_t full_GEN9_GA3[] = { static uint16_t full_GEN10_GA1[] = { S390_FEAT_EDAT, S390_FEAT_CONFIGURATION_TOPOLOGY, + S390_FEAT_DISABLE_CPU_TOPOLOGY, S390_FEAT_GROUP_MSA_EXT_2, S390_FEAT_ESOP, S390_FEAT_SIE_PFMFI, @@ -605,6 +606,8 @@ static uint16_t default_GEN9_GA1[] = { static uint16_t default_GEN10_GA1[] = { S390_FEAT_EDAT, S390_FEAT_GROUP_MSA_EXT_2, + S390_FEAT_DISABLE_CPU_TOPOLOGY, + S390_FEAT_CONFIGURATION_TOPOLOGY, }; #define default_GEN10_GA2 EmptyFeat diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index a79fdf1c79..ec2c9fd8fa 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2471,6 +2471,20 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) set_bit(S390_FEAT_UNPACK, model->features); } + /* + * If we have support for CPU Topology prevent overrule + * S390_FEAT_CONFIGURATION_TOPOLOGY with S390_FEAT_DISABLE_CPU_TOPOLOGY + * implemented in KVM, activate the CPU TOPOLOGY feature. + */ + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) { + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) { + error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY"); + return; + } + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features); + set_bit(S390_FEAT_DISABLE_CPU_TOPOLOGY, model->features); + } + /* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit(S390_FEAT_ZPCI, model->features); set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features); From patchwork Tue Nov 29 17:42:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13058919 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8D42C433FE for ; Tue, 29 Nov 2022 17:42:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236670AbiK2Rmn (ORCPT ); Tue, 29 Nov 2022 12:42:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236664AbiK2Rmg (ORCPT ); Tue, 29 Nov 2022 12:42:36 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62FFF2BF4 for ; Tue, 29 Nov 2022 09:42:35 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2ATGx8xL013084; Tue, 29 Nov 2022 17:42:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=zRgxJwhdvMuf0/S6nTCkxTz5a4usYMPGRTMHVCwL9So=; b=E6NOUSkEUs5yk1iBQGPREyXt31R5fGWZ4g8RZQQ+OfyQLr5zqdfvGkWwEIv1axczNwQE F1TUfNSiCOfe/V14kNiOaGw7igClWraMTQ/vkRHwb3TNGye2oRvq3K5dmP1WOlnOJWvt KJdxX+hs8LGOFkUQMRRyHRvjafbqsdbLd0IcOdk4HlWopig0QL6efN4kdB5+PfMKvswk lsExxsoPqyurkFNpyz2dykVRGiDqA6OjQUaRcECIWGR2t2B1KG81h19zlUvALIknAoYU VVsy54jdluVyt1Q3w+L2Ah81z4ocnVLULrbFc/+lDQPFBNHkhUoZfhbhLhGnI4C7ngWC YQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5kmv6nu7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:22 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2ATGx7XZ013054; Tue, 29 Nov 2022 17:42:22 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3m5kmv6ntc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:21 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2ATHZN73019130; Tue, 29 Nov 2022 17:42:19 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma03ams.nl.ibm.com with ESMTP id 3m3ae9cfey-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 29 Nov 2022 17:42:19 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2ATHgxj464422286 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Nov 2022 17:42:59 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F4BEA404D; Tue, 29 Nov 2022 17:42:16 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F0F1A4040; Tue, 29 Nov 2022 17:42:15 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.89.107]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 29 Nov 2022 17:42:15 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v12 7/7] docs/s390x: document s390x cpu topology Date: Tue, 29 Nov 2022 18:42:06 +0100 Message-Id: <20221129174206.84882-8-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221129174206.84882-1-pmorel@linux.ibm.com> References: <20221129174206.84882-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: XsptuCgLKPfvCqgGefrwAoIZ-KrupQhD X-Proofpoint-ORIG-GUID: wVBJDpKjGPNYHQvzk1e33pRkioveiNM7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-29_11,2022-11-29_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 suspectscore=0 adultscore=0 phishscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 spamscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000 definitions=main-2211290097 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add some basic examples for the definition of cpu topology in s390x. Signed-off-by: Pierre Morel --- docs/system/s390x/cpu-topology.rst | 80 ++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) create mode 100644 docs/system/s390x/cpu-topology.rst diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst new file mode 100644 index 0000000000..2fad28453c --- /dev/null +++ b/docs/system/s390x/cpu-topology.rst @@ -0,0 +1,80 @@ +CPU Topology on s390x +===================== + +CPU Topology on S390x provides up to 5 levels of topology containers: +nodes, drawers, books, sockets and CPUs. +While the higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the parent container. + +Currently QEMU CPU topology uses a single level of container: the sockets. + +For backward compatibility, threads can be declared on the ``-smp`` command +line. They will be seen as CPUs by the guest as long as multithreading +is not really supported by QEMU for S390. + +Prerequisites +------------- + +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version +of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM +QEMU use of the CPU Topology. + +Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments. + +Like in : + +.. code-block:: sh + -smp cpus=5,sockets=8,cores=2,threads=2,maxcpus=32 + -device host-s390x-cpu,core-id=14 + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: sh + (qemu) device_add host-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 2 threads in 2 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will + be placed in socket 2, with core-id 9 + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be: + +* in socket 0: 0xf0000000 (core id 0,1,2,3) +* in socket 1: 0x00400000 (core id 9) +* in socket 1: 0x00020000 (core id 14) + +Enabling CPU topology +--------------------- + +Currently, CPU topology is disabled by default. + +Enabling CPU topology can be done by setting the feature flag +``disable-topology`` to ``off`` like in: + +.. code-block:: sh + -cpu gen16b,disable-topology=off + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags.