From patchwork Wed Feb 1 13:20:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AF53C636CD for ; Wed, 1 Feb 2023 13:21:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231778AbjBANVQ (ORCPT ); Wed, 1 Feb 2023 08:21:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229451AbjBANVP (ORCPT ); Wed, 1 Feb 2023 08:21:15 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7149813DDA for ; Wed, 1 Feb 2023 05:21:13 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311BvkcA015790; Wed, 1 Feb 2023 13:21:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=nfyZvxKs4mIjgRK2bDgZ60HbGU4YEVP4EliC+Vlaurg=; b=PRCghMXhA1YeSJI+HLQvOYRjnfRkdFoxx8gAq6UuO4MYsRdDKsjF69uazSUwNfVGHMQp VAgQYWucKUb+L6y/KypaXyl3Gq16qFzuWwSjKKfe6kDnXFnImm4VPdQ9ssRFY4i7FN3A nhkqnm43hQDyF/ObM+5qfaBc2p9KngEzsJ3QiV2Qe1VhtWG4IOkd3JCkqnv7/Rmw5Zfq EhvYqYHDJHcPUUqgjVHMAvR0895rGZr3r1DkJLFjVZZxrlX77J+QMQytT86KocFPRuD6 T6uPX94DUFkKJuLvygfJarn6dEMf/InXmPsxdkgqbt5aC6ZYeaS0X8i+zhqV+Cls+JQC bw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqnwj1x7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:03 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311CTU3t032063; Wed, 1 Feb 2023 13:21:02 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqnwj1wd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:02 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3116BqjI026906; Wed, 1 Feb 2023 13:21:01 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3ncvs7mw7a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:00 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DKvsU46137698 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:20:57 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 49ED920040; Wed, 1 Feb 2023 13:20:57 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D278820043; Wed, 1 Feb 2023 13:20:55 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:20:55 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Date: Wed, 1 Feb 2023 14:20:41 +0100 Message-Id: <20230201132051.126868-2-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: YVctsc91v2CyFjlYJ1h5x-gOgU0X-rZ5 X-Proofpoint-ORIG-GUID: m6RfHxREouUPYR5gH2DijKRsvgzoI9pk X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 phishscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org S390 adds two new SMP levels, drawers and books to the CPU topology. The S390 CPU have specific toplogy features like dedication and polarity to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs. Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and polarity, Signed-off-by: Pierre Morel Reviewed-by: Thomas Huth --- qapi/machine.json | 14 ++++++++-- include/hw/boards.h | 10 ++++++- include/hw/s390x/cpu-topology.h | 24 +++++++++++++++++ target/s390x/cpu.h | 5 ++++ hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++----- hw/core/machine.c | 4 +++ hw/s390x/s390-virtio-ccw.c | 2 ++ softmmu/vl.c | 6 +++++ target/s390x/cpu.c | 7 +++++ qemu-options.hx | 7 +++-- 10 files changed, 115 insertions(+), 12 deletions(-) create mode 100644 include/hw/s390x/cpu-topology.h diff --git a/qapi/machine.json b/qapi/machine.json index b9228a5e46..3036117059 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -900,13 +900,15 @@ # a CPU is being hotplugged. # # @node-id: NUMA node ID the CPU belongs to -# @socket-id: socket number within node/board the CPU belongs to +# @drawer-id: drawer number within node/board the CPU belongs to (since 8.0) +# @book-id: book number within drawer/node/board the CPU belongs to (since 8.0) +# @socket-id: socket number within book/node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1) # @cluster-id: cluster number within die the CPU belongs to (since 7.1) # @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present +# Note: currently there are 8 properties that could be present # but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in @@ -916,6 +918,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int', @@ -1465,6 +1469,10 @@ # # @cpus: number of virtual CPUs in the virtual machine # +# @drawers: number of drawers in the CPU topology (since 8.0) +# +# @books: number of books in the CPU topology (since 8.0) +# # @sockets: number of sockets in the CPU topology # # @dies: number of dies per socket in the CPU topology @@ -1481,6 +1489,8 @@ ## { 'struct': 'SMPConfiguration', 'data': { '*cpus': 'int', + '*drawers': 'int', + '*books': 'int', '*sockets': 'int', '*dies': 'int', '*clusters': 'int', diff --git a/include/hw/boards.h b/include/hw/boards.h index 6fbbfd56c8..9ef0bb76cf 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -131,12 +131,16 @@ typedef struct { * @clusters_supported - whether clusters are supported by the machine * @has_clusters - whether clusters are explicitly specified in the user * provided SMP configuration + * @books_supported - whether books are supported by the machine + * @drawers_supported - whether drawers are supported by the machine */ typedef struct { bool prefer_sockets; bool dies_supported; bool clusters_supported; bool has_clusters; + bool books_supported; + bool drawers_supported; } SMPCompatProps; /** @@ -301,7 +305,9 @@ typedef struct DeviceMemoryState { /** * CpuTopology: * @cpus: the number of present logical processors on the machine - * @sockets: the number of sockets on the machine + * @drawers: the number of drawers on the machine + * @books: the number of books in one drawer + * @sockets: the number of sockets in one book * @dies: the number of dies in one socket * @clusters: the number of clusters in one die * @cores: the number of cores in one cluster @@ -310,6 +316,8 @@ typedef struct DeviceMemoryState { */ typedef struct CpuTopology { unsigned int cpus; + unsigned int drawers; + unsigned int books; unsigned int sockets; unsigned int dies; unsigned int clusters; diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h new file mode 100644 index 0000000000..7a84b30a21 --- /dev/null +++ b/include/hw/s390x/cpu-topology.h @@ -0,0 +1,24 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H + +#define S390_TOPOLOGY_CPU_IFL 0x03 + +enum s390_topology_polarity { + POLARITY_HORIZONTAL, + POLARITY_VERTICAL, + POLARITY_VERTICAL_LOW = 1, + POLARITY_VERTICAL_MEDIUM, + POLARITY_VERTICAL_HIGH, + POLARITY_MAX, +}; + +#endif diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 7d6d01325b..d654267a71 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -131,6 +131,11 @@ struct CPUArchState { #if !defined(CONFIG_USER_ONLY) uint32_t core_id; /* PoP "CPU address", same as cpu_index */ + int32_t socket_id; + int32_t book_id; + int32_t drawer_id; + bool dedicated; + uint8_t entitlement; /* Used only for vertical polarization */ uint64_t cpuid; #endif diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c index c3dab007da..b8233df5a9 100644 --- a/hw/core/machine-smp.c +++ b/hw/core/machine-smp.c @@ -31,6 +31,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms) MachineClass *mc = MACHINE_GET_CLASS(ms); GString *s = g_string_new(NULL); + if (mc->smp_props.drawers_supported) { + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers); + } + + if (mc->smp_props.books_supported) { + g_string_append_printf(s, " * books (%u)", ms->smp.books); + } + g_string_append_printf(s, "sockets (%u)", ms->smp.sockets); if (mc->smp_props.dies_supported) { @@ -73,6 +81,8 @@ void machine_parse_smp_config(MachineState *ms, { MachineClass *mc = MACHINE_GET_CLASS(ms); unsigned cpus = config->has_cpus ? config->cpus : 0; + unsigned drawers = config->has_drawers ? config->drawers : 0; + unsigned books = config->has_books ? config->books : 0; unsigned sockets = config->has_sockets ? config->sockets : 0; unsigned dies = config->has_dies ? config->dies : 0; unsigned clusters = config->has_clusters ? config->clusters : 0; @@ -85,6 +95,8 @@ void machine_parse_smp_config(MachineState *ms, * explicit configuration like "cpus=0" is not allowed. */ if ((config->has_cpus && config->cpus == 0) || + (config->has_drawers && config->drawers == 0) || + (config->has_books && config->books == 0) || (config->has_sockets && config->sockets == 0) || (config->has_dies && config->dies == 0) || (config->has_clusters && config->clusters == 0) || @@ -111,6 +123,19 @@ void machine_parse_smp_config(MachineState *ms, dies = dies > 0 ? dies : 1; clusters = clusters > 0 ? clusters : 1; + if (!mc->smp_props.books_supported && books > 1) { + error_setg(errp, "books not supported by this machine's CPU topology"); + return; + } + books = books > 0 ? books : 1; + + if (!mc->smp_props.drawers_supported && drawers > 1) { + error_setg(errp, + "drawers not supported by this machine's CPU topology"); + return; + } + drawers = drawers > 0 ? drawers : 1; + /* compute missing values based on the provided ones */ if (cpus == 0 && maxcpus == 0) { sockets = sockets > 0 ? sockets : 1; @@ -124,33 +149,41 @@ void machine_parse_smp_config(MachineState *ms, if (sockets == 0) { cores = cores > 0 ? cores : 1; threads = threads > 0 ? threads : 1; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); } else if (cores == 0) { threads = threads > 0 ? threads : 1; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); } } else { /* prefer cores over sockets since 6.2 */ if (cores == 0) { sockets = sockets > 0 ? sockets : 1; threads = threads > 0 ? threads : 1; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); } else if (sockets == 0) { threads = threads > 0 ? threads : 1; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); } } /* try to calculate omitted threads at last */ if (threads == 0) { - threads = maxcpus / (sockets * dies * clusters * cores); + threads = maxcpus / + (drawers * books * sockets * dies * clusters * cores); } } - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads; + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies * + clusters * cores * threads; cpus = cpus > 0 ? cpus : maxcpus; ms->smp.cpus = cpus; + ms->smp.drawers = drawers; + ms->smp.books = books; ms->smp.sockets = sockets; ms->smp.dies = dies; ms->smp.clusters = clusters; @@ -161,7 +194,8 @@ void machine_parse_smp_config(MachineState *ms, mc->smp_props.has_clusters = config->has_clusters; /* sanity-check of the computed topology */ - if (sockets * dies * clusters * cores * threads != maxcpus) { + if (drawers * books * sockets * dies * clusters * cores * threads != + maxcpus) { g_autofree char *topo_msg = cpu_hierarchy_to_string(ms); error_setg(errp, "Invalid CPU topology: " "product of the hierarchy must match maxcpus: " diff --git a/hw/core/machine.c b/hw/core/machine.c index 616f3a207c..e38f99230b 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -828,6 +828,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name, MachineState *ms = MACHINE(obj); SMPConfiguration *config = &(SMPConfiguration){ .has_cpus = true, .cpus = ms->smp.cpus, + .has_drawers = true, .drawers = ms->smp.drawers, + .has_books = true, .books = ms->smp.books, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters, @@ -1093,6 +1095,8 @@ static void machine_initfn(Object *obj) /* default to mc->default_cpus */ ms->smp.cpus = mc->default_cpus; ms->smp.max_cpus = mc->default_cpus; + ms->smp.drawers = 1; + ms->smp.books = 1; ms->smp.sockets = 1; ms->smp.dies = 1; ms->smp.clusters = 1; diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index f22f61b8b6..f3cc845d3b 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -733,6 +733,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) mc->no_sdcard = 1; mc->max_cpus = S390_MAX_CPUS; mc->has_hotpluggable_cpus = true; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true; assert(!mc->get_hotplug_handler); mc->get_hotplug_handler = s390_get_hotplug_handler; mc->cpu_index_to_instance_props = s390_cpu_index_to_props; diff --git a/softmmu/vl.c b/softmmu/vl.c index 9177d95d4e..05e1f33fa5 100644 --- a/softmmu/vl.c +++ b/softmmu/vl.c @@ -727,6 +727,12 @@ static QemuOptsList qemu_smp_opts = { { .name = "cpus", .type = QEMU_OPT_NUMBER, + }, { + .name = "drawers", + .type = QEMU_OPT_NUMBER, + }, { + .name = "books", + .type = QEMU_OPT_NUMBER, }, { .name = "sockets", .type = QEMU_OPT_NUMBER, diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c index b10a8541ff..828e8b8fce 100644 --- a/target/s390x/cpu.c +++ b/target/s390x/cpu.c @@ -37,6 +37,7 @@ #ifndef CONFIG_USER_ONLY #include "sysemu/reset.h" #endif +#include "hw/s390x/cpu-topology.h" #define CR0_RESET 0xE0UL #define CR14_RESET 0xC2000000UL; @@ -259,6 +260,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs) static Property s390x_cpu_properties[] = { #if !defined(CONFIG_USER_ONLY) DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0), + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1), + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1), + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1), + DEFINE_PROP_BOOL("dedicated", S390CPU, env.dedicated, false), + DEFINE_PROP_UINT8("polarity", S390CPU, env.entitlement, + POLARITY_VERTICAL_MEDIUM), #endif DEFINE_PROP_END_OF_LIST() }; diff --git a/qemu-options.hx b/qemu-options.hx index d59d19704b..131687c6dc 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -250,11 +250,14 @@ SRST ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, - "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n" + " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n" - " sockets= number of sockets on the machine board\n" + " drawers= number of drawers on the machine board\n" + " books= number of books in one drawer\n" + " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" " cores= number of cores in one cluster\n" From patchwork Wed Feb 1 13:20:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 904BFC05027 for ; Wed, 1 Feb 2023 13:21:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231964AbjBANVW (ORCPT ); Wed, 1 Feb 2023 08:21:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231835AbjBANVV (ORCPT ); Wed, 1 Feb 2023 08:21:21 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 494EE1E29D for ; Wed, 1 Feb 2023 05:21:19 -0800 (PST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311Cmn2d016436; Wed, 1 Feb 2023 13:21:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=m6kEUNgOMiCoc4olUf64ZQBKhmiWrd0cl9q3qLoru5Q=; b=jugprLDzfz0wW/8HVZVR+Z/3kMoE9VRWmQCYop5zTzJfLS3KE31D7G7tGHKZD+zM5lEe ZaoaiisrQva8weG0QH/wpDKfvF+VYzSK+X55YG22DzZ3FE/RlUSvYBNrwO+HYq9A9Ihg sIYCEGfg/mYf9eVUr3l5p57k8nBdxGzWIXMFxvyAub7kgxqFFjrHn/Pms6OnjHCNQkZP wHaFng1qcgskbZG1cCHar61iyZQB4P/Ibs5xcuULdR7dKU+w1mIcg04V3oLd3auFrE4e rRvU+L7qqS4Ui9AFPY2p31hBujq7Gl9DwZgur51WIeXC0Nl/wowJAKVzBtRPzdTsWk8x LQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrdugryc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:05 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311CpQWU028295; Wed, 1 Feb 2023 13:21:05 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrdugrxb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:05 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3111nv5B023655; Wed, 1 Feb 2023 13:21:02 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma06fra.de.ibm.com (PPS) with ESMTPS id 3ncvugkggd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:02 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DKxNh50463088 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:20:59 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0805320043; Wed, 1 Feb 2023 13:20:59 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A903620040; Wed, 1 Feb 2023 13:20:57 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:20:57 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 02/11] s390x/cpu topology: add topology entries on CPU hotplug Date: Wed, 1 Feb 2023 14:20:42 +0100 Message-Id: <20230201132051.126868-3-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: ab5-zZDtWUhGYC8r29lyLt4sHyu-1pLz X-Proofpoint-ORIG-GUID: JCU93Gcfy7W-EqAM9YCmiaIqFBRFmTtl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 suspectscore=0 clxscore=1015 malwarescore=0 spamscore=0 mlxlogscore=999 mlxscore=0 adultscore=0 impostorscore=0 priorityscore=1501 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The topology information are attributes of the CPU and are specified during the CPU device creation. On hot plug we: - calculate the default values for the topology for drawers, books and sockets in the case they are not specified. - verify the CPU attributes - check that we have still room on the desired socket The possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer. If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes. This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 24 +++ hw/s390x/cpu-topology.c | 256 ++++++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 23 ++- hw/s390x/meson.build | 1 + 4 files changed, 302 insertions(+), 2 deletions(-) create mode 100644 hw/s390x/cpu-topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index 7a84b30a21..9b6f889ad4 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -10,6 +10,9 @@ #ifndef HW_S390X_CPU_TOPOLOGY_H #define HW_S390X_CPU_TOPOLOGY_H +#include "qemu/queue.h" +#include "hw/boards.h" + #define S390_TOPOLOGY_CPU_IFL 0x03 enum s390_topology_polarity { @@ -21,4 +24,25 @@ enum s390_topology_polarity { POLARITY_MAX, }; +typedef struct S390Topology { + uint8_t *cores_per_socket; + CpuTopology *smp; +} S390Topology; + +#ifdef CONFIG_KVM +bool s390_has_topology(void); +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp); +#else +static inline bool s390_has_topology(void) +{ + return false; +} +static inline void s390_topology_set_cpu(MachineState *ms, + S390CPU *cpu, + Error **errp) {} +#endif + +extern S390Topology s390_topology; +int s390_socket_nb(S390CPU *cpu); + #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c new file mode 100644 index 0000000000..12df4eca6c --- /dev/null +++ b/hw/s390x/cpu-topology.c @@ -0,0 +1,256 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/* + * s390_topology is used to keep the topology information. + * .list: queue the topology entries inside which + * we keep the information on the CPU topology. + * .socket: tracks information on the count of cores per socket. + * .smp: keeps track of the machine topology. + * + */ +S390Topology s390_topology = { + /* will be initialized after the cpu model is realized */ + .cores_per_socket = NULL, + .smp = NULL, +}; + +/** + * s390_socket_nb: + * @cpu: s390x CPU + * + * Returns the socket number used inside the cores_per_socket array + * for a cpu. + */ +int s390_socket_nb(S390CPU *cpu) +{ + return (cpu->env.drawer_id * s390_topology.smp->books + cpu->env.book_id) * + s390_topology.smp->sockets + cpu->env.socket_id; +} + +/** + * s390_has_topology: + * + * Return value: if the topology is supported by the machine. + */ +bool s390_has_topology(void) +{ + return false; +} + +/** + * s390_topology_init: + * @ms: the machine state where the machine topology is defined + * + * Keep track of the machine topology. + * + * Allocate an array to keep the count of cores per socket. + * The index of the array starts at socket 0 from book 0 and + * drawer 0 up to the maximum allowed by the machine topology. + */ +static void s390_topology_init(MachineState *ms) +{ + CpuTopology *smp = &ms->smp; + + s390_topology.smp = smp; + s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets * + smp->books * smp->drawers); +} + +/** + * s390_topology_cpu_default: + * @cpu: pointer to a S390CPU + * @errp: Error pointer + * + * Setup the default topology for unset attributes. + * + * The function accept only all all default values or all set values + * for the geometry topology. + * + * The function calculates the (drawer_id, book_id, socket_id) + * topology by filling the cores starting from the first socket + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets). + * + * CPU type, polarity and dedication have defaults values set in the + * s390x_cpu_properties. + */ +static void s390_topology_cpu_default(S390CPU *cpu, Error **errp) +{ + CpuTopology *smp = s390_topology.smp; + CPUS390XState *env = &cpu->env; + + /* All geometry topology attributes must be set or all unset */ + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) && + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) { + error_setg(errp, + "Please define all or none of the topology geometry attributes"); + return; + } + + /* Check if one of the geometry topology is unset */ + if (env->socket_id < 0) { + /* Calculate default geometry topology attributes */ + env->socket_id = (env->core_id / smp->cores) % smp->sockets; + env->book_id = (env->core_id / (smp->sockets * smp->cores)) % + smp->books; + env->drawer_id = (env->core_id / + (smp->books * smp->sockets * smp->cores)) % + smp->drawers; + } +} + +/** + * s390_topology_check: + * @cpu: s390x CPU to be verified + * @errp: Error pointer + * + * The function first setup default values and then checks if the cpu + * fits inside the system topology. + */ +static void s390_topology_check(S390CPU *cpu, Error **errp) +{ + CpuTopology *smp = s390_topology.smp; + ERRP_GUARD(); + + s390_topology_cpu_default(cpu, errp); + if (*errp) { + return; + } + + if (cpu->env.socket_id > smp->sockets) { + error_setg(errp, "Unavailable socket: %d", cpu->env.socket_id); + return; + } + if (cpu->env.book_id > smp->books) { + error_setg(errp, "Unavailable book: %d", cpu->env.book_id); + return; + } + if (cpu->env.drawer_id > smp->drawers) { + error_setg(errp, "Unavailable drawer: %d", cpu->env.drawer_id); + return; + } + if (cpu->env.entitlement >= POLARITY_MAX) { + error_setg(errp, "Unknown polarity: %d", cpu->env.entitlement); + return; + } + + /* Dedication, boolean, can not be wrong. */ +} + +/** + * s390_set_core_in_socket: + * @cpu: the new S390CPU to insert in the topology structure + * @drawer_id: new drawer_id + * @book_id: new book_id + * @socket_id: new socket_id + * @creation: if is true the CPU is a new CPU and there is no old socket + * to handle. + * if is false, this is a moving the CPU and old socket count + * must be decremented. + * @errp: the error pointer + * + */ +static void s390_set_core_in_socket(S390CPU *cpu, int drawer_id, int book_id, + int socket_id, bool creation, Error **errp) +{ + int old_socket = s390_socket_nb(cpu); + int new_socket; + + if (creation) { + new_socket = old_socket; + } else { + new_socket = drawer_id * s390_topology.smp->books + + book_id * s390_topology.smp->sockets + + socket_id; + } + + /* Check for space on new socket */ + if ((new_socket != old_socket) && + (s390_topology.cores_per_socket[new_socket] >= + s390_topology.smp->cores)) { + error_setg(errp, "No more space on this socket"); + return; + } + + /* Update the count of cores in sockets */ + s390_topology.cores_per_socket[new_socket] += 1; + if (!creation) { + s390_topology.cores_per_socket[old_socket] -= 1; + } +} + +/** + * s390_update_cpu_props: + * @ms: the machine state + * @cpu: the CPU for which to update the properties from the environment. + * + */ +static void s390_update_cpu_props(MachineState *ms, S390CPU *cpu) +{ + CpuInstanceProperties *props; + + props = &ms->possible_cpus->cpus[cpu->env.core_id].props; + + props->socket_id = cpu->env.socket_id; + props->book_id = cpu->env.book_id; + props->drawer_id = cpu->env.drawer_id; +} + +/** + * s390_topology_set_cpu: + * @ms: MachineState used to initialize the topology structure on + * first call. + * @cpu: the new S390CPU to insert in the topology structure + * @errp: the error pointer + * + * Called from CPU Hotplug to check and setup the CPU attributes + * before to insert the CPU in the topology. + */ +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp) +{ + ERRP_GUARD(); + + /* + * We do not want to initialize the topology if the cpu model + * does not support topology consequently, we have to wait for + * the first CPU to be realized, which realizes the CPU model + * to initialize the topology structures. + * + * s390_topology_set_cpu() is called from the cpu hotplug. + */ + if (!s390_topology.cores_per_socket) { + s390_topology_init(ms); + } + + s390_topology_check(cpu, errp); + if (*errp) { + return; + } + + /* Set the CPU inside the socket */ + s390_set_core_in_socket(cpu, 0, 0, 0, true, errp); + if (*errp) { + return; + } + + /* topology tree is reflected in props */ + s390_update_cpu_props(ms, cpu); +} diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index f3cc845d3b..9bc51a83f4 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -44,6 +44,7 @@ #include "hw/s390x/pv.h" #include "migration/blocker.h" #include "qapi/visitor.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker; @@ -310,10 +311,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev, { MachineState *ms = MACHINE(hotplug_dev); S390CPU *cpu = S390_CPU(dev); + ERRP_GUARD(); g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev); + if (s390_has_topology()) { + s390_topology_set_cpu(ms, cpu, errp); + if (*errp) { + return; + } + } + if (dev->hotplugged) { raise_irq_cpu_hotplug(); } @@ -551,11 +560,21 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms) sizeof(CPUArchId) * max_cpus); ms->possible_cpus->len = max_cpus; for (i = 0; i < ms->possible_cpus->len; i++) { + CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props; + ms->possible_cpus->cpus[i].type = ms->cpu_type; ms->possible_cpus->cpus[i].vcpus_count = 1; ms->possible_cpus->cpus[i].arch_id = i; - ms->possible_cpus->cpus[i].props.has_core_id = true; - ms->possible_cpus->cpus[i].props.core_id = i; + + props->has_core_id = true; + props->core_id = i; + props->has_socket_id = true; + props->socket_id = i / ms->smp.cores; + props->has_book_id = true; + props->book_id = i / (ms->smp.cores * ms->smp.sockets); + props->has_drawer_id = true; + props->drawer_id = i / + (ms->smp.cores * ms->smp.sockets * ms->smp.books); } return ms->possible_cpus; diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build index f291016fee..58dfbdff4f 100644 --- a/hw/s390x/meson.build +++ b/hw/s390x/meson.build @@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( 's390-stattrib-kvm.c', 'pv.c', 's390-pci-kvm.c', + 'cpu-topology.c', )) s390x_ss.add(when: 'CONFIG_TCG', if_true: files( 'tod-tcg.c', From patchwork Wed Feb 1 13:20:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48BEFC636D3 for ; Wed, 1 Feb 2023 13:21:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232108AbjBANVa (ORCPT ); Wed, 1 Feb 2023 08:21:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232090AbjBANV0 (ORCPT ); Wed, 1 Feb 2023 08:21:26 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 207743C2B2 for ; Wed, 1 Feb 2023 05:21:21 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311BvrQD016040; Wed, 1 Feb 2023 13:21:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=wfQa3RzvnJQklH2X5jmW/rDPAVA/tDyZ+ssWCRvrA0A=; b=BOe6rOhQVfdRmRv/Tk6dc/r69tx0YGigeESMk9OTam5WQkFbx3RNtsHX70V5VplzYIAT z3sC++lTKe/iT5OLATtCcFFxjtD8LEuwQ4cpo+GM9zn36zEyvi9R8kJz4NtDyOYYwlIx 47uJEjyvK4rV6QArWtwsyHxQmG6olsfuFrMBCly2DcbpyQSRgtUZsksadGYDVMuEPwVe n8xy9CCBApCbK1Fzuu7lTs9Oi4wBWr4TQuwJDJQ9xZNx+ONAh2NzlSR/NJhHeRk8Ci0l EhHxYXtvGzK4xm4V/UVhEvk0VaLXD8OYtFTuvZZtzcl2jPoghk99pWau0j9E2i3TnFpY jw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqnwj20j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:06 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311ClXkm020243; Wed, 1 Feb 2023 13:21:06 GMT Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqnwj1yk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:06 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30VNLCfH028705; Wed, 1 Feb 2023 13:21:04 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma01fra.de.ibm.com (PPS) with ESMTPS id 3ncvuquh1m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:04 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL0Le45941202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:00 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 79D0220043; Wed, 1 Feb 2023 13:21:00 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 23EEB20040; Wed, 1 Feb 2023 13:20:59 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:20:59 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Date: Wed, 1 Feb 2023 14:20:43 +0100 Message-Id: <20230201132051.126868-4-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: BT-Y_yQgCfhZstUXECYWKRrMfJq8oFOl X-Proofpoint-ORIG-GUID: K6QuBE7PYzDcU-vTJaZjyZr-2P2Leb8Q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 phishscore=0 priorityscore=1501 bulkscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 adultscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 22 +++ include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 72 +++++++ hw/s390x/cpu-topology.c | 10 + target/s390x/kvm/cpu_topology.c | 335 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/meson.build | 3 +- 7 files changed, 446 insertions(+), 2 deletions(-) create mode 100644 target/s390x/kvm/cpu_topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index 9b6f889ad4..1ae7e7c5e3 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -24,9 +24,31 @@ enum s390_topology_polarity { POLARITY_MAX, }; +typedef union s390_topology_id { + uint64_t id; + struct { + uint8_t level5; + uint8_t drawer; + uint8_t book; + uint8_t socket; + uint8_t dedicated; + uint8_t polarity; + uint8_t type; + uint8_t origin; + }; +} s390_topology_id; + +typedef struct S390TopologyEntry { + QTAILQ_ENTRY(S390TopologyEntry) next; + s390_topology_id id; + uint64_t mask; +} S390TopologyEntry; + typedef struct S390Topology { uint8_t *cores_per_socket; + QTAILQ_HEAD(, S390TopologyEntry) list; CpuTopology *smp; + uint8_t polarity; } S390Topology; #ifdef CONFIG_KVM diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h index d3ade40a5a..712fd68123 100644 --- a/include/hw/s390x/sclp.h +++ b/include/hw/s390x/sclp.h @@ -112,6 +112,7 @@ typedef struct CPUEntry { } QEMU_PACKED CPUEntry; #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 +#define SCLP_READ_SCP_INFO_MNEST 2 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax; diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index d654267a71..e1f6925856 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -560,6 +560,25 @@ typedef struct SysIB_322 { } SysIB_322; QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096); +#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[]; +} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x; +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16); + typedef union SysIB { SysIB_111 sysib_111; SysIB_121 sysib_121; @@ -567,9 +586,62 @@ typedef union SysIB { SysIB_221 sysib_221; SysIB_222 sysib_222; SysIB_322 sysib_322; + SysIB_151x sysib_151x; } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096); +/* + * CPU Topology List provided by STSI with fc=15 provides a list + * of two different Topology List Entries (TLE) types to specify + * the topology hierarchy. + * + * - Container Topology List Entry + * Defines a container to contain other Topology List Entries + * of any type, nested containers or CPU. + * - CPU Topology List Entry + * Specifies the CPUs position, type, entitlement and polarization + * of the CPUs contained in the last Container TLE. + * + * There can be theoretically up to five levels of containers, QEMU + * uses only three levels, the drawer's, book's and socket's level. + * + * A container of with a nesting level (NL) greater than 1 can only + * contain another container of nesting level NL-1. + * + * A container of nesting level 1 (socket), contains as many CPU TLE + * as needed to describe the position and qualities of all CPUs inside + * the container. + * The qualities of a CPU are polarization, entitlement and type. + * + * The CPU TLE defines the position of the CPUs of identical qualities + * using a 64bits mask which first bit has its offset defined by + * the CPU address orgin field of the CPU TLE like in: + * CPU address = origin * 64 + bit position within the mask + * + */ +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8); + +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; +#define SYSIB_TLE_POLARITY_MASK 0x03 +#define SYSIB_TLE_DEDICATED 0x04 + uint8_t entitlement; + uint8_t type; + uint16_t origin; + uint64_t mask; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16); + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar); + /* MMU defines */ #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ #define ASCE_SUBSPACE 0x200 /* subspace group control */ diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 12df4eca6c..a80a1ebf22 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -31,6 +31,8 @@ S390Topology s390_topology = { /* will be initialized after the cpu model is realized */ .cores_per_socket = NULL, .smp = NULL, + .list = QTAILQ_HEAD_INITIALIZER(s390_topology.list), + .polarity = POLARITY_HORIZONTAL, }; /** @@ -65,14 +67,22 @@ bool s390_has_topology(void) * Allocate an array to keep the count of cores per socket. * The index of the array starts at socket 0 from book 0 and * drawer 0 up to the maximum allowed by the machine topology. + * + * Insert a sentinel entry using unused level5 with its maximum value. + * This entry will never be free. */ static void s390_topology_init(MachineState *ms) { CpuTopology *smp = &ms->smp; + S390TopologyEntry *entry; s390_topology.smp = smp; s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets * smp->books * smp->drawers); + + entry = g_malloc0(sizeof(S390TopologyEntry)); + entry->id.level5 = 0xff; + QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next); } /** diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c new file mode 100644 index 0000000000..aba141fb66 --- /dev/null +++ b/target/s390x/kvm/cpu_topology.c @@ -0,0 +1,335 @@ +/* + * QEMU S390x CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/sclp.h" +#include "hw/s390x/cpu-topology.h" + +/** + * fill_container: + * @p: The address of the container TLE to fill + * @level: The level of nesting for this container + * @id: The container receives a uniq ID inside its own container + * + * Returns the next free TLE entry. + */ +static char *fill_container(char *p, int level, int id) +{ + SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; + return p + sizeof(*tle); +} + +/** + * fill_tle_cpu: + * @p: The address of the CPU TLE to fill + * @entry: a pointer to the S390TopologyEntry defining this + * CPU container. + * + * Returns the next free TLE entry. + */ +static char *fill_tle_cpu(char *p, S390TopologyEntry *entry) +{ + SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + s390_topology_id topology_id = entry->id; + + tle->nl = 0; + if (topology_id.dedicated) { + tle->entitlement = SYSIB_TLE_DEDICATED; + } + tle->entitlement |= topology_id.polarity; + tle->type = topology_id.type; + tle->origin = cpu_to_be16(topology_id.origin * 64); + tle->mask = cpu_to_be64(entry->mask); + return p + sizeof(*tle); +} + +/* + * Macro to check that the size of data after increment + * will not get bigger than the size of the SysIB. + */ +#define SYSIB_GUARD(data, x) do { \ + data += x; \ + if (data > sizeof(SysIB)) { \ + return -ENOSPC; \ + } \ + } while (0) + +/** + * stsi_set_tle: + * @p: A pointer to the position of the first TLE + * @level: The nested level wanted by the guest + * + * Loop inside the s390_topology.list until the sentinelle entry + * is found and for each entry: + * - Check using SYSIB_GUARD() that the size of the SysIB is not + * reached. + * - Add all the container TLE needed for the level + * - Add the CPU TLE. + * + * Return value: + * s390_top_set_level returns the size of the SysIB_15x after being + * filled with TLE on success. + * It returns -ENOSPC in the case we would overrun the end of the SysIB. + */ +static int stsi_set_tle(char *p, int level) +{ + S390TopologyEntry *entry; + int last_drawer = -1; + int last_book = -1; + int last_socket = -1; + int drawer_id = 0; + int book_id = 0; + int socket_id = 0; + int n = sizeof(SysIB_151x); + + QTAILQ_FOREACH(entry, &s390_topology.list, next) { + int current_drawer = entry->id.drawer; + int current_book = entry->id.book; + int current_socket = entry->id.socket; + bool drawer_change = last_drawer != current_drawer; + bool book_change = drawer_change || last_book != current_book; + bool socket_change = book_change || last_socket != current_socket; + + /* If we reach the guard get out */ + if (entry->id.level5) { + break; + } + + if (level > 3 && drawer_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 3, drawer_id++); + book_id = 0; + } + if (level > 2 && book_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 2, book_id++); + socket_id = 0; + } + if (socket_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 1, socket_id++); + } + + SYSIB_GUARD(n, sizeof(SysIBTl_cpu)); + p = fill_tle_cpu(p, entry); + last_drawer = entry->id.drawer; + last_book = entry->id.book; + last_socket = entry->id.socket; + } + + return n; +} + +/** + * setup_stsi: + * sysib: pointer to a SysIB to be filled with SysIB_151x data + * level: Nested level specified by the guest + * + * Setup the SysIB_151x header before calling stsi_set_tle with + * a pointer to the first TLE entry. + */ +static int setup_stsi(SysIB_151x *sysib, int level) +{ + sysib->mnest = level; + switch (level) { + case 4: + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers; + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 3: + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers * + current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 2: + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers * + current_machine->smp.books * + current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + } + + return stsi_set_tle(sysib->tle, level); +} + +/** + * s390_topology_add_cpu_to_entry: + * @entry: Topology entry to setup + * @cpu: the S390CPU to add + * + * Set the core bit inside the topology mask and + * increments the number of cores for the socket. + */ +static void s390_topology_add_cpu_to_entry(S390TopologyEntry *entry, + S390CPU *cpu) +{ + set_bit(63 - (cpu->env.core_id % 64), &entry->mask); +} + +/** + * s390_topology_new_entry: + * @id: s390_topology_id to add + * @cpu: the S390CPU to add + * + * Allocate a new entry and initialize it. + * + * returns the newly allocated entry. + */ +static S390TopologyEntry *s390_topology_new_entry(s390_topology_id id, + S390CPU *cpu) +{ + S390TopologyEntry *entry; + + entry = g_malloc0(sizeof(S390TopologyEntry)); + entry->id.id = id.id; + s390_topology_add_cpu_to_entry(entry, cpu); + + return entry; +} + +/** + * s390_topology_from_cpu: + * @cpu: The S390CPU + * + * Initialize the topology id from the CPU environment. + */ +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu) +{ + s390_topology_id topology_id = {0}; + + topology_id.drawer = cpu->env.drawer_id; + topology_id.book = cpu->env.book_id; + topology_id.socket = cpu->env.socket_id; + topology_id.origin = cpu->env.core_id / 64; + topology_id.type = S390_TOPOLOGY_CPU_IFL; + topology_id.dedicated = cpu->env.dedicated; + + if (s390_topology.polarity == POLARITY_VERTICAL) { + /* + * Vertical polarity with dedicated CPU implies + * vertical high entitlement. + */ + if (topology_id.dedicated) { + topology_id.polarity |= POLARITY_VERTICAL_HIGH; + } else { + topology_id.polarity |= cpu->env.entitlement; + } + } + + return topology_id; +} + +/** + * s390_topology_insert: + * @cpu: s390CPU insert. + * + * Parse the topology list to find if the entry already + * exist and add the core in it. + * If it does not exist, allocate a new entry and insert + * it in the queue from lower id to greater id. + */ +static void s390_topology_insert(S390CPU *cpu) +{ + s390_topology_id id = s390_topology_from_cpu(cpu); + S390TopologyEntry *entry = NULL; + S390TopologyEntry *tmp = NULL; + + QTAILQ_FOREACH(tmp, &s390_topology.list, next) { + if (id.id == tmp->id.id) { + s390_topology_add_cpu_to_entry(tmp, cpu); + return; + } else if (id.id < tmp->id.id) { + entry = s390_topology_new_entry(id, cpu); + QTAILQ_INSERT_BEFORE(tmp, entry, next); + return; + } + } +} + +/** + * s390_order_tle: + * + * Loop over all CPU and insert it at the right place + * inside the TLE entry list. + */ +static void s390_order_tle(void) +{ + CPUState *cs; + + CPU_FOREACH(cs) { + s390_topology_insert(S390_CPU(cs)); + } +} + +/** + * s390_free_tle: + * + * Loop over all TLE entries and free them. + * Keep the sentinelle which is the only one with level5 != 0 + */ +static void s390_free_tle(void) +{ + S390TopologyEntry *entry = NULL; + S390TopologyEntry *tmp = NULL; + + QTAILQ_FOREACH_SAFE(entry, &s390_topology.list, next, tmp) { + if (!entry->id.level5) { + QTAILQ_REMOVE(&s390_topology.list, entry, next); + g_free(entry); + } + } +} + +/** + * insert_stsi_15_1_x: + * cpu: the CPU doing the call for which we set CC + * sel2: the selector 2, containing the nested level + * addr: Guest logical address of the guest SysIB + * ar: the access register number + * + * Reserve a zeroed SysIB, let setup_stsi to fill it and + * copy the SysIB to the guest memory. + * + * In case of overflow set CC(3) and no copy is done. + */ +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) +{ + SysIB sysib = {0}; + int len; + + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) { + setcc(cpu, 3); + return; + } + + s390_order_tle(); + + len = setup_stsi(&sysib.sysib_151x, sel2); + + if (len < 0) { + setcc(cpu, 3); + return; + } + + sysib.sysib_151x.length = cpu_to_be16(len); + s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, len); + setcc(cpu, 0); + + s390_free_tle(); +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 3ac7ec9acf..5ea358cbb0 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu) if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) { return 0; } - /* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar); return 0; + case 15: + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); + return 0; default: return 0; } diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build index aef52b6686..5daa5c6033 100644 --- a/target/s390x/kvm/meson.build +++ b/target/s390x/kvm/meson.build @@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'cpu_topology.c' ), if_false: files( 'stubs.c' )) From patchwork Wed Feb 1 13:20:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07B05C05027 for ; Wed, 1 Feb 2023 13:21:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232147AbjBANVc (ORCPT ); Wed, 1 Feb 2023 08:21:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232115AbjBANVa (ORCPT ); Wed, 1 Feb 2023 08:21:30 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 349C163874 for ; Wed, 1 Feb 2023 05:21:27 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311DCHNb038656; Wed, 1 Feb 2023 13:21:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=73cIIRBMqeKPbw9ck1Kxwmq2wUO190y/qjLxyajLo90=; b=lQ8AwLB7W23Vx7SHOaeQxcwSL0n/2gw/Djq/2SyWMt9/YmTHmzlczuI3oi3Cyd68AlBp t4K4hIXLk7HcZoswIezU2cjLPCxgi/ORMxqZIyqcuP4ADWbiVtzg+jHFphtTjR47CeNR t54MTMc79ejQr6d8rzgz7Rtj0NZzfG8xmlLJkRldcNKOTn3PYKH2IqM0KPtcxuFNd96B Q4LcEVJ7w1O63jMDeta1VjjO16wBFstvo4bK9Vwdkd6Q2aP7f7ve9OCcvUjDqQDSStvD scTfwyEZOuGvexCnummAVqoxOrKtjj6hYTQr/7WU0twQlyQ9Ina5CaMuKxcNTI5xF0NP Rw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrrk07j7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:08 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311DCKS8038852; Wed, 1 Feb 2023 13:21:08 GMT Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrrk07gy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:08 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30VJIRYV017434; Wed, 1 Feb 2023 13:21:05 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma04fra.de.ibm.com (PPS) with ESMTPS id 3ndn6uas5e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:05 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL20N44695988 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:02 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0137820043; Wed, 1 Feb 2023 13:21:02 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 94B5120040; Wed, 1 Feb 2023 13:21:00 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:00 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 04/11] s390x/sclp: reporting the maximum nested topology entries Date: Wed, 1 Feb 2023 14:20:44 +0100 Message-Id: <20230201132051.126868-5-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: GoxGfhF5mxlcp3ecMRT3GIWXq2ncr2xV X-Proofpoint-GUID: 8l3oxecGDzP-t0iiGFj3Ut6w3fJGbj6a X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=889 lowpriorityscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 adultscore=0 malwarescore=0 clxscore=1015 bulkscore=0 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The maximum nested topology entries is used by the guest to know how many nested topology are available on the machine. Let change the MNEST value from 2 to 4 in the SCLP READ INFO structure now that we support books and drawers. Signed-off-by: Pierre Morel Reviewed-by: Nina Schoetterl-Glausch Reviewed-by: Thomas Huth --- include/hw/s390x/sclp.h | 5 +++-- hw/s390x/sclp.c | 5 +++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h index 712fd68123..902252b319 100644 --- a/include/hw/s390x/sclp.h +++ b/include/hw/s390x/sclp.h @@ -112,12 +112,13 @@ typedef struct CPUEntry { } QEMU_PACKED CPUEntry; #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 -#define SCLP_READ_SCP_INFO_MNEST 2 +#define SCLP_READ_SCP_INFO_MNEST 4 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax; uint8_t rnsize; - uint8_t _reserved1[16 - 11]; /* 11-15 */ + uint8_t _reserved1[15 - 11]; /* 11-14 */ + uint8_t stsi_parm; /* 15-15 */ uint16_t entries_cpu; /* 16-17 */ uint16_t offset_cpu; /* 18-19 */ uint8_t _reserved2[24 - 20]; /* 20-23 */ diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c index eff74479f4..d339cbb7e4 100644 --- a/hw/s390x/sclp.c +++ b/hw/s390x/sclp.c @@ -20,6 +20,7 @@ #include "hw/s390x/event-facility.h" #include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/ipl.h" +#include "hw/s390x/cpu-topology.h" static inline SCLPDevice *get_sclp_device(void) { @@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb) return; } + if (s390_has_topology()) { + read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST; + } + /* CPU information */ prepare_cpu_entries(machine, entries_start, &cpu_count); read_info->entries_cpu = cpu_to_be16(cpu_count); From patchwork Wed Feb 1 13:20:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31810C636D7 for ; Wed, 1 Feb 2023 13:21:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232157AbjBANVd (ORCPT ); Wed, 1 Feb 2023 08:21:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbjBANVb (ORCPT ); Wed, 1 Feb 2023 08:21:31 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34D743B0D8 for ; Wed, 1 Feb 2023 05:21:27 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311DCMed039294; Wed, 1 Feb 2023 13:21:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=GaUsV/D0kudZE0kv5u/leV1kXfbW5VjwFTN6wOslPus=; b=q/Hpf+anNZYYZLg4u4OAuWSDKEM3xhZjMe/8A4NhndnazjA2Rv7TgKZ7LrKFfhTQfaLe jvVfYf+ziXmVpKClMKgnHOj/4JaMU2sxXlMy34qffrAGCNj7eonDq+7MEoIWKKL/eV1p T+SReAXzlT9X1ScxzS3Xl1948GBz/LlJREyfA4rPw8qEV9x0CycN0ldgXSXFgxmrIaww lBfFVtRKnNwKSDi4aawxKg5OVSoS3BF6KerWC4f3/gARMTx28/gyw8jgX6zoQTVgM5li Sf9fdZk6s3ik3GD7AcUd1/fmoxyx8oKIkhKtvb04nRVnSSrWpm5UIS/hkcHrEkjOElT3 RA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrrk07jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:10 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311DDf6P004068; Wed, 1 Feb 2023 13:21:09 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrrk07hq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:09 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3116nccC014946; Wed, 1 Feb 2023 13:21:07 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3ncvttvvw6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:06 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL3Kg22217118 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:03 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6B2812004B; Wed, 1 Feb 2023 13:21:03 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1DF7620040; Wed, 1 Feb 2023 13:21:02 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:02 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 05/11] s390x/cpu topology: resetting the Topology-Change-Report Date: Wed, 1 Feb 2023 14:20:45 +0100 Message-Id: <20230201132051.126868-6-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 7d9Vji3HF8X3u1lm-23-02D5pgfGUt5B X-Proofpoint-GUID: 9JlYBxCZeK96iV49L6IS_zplt5cXYJO8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 lowpriorityscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 adultscore=0 malwarescore=0 clxscore=1015 bulkscore=0 phishscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org During a subsystem reset the Topology-Change-Report is cleared by the machine. Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset. Signed-off-by: Pierre Morel Reviewed-by: Thomas Huth --- include/hw/s390x/cpu-topology.h | 1 + target/s390x/cpu.h | 1 + target/s390x/kvm/kvm_s390x.h | 1 + hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 3 +++ target/s390x/cpu-sysemu.c | 13 +++++++++++++ target/s390x/kvm/kvm.c | 17 +++++++++++++++++ 7 files changed, 48 insertions(+) diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index 1ae7e7c5e3..60e0b9fbfa 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -66,5 +66,6 @@ static inline void s390_topology_set_cpu(MachineState *ms, extern S390Topology s390_topology; int s390_socket_nb(S390CPU *cpu); +void s390_topology_reset(void); #endif diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index e1f6925856..848314d2a9 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -641,6 +641,7 @@ typedef struct SysIBTl_cpu { QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16); void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar); +void s390_cpu_topology_reset(void); /* MMU defines */ #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h index f9785564d0..649dae5948 100644 --- a/target/s390x/kvm/kvm_s390x.h +++ b/target/s390x/kvm/kvm_s390x.h @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void); void kvm_s390_restart_interrupt(S390CPU *cpu); void kvm_s390_stop_interrupt(S390CPU *cpu); void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info); +int kvm_s390_topology_set_mtcr(uint64_t attr); #endif /* KVM_S390X_H */ diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index a80a1ebf22..cf63f3dd01 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -85,6 +85,18 @@ static void s390_topology_init(MachineState *ms) QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next); } +/** + * s390_topology_reset: + * + * Generic reset for CPU topology, calls s390_topology_reset() + * s390_topology_reset() to reset the kernel Modified Topology + * change record. + */ +void s390_topology_reset(void) +{ + s390_cpu_topology_reset(); +} + /** * s390_topology_cpu_default: * @cpu: pointer to a S390CPU diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index 9bc51a83f4..30fdfe41fa 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -122,6 +122,9 @@ static void subsystem_reset(void) device_cold_reset(dev); } } + if (s390_has_topology()) { + s390_topology_reset(); + } } static int virtio_ccw_hcall_notify(const uint64_t *args) diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index 948e4bd3e0..e27864c5f5 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg) kvm_s390_set_diag318(cs, arg.host_ulong); } } + +void s390_cpu_topology_reset(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(0); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 5ea358cbb0..bc953151ce 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void) return cap_zpci_op; } +int kvm_s390_topology_set_mtcr(uint64_t attr) +{ + struct kvm_device_attr attribute = { + .group = KVM_S390_VM_CPU_TOPOLOGY, + .attr = attr, + }; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + return 0; + } + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) { + return -ENOTSUP; + } + + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute); +} + void kvm_arch_accel_class_init(ObjectClass *oc) { } From patchwork Wed Feb 1 13:20:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF3D1C636CD for ; Wed, 1 Feb 2023 13:21:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232173AbjBANVj (ORCPT ); Wed, 1 Feb 2023 08:21:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232149AbjBANVc (ORCPT ); Wed, 1 Feb 2023 08:21:32 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 673F03EFEA for ; Wed, 1 Feb 2023 05:21:27 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311Ctj1V030275; Wed, 1 Feb 2023 13:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=0rQpBWnSCSg+hTCpttVtfiOSflMJCbv1cXL8T/yBGQQ=; b=ak53fYxXSNZLvnsr2TrNmICxht2b4muywYB6GMGhnY9fRlYEUKyZtOcnZ7r/EjWCk3Ez dzeAEdkjzq8A0xaxnjY1FgnfEN9qxBvCV5pdv+2zH+Qz0tQyzktXo/efZqSqjFmgaavE w48eL+D+uLeSOqqNPuj8HCqbJg2RuZ6RdPTRzxf1QnAEdefTmk1CnXM2wx9GLdIbp/Gf /lzyabhgolsnvxWpNTSHAZsoMV4v7xQekUMFgrFr3TJmwTpCsO2qGe0l9qF6t5C5b4Hk 9HdqdyhUq+tnFxLLZHrxZ8uhpENpcK1e3tc8l3wLVCfjxO1NqAjq7+5E6LTf8EVuqQkY IQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrh3grnr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:11 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311CujbN003850; Wed, 1 Feb 2023 13:21:11 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrh3grm9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:11 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3116ieXO014744; Wed, 1 Feb 2023 13:21:08 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3ncvttvvw8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:08 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL4lo44302624 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:05 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CF42D2004B; Wed, 1 Feb 2023 13:21:04 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 87C7220040; Wed, 1 Feb 2023 13:21:03 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:03 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 06/11] s390x/cpu topology: interception of PTF instruction Date: Wed, 1 Feb 2023 14:20:46 +0100 Message-Id: <20230201132051.126868-7-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: bxH3NQmoaBKQ-m6YCkbZHWIK7Zmnr86i X-Proofpoint-GUID: swzgAWBcr_vMJg3TubiYflSJI92SJi-n X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 bulkscore=0 phishscore=0 mlxscore=0 suspectscore=0 adultscore=0 mlxlogscore=999 clxscore=1015 malwarescore=0 lowpriorityscore=0 priorityscore=1501 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension. The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor. During RESET all CPU of the configuration are placed in horizontal polarity. Signed-off-by: Pierre Morel --- include/hw/s390x/s390-virtio-ccw.h | 6 ++ target/s390x/cpu.h | 1 + hw/s390x/cpu-topology.c | 103 +++++++++++++++++++++++++++++ target/s390x/cpu-sysemu.c | 14 ++++ target/s390x/kvm/kvm.c | 11 +++ 5 files changed, 135 insertions(+) diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h index 9bba21a916..c1d46e78af 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -30,6 +30,12 @@ struct S390CcwMachineState { uint8_t loadparm[8]; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra); + struct S390CcwMachineClass { /*< private >*/ MachineClass parent_class; diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 848314d2a9..f6e207afde 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -857,6 +857,7 @@ void s390_enable_css_support(S390CPU *cpu); void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); +void s390_cpu_topology_set_modified(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index cf63f3dd01..1028bf4476 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -85,16 +85,104 @@ static void s390_topology_init(MachineState *ms) QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next); } +/** + * s390_topology_set_cpus_polarity: + * @polarity: polarity requested by the caller + * + * Set all CPU entitlement according to polarity and + * dedication. + * Default vertical entitlement is POLARITY_VERTICAL_MEDIUM as + * it does not require host modification of the CPU provisioning + * until the host decide to modify individual CPU provisioning + * using QAPI interface. + * However a dedicated vCPU will have a POLARITY_VERTICAL_HIGH + * entitlement. + */ +static void s390_topology_set_cpus_polarity(int polarity) +{ + CPUState *cs; + + CPU_FOREACH(cs) { + if (polarity == POLARITY_HORIZONTAL) { + S390_CPU(cs)->env.entitlement = 0; + } else if (S390_CPU(cs)->env.dedicated) { + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_HIGH; + } else { + S390_CPU(cs)->env.entitlement = POLARITY_VERTICAL_MEDIUM; + } + } +} + +/* + * s390_handle_ptf: + * + * @register 1: contains the function code + * + * Function codes 0 (horizontal) and 1 (vertical) define the CPU + * polarization requested by the guest. + * + * Verify that the polarization really need to change and call + * s390_topology_set_cpus_polarity() specifying the requested polarity + * to set for all CPUs. + * + * Function code 2 is handling topology changes and is interpreted + * by the SIE. + */ +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) +{ + CPUS390XState *env = &cpu->env; + uint64_t reg = env->regs[r1]; + int fc = reg & S390_TOPO_FC_MASK; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + s390_program_interrupt(env, PGM_OPERATION, ra); + return; + } + + if (env->psw.mask & PSW_MASK_PSTATE) { + s390_program_interrupt(env, PGM_PRIVILEGED, ra); + return; + } + + if (reg & ~S390_TOPO_FC_MASK) { + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + return; + } + + switch (fc) { + case POLARITY_VERTICAL: + case POLARITY_HORIZONTAL: + if (s390_topology.polarity == fc) { + env->regs[r1] |= S390_PTF_REASON_DONE; + setcc(cpu, 2); + } else { + s390_topology.polarity = fc; + s390_cpu_topology_set_modified(); + s390_topology_set_cpus_polarity(fc); + setcc(cpu, 0); + } + break; + default: + /* Note that fc == 2 is interpreted by the SIE */ + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + } +} + /** * s390_topology_reset: * * Generic reset for CPU topology, calls s390_topology_reset() * s390_topology_reset() to reset the kernel Modified Topology * change record. + * Then set global and all CPUs polarity to POLARITY_HORIZONTAL. */ void s390_topology_reset(void) { s390_cpu_topology_reset(); + /* Set global polarity to POLARITY_HORIZONTAL */ + s390_topology.polarity = POLARITY_HORIZONTAL; + /* Set all CPU polarity to POLARITY_HORIZONTAL */ + s390_topology_set_cpus_polarity(POLARITY_HORIZONTAL); } /** @@ -137,6 +225,21 @@ static void s390_topology_cpu_default(S390CPU *cpu, Error **errp) (smp->books * smp->sockets * smp->cores)) % smp->drawers; } + + /* + * Machine polarity is set inside the global s390_topology structure. + * In the case the polarity is set as horizontal set the entitlement + * to POLARITY_VERTICAL_MEDIUM which is the better equivalent when + * machine polarity is set to vertical or POLARITY_VERTICAL_HIGH if + * the vCPU is dedicated. + */ + if (s390_topology.polarity && !env->entitlement) { + if (env->dedicated) { + env->entitlement = POLARITY_VERTICAL_HIGH; + } else { + env->entitlement = POLARITY_VERTICAL_MEDIUM; + } + } } /** diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index e27864c5f5..82e3f3891e 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -37,6 +37,7 @@ #include "sysemu/sysemu.h" #include "sysemu/tcg.h" #include "hw/core/sysemu-cpu-ops.h" +#include "hw/s390x/cpu-topology.h" /* S390CPUClass::load_normal() */ static void s390_cpu_load_normal(CPUState *s) @@ -319,3 +320,16 @@ void s390_cpu_topology_reset(void) } } } + +void s390_cpu_topology_set_modified(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(1); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index bc953151ce..fb63be41b7 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -96,6 +96,7 @@ #define PRIV_B9_EQBS 0x9c #define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2 #define PRIV_B9_PCISTG 0xd0 #define PRIV_B9_PCILG 0xd2 #define PRIV_B9_RPCIT 0xd3 @@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run) +{ + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f; + + s390_handle_ptf(cpu, r1, RA_IGNORED); +} + static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) { int r = 0; @@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) case PRIV_B9_RPCIT: r = kvm_rpcit_service_call(cpu, run); break; + case PRIV_B9_PTF: + kvm_handle_ptf(cpu, run); + break; case PRIV_B9_EQBS: /* just inject exception */ r = -1; From patchwork Wed Feb 1 13:20:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEBDEC636CD for ; Wed, 1 Feb 2023 13:21:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232164AbjBANVg (ORCPT ); Wed, 1 Feb 2023 08:21:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232148AbjBANVc (ORCPT ); Wed, 1 Feb 2023 08:21:32 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EBC1646B4 for ; Wed, 1 Feb 2023 05:21:30 -0800 (PST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311Cfc09013010; Wed, 1 Feb 2023 13:21:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Ygi3pyeH0tCI6Vskdugv41wNQenFvDnoVAqb8Eh5WnU=; b=EPocf6jNi0/P50sAiMmbj2vBnIXj6/jOkOMfVqj5hHGw5BH1Dp7gHqnS+ybWeUzCdrCH rNtjjtjredIj/b92b+hKdn1p1pIMTDCAxVHePra5ZYV7BVPACyAtaPBN3LAskYvy5fL/ PaNNSv8GSvf4V++WSxBynohjK2/MdsSKV/YM04ekvWbM6L3w6w2O2ZgZXA5jwIRSJY1E 3meJO0Yuoorol5nbaY9vxbOlrzXZx8ewiItmNIVyMfLule+Ay7IsJ12+SQkbtRGHwfc7 TuaxRT3HW9le0r05kCz5JA2i71geGt7/VzIjEILEdj4YfWfGskJgZ7jsjQuJ0dUrzb3H tw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfq452kv1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:12 +0000 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311D37E1000899; Wed, 1 Feb 2023 13:21:12 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfq452kty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:11 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3117QgLm026916; Wed, 1 Feb 2023 13:21:10 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3ncvs7mw7c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:09 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL6b446989760 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:06 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 483FB2004B; Wed, 1 Feb 2023 13:21:06 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA6B920040; Wed, 1 Feb 2023 13:21:04 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:04 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 07/11] target/s390x/cpu topology: activating CPU topology Date: Wed, 1 Feb 2023 14:20:47 +0100 Message-Id: <20230201132051.126868-8-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: RORa1BacgHFAiNdExTefbMUsbJAZEXQX X-Proofpoint-ORIG-GUID: 18z8n6z2x3dyATIFQ_GgKok13N9qwwai X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 clxscore=1015 phishscore=0 mlxlogscore=999 bulkscore=0 adultscore=0 impostorscore=0 mlxscore=0 priorityscore=1501 spamscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility in the host CPU model for the guest in the case the topology is available in QEMU and in KVM. The feature is disabled by default and fenced for SE (secure execution). Signed-off-by: Pierre Morel --- hw/s390x/cpu-topology.c | 2 +- target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 12 ++++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 1028bf4476..c33378577b 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -55,7 +55,7 @@ int s390_socket_nb(S390CPU *cpu) */ bool s390_has_topology(void) { - return false; + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY); } /** diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index 065ec6d66c..aca2c5c96b 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -254,6 +254,7 @@ bool s390_has_feat(S390Feat feat) case S390_FEAT_SIE_CMMA: case S390_FEAT_SIE_PFMFI: case S390_FEAT_SIE_IBS: + case S390_FEAT_CONFIGURATION_TOPOLOGY: return false; break; default: diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index fb63be41b7..808e35a7bd 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2470,6 +2470,18 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) set_bit(S390_FEAT_UNPACK, model->features); } + /* + * If we have kernel support for CPU Topology indicate the + * configuration-topology facility. + */ + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) { + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) { + error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY"); + return; + } + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features); + } + /* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit(S390_FEAT_ZPCI, model->features); set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features); From patchwork Wed Feb 1 13:20:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5D60C05027 for ; Wed, 1 Feb 2023 13:21:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232141AbjBANV1 (ORCPT ); Wed, 1 Feb 2023 08:21:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232078AbjBANVZ (ORCPT ); Wed, 1 Feb 2023 08:21:25 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B97563D0BD for ; Wed, 1 Feb 2023 05:21:20 -0800 (PST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311CHQx3014738; Wed, 1 Feb 2023 13:21:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Z31FhKDemu6fYZ6eIL5PKE2UxCO1+PQcrRdfmfgn4U8=; b=T6J53/Of2xIvwMFEat5t4hC27Yo4yp3oYY4ndDNFDPwD0XwRe21a6J2lWc3bniM9qay1 DqRkqtpW+OZy6o/GANfeNwh7ZxsBAcTfV1/wMba9O8UabpVfTy3ElEbKe/Qn+CjHtYJx 5eA2qxIRo/jA3AlXTtpU7icMtYmlJdrpaiKDm4ISwFXTZBONK7lIODBGC2dhaKPr6EDt lwrRrp3Dconk3LRWmBUSAK5HiI5PHI1ypJfvVgY8o77qIeT7q1WrFwBLyJontRulNdBZ 8qJg9eJ8+FAMyqr23dbO4J0AbWsalploB7bd8Ocjdp/trs1c42S6KUnUK5btxcpE9VXS eQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqy4hkft-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:14 +0000 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311DAnaN014011; Wed, 1 Feb 2023 13:21:13 GMT Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com [149.81.74.107]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfqy4hkf4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:13 +0000 Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1]) by ppma03fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30VM5qZR009107; Wed, 1 Feb 2023 13:21:11 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma03fra.de.ibm.com (PPS) with ESMTPS id 3ncvshbga5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:11 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL7EQ50266458 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:07 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BF26620043; Wed, 1 Feb 2023 13:21:07 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63CFA20040; Wed, 1 Feb 2023 13:21:06 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:06 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 08/11] qapi/s390x/cpu topology: x-set-cpu-topology monitor command Date: Wed, 1 Feb 2023 14:20:48 +0100 Message-Id: <20230201132051.126868-9-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: YJKhyzWMUnffALxAzbaz_gK9MFs-8ovw X-Proofpoint-GUID: 2Etup9bQOnZMsjDkWCESr5oypRblYULm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 clxscore=1015 mlxlogscore=999 spamscore=0 impostorscore=0 mlxscore=0 adultscore=0 priorityscore=1501 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The modification of the CPU attributes are done through a monitor command. It allows to move the core inside the topology tree to optimise the cache usage in the case the host's hypervisor previously moved the CPU. The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU. With this knowledge the guest has the possibility to optimize the usage of the vCPUs. The command is made experimental for the moment. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 29 +++++++++++++ include/monitor/hmp.h | 1 + hw/s390x/cpu-topology.c | 88 ++++++++++++++++++++++++++++++++++++++++ hmp-commands.hx | 16 ++++++++ 4 files changed, 134 insertions(+) diff --git a/qapi/machine-target.json b/qapi/machine-target.json index 2e267fa458..58df0f5061 100644 --- a/qapi/machine-target.json +++ b/qapi/machine-target.json @@ -342,3 +342,32 @@ 'TARGET_S390X', 'TARGET_MIPS', 'TARGET_LOONGARCH64' ] } } + +## +# @x-set-cpu-topology: +# +# @core: the vCPU ID to be moved +# @socket: the destination socket where to move the vCPU +# @book: the destination book where to move the vCPU +# @drawer: the destination drawer where to move the vCPU +# @polarity: optional polarity, default is last polarity set by the guest +# @dedicated: optional, if the vCPU is dedicated to a real CPU +# +# Modifies the topology by moving the CPU inside the topology +# tree or by changing a modifier attribute of a CPU. +# +# Returns: Nothing on success, the reason on failure. +# +# Since: +## +{ 'command': 'x-set-cpu-topology', + 'data': { + 'core': 'int', + 'socket': 'int', + 'book': 'int', + 'drawer': 'int', + '*polarity': 'int', + '*dedicated': 'bool' + }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h index 1b3bdcb446..12827479cf 100644 --- a/include/monitor/hmp.h +++ b/include/monitor/hmp.h @@ -151,5 +151,6 @@ void hmp_human_readable_text_helper(Monitor *mon, HumanReadableText *(*qmp_handler)(Error **)); void hmp_info_stats(Monitor *mon, const QDict *qdict); void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict); +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict); #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index c33378577b..6c50050991 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -18,6 +18,10 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "qapi/qapi-commands-machine-target.h" +#include "qapi/qmp/qdict.h" +#include "monitor/hmp.h" +#include "monitor/monitor.h" /* * s390_topology is used to keep the topology information. @@ -379,3 +383,87 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp) /* topology tree is reflected in props */ s390_update_cpu_props(ms, cpu); } + +/* + * qmp and hmp implementations + */ + +static void s390_change_topology(int64_t core_id, int64_t socket_id, + int64_t book_id, int64_t drawer_id, + int64_t polarity, bool dedicated, + Error **errp) +{ + MachineState *ms = current_machine; + S390CPU *cpu; + ERRP_GUARD(); + + cpu = (S390CPU *)ms->possible_cpus->cpus[core_id].cpu; + if (!cpu) { + error_setg(errp, "Core-id %ld does not exist!", core_id); + return; + } + + /* Verify the new topology */ + s390_topology_check(cpu, errp); + if (*errp) { + return; + } + + /* Move the CPU into its new socket */ + s390_set_core_in_socket(cpu, drawer_id, book_id, socket_id, true, errp); + + /* All checks done, report topology in environment */ + cpu->env.drawer_id = drawer_id; + cpu->env.book_id = book_id; + cpu->env.socket_id = socket_id; + cpu->env.dedicated = dedicated; + cpu->env.entitlement = polarity; + + /* topology tree is reflected in props */ + s390_update_cpu_props(ms, cpu); + + /* Advertise the topology change */ + s390_cpu_topology_set_modified(); +} + +void qmp_x_set_cpu_topology(int64_t core, int64_t socket, + int64_t book, int64_t drawer, + bool has_polarity, int64_t polarity, + bool has_dedicated, bool dedicated, + Error **errp) +{ + ERRP_GUARD(); + + if (!s390_has_topology()) { + error_setg(errp, "This machine doesn't support topology"); + return; + } + if (!has_polarity) { + polarity = POLARITY_VERTICAL_MEDIUM; + } + if (!has_dedicated) { + dedicated = false; + } + s390_change_topology(core, socket, book, drawer, polarity, dedicated, errp); +} + +void hmp_x_set_cpu_topology(Monitor *mon, const QDict *qdict) +{ + const int64_t core = qdict_get_int(qdict, "core"); + const int64_t socket = qdict_get_int(qdict, "socket"); + const int64_t book = qdict_get_int(qdict, "book"); + const int64_t drawer = qdict_get_int(qdict, "drawer"); + bool has_polarity = qdict_haskey(qdict, "polarity"); + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0); + bool has_dedicated = qdict_haskey(qdict, "dedicated"); + const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false); + Error *local_err = NULL; + + qmp_x_set_cpu_topology(core, socket, book, drawer, + has_polarity, polarity, + has_dedicated, dedicated, + &local_err); + if (hmp_handle_error(mon, local_err)) { + return; + } +} diff --git a/hmp-commands.hx b/hmp-commands.hx index 673e39a697..bb3c908356 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1815,3 +1815,19 @@ SRST Dump the FDT in dtb format to *filename*. ERST #endif + +#if defined(TARGET_S390X) && defined(CONFIG_KVM) + { + .name = "x-set-cpu-topology", + .args_type = "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?", + .params = "core socket book drawer [polarity] [dedicated]", + .help = "Move CPU 'core' to 'socket/book/drawer' " + "optionaly modifies polarity and dedication", + .cmd = hmp_x_set_cpu_topology, + }, + +SRST +``x-set-cpu-topology`` *core* *socket* *book* *drawer* *polarity* *dedicated* + Moves the CPU *core* to *socket* *book* *drawer* with *polarity* *dedicated*. +ERST +#endif From patchwork Wed Feb 1 13:20:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18244C636CD for ; Wed, 1 Feb 2023 13:21:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232068AbjBANV3 (ORCPT ); Wed, 1 Feb 2023 08:21:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231835AbjBANV0 (ORCPT ); Wed, 1 Feb 2023 08:21:26 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C10AE43475 for ; Wed, 1 Feb 2023 05:21:22 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311DDnm2032990; Wed, 1 Feb 2023 13:21:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=VAdg3XqGhSgAl6+Ag+TOciEbg4tAAoOJnuiiN7YQGMw=; b=OQw2w7XWnXNX8xB2inunGJDiKwaY/xZLnCWIjnZwlvbTj3EvHKwHqVO91EfYg1pRRU1c 4O1qDzvZNltqrEHxuPyZgWeuNmU/K+7Eg1oUKl8YmE7UOQl17gQPab+ZkEwp0o4q+gSj HQ5GxuOw/PrMgeNylx2VZPN/XR2anx9ELDsqiqxOJdrA+j0Q9CA9RW8MBTsomC9u8WYy 2NObx6JoLk35MqlH0nWAt73E/rct+njBos5E76BWZ9gLgFzD3oaaF1F5EZS1F+vU9+Pn 053PSKhP+r3VU8JXn+Ki9FELue9dnxKp/Pj8/OTV1PjVANc7rsR7vtd/U9WqtK7G5h0k LA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrscr68w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:15 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311DIln3013810; Wed, 1 Feb 2023 13:21:14 GMT Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrscr686-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:14 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 311A4ws4026038; Wed, 1 Feb 2023 13:21:12 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma04fra.de.ibm.com (PPS) with ESMTPS id 3ndn6uas5k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:12 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DL9aB43385282 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:09 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4037820043; Wed, 1 Feb 2023 13:21:09 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DDBC220040; Wed, 1 Feb 2023 13:21:07 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:07 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 09/11] machine: adding s390 topology to query-cpu-fast Date: Wed, 1 Feb 2023 14:20:49 +0100 Message-Id: <20230201132051.126868-10-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 7zvOXTXUplLM8fi2a2M0FEGBs613bN7i X-Proofpoint-GUID: Q6VWJMsvsAkRlK9Xtpg5KrwB4tPCJuDF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 clxscore=1015 malwarescore=0 bulkscore=0 mlxlogscore=999 phishscore=0 priorityscore=1501 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org S390x provides two more topology containers above the sockets, books and drawers. Let's add these CPU attributes to the QAPI command query-cpu-fast. Signed-off-by: Pierre Morel Reviewed-by: Nina Schoetterl-Glausch if you fix the issues in Thomas' first reply. --- qapi/machine.json | 13 ++++++++++--- hw/core/machine-qmp-cmds.c | 2 ++ 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/qapi/machine.json b/qapi/machine.json index 3036117059..e36c39e258 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -53,11 +53,18 @@ # # Additional information about a virtual S390 CPU # -# @cpu-state: the virtual CPU's state +# @cpu-state: the virtual CPU's state (since 2.12) +# @dedicated: the virtual CPU's dedication (since 8.0) +# @polarity: the virtual CPU's polarity (since 8.0) # # Since: 2.12 ## -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } } +{ 'struct': 'CpuInfoS390', + 'data': { 'cpu-state': 'CpuS390State', + 'dedicated': 'bool', + 'polarity': 'int' + } +} ## # @CpuInfoFast: @@ -70,7 +77,7 @@ # # @thread-id: ID of the underlying host thread # -# @props: properties describing to which node/socket/core/thread +# @props: properties describing to which node/drawer/book/socket/core/thread # virtual CPU belongs to, provided if supported by board # # @target: the QEMU system emulation target, which determines which diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c index 80d5e59651..e6d93cf2a0 100644 --- a/hw/core/machine-qmp-cmds.c +++ b/hw/core/machine-qmp-cmds.c @@ -30,6 +30,8 @@ static void cpustate_to_cpuinfo_s390(CpuInfoS390 *info, const CPUState *cpu) CPUS390XState *env = &s390_cpu->env; info->cpu_state = env->cpu_state; + info->dedicated = env->dedicated; + info->polarity = env->entitlement; #else abort(); #endif From patchwork Wed Feb 1 13:20:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D7DC05027 for ; Wed, 1 Feb 2023 13:21:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232078AbjBANVe (ORCPT ); Wed, 1 Feb 2023 08:21:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232138AbjBANVc (ORCPT ); Wed, 1 Feb 2023 08:21:32 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DAB53D0BD for ; Wed, 1 Feb 2023 05:21:29 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311Cvn54002633; Wed, 1 Feb 2023 13:21:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=eZhbegBou4I7xPYYPIaB/7qY1+19K7m+dR0rwf7gbu8=; b=F4u5rYzWW6SQXL71KmxbkJSl59WkZB6QxYbkVrwZ8J7ofx1NRmQAJjeWMsNv9+Z8afw/ SvXVbxSxL0MtO4LNnWKDgGOSpo0CvjiDyfhB6qZZTykUicUKxYKG3IFL+/3+fHN3DO3H Y0kmubVk6lC1qb+9IPCliHTSnrOsR1/G2RF5PnQwdqT7AycNtPSJefT8j4+0z+EoDEmB Nz2jBYJZzsW8h+RblkJx3s/PooIQyOpONSOAmz0ygbS5BpJcldzdAXtEmEaZMC1tBZIb RNe4T+zqegEC9w8KCEV8PWaiWuykAiEkccEKL4QG5lQmKP3E2IMaCfjeLSJJKR1OsWrF 6A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrj28qdy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:17 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311Cxhbx012610; Wed, 1 Feb 2023 13:21:17 GMT Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com [149.81.74.107]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrj28qct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:16 +0000 Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1]) by ppma03fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3116j6oW013346; Wed, 1 Feb 2023 13:21:14 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma03fra.de.ibm.com (PPS) with ESMTPS id 3ncvshbga8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:14 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DLAUm23069036 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:10 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AEF4F2004B; Wed, 1 Feb 2023 13:21:10 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5A11920040; Wed, 1 Feb 2023 13:21:09 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:09 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 10/11] qapi/s390x/cpu topology: CPU_POLARITY_CHANGE qapi event Date: Wed, 1 Feb 2023 14:20:50 +0100 Message-Id: <20230201132051.126868-11-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: jlGWZjg8vhTlHqryaZiFzl0pihTosTEJ X-Proofpoint-ORIG-GUID: aYSiZbwyy4nhh29NqVHmYL65zi_2Wsfl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 spamscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 adultscore=0 priorityscore=1501 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the guest asks to change the polarity this change is forwarded to the admin using QAPI. The admin is supposed to take according decisions concerning CPU provisioning. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 30 ++++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 2 ++ 2 files changed, 32 insertions(+) diff --git a/qapi/machine-target.json b/qapi/machine-target.json index 58df0f5061..5883c3b020 100644 --- a/qapi/machine-target.json +++ b/qapi/machine-target.json @@ -371,3 +371,33 @@ }, 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @CPU_POLARITY_CHANGE: +# +# Emitted when the guest asks to change the polarity. +# +# @polarity: polarity specified by the guest +# +# The guest can tell the host (via the PTF instruction) whether the +# CPUs should be provisioned using horizontal or vertical polarity. +# +# On horizontal polarity the host is expected to provision all vCPUs +# equally. +# On vertical polarity the host can provision each vCPU differently. +# The guest will get information on the details of the provisioning +# the next time it uses the STSI(15) instruction. +# +# Since: 8.0 +# +# Example: +# +# <- { "event": "CPU_POLARITY_CHANGE", +# "data": { "polarity": 0 }, +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } } +# +## +{ 'event': 'CPU_POLARITY_CHANGE', + 'data': { 'polarity': 'int' }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] } +} diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 6c50050991..2f8e1b60cf 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -19,6 +19,7 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "qapi/qapi-commands-machine-target.h" +#include "qapi/qapi-events-machine-target.h" #include "qapi/qmp/qdict.h" #include "monitor/hmp.h" #include "monitor/monitor.h" @@ -163,6 +164,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) s390_topology.polarity = fc; s390_cpu_topology_set_modified(); s390_topology_set_cpus_polarity(fc); + qapi_event_send_cpu_polarity_change(fc); setcc(cpu, 0); } break; From patchwork Wed Feb 1 13:20:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13124272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 316CCC636CD for ; Wed, 1 Feb 2023 13:21:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232191AbjBANVn (ORCPT ); Wed, 1 Feb 2023 08:21:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232179AbjBANVl (ORCPT ); Wed, 1 Feb 2023 08:21:41 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1330942DE1 for ; Wed, 1 Feb 2023 05:21:30 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 311DDrb0033158; Wed, 1 Feb 2023 13:21:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=SYHjq1/6eI/ESLEJN/HQ4//q0+l0VZZql8wdWFOy+eU=; b=W5xKsDyfS++UEbU+F9yxS1TVjdNo5+Cm5P6GGq1DDh9YtqHVIzv0/ILXMJrRier6x0OR mMfsuORvX5j2It86uSzZCG3wiw93qZk4RtBTJwjIIIQqarAZujm+YcA/ayFl98vjOnxl Gi1/UOC0K4WvGBK96hzdaIstRXqkpKIfg3xGT1sLsPoz/IC6VDxWmXFYn2aPvWrRDsyT WqkxJo1Nxn18FGXreSCBil2BysrFoCqkH7hWFrLoRsg81VbBCPpQpfnlwN1fNnLkyxr5 JsVYz7F1CCbkSBHsUmK+7BYsZBe/SAzynxyMVucBljnsLamcUSMNCh6bPB6K+KVbn1GQ jg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrscr6ad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:18 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 311DF78l037151; Wed, 1 Feb 2023 13:21:17 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3nfrscr69n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:17 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30VMi3Zg012460; Wed, 1 Feb 2023 13:21:15 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3ncvs7mw7f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 01 Feb 2023 13:21:15 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 311DLCTE23724618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 Feb 2023 13:21:12 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 274F22004D; Wed, 1 Feb 2023 13:21:12 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC61E20040; Wed, 1 Feb 2023 13:21:10 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.179.4.198]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 1 Feb 2023 13:21:10 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, nsg@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v15 11/11] docs/s390x/cpu topology: document s390x cpu topology Date: Wed, 1 Feb 2023 14:20:51 +0100 Message-Id: <20230201132051.126868-12-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230201132051.126868-1-pmorel@linux.ibm.com> References: <20230201132051.126868-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: fqd9kZOWhFp--yRQwvEqZT_-WCnu5H4k X-Proofpoint-GUID: C8p4QX4X4Ryzoml1y-Bijj1IKAldgY0q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1 definitions=2023-02-01_04,2023-01-31_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 clxscore=1015 malwarescore=0 bulkscore=0 mlxlogscore=999 phishscore=0 priorityscore=1501 mlxscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302010112 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add some basic examples for the definition of cpu topology in s390x. Signed-off-by: Pierre Morel --- docs/system/s390x/cpu-topology.rst | 294 +++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 + 2 files changed, 295 insertions(+) create mode 100644 docs/system/s390x/cpu-topology.rst diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst new file mode 100644 index 0000000000..e2190318c0 --- /dev/null +++ b/docs/system/s390x/cpu-topology.rst @@ -0,0 +1,294 @@ +CPU topology on s390x +===================== + +Since QEMU 8.0, CPU topology on s390x provides up to 4 levels of +topology containers: drawers, books, sockets and cores. + +The first three containers define a tree hierarchy, the last one +provides the placement of the CPUs inside the parent container and +3 CPU attributes: + +- CPU type +- polarity entitlement +- dedication + +Note also that since 7.2 threads are no longer supported in the topology +and the ``-smp`` command line argument accepts only ``threads=1``. + +Prerequisites +------------- + +To use CPU topology a Linux QEMU/KVM machine providing the CPU topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version +of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM +that QEMU is supporting CPU topology. + +Enabling CPU topology +--------------------- + +Currently, CPU topology is only enabled in the host model by default. + +Enabling CPU topology in a CPU model is done by setting the CPU flag +``ctop`` to ``on`` like in: + +.. code-block:: bash + + -cpu gen16b,ctop=on + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags. + +Default topology usage +---------------------- + +The CPU topology, can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments +without using any new attributes. +In this case, the topology will be calculated by simply adding +to the topology the cores based on the core-id starting with +core-0 at position 0 of socket-0, book-0, drawer-0 with default +modifier attributes: horizontal polarity and no dedication. + +In the following machine we define 8 sockets with 4 cores each. +Note that s390x QEMU machines do not implement multithreading. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ + -device host-s390x-cpu,core-id=14 \ + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 4 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will + be placed in socket 2, with core-id 9 + +Polarity and dedication +----------------------- + +Polarity can be of two types: horizontal or vertical. + +The horizontal polarization specifies that all guest's vCPUs get +almost the same amount of provisioning of real CPU by the host. + +The vertical polarization specifies that guest's vCPU can get +different real CPU provisions: + +- a vCPU with Vertical high entitlement specifies that this + vCPU gets 100% of the real CPU provisioning. + +- a vCPU with Vertical medium entitlement specifies that this + vCPU shares the real CPU with other vCPUs. + +- a vCPU with Vertical low entitlement specifies that this + vCPU only get real CPU provisioning when no other vCPU need it. + +In the case a vCPU with vertical high entitlement does not use +the real CPU, the unused "slack" can be dispatched to other vCPU +with medium or low entitlement. + +A subsystem reset puts all vCPU of the configuration into the +horizontal polarization. + +The admin specifies the dedicated bit when the vCPU is dedicated +to a single real CPU. + +As for the Linux admin, the dedicated bit is an indication on the +affinity of a vCPU for a real CPU while the entitlement indicates the +sharing or exclusivity of use. + +Defining the topology on command line +------------------------------------- + +The topology can be defined entirely during the CPU definition, +with the exception of CPU 0 which must be defined with the -smp +argument. + +For example, here we set the position of the cores 1,2,3 on +drawer 1, book 1, socket 2 and cores 0,9 and 14 on drawer 0, +book 0, socket 0 with all horizontal polarity and not dedicated. +The core 4, will be set on its default position on socket 1 +(since we have 4 core per socket) and we define it with dedication and +vertical high entitlement. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \ + \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \ + \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \ + \ + -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarity=3 \ + +QAPI interface for topology +--------------------------- + +Let's start QEMU with the following command: + +.. code-block:: bash + + sudo /usr/local/bin/qemu-system-s390x \ + -enable-kvm \ + -cpu z14,ctop=on \ + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \ + \ + -device z14-s390x-cpu,core-id=19,polarity=3 \ + -device z14-s390x-cpu,core-id=11,polarity=1 \ + -device z14-s390x-cpu,core-id=112,polarity=3 \ + ... + +and see the result when using of the QAPI interface. + +addons to query-cpus-fast ++++++++++++++++++++++++++ + +The command query-cpus-fast allows the admin to query the topology +tree and modifiers for all configured vCPU. + +.. code-block:: QMP + + -> { "execute": "query-cpus-fast" } + { + "return": [ + { + "dedicated": false, + "thread-id": 3631238, + "props": { + "core-id": 0, + "socket-id": 0, + "drawer-id": 0, + "book-id": 0 + }, + "cpu-state": "operating", + "qom-path": "/machine/unattached/device[0]", + "polarity": 2, + "cpu-index": 0, + "target": "s390x" + }, + { + "dedicated": false, + "thread-id": 3631248, + "props": { + "core-id": 19, + "socket-id": 9, + "drawer-id": 0, + "book-id": 2 + }, + "cpu-state": "operating", + "qom-path": "/machine/peripheral-anon/device[0]", + "polarity": 3, + "cpu-index": 19, + "target": "s390x" + }, + { + "dedicated": false, + "thread-id": 3631249, + "props": { + "core-id": 11, + "socket-id": 5, + "drawer-id": 0, + "book-id": 1 + }, + "cpu-state": "operating", + "qom-path": "/machine/peripheral-anon/device[1]", + "polarity": 1, + "cpu-index": 11, + "target": "s390x" + }, + { + "dedicated": true, + "thread-id": 3631250, + "props": { + "core-id": 112, + "socket-id": 56, + "drawer-id": 3, + "book-id": 14 + }, + "cpu-state": "operating", + "qom-path": "/machine/peripheral-anon/device[2]", + "polarity": 3, + "cpu-index": 112, + "target": "s390x" + } + ] + } + +x-set-cpu-topology +++++++++++++++++++ + +The command x-set-cpu-topology allows the admin to modify the topology +tree or the topology modifiers of a vCPU in the configuration. + +.. code-block:: QMP + + -> { "execute": "x-set-cpu-topology", + "arguments": { + "core": 11, + "socket": 0, + "book": 0, + "drawer": 0, + "polarity": 0, + "dedicated": false + } + } + <- {"return": {}} + + +event CPU_POLARITY_CHANGE ++++++++++++++++++++++++++ + +When a guest is requesting a modification of the polarity, +QEMU sends a CPU_POLARITY_CHANGE event. + +When requesting the change, the guest only specifies horizontal or +vertical polarity. +The dedication and fine grain vertical entitlement depends on admin +to set according to its response to this event. + +Note that a vertical polarized dedicated vCPU can only have a high +entitlement, this gives 6 possibilities for a vCPU polarity: + +- Horizontal +- Horizontal dedicated +- Vertical low +- Vertical medium +- Vertical high +- Vertical high dedicated + +Example of the event received when the guest issues the CPU instruction +Perform Topology Function PTF(0) to request an horizontal polarity: + +.. code-block:: QMP + + <- { "event": "CPU_POLARITY_CHANGE", + "data": { "polarity": 0 }, + "timestamp": { "seconds": 1401385907, "microseconds": 422329 } } + + diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst index c636f64113..ff0ffe04f3 100644 --- a/docs/system/target-s390x.rst +++ b/docs/system/target-s390x.rst @@ -33,3 +33,4 @@ Architectural features .. toctree:: s390x/bootdevices s390x/protvirt + s390x/cpu-topology