From patchwork Thu Jan 5 14:53:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6A4DC53210 for ; Thu, 5 Jan 2023 14:54:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234190AbjAEOx7 (ORCPT ); Thu, 5 Jan 2023 09:53:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234200AbjAEOxi (ORCPT ); Thu, 5 Jan 2023 09:53:38 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3DBF5B175 for ; Thu, 5 Jan 2023 06:53:35 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EpjjH022529; Thu, 5 Jan 2023 14:53:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=yM9j2uyN7Bpwo18nIstSzp/AMscgnmpHRKPEc1HY7XY=; b=sj5QunvCgQ1X0rJcbj+G4QN7iVSZxAf83ERDbNfsKP2ypDALPGKSYojFPMlHNZ77yJ3A /p0LKV8bpedp96ojrfQyQUJgez4IZLX2gXF7dssjBI6xl9Ow9z/I4/yPh5ywYRX40x3j 48Ski/sYLkNhnZ7A1G66BfF+8ia9+t5X7qkN97sYQ+3mPeXw0Tj0/j0cvw5factz95fz Gmz6HplnNKbQITr/NoIuPrXMduxCh+3IxSqZdXlbpQxg8PodxcuB5ZXmswdHwEV1YDDz ReIcIl0Bcqbr/yA/8c1IJUHP3MAstvyUXuw7WHLiVDKZp2MyMlEz8PYRLkl7TBhOwhuJ og== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0pfg136-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:24 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305Er4jY028552; Thu, 5 Jan 2023 14:53:23 GMT Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com [149.81.74.108]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0pfg12e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:23 +0000 Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1]) by ppma05fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3059IZQ3026929; Thu, 5 Jan 2023 14:53:21 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma05fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6n2yj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:21 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErHOf20447610 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:18 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D234C2004B; Thu, 5 Jan 2023 14:53:16 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CA2F320049; Thu, 5 Jan 2023 14:53:15 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:15 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 01/11] s390x/cpu topology: adding s390 specificities to CPU topology Date: Thu, 5 Jan 2023 15:53:03 +0100 Message-Id: <20230105145313.168489-2-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: -QljFR5Ep3TiHbzifM9d0O0eKlpxH-Gp X-Proofpoint-GUID: JglB97A7WW5t8aLdViL2vS-oKDfI2wro X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 phishscore=0 lowpriorityscore=0 suspectscore=0 malwarescore=0 mlxscore=0 bulkscore=0 spamscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org S390 adds two new SMP levels, drawers and books to the CPU topology. The S390 CPU have specific toplogy features like dedication and polarity to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs. Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and polarity, Signed-off-by: Pierre Morel --- qapi/machine.json | 14 ++++++++-- include/hw/boards.h | 10 ++++++- include/hw/s390x/cpu-topology.h | 23 ++++++++++++++++ target/s390x/cpu.h | 6 +++++ hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++----- hw/core/machine.c | 4 +++ hw/s390x/s390-virtio-ccw.c | 2 ++ softmmu/vl.c | 6 +++++ target/s390x/cpu.c | 10 +++++++ qemu-options.hx | 6 +++-- 10 files changed, 117 insertions(+), 12 deletions(-) create mode 100644 include/hw/s390x/cpu-topology.h diff --git a/qapi/machine.json b/qapi/machine.json index b9228a5e46..ff8f2b0e84 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -900,13 +900,15 @@ # a CPU is being hotplugged. # # @node-id: NUMA node ID the CPU belongs to -# @socket-id: socket number within node/board the CPU belongs to +# @drawer-id: drawer number within node/board the CPU belongs to +# @book-id: book number within drawer/node/board the CPU belongs to +# @socket-id: socket number within book/node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1) # @cluster-id: cluster number within die the CPU belongs to (since 7.1) # @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present +# Note: currently there are 8 properties that could be present # but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in @@ -916,6 +918,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int', @@ -1465,6 +1469,10 @@ # # @cpus: number of virtual CPUs in the virtual machine # +# @drawers: number of drawers in the CPU topology +# +# @books: number of books in the CPU topology +# # @sockets: number of sockets in the CPU topology # # @dies: number of dies per socket in the CPU topology @@ -1481,6 +1489,8 @@ ## { 'struct': 'SMPConfiguration', 'data': { '*cpus': 'int', + '*drawers': 'int', + '*books': 'int', '*sockets': 'int', '*dies': 'int', '*clusters': 'int', diff --git a/include/hw/boards.h b/include/hw/boards.h index d18d6d0073..239fc017b0 100644 --- a/include/hw/boards.h +++ b/include/hw/boards.h @@ -130,11 +130,15 @@ typedef struct { * @prefer_sockets - whether sockets are preferred over cores in smp parsing * @dies_supported - whether dies are supported by the machine * @clusters_supported - whether clusters are supported by the machine + * @books_supported - whether books are supported by the machine + * @drawers_supported - whether drawers are supported by the machine */ typedef struct { bool prefer_sockets; bool dies_supported; bool clusters_supported; + bool books_supported; + bool drawers_supported; } SMPCompatProps; /** @@ -299,7 +303,9 @@ typedef struct DeviceMemoryState { /** * CpuTopology: * @cpus: the number of present logical processors on the machine - * @sockets: the number of sockets on the machine + * @drawers: the number of drawers on the machine + * @books: the number of books in one drawer + * @sockets: the number of sockets in one book * @dies: the number of dies in one socket * @clusters: the number of clusters in one die * @cores: the number of cores in one cluster @@ -308,6 +314,8 @@ typedef struct DeviceMemoryState { */ typedef struct CpuTopology { unsigned int cpus; + unsigned int drawers; + unsigned int books; unsigned int sockets; unsigned int dies; unsigned int clusters; diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h new file mode 100644 index 0000000000..d945b57fc3 --- /dev/null +++ b/include/hw/s390x/cpu-topology.h @@ -0,0 +1,23 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H + +#define S390_TOPOLOGY_CPU_IFL 0x03 + +#define S390_TOPOLOGY_POLARITY_HORIZONTAL 0x00 +#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW 0x01 +#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02 +#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH 0x03 + +#define S390_TOPOLOGY_SHARED 0x00 +#define S390_TOPOLOGY_DEDICATED 0x01 + +#endif diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 7d6d01325b..39ea63a416 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -131,6 +131,12 @@ struct CPUArchState { #if !defined(CONFIG_USER_ONLY) uint32_t core_id; /* PoP "CPU address", same as cpu_index */ + int32_t socket_id; + int32_t book_id; + int32_t drawer_id; + int32_t dedicated; + int32_t polarity; + int32_t cpu_type; uint64_t cpuid; #endif diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c index b39ed21e65..edabfdb67e 100644 --- a/hw/core/machine-smp.c +++ b/hw/core/machine-smp.c @@ -31,6 +31,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms) MachineClass *mc = MACHINE_GET_CLASS(ms); GString *s = g_string_new(NULL); + if (mc->smp_props.drawers_supported) { + g_string_append_printf(s, " * drawers (%u)", ms->smp.drawers); + } + + if (mc->smp_props.books_supported) { + g_string_append_printf(s, " * books (%u)", ms->smp.books); + } + g_string_append_printf(s, "sockets (%u)", ms->smp.sockets); if (mc->smp_props.dies_supported) { @@ -73,6 +81,8 @@ void machine_parse_smp_config(MachineState *ms, { MachineClass *mc = MACHINE_GET_CLASS(ms); unsigned cpus = config->has_cpus ? config->cpus : 0; + unsigned drawers = config->has_drawers ? config->drawers : 0; + unsigned books = config->has_books ? config->books : 0; unsigned sockets = config->has_sockets ? config->sockets : 0; unsigned dies = config->has_dies ? config->dies : 0; unsigned clusters = config->has_clusters ? config->clusters : 0; @@ -85,6 +95,8 @@ void machine_parse_smp_config(MachineState *ms, * explicit configuration like "cpus=0" is not allowed. */ if ((config->has_cpus && config->cpus == 0) || + (config->has_drawers && config->drawers == 0) || + (config->has_books && config->books == 0) || (config->has_sockets && config->sockets == 0) || (config->has_dies && config->dies == 0) || (config->has_clusters && config->clusters == 0) || @@ -111,6 +123,19 @@ void machine_parse_smp_config(MachineState *ms, dies = dies > 0 ? dies : 1; clusters = clusters > 0 ? clusters : 1; + if (!mc->smp_props.books_supported && books > 1) { + error_setg(errp, "books not supported by this machine's CPU topology"); + return; + } + books = books > 0 ? books : 1; + + if (!mc->smp_props.drawers_supported && drawers > 1) { + error_setg(errp, + "drawers not supported by this machine's CPU topology"); + return; + } + drawers = drawers > 0 ? drawers : 1; + /* compute missing values based on the provided ones */ if (cpus == 0 && maxcpus == 0) { sockets = sockets > 0 ? sockets : 1; @@ -124,33 +149,41 @@ void machine_parse_smp_config(MachineState *ms, if (sockets == 0) { cores = cores > 0 ? cores : 1; threads = threads > 0 ? threads : 1; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); } else if (cores == 0) { threads = threads > 0 ? threads : 1; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); } } else { /* prefer cores over sockets since 6.2 */ if (cores == 0) { sockets = sockets > 0 ? sockets : 1; threads = threads > 0 ? threads : 1; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); } else if (sockets == 0) { threads = threads > 0 ? threads : 1; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); } } /* try to calculate omitted threads at last */ if (threads == 0) { - threads = maxcpus / (sockets * dies * clusters * cores); + threads = maxcpus / + (drawers * books * sockets * dies * clusters * cores); } } - maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads; + maxcpus = maxcpus > 0 ? maxcpus : drawers * books * sockets * dies * + clusters * cores * threads; cpus = cpus > 0 ? cpus : maxcpus; ms->smp.cpus = cpus; + ms->smp.drawers = drawers; + ms->smp.books = books; ms->smp.sockets = sockets; ms->smp.dies = dies; ms->smp.clusters = clusters; @@ -159,7 +192,8 @@ void machine_parse_smp_config(MachineState *ms, ms->smp.max_cpus = maxcpus; /* sanity-check of the computed topology */ - if (sockets * dies * clusters * cores * threads != maxcpus) { + if (drawers * books * sockets * dies * clusters * cores * threads != + maxcpus) { g_autofree char *topo_msg = cpu_hierarchy_to_string(ms); error_setg(errp, "Invalid CPU topology: " "product of the hierarchy must match maxcpus: " diff --git a/hw/core/machine.c b/hw/core/machine.c index f589b92909..3c781fed07 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -827,6 +827,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name, MachineState *ms = MACHINE(obj); SMPConfiguration *config = &(SMPConfiguration){ .has_cpus = true, .cpus = ms->smp.cpus, + .has_drawers = true, .drawers = ms->smp.drawers, + .has_books = true, .books = ms->smp.books, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters, @@ -1092,6 +1094,8 @@ static void machine_initfn(Object *obj) /* default to mc->default_cpus */ ms->smp.cpus = mc->default_cpus; ms->smp.max_cpus = mc->default_cpus; + ms->smp.drawers = 1; + ms->smp.books = 1; ms->smp.sockets = 1; ms->smp.dies = 1; ms->smp.clusters = 1; diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index f22f61b8b6..f3cc845d3b 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -733,6 +733,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) mc->no_sdcard = 1; mc->max_cpus = S390_MAX_CPUS; mc->has_hotpluggable_cpus = true; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true; assert(!mc->get_hotplug_handler); mc->get_hotplug_handler = s390_get_hotplug_handler; mc->cpu_index_to_instance_props = s390_cpu_index_to_props; diff --git a/softmmu/vl.c b/softmmu/vl.c index 798e1dc933..e894ab32b0 100644 --- a/softmmu/vl.c +++ b/softmmu/vl.c @@ -724,6 +724,12 @@ static QemuOptsList qemu_smp_opts = { { .name = "cpus", .type = QEMU_OPT_NUMBER, + }, { + .name = "drawers", + .type = QEMU_OPT_NUMBER, + }, { + .name = "books", + .type = QEMU_OPT_NUMBER, }, { .name = "sockets", .type = QEMU_OPT_NUMBER, diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c index 96562c516d..c9f980dbb5 100644 --- a/target/s390x/cpu.c +++ b/target/s390x/cpu.c @@ -35,6 +35,7 @@ #include "fpu/softfloat-helpers.h" #include "disas/capstone.h" #include "sysemu/tcg.h" +#include "hw/s390x/cpu-topology.h" #define CR0_RESET 0xE0UL #define CR14_RESET 0xC2000000UL; @@ -257,6 +258,15 @@ static gchar *s390_gdb_arch_name(CPUState *cs) static Property s390x_cpu_properties[] = { #if !defined(CONFIG_USER_ONLY) DEFINE_PROP_UINT32("core-id", S390CPU, env.core_id, 0), + DEFINE_PROP_INT32("socket-id", S390CPU, env.socket_id, -1), + DEFINE_PROP_INT32("book-id", S390CPU, env.book_id, -1), + DEFINE_PROP_INT32("drawer-id", S390CPU, env.drawer_id, -1), + DEFINE_PROP_INT32("dedicated", S390CPU, env.dedicated, + S390_TOPOLOGY_SHARED), + DEFINE_PROP_INT32("polarity", S390CPU, env.polarity, + S390_TOPOLOGY_POLARITY_HORIZONTAL), + DEFINE_PROP_INT32("cpu-type", S390CPU, env.cpu_type, + S390_TOPOLOGY_CPU_IFL), #endif DEFINE_PROP_END_OF_LIST() }; diff --git a/qemu-options.hx b/qemu-options.hx index 7f99d15b23..8dc9a4c052 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -250,11 +250,13 @@ SRST ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, - "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n" - " sockets= number of sockets on the machine board\n" + " drawers= number of drawers on the machine board\n" + " books= number of books in one drawer\n" + " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" " cores= number of cores in one cluster\n" From patchwork Thu Jan 5 14:53:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBC63C54E76 for ; Thu, 5 Jan 2023 14:54:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234252AbjAEOyD (ORCPT ); Thu, 5 Jan 2023 09:54:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232923AbjAEOxh (ORCPT ); Thu, 5 Jan 2023 09:53:37 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30B1F5AC57 for ; Thu, 5 Jan 2023 06:53:33 -0800 (PST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305Dsupi008123; Thu, 5 Jan 2023 14:53:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=tuIPbZSeNUotYslDIY1M1kwHksA4aozwFbUiiQWbRh8=; b=ZNfuZXwqhZE7qyvTNZKKHHecg6rZcd1BHY6P+ZUipg9bFaBpKA3s59NO4r+jcUbhGIN3 BPM6dIUTxQtBj0Ozc0iG6w3P2/epc5tmexVJDYXCWv5smaE3gHlWNfoytRUPktZ9dsnL TuXj6jhLWYC9PPErW7lTRq83Nthuq6SyGbQT6APEQU0ig8fTXKY9H5QVUuOFI2Y3Meuu 5yWdEsaba9HWS+OgO02vLLdybS/xJ4/8bNXleX8Uo1U2SaJDDAshxATwO4DYg6HFsMPz 971+g/yurXVRW0vmO6DVdPt7ef55P38lSaFX2H3ySbtkV6x+xs5mvqoBG4tqkjCfC2Ru vw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mwyuq9gxd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:24 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305Ekb9A030966; Thu, 5 Jan 2023 14:53:23 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mwyuq9gwp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:23 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3054vfuD020172; Thu, 5 Jan 2023 14:53:21 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3mtcq6ewa1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:21 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErHwU48759176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:18 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D524520049; Thu, 5 Jan 2023 14:53:17 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E71BE20040; Thu, 5 Jan 2023 14:53:16 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:16 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 02/11] s390x/cpu topology: add topology entries on CPU hotplug Date: Thu, 5 Jan 2023 15:53:04 +0100 Message-Id: <20230105145313.168489-3-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: DV6_PXItAycBkssK5t_mWhdSIadoEnCZ X-Proofpoint-GUID: Wnt4p6K5KUBdI6HqFstKSVO_dIuYGIa- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 lowpriorityscore=0 priorityscore=1501 impostorscore=0 bulkscore=0 clxscore=1015 mlxscore=0 malwarescore=0 phishscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The topology information are attributes of the CPU and are specified during the CPU device creation. On hot plug, we gather the topology information on the core, creates a list of topology entries, each entry contains a single core mask of each core with identical topology and finaly we orders the list in topological order. The topological order is, from higher to lower priority: - physical topology - drawer - book - socket - core origin, offset in 64bit increment from core 0. - modifier attributes - CPU type - polarization entitlement - dedication The possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer. If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes. This way, starting QEMU without specifying the topology can still get some adventage of the CPU topology. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 48 ++++++ hw/s390x/cpu-topology.c | 293 ++++++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 10 ++ hw/s390x/meson.build | 1 + 4 files changed, 352 insertions(+) create mode 100644 hw/s390x/cpu-topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index d945b57fc3..b3fd752d8d 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -10,7 +10,11 @@ #ifndef HW_S390X_CPU_TOPOLOGY_H #define HW_S390X_CPU_TOPOLOGY_H +#include "qemu/queue.h" +#include "hw/boards.h" + #define S390_TOPOLOGY_CPU_IFL 0x03 +#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64) #define S390_TOPOLOGY_POLARITY_HORIZONTAL 0x00 #define S390_TOPOLOGY_POLARITY_VERTICAL_LOW 0x01 @@ -20,4 +24,48 @@ #define S390_TOPOLOGY_SHARED 0x00 #define S390_TOPOLOGY_DEDICATED 0x01 +typedef union s390_topology_id { + uint64_t id; + struct { + uint64_t level_6:8; /* byte 0 BE */ + uint64_t level_5:8; /* byte 1 BE */ + uint64_t drawer:8; /* byte 2 BE */ + uint64_t book:8; /* byte 3 BE */ + uint64_t socket:8; /* byte 4 BE */ + uint64_t rsrv:5; + uint64_t d:1; + uint64_t p:2; /* byte 5 BE */ + uint64_t type:8; /* byte 6 BE */ + uint64_t origin:2; + uint64_t core:6; /* byte 7 BE */ + }; +} s390_topology_id; +#define TOPO_CPU_MASK 0x000000000000003fUL + +typedef struct S390TopologyEntry { + s390_topology_id id; + QTAILQ_ENTRY(S390TopologyEntry) next; + uint64_t mask; +} S390TopologyEntry; + +typedef struct S390Topology { + QTAILQ_HEAD(, S390TopologyEntry) list; + uint8_t *sockets; + CpuTopology *smp; +} S390Topology; + +#ifdef CONFIG_KVM +bool s390_has_topology(void); +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp); +#else +static inline bool s390_has_topology(void) +{ + return false; +} +static inline void s390_topology_set_cpu(MachineState *ms, + S390CPU *cpu, + Error **errp) {} +#endif +extern S390Topology s390_topology; + #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c new file mode 100644 index 0000000000..438055c612 --- /dev/null +++ b/hw/s390x/cpu-topology.c @@ -0,0 +1,293 @@ +/* + * CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/* + * s390_topology is used to keep the topology information. + * .list: queue the topology entries inside which + * we keep the information on the CPU topology. + * + * .smp: keeps track of the machine topology. + * + * .socket: tracks information on the count of cores per socket. + * + */ +S390Topology s390_topology = { + .list = QTAILQ_HEAD_INITIALIZER(s390_topology.list), + .sockets = NULL, /* will be initialized after the cpu model is realized */ +}; + +/** + * s390_socket_nb: + * @id: s390_topology_id + * + * Returns the socket number used inside the socket array. + */ +static int s390_socket_nb(s390_topology_id id) +{ + return (id.socket + 1) * (id.book + 1) * (id.drawer + 1); +} + +/** + * s390_has_topology: + * + * Return value: if the topology is supported by the machine. + */ +bool s390_has_topology(void) +{ + return false; +} + +/** + * s390_topology_init: + * @ms: the machine state where the machine topology is defined + * + * Keep track of the machine topology. + * Allocate an array to keep the count of cores per socket. + * The index of the array starts at socket 0 from book 0 and + * drawer 0 up to the maximum allowed by the machine topology. + */ +static void s390_topology_init(MachineState *ms) +{ + CpuTopology *smp = &ms->smp; + + s390_topology.smp = smp; + if (!s390_topology.sockets) { + s390_topology.sockets = g_new0(uint8_t, smp->sockets * + smp->books * smp->drawers); + } +} + +/** + * s390_topology_from_cpu: + * @cpu: The S390CPU + * + * Initialize the topology id from the CPU environment. + */ +static s390_topology_id s390_topology_from_cpu(S390CPU *cpu) +{ + s390_topology_id topology_id; + + topology_id.core = cpu->env.core_id; + topology_id.type = cpu->env.cpu_type; + topology_id.p = cpu->env.polarity; + topology_id.d = cpu->env.dedicated; + topology_id.socket = cpu->env.socket_id; + topology_id.book = cpu->env.book_id; + topology_id.drawer = cpu->env.drawer_id; + + return topology_id; +} + +/** + * s390_topology_set_entry: + * @entry: Topology entry to setup + * @id: topology id to use for the setup + * + * Set the core bit inside the topology mask and + * increments the number of cores for the socket. + */ +static void s390_topology_set_entry(S390TopologyEntry *entry, + s390_topology_id id) +{ + set_bit(63 - id.core, &entry->mask); + s390_topology.sockets[s390_socket_nb(id)]++; +} + +/** + * s390_topology_new_entry: + * @id: s390_topology_id to add + * + * Allocate a new entry and initialize it. + * + * returns the newly allocated entry. + */ +static S390TopologyEntry *s390_topology_new_entry(s390_topology_id id) +{ + S390TopologyEntry *entry; + + entry = g_malloc0(sizeof(S390TopologyEntry)); + entry->id.id = id.id & ~TOPO_CPU_MASK; + s390_topology_set_entry(entry, id); + + return entry; +} + +/** + * s390_topology_insert: + * + * @id: s390_topology_id to insert. + * + * Parse the topology list to find if the entry already + * exist and add the core in it. + * If it does not exist, allocate a new entry and insert + * it in the queue from lower id to greater id. + */ +static void s390_topology_insert(s390_topology_id id) +{ + S390TopologyEntry *entry; + S390TopologyEntry *tmp = NULL; + uint64_t new_id; + + new_id = id.id & ~TOPO_CPU_MASK; + + /* First CPU to add to an entry */ + if (QTAILQ_EMPTY(&s390_topology.list)) { + entry = s390_topology_new_entry(id); + QTAILQ_INSERT_HEAD(&s390_topology.list, entry, next); + return; + } + + QTAILQ_FOREACH(tmp, &s390_topology.list, next) { + if (new_id == tmp->id.id) { + s390_topology_set_entry(tmp, id); + return; + } else if (new_id < tmp->id.id) { + entry = s390_topology_new_entry(id); + QTAILQ_INSERT_BEFORE(tmp, entry, next); + return; + } + } + + entry = s390_topology_new_entry(id); + QTAILQ_INSERT_TAIL(&s390_topology.list, entry, next); +} + +/** + * s390_topology_check: + * @errp: Error pointer + * id: s390_topology_id to be verified + * + * The function checks if the topology id fits inside the + * system topology. + */ +static void s390_topology_check(Error **errp, s390_topology_id id) +{ + CpuTopology *smp = s390_topology.smp; + + if (id.socket > smp->sockets) { + error_setg(errp, "Unavailable socket: %d", id.socket); + return; + } + if (id.book > smp->books) { + error_setg(errp, "Unavailable book: %d", id.book); + return; + } + if (id.drawer > smp->drawers) { + error_setg(errp, "Unavailable drawer: %d", id.drawer); + return; + } + if (id.type != S390_TOPOLOGY_CPU_IFL) { + error_setg(errp, "Unknown cpu type: %d", id.type); + return; + } + /* Polarity and dedication can never be wrong */ +} + +/** + * s390_topology_cpu_default: + * @errp: Error pointer + * @cpu: pointer to a S390CPU + * + * Setup the default topology for unset attributes. + * + * The function accept only all all default values or all set values + * for the geometry topology. + * + * The function calculates the (drawer_id, book_id, socket_id) + * topology by filling the cores starting from the first socket + * (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets). + * + */ +static void s390_topology_cpu_default(Error **errp, S390CPU *cpu) +{ + CpuTopology *smp = s390_topology.smp; + CPUS390XState *env = &cpu->env; + + /* All geometry topology attributes must be set or all unset */ + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id < 0) && + (env->socket_id >= 0 || env->book_id >= 0 || env->drawer_id >= 0)) { + error_setg(errp, + "Please define all or none of the topology geometry attributes"); + return; + } + + /* Check if one of the geometry topology is unset */ + if (env->socket_id < 0) { + /* Calculate default geometry topology attributes */ + env->socket_id = (env->core_id / smp->cores) % smp->sockets; + env->book_id = (env->core_id / (smp->sockets * smp->cores)) % + smp->books; + env->drawer_id = (env->core_id / + (smp->books * smp->sockets * smp->cores)) % + smp->drawers; + } +} + +/** + * s390_topology_set_cpu: + * @ms: MachineState used to initialize the topology structure on + * first call. + * @cpu: the new S390CPU to insert in the topology structure + * @errp: the error pointer + * + * Called from CPU Hotplug to check and setup the CPU attributes + * before to insert the CPU in the topology. + */ +void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp) +{ + Error *local_error = NULL; + s390_topology_id id; + + /* + * We do not want to initialize the topology if the cpu model + * does not support topology consequently, we have to wait for + * the first CPU to be realized, which realizes the CPU model + * to initialize the topology structures. + * + * s390_topology_set_cpu() is called from the cpu hotplug. + */ + if (!s390_topology.sockets) { + s390_topology_init(ms); + } + + s390_topology_cpu_default(&local_error, cpu); + if (local_error) { + error_propagate(errp, local_error); + return; + } + + id = s390_topology_from_cpu(cpu); + + /* Check for space on the socket */ + if (s390_topology.sockets[s390_socket_nb(id)] >= + s390_topology.smp->sockets) { + error_setg(&local_error, "No more space on socket"); + return; + } + + s390_topology_check(&local_error, id); + if (local_error) { + error_propagate(errp, local_error); + return; + } + + s390_topology_insert(id); +} diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index f3cc845d3b..c98b93a15f 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -44,6 +44,7 @@ #include "hw/s390x/pv.h" #include "migration/blocker.h" #include "qapi/visitor.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker; @@ -310,10 +311,19 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev, { MachineState *ms = MACHINE(hotplug_dev); S390CPU *cpu = S390_CPU(dev); + Error *local_err = NULL; g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev); + if (s390_has_topology()) { + s390_topology_set_cpu(ms, cpu, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } + if (dev->hotplugged) { raise_irq_cpu_hotplug(); } diff --git a/hw/s390x/meson.build b/hw/s390x/meson.build index f291016fee..58dfbdff4f 100644 --- a/hw/s390x/meson.build +++ b/hw/s390x/meson.build @@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( 's390-stattrib-kvm.c', 'pv.c', 's390-pci-kvm.c', + 'cpu-topology.c', )) s390x_ss.add(when: 'CONFIG_TCG', if_true: files( 'tod-tcg.c', From patchwork Thu Jan 5 14:53:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 816D5C3DA7D for ; Thu, 5 Jan 2023 14:53:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234234AbjAEOxj (ORCPT ); Thu, 5 Jan 2023 09:53:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234421AbjAEOxg (ORCPT ); Thu, 5 Jan 2023 09:53:36 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98CED5AC63 for ; Thu, 5 Jan 2023 06:53:32 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EBWdP028051; Thu, 5 Jan 2023 14:53:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=wr11b4bT864+Aocj/J0rZ2emjh5kRsQlacctMQpZ7gk=; b=ZIUaJ0rRFwNewMbApH81OUScY9TklkaoWNRNmjrRN9Lht/rr51P0bNULD2kg/bQBlypA gKvDIBOP2z5gV3Wya5Fn44BLXobv3QJgCto3akxCPlKwOgT3B29N5ZjxvWvPUSmDPdhA VFpZBGYggC1DHMXP9Pf3R44Ojx6iOMa5fkLAkV6ze3ROSyQdFdw9bsgc/lnv5cap8fiP KthFEkKHE6L9bhxP8Ew8qdelEP6rKO9taCgtlCT0pUKLd/qtZl+D4qEXUgqqIFMXkMqo WtdovR5M2/x3ayks3ChsWzA6LtnyvvoFNe35RiCMfKOQApEwvh0eDIqjMQ6KulFFA6PB Og== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1k9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:25 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EDCle000923; Thu, 5 Jan 2023 14:53:25 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1jd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:25 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 304NvbRu000998; Thu, 5 Jan 2023 14:53:22 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma06fra.de.ibm.com (PPS) with ESMTPS id 3mtcbfn3bp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:22 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErJed30933248 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:19 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EB0C720043; Thu, 5 Jan 2023 14:53:18 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F2B2420040; Thu, 5 Jan 2023 14:53:17 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:17 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 03/11] target/s390x/cpu topology: handle STSI(15) and build the SYSIB Date: Thu, 5 Jan 2023 15:53:05 +0100 Message-Id: <20230105145313.168489-4-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: kyCSphRxz0LpBe9cIgPgNyDroZfkSnkS X-Proofpoint-GUID: aMD1ISyO6SYnc8UUi1shdIS7a-flQb6a X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 clxscore=1015 adultscore=0 priorityscore=1501 malwarescore=0 phishscore=0 spamscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 3 + include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 78 ++++++++++++++++++ target/s390x/kvm/cpu_topology.c | 136 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/meson.build | 3 +- 6 files changed, 224 insertions(+), 2 deletions(-) create mode 100644 target/s390x/kvm/cpu_topology.c diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index b3fd752d8d..9571aa70e5 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -41,6 +41,9 @@ typedef union s390_topology_id { }; } s390_topology_id; #define TOPO_CPU_MASK 0x000000000000003fUL +#define TOPO_SOCKET_MASK 0x0000ffffff000000UL +#define TOPO_BOOK_MASK 0x0000ffff00000000UL +#define TOPO_DRAWER_MASK 0x0000ff0000000000UL typedef struct S390TopologyEntry { s390_topology_id id; diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h index d3ade40a5a..712fd68123 100644 --- a/include/hw/s390x/sclp.h +++ b/include/hw/s390x/sclp.h @@ -112,6 +112,7 @@ typedef struct CPUEntry { } QEMU_PACKED CPUEntry; #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 +#define SCLP_READ_SCP_INFO_MNEST 2 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax; diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 39ea63a416..78988048dd 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -561,6 +561,25 @@ typedef struct SysIB_322 { } SysIB_322; QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096); +#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[]; +} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x; +QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16); + typedef union SysIB { SysIB_111 sysib_111; SysIB_121 sysib_121; @@ -568,9 +587,68 @@ typedef union SysIB { SysIB_221 sysib_221; SysIB_222 sysib_222; SysIB_322 sysib_322; + SysIB_151x sysib_151x; } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096); +/* + * CPU Topology List provided by STSI with fc=15 provides a list + * of two different Topology List Entries (TLE) types to specify + * the topology hierarchy. + * + * - Container Topology List Entry + * Defines a container to contain other Topology List Entries + * of any type, nested containers or CPU. + * - CPU Topology List Entry + * Specifies the CPUs position, type, entitlement and polarization + * of the CPUs contained in the last Container TLE. + * + * There can be theoretically up to five levels of containers, QEMU + * uses only one level, the socket level. + * + * A container of with a nesting level (NL) greater than 1 can only + * contain another container of nesting level NL-1. + * + * A container of nesting level 1 (socket), contains as many CPU TLE + * as needed to describe the position and qualities of all CPUs inside + * the container. + * The qualities of a CPU are polarization, entitlement and type. + * + * The CPU TLE defines the position of the CPUs of identical qualities + * using a 64bits mask which first bit has its offset defined by + * the CPU address orgin field of the CPU TLE like in: + * CPU address = origin * 64 + bit position within the mask + * + */ +/* Container type Topology List Entry */ +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8); + +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; + uint8_t reserved1:5; + uint8_t dedicated:1; + uint8_t polarity:2; + uint8_t type; + uint16_t origin; + uint64_t mask; +} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu; +QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16); + +/* Max size of a SYSIB structure is when all CPU are alone in a container */ +#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x) + \ + S390_MAX_CPUS * (sizeof(SysIBTl_container) + \ + sizeof(SysIBTl_cpu))) + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar); + /* MMU defines */ #define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ #define ASCE_SUBSPACE 0x200 /* subspace group control */ diff --git a/target/s390x/kvm/cpu_topology.c b/target/s390x/kvm/cpu_topology.c new file mode 100644 index 0000000000..3831a3264c --- /dev/null +++ b/target/s390x/kvm/cpu_topology.c @@ -0,0 +1,136 @@ +/* + * QEMU S390x CPU Topology + * + * Copyright IBM Corp. 2022 + * Author(s): Pierre Morel + * + * This work is licensed under the terms of the GNU GPL, version 2 or (at + * your option) any later version. See the COPYING file in the top-level + * directory. + */ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/sclp.h" +#include "hw/s390x/cpu-topology.h" + +static char *fill_container(char *p, int level, int id) +{ + SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; + return p + sizeof(*tle); +} + +static char *fill_tle_cpu(char *p, S390TopologyEntry *entry) +{ + SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + s390_topology_id topology_id = entry->id; + + tle->nl = 0; + tle->dedicated = topology_id.d; + tle->polarity = topology_id.p; + tle->type = topology_id.type; + tle->origin = topology_id.origin; + tle->mask = cpu_to_be64(entry->mask); + return p + sizeof(*tle); +} + +static char *s390_top_set_level(char *p, int level) +{ + S390TopologyEntry *entry; + uint64_t last_socket = -1UL; + uint64_t last_book = -1UL; + uint64_t last_drawer = -1UL; + int drawer_cnt = 0; + int book_cnt = 0; + int socket_cnt = 0; + + QTAILQ_FOREACH(entry, &s390_topology.list, next) { + + if (level > 3 && (last_drawer != entry->id.drawer)) { + book_cnt = 0; + socket_cnt = 0; + p = fill_container(p, 3, drawer_cnt++); + last_drawer = entry->id.id & TOPO_DRAWER_MASK; + p = fill_container(p, 2, book_cnt++); + last_book = entry->id.id & TOPO_BOOK_MASK; + p = fill_container(p, 1, socket_cnt++); + last_socket = entry->id.id & TOPO_SOCKET_MASK; + p = fill_tle_cpu(p, entry); + } else if (level > 2 && (last_book != + (entry->id.id & TOPO_BOOK_MASK))) { + socket_cnt = 0; + p = fill_container(p, 2, book_cnt++); + last_book = entry->id.id & TOPO_BOOK_MASK; + p = fill_container(p, 1, socket_cnt++); + last_socket = entry->id.id & TOPO_SOCKET_MASK; + p = fill_tle_cpu(p, entry); + } else if (last_socket != (entry->id.id & TOPO_SOCKET_MASK)) { + p = fill_container(p, 1, socket_cnt++); + last_socket = entry->id.id & TOPO_SOCKET_MASK; + p = fill_tle_cpu(p, entry); + } else { + p = fill_tle_cpu(p, entry); + } + } + + return p; +} + +static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level) +{ + char *p = sysib->tle; + + sysib->mnest = level; + switch (level) { + case 4: + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers; + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 3: + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers * + current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 2: + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers * + current_machine->smp.books * + current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + } + p = s390_top_set_level(p, level); + + return p - (char *)sysib; +} + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) +{ + union { + char place_holder[S390_TOPOLOGY_SYSIB_SIZE]; + SysIB_151x sysib; + } buffer QEMU_ALIGNED(8) = {}; + int len; + + if (!s390_has_topology() || sel2 < 2 || sel2 > SCLP_READ_SCP_INFO_MNEST) { + setcc(cpu, 3); + return; + } + + len = setup_stsi(cpu, &buffer.sysib, sel2); + + if (len > 4096) { + setcc(cpu, 3); + return; + } + + buffer.sysib.length = cpu_to_be16(len); + s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len); + setcc(cpu, 0); +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 3ac7ec9acf..5ea358cbb0 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu) if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) { return 0; } - /* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar); return 0; + case 15: + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); + return 0; default: return 0; } diff --git a/target/s390x/kvm/meson.build b/target/s390x/kvm/meson.build index aef52b6686..5daa5c6033 100644 --- a/target/s390x/kvm/meson.build +++ b/target/s390x/kvm/meson.build @@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'cpu_topology.c' ), if_false: files( 'stubs.c' )) From patchwork Thu Jan 5 14:53:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72C8BC53210 for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234185AbjAEOyi (ORCPT ); Thu, 5 Jan 2023 09:54:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234244AbjAEOyB (ORCPT ); Thu, 5 Jan 2023 09:54:01 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDDB15BA06 for ; Thu, 5 Jan 2023 06:54:00 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305Eolns021339; Thu, 5 Jan 2023 14:53:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=4XbdqSg29A4QtjfzyCWmXNUKMNG4NiwGYUouTXfz8gI=; b=c8kYstzS5XzU2kSsyjmoF97LczlK/ssA3l7gjCfLC+OUboKT1tFMyg88/54qMW5+gK2z Ndn1ZglXTLs1aQU9dMRJE7pjVfaC/QfsKg3lfNRfbRKvkKgsAuXE44lm4I60V77EO/fn 6UqYG3PwSsU0ytqU6MF9/1Qt4s/fw+ZF/8md1hT040k2h+E1EEy5f0ZXoClS3UrjXHvM 4VkYKaQP8FW4MFJO8aXkyU9C4QngzSyeYreSv+WcV0GbzLkzpMT7gRQKJIFtjnL7KXsP 7g/zz52mRlNloOG2KYCTXA0vcFQzV02Tul4VQ+0OirJwiKMVnEEC6giUsOUZscGpOVjY RQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0p101sx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:26 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305Eq8uf028646; Thu, 5 Jan 2023 14:53:26 GMT Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0p101s5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:26 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 305DESrU028726; Thu, 5 Jan 2023 14:53:23 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma06fra.de.ibm.com (PPS) with ESMTPS id 3mtcbfn3bq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:23 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErKs135062140 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:20 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4F3A2004B; Thu, 5 Jan 2023 14:53:19 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0A02520049; Thu, 5 Jan 2023 14:53:19 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:18 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 04/11] s390x/sclp: reporting the maximum nested topology entries Date: Thu, 5 Jan 2023 15:53:06 +0100 Message-Id: <20230105145313.168489-5-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 9etx2hSWC-Isj-F9PiE5d2FoL2wRYysp X-Proofpoint-ORIG-GUID: GtZBef6pTThgUVnEjTQVHePTHE1ntGwe X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 mlxscore=0 clxscore=1015 phishscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 mlxlogscore=880 spamscore=0 priorityscore=1501 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The maximum nested topology entries is used by the guest to know how many nested topology are available on the machine. Currently, SCLP READ SCP INFO reports MNEST = 0, which is the equivalent of reporting the default value of 2. Let's use the default SCLP value of 2 and increase this value in the future patches implementing higher levels. Signed-off-by: Pierre Morel Reviewed-by: Nina Schoetterl-Glausch --- include/hw/s390x/sclp.h | 5 +++-- hw/s390x/sclp.c | 4 ++++ 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h index 712fd68123..4ce852473c 100644 --- a/include/hw/s390x/sclp.h +++ b/include/hw/s390x/sclp.h @@ -112,12 +112,13 @@ typedef struct CPUEntry { } QEMU_PACKED CPUEntry; #define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 -#define SCLP_READ_SCP_INFO_MNEST 2 +#define SCLP_READ_SCP_INFO_MNEST 4 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax; uint8_t rnsize; - uint8_t _reserved1[16 - 11]; /* 11-15 */ + uint8_t _reserved1[15 - 11]; /* 11-14 */ + uint8_t stsi_parm; /* 15-16 */ uint16_t entries_cpu; /* 16-17 */ uint16_t offset_cpu; /* 18-19 */ uint8_t _reserved2[24 - 20]; /* 20-23 */ diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c index eff74479f4..07e3cb4cac 100644 --- a/hw/s390x/sclp.c +++ b/hw/s390x/sclp.c @@ -20,6 +20,7 @@ #include "hw/s390x/event-facility.h" #include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/ipl.h" +#include "hw/s390x/cpu-topology.h" static inline SCLPDevice *get_sclp_device(void) { @@ -125,6 +126,9 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb) /* CPU information */ prepare_cpu_entries(machine, entries_start, &cpu_count); + if (s390_has_topology()) { + read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST; + } read_info->entries_cpu = cpu_to_be16(cpu_count); read_info->offset_cpu = cpu_to_be16(offset_cpu); read_info->highest_cpu = cpu_to_be16(machine->smp.max_cpus - 1); From patchwork Thu Jan 5 14:53:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 291DAC54EF1 for ; Thu, 5 Jan 2023 14:54:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234350AbjAEOyH (ORCPT ); Thu, 5 Jan 2023 09:54:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234249AbjAEOxk (ORCPT ); Thu, 5 Jan 2023 09:53:40 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D94113CF9 for ; Thu, 5 Jan 2023 06:53:38 -0800 (PST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EMo1m024360; Thu, 5 Jan 2023 14:53:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Xt/EKmvStB+miA5JBeu2p0ocnBSyQ1aZzse3MRhbNss=; b=U2RMUiEKC/8KRz0ggf4h3WMaq8kpTTAKHH71EHR5ardvDBsq0c3qqVccG6sHfNJEaf/c xkOAICt7nKL3vjI/HJ8kNsM7FDGXlkFh4Dlj4J66DTYiv+iNxhJsKLUw3izL+OkLwNfY FAmcI2D5wtRnQtKuWg7pwuYXt7md0rJlPgQf8VgBg2FU42TkHLgFh5A592h+KunHIHe/ 9/d6K51hSzUuXFd4LgvtY1t3cJePnC/25fzMltkQtbVOKXUttiqGOXIhd/pXj9Ik8CnL +5XXlYHRk7pel1ROaGs5zaORbGFd2CZga//Gnm3iX5/U4PXoKpgd7x36nXVneh6QHc0f Lw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx08w0kdt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:27 +0000 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305Ej9Pe011697; Thu, 5 Jan 2023 14:53:27 GMT Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx08w0kd9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:26 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3059fGYS022351; Thu, 5 Jan 2023 14:53:24 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma01fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6d20p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:24 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErLV421037752 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:21 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EE77B20049; Thu, 5 Jan 2023 14:53:20 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03D2820040; Thu, 5 Jan 2023 14:53:20 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:19 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 05/11] s390x/cpu topology: resetting the Topology-Change-Report Date: Thu, 5 Jan 2023 15:53:07 +0100 Message-Id: <20230105145313.168489-6-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 4oJUK3x1xkRRs0BggJrvRP0D_QqfgoZc X-Proofpoint-ORIG-GUID: sBRI_A2q_11_2wewJIiluuZMiEdHTnE5 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 suspectscore=0 mlxscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org During a subsystem reset the Topology-Change-Report is cleared by the machine. Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset. Signed-off-by: Pierre Morel Reviewed-by: Thomas Huth --- target/s390x/cpu.h | 1 + target/s390x/kvm/kvm_s390x.h | 1 + hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 13 +++++++++++++ target/s390x/kvm/kvm.c | 17 +++++++++++++++++ 5 files changed, 33 insertions(+) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 78988048dd..01ade07009 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -863,6 +863,7 @@ void s390_enable_css_support(S390CPU *cpu); void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); +void s390_cpu_topology_reset(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h index f9785564d0..649dae5948 100644 --- a/target/s390x/kvm/kvm_s390x.h +++ b/target/s390x/kvm/kvm_s390x.h @@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void); void kvm_s390_restart_interrupt(S390CPU *cpu); void kvm_s390_stop_interrupt(S390CPU *cpu); void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info); +int kvm_s390_topology_set_mtcr(uint64_t attr); #endif /* KVM_S390X_H */ diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c index c98b93a15f..14798ca305 100644 --- a/hw/s390x/s390-virtio-ccw.c +++ b/hw/s390x/s390-virtio-ccw.c @@ -122,6 +122,7 @@ static void subsystem_reset(void) device_cold_reset(dev); } } + s390_cpu_topology_reset(); } static int virtio_ccw_hcall_notify(const uint64_t *args) diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index 948e4bd3e0..e27864c5f5 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg) kvm_s390_set_diag318(cs, arg.host_ulong); } } + +void s390_cpu_topology_reset(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(0); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index 5ea358cbb0..bc953151ce 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void) return cap_zpci_op; } +int kvm_s390_topology_set_mtcr(uint64_t attr) +{ + struct kvm_device_attr attribute = { + .group = KVM_S390_VM_CPU_TOPOLOGY, + .attr = attr, + }; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + return 0; + } + if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) { + return -ENOTSUP; + } + + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute); +} + void kvm_arch_accel_class_init(ObjectClass *oc) { } From patchwork Thu Jan 5 14:53:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FD3AC3DA7A for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233867AbjAEOyb (ORCPT ); Thu, 5 Jan 2023 09:54:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232923AbjAEOyE (ORCPT ); Thu, 5 Jan 2023 09:54:04 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F6285AC57 for ; Thu, 5 Jan 2023 06:54:02 -0800 (PST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EBVqt027992; Thu, 5 Jan 2023 14:53:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=xOnc2dhPoENldXYuGgZw2NckHDZA7uyD6qOXKhsbNiY=; b=nQo+Q7FxBgOoma3YOXEG0cxWCcGPDp5VPO9s0LYPjQn/UXSmChamPVd6QglrV2N7JoU5 ZcmmAYs6yRZE3+mxHWjlmX2HsKpNBZr+64jSFOZbiOaTRKPYDsgQoUnu7cacBOdPY6gx uZmtp9LpqqKVylfQE2KLr/TpmYBGtg0YYqPbvgM9ECb/FCarjiMpj3BHF0ShaAuH9eD6 QFLiXChokjfRujyTOTPHgeK4BHwW9n/YhraKI59a3TfOdAY2jMyuXOHbq6wQuzUirrBE HHnrlB5rhEAvwVqVnR2AOIPkK0N9g4PGNf6teYDBqPEDCOpS9KvLx7ksG8biVa4enJOi fw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1ma-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:28 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EEak5011559; Thu, 5 Jan 2023 14:53:28 GMT Received: from ppma04ams.nl.ibm.com (63.31.33a9.ip4.static.sl-reverse.com [169.51.49.99]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx03hh1kk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:28 +0000 Received: from pps.filterd (ppma04ams.nl.ibm.com [127.0.0.1]) by ppma04ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30545tiK021071; Thu, 5 Jan 2023 14:53:25 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma04ams.nl.ibm.com (PPS) with ESMTPS id 3mtcq6ewa5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:25 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErM5045678928 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:22 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0CEDE20043; Thu, 5 Jan 2023 14:53:22 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1392F20040; Thu, 5 Jan 2023 14:53:21 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:20 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 06/11] s390x/cpu topology: interception of PTF instruction Date: Thu, 5 Jan 2023 15:53:08 +0100 Message-Id: <20230105145313.168489-7-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: My3thOFCQpxrJmTR1GOUTfD3fggHOe-7 X-Proofpoint-GUID: Wr7CgBCJMUwZ0pCRnpLYdrgZhi2Ts90u X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 clxscore=1015 adultscore=0 priorityscore=1501 malwarescore=0 phishscore=0 spamscore=0 impostorscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension. The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor. During RESET all CPU of the configuration are placed in horizontal polarity. Signed-off-by: Pierre Morel --- include/hw/s390x/cpu-topology.h | 3 + include/hw/s390x/s390-virtio-ccw.h | 6 ++ target/s390x/cpu.h | 1 + hw/s390x/cpu-topology.c | 92 ++++++++++++++++++++++++++++++ target/s390x/cpu-sysemu.c | 16 ++++++ target/s390x/kvm/kvm.c | 11 ++++ 6 files changed, 129 insertions(+) diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h index 9571aa70e5..33e23d78b9 100644 --- a/include/hw/s390x/cpu-topology.h +++ b/include/hw/s390x/cpu-topology.h @@ -55,11 +55,13 @@ typedef struct S390Topology { QTAILQ_HEAD(, S390TopologyEntry) list; uint8_t *sockets; CpuTopology *smp; + int polarity; } S390Topology; #ifdef CONFIG_KVM bool s390_has_topology(void); void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp); +void s390_topology_set_polarity(int polarity); #else static inline bool s390_has_topology(void) { @@ -68,6 +70,7 @@ static inline bool s390_has_topology(void) static inline void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp) {} +static inline void s390_topology_set_polarity(int polarity) {} #endif extern S390Topology s390_topology; diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h index 9bba21a916..c1d46e78af 100644 --- a/include/hw/s390x/s390-virtio-ccw.h +++ b/include/hw/s390x/s390-virtio-ccw.h @@ -30,6 +30,12 @@ struct S390CcwMachineState { uint8_t loadparm[8]; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra); + struct S390CcwMachineClass { /*< private >*/ MachineClass parent_class; diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index 01ade07009..5da4041576 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -864,6 +864,7 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg); int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id, int vq, bool assign); void s390_cpu_topology_reset(void); +void s390_cpu_topology_set(void); #ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu); #else diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 438055c612..e6b4692581 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -97,6 +97,98 @@ static s390_topology_id s390_topology_from_cpu(S390CPU *cpu) } /** + * s390_topology_set_polarity + * @polarity: horizontal or vertical + * + * Changes the polarity of all the CPU in the configuration. + * + * If the dedicated CPU modifier attribute is set a vertical + * polarization is always high (Architecture). + * Otherwise we decide to set it as medium. + * + * Once done, advertise a topology change. + */ +void s390_topology_set_polarity(int polarity) +{ + S390TopologyEntry *entry; + + QTAILQ_FOREACH(entry, &s390_topology.list, next) { + if (polarity == S390_TOPOLOGY_POLARITY_HORIZONTAL) { + entry->id.p = polarity; + } else { + if (entry->id.d) { + entry->id.p = S390_TOPOLOGY_POLARITY_VERTICAL_HIGH; + } else { + entry->id.p = S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM; + } + } + } + s390_cpu_topology_set(); +} + +/* + * s390_handle_ptf: + * + * @register 1: contains the function code + * + * Function codes 0 and 1 handle the CPU polarization. + * We assume an horizontal topology, the only one supported currently + * by Linux, consequently we answer to function code 0, requesting + * horizontal polarization that it is already the current polarization + * and reject vertical polarization request without further explanation. + * + * Function code 2 is handling topology changes and is interpreted + * by the SIE. + */ +void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) +{ + CPUS390XState *env = &cpu->env; + uint64_t reg = env->regs[r1]; + uint8_t fc = reg & S390_TOPO_FC_MASK; + + if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) { + s390_program_interrupt(env, PGM_OPERATION, ra); + return; + } + + if (env->psw.mask & PSW_MASK_PSTATE) { + s390_program_interrupt(env, PGM_PRIVILEGED, ra); + return; + } + + if (reg & ~S390_TOPO_FC_MASK) { + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + return; + } + + switch (fc) { + case 0: /* Horizontal polarization is already set */ + if (s390_topology.polarity == S390_TOPOLOGY_POLARITY_HORIZONTAL) { + env->regs[r1] |= S390_PTF_REASON_DONE; + setcc(cpu, 2); + } else { + s390_topology_set_polarity(S390_TOPOLOGY_POLARITY_HORIZONTAL); + s390_topology.polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL; + setcc(cpu, 0); + } + break; + case 1: /* Vertical polarization is not supported */ + if (s390_topology.polarity != S390_TOPOLOGY_POLARITY_HORIZONTAL) { + env->regs[r1] |= S390_PTF_REASON_DONE; + setcc(cpu, 2); + } else { + s390_topology_set_polarity(S390_TOPOLOGY_POLARITY_VERTICAL_LOW); + s390_topology.polarity = S390_TOPOLOGY_POLARITY_VERTICAL_LOW; + setcc(cpu, 0); + } + break; + default: + /* Note that fc == 2 is interpreted by the SIE */ + s390_program_interrupt(env, PGM_SPECIFICATION, ra); + } +} + + /** * s390_topology_set_entry: * @entry: Topology entry to setup * @id: topology id to use for the setup diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c index e27864c5f5..8643320947 100644 --- a/target/s390x/cpu-sysemu.c +++ b/target/s390x/cpu-sysemu.c @@ -37,6 +37,7 @@ #include "sysemu/sysemu.h" #include "sysemu/tcg.h" #include "hw/core/sysemu-cpu-ops.h" +#include "hw/s390x/cpu-topology.h" /* S390CPUClass::load_normal() */ static void s390_cpu_load_normal(CPUState *s) @@ -311,6 +312,8 @@ void s390_cpu_topology_reset(void) { int ret; + s390_topology_set_polarity(S390_TOPOLOGY_POLARITY_HORIZONTAL); + if (kvm_enabled()) { ret = kvm_s390_topology_set_mtcr(0); if (ret) { @@ -319,3 +322,16 @@ void s390_cpu_topology_reset(void) } } } + +void s390_cpu_topology_set(void) +{ + int ret; + + if (kvm_enabled()) { + ret = kvm_s390_topology_set_mtcr(1); + if (ret) { + error_report("Failed to set Modified Topology Change Report: %s", + strerror(-ret)); + } + } +} diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index bc953151ce..fb63be41b7 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -96,6 +96,7 @@ #define PRIV_B9_EQBS 0x9c #define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2 #define PRIV_B9_PCISTG 0xd0 #define PRIV_B9_PCILG 0xd2 #define PRIV_B9_RPCIT 0xd3 @@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run) +{ + uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f; + + s390_handle_ptf(cpu, r1, RA_IGNORED); +} + static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) { int r = 0; @@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1) case PRIV_B9_RPCIT: r = kvm_rpcit_service_call(cpu, run); break; + case PRIV_B9_PTF: + kvm_handle_ptf(cpu, run); + break; case PRIV_B9_EQBS: /* just inject exception */ r = -1; From patchwork Thu Jan 5 14:53:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FA62C54E76 for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234241AbjAEOye (ORCPT ); Thu, 5 Jan 2023 09:54:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234258AbjAEOyD (ORCPT ); Thu, 5 Jan 2023 09:54:03 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36FD35BA02 for ; Thu, 5 Jan 2023 06:54:01 -0800 (PST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305Epi2P022468; Thu, 5 Jan 2023 14:53:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=SxbuPdOWXGoof9kp9R4zjXTiOzZkOwyrjxz+z9AhaYQ=; b=bGdqpOCVba0/BRwvi2fXTS0Qip7MYFiMv+rgg2yyli3aV36v/bGIQNtDo4JuOLGUOI2s bgnK24JU+8pqJ1Ss8gGayZca6O0r/BGa6/h0GenGdq8k5yvX4vNtytA1t2ZPPgaU6y4h WzydoeUiBT4qUlvQqvjRuS1PCllB24hlDVkdg1lMPOEN1GxShjr0nImJUqDSK/SOPk5o 86yexXgiW5Os2FEWTG2M6gaCcSJy7o6PFHBZlj+wLlUyF0R9mtSMpQDg7taKdypaeYgJ F/I55DT6pk7Di3LotUWjAtQEsJAfN6jiy74WPB45//XnG7XHgOzBC0fGemGwv+NwS/Yq vQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0pfg152-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:29 +0000 Received: from m0127361.ppops.net (m0127361.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305Er4jb028552; Thu, 5 Jan 2023 14:53:28 GMT Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0pfg14d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:28 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3051kSQE024684; Thu, 5 Jan 2023 14:53:26 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma01fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6d20q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:26 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErN6021692726 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:23 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 07C832004D; Thu, 5 Jan 2023 14:53:23 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1F08320049; Thu, 5 Jan 2023 14:53:22 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:22 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 07/11] target/s390x/cpu topology: activating CPU topology Date: Thu, 5 Jan 2023 15:53:09 +0100 Message-Id: <20230105145313.168489-8-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: pdTodLx_1JY0KDVqf9b0TiW1uEOtEAqs X-Proofpoint-GUID: SZqsnV2NOC1ugrFzqDpTUIgVW94-qQe3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxlogscore=999 clxscore=1015 phishscore=0 lowpriorityscore=0 suspectscore=0 malwarescore=0 mlxscore=0 bulkscore=0 spamscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The KVM capability, KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility for the guest in the case the topology is available in QEMU and in KVM. The feature is disabled by default and fenced for SE (secure execution). Signed-off-by: Pierre Morel --- hw/s390x/cpu-topology.c | 2 +- target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 13 +++++++++++++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index e6b4692581..b69955a1cd 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -52,7 +52,7 @@ static int s390_socket_nb(s390_topology_id id) */ bool s390_has_topology(void) { - return false; + return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY); } /** diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c index c3a4f80633..3f05e05fd3 100644 --- a/target/s390x/cpu_models.c +++ b/target/s390x/cpu_models.c @@ -253,6 +253,7 @@ bool s390_has_feat(S390Feat feat) case S390_FEAT_SIE_CMMA: case S390_FEAT_SIE_PFMFI: case S390_FEAT_SIE_IBS: + case S390_FEAT_CONFIGURATION_TOPOLOGY: return false; break; default: diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c index fb63be41b7..4e2a2ff516 100644 --- a/target/s390x/kvm/kvm.c +++ b/target/s390x/kvm/kvm.c @@ -2470,6 +2470,19 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) set_bit(S390_FEAT_UNPACK, model->features); } + /* + * If we have support for CPU Topology prevent overrule + * S390_FEAT_CONFIGURATION_TOPOLOGY with S390_FEAT_DISABLE_CPU_TOPOLOGY + * implemented in KVM, activate the CPU TOPOLOGY feature. + */ + if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) { + if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) { + error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY"); + return; + } + set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features); + } + /* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit(S390_FEAT_ZPCI, model->features); set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features); From patchwork Thu Jan 5 14:53:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 987BEC54EBD for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234422AbjAEOyp (ORCPT ); Thu, 5 Jan 2023 09:54:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234333AbjAEOyG (ORCPT ); Thu, 5 Jan 2023 09:54:06 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DD2C5BA01 for ; Thu, 5 Jan 2023 06:54:05 -0800 (PST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EKf0f009627; Thu, 5 Jan 2023 14:53:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=ethdufynlW1MEixRs+2AXSiySCmWBFk8ra2McLsJN2w=; b=QiWszAf4qE3F3ntXCbwv+wpRdhHbAhLOhfw75gqCfd8Ytvt+F8kafwxBULRF/p0kGpDy u9KwQSq/L7MNjE0mYFv3qDhFJ3L6JvR1j6p6G3ZiL+1oHxNbAkpPBGZ22skhmCGqvsgI UV9EbcbJqIDLax/DJvkQPw3vFp3w5B638Vn6q471UbVgmTM1OtBOlnPfUe6p09DWcB0U iTj4vZt20Ztld5jmrzumx56+EJyViSLJrDFUljoeooCUzHDYq4MuE90EgbfOpWNthzup tFEnVUJKpxGtI9tM14cB4pZYNgL8N/HEZYIZrwvYTtthb1afOSxBYU98brtxlH8oAzDg zQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx07wrsd9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:31 +0000 Received: from m0187473.ppops.net (m0187473.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EL1p0011135; Thu, 5 Jan 2023 14:53:30 GMT Received: from ppma04fra.de.ibm.com (6a.4a.5195.ip4.static.sl-reverse.com [149.81.74.106]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx07wrscg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:30 +0000 Received: from pps.filterd (ppma04fra.de.ibm.com [127.0.0.1]) by ppma04fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3052VGxW010510; Thu, 5 Jan 2023 14:53:28 GMT Received: from smtprelay07.fra02v.mail.ibm.com ([9.218.2.229]) by ppma04fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6d2b9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:27 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay07.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErOVg49414634 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:24 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2598120040; Thu, 5 Jan 2023 14:53:24 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 217BA20049; Thu, 5 Jan 2023 14:53:23 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:23 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 08/11] qapi/s390/cpu topology: change-topology monitor command Date: Thu, 5 Jan 2023 15:53:10 +0100 Message-Id: <20230105145313.168489-9-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: kaN2e_OFZoK6BbQPjgAzMn5S4LU9B7Im X-Proofpoint-ORIG-GUID: xefNCUmSMyiOBuSaxryUHpmIPiDQcEqm X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_04,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 adultscore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The modification of the CPU attributes are done through a monitor commands. It allows to move the core inside the topology tree to optimise the cache usage in the case the host's hypervizor previously moved the CPU. The same command allows to modifiy the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU. With this knowledge the guest has the possibility to optimize the usage of the vCPUs. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 29 ++++++++ include/monitor/hmp.h | 1 + hw/s390x/cpu-topology.c | 141 +++++++++++++++++++++++++++++++++++++++ hmp-commands.hx | 16 +++++ 4 files changed, 187 insertions(+) diff --git a/qapi/machine-target.json b/qapi/machine-target.json index 2e267fa458..75b0aa254d 100644 --- a/qapi/machine-target.json +++ b/qapi/machine-target.json @@ -342,3 +342,32 @@ 'TARGET_S390X', 'TARGET_MIPS', 'TARGET_LOONGARCH64' ] } } + +## +# @change-topology: +# +# @core: the vCPU ID to be moved +# @socket: the destination socket where to move the vCPU +# @book: the destination book where to move the vCPU +# @drawer: the destination drawer where to move the vCPU +# @polarity: optional polarity, default is last polarity set by the guest +# @dedicated: optional, if the vCPU is dedicated to a real CPU +# +# Modifies the topology by moving the CPU inside the topology +# tree or by changing a modifier attribute of a CPU. +# +# Returns: Nothing on success, the reason on failure. +# +# Since: +## +{ 'command': 'change-topology', + 'data': { + 'core': 'int', + 'socket': 'int', + 'book': 'int', + 'drawer': 'int', + '*polarity': 'int', + '*dedicated': 'bool' + }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h index 27f86399f7..15c36bf549 100644 --- a/include/monitor/hmp.h +++ b/include/monitor/hmp.h @@ -144,5 +144,6 @@ void hmp_human_readable_text_helper(Monitor *mon, HumanReadableText *(*qmp_handler)(Error **)); void hmp_info_stats(Monitor *mon, const QDict *qdict); void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict); +void hmp_change_topology(Monitor *mon, const QDict *qdict); #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index b69955a1cd..0faffe657e 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -18,6 +18,10 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "qapi/qapi-commands-machine-target.h" +#include "qapi/qmp/qdict.h" +#include "monitor/hmp.h" +#include "monitor/monitor.h" /* * s390_topology is used to keep the topology information. @@ -203,6 +207,21 @@ static void s390_topology_set_entry(S390TopologyEntry *entry, s390_topology.sockets[s390_socket_nb(id)]++; } +/** + * s390_topology_clear_entry: + * @entry: Topology entry to setup + * @id: topology id to use for the setup + * + * Clear the core bit inside the topology mask and + * decrements the number of cores for the socket. + */ +static void s390_topology_clear_entry(S390TopologyEntry *entry, + s390_topology_id id) +{ + clear_bit(63 - id.core, &entry->mask); + s390_topology.sockets[s390_socket_nb(id)]--; +} + /** * s390_topology_new_entry: * @id: s390_topology_id to add @@ -383,3 +402,125 @@ void s390_topology_set_cpu(MachineState *ms, S390CPU *cpu, Error **errp) s390_topology_insert(id); } + +/* + * qmp and hmp implementations + */ + +static S390TopologyEntry *s390_topology_core_to_entry(int core) +{ + S390TopologyEntry *entry; + + QTAILQ_FOREACH(entry, &s390_topology.list, next) { + if (entry->mask & (1UL << (63 - core))) { + return entry; + } + } + return NULL; +} + +static void s390_change_topology(Error **errp, int64_t core, int64_t socket, + int64_t book, int64_t drawer, + int64_t polarity, bool dedicated) +{ + S390TopologyEntry *entry; + s390_topology_id new_id; + s390_topology_id old_id; + Error *local_error = NULL; + + /* Get the old entry */ + entry = s390_topology_core_to_entry(core); + if (!entry) { + error_setg(errp, "No core %ld", core); + return; + } + + /* Compute old topology id */ + old_id = entry->id; + old_id.core = core; + + /* Compute new topology id */ + new_id = entry->id; + new_id.core = core; + new_id.socket = socket; + new_id.book = book; + new_id.drawer = drawer; + new_id.p = polarity; + new_id.d = dedicated; + new_id.type = S390_TOPOLOGY_CPU_IFL; + + /* Same topology entry, nothing to do */ + if (entry->id.id == new_id.id) { + return; + } + + /* Check for space on the socket if ids are different */ + if ((s390_socket_nb(old_id) != s390_socket_nb(new_id)) && + (s390_topology.sockets[s390_socket_nb(new_id)] >= + s390_topology.smp->sockets)) { + error_setg(errp, "No more space on this socket"); + return; + } + + /* Verify the new topology */ + s390_topology_check(&local_error, new_id); + if (local_error) { + error_propagate(errp, local_error); + return; + } + + /* Clear the old topology */ + s390_topology_clear_entry(entry, old_id); + + /* Insert the new topology */ + s390_topology_insert(new_id); + + /* Remove unused entry */ + if (!entry->mask) { + QTAILQ_REMOVE(&s390_topology.list, entry, next); + g_free(entry); + } + + /* Advertise the topology change */ + s390_cpu_topology_set(); +} + +void qmp_change_topology(int64_t core, int64_t socket, + int64_t book, int64_t drawer, + bool has_polarity, int64_t polarity, + bool has_dedicated, bool dedicated, + Error **errp) +{ + Error *local_err = NULL; + + if (!s390_has_topology()) { + error_setg(&local_err, "This machine doesn't support topology"); + return; + } + s390_change_topology(&local_err, core, socket, book, drawer, + polarity, dedicated); + if (local_err) { + error_propagate(errp, local_err); + } +} + +void hmp_change_topology(Monitor *mon, const QDict *qdict) +{ + const int64_t core = qdict_get_int(qdict, "core"); + const int64_t socket = qdict_get_int(qdict, "socket"); + const int64_t book = qdict_get_int(qdict, "book"); + const int64_t drawer = qdict_get_int(qdict, "drawer"); + bool has_polarity = qdict_haskey(qdict, "polarity"); + const int64_t polarity = qdict_get_try_int(qdict, "polarity", 0); + bool has_dedicated = qdict_haskey(qdict, "dedicated"); + const bool dedicated = qdict_get_try_bool(qdict, "dedicated", false); + Error *local_err = NULL; + + qmp_change_topology(core, socket, book, drawer, + has_polarity, polarity, + has_dedicated, dedicated, + &local_err); + if (hmp_handle_error(mon, local_err)) { + return; + } +} diff --git a/hmp-commands.hx b/hmp-commands.hx index 673e39a697..a617cfed0d 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx @@ -1815,3 +1815,19 @@ SRST Dump the FDT in dtb format to *filename*. ERST #endif + +#if defined(TARGET_S390X) && defined(CONFIG_KVM) + { + .name = "change-topology", + .args_type = "core:l,socket:l,book:l,drawer:l,polarity:l?,dedicated:b?", + .params = "core socket book drawer [polarity] [dedicated]", + .help = "Move CPU 'core' to 'socket/book/drawer' " + "optionaly modifies polarity and dedication", + .cmd = hmp_change_topology, + }, + +SRST +``change-topology`` *core* *socket* *book* *drawer* *polarity* *dedicated* + Moves the CPU *core* to *socket* *book* *drawer* with *polarity* *dedicated*. +ERST +#endif From patchwork Thu Jan 5 14:53:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA54EC54EBF for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234467AbjAEOyt (ORCPT ); Thu, 5 Jan 2023 09:54:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234326AbjAEOyG (ORCPT ); Thu, 5 Jan 2023 09:54:06 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F23EF5BA3D for ; Thu, 5 Jan 2023 06:54:04 -0800 (PST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305Eko0G030317; Thu, 5 Jan 2023 14:53:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=1jMdpFXY9jo4vujFoe8Cvpy+OvJwfSPVhM2ZXZ/Mww4=; b=kKkDxzRoKSOOTCg/BNHkwoOYztDIgtAnOHLV+jhIkNvm3cPUeRK5kNRRqL6gjTjG3pcR h3crQu06/iakratBB8ktMWWiBagamqgIr9syNJ9Y09eiqRH/+F/NlOL6phcL0wEdY1KM OfD834zW3QFZny57cFqXNzl4ePVkmrSLBUy/sFDc1jCt3Z4qot7gcP97ggl4lE40Wwz/ 7c498tg7LHanpC2SfWNKi/c7UJxsdqxcKpId229DKopuHHFRcA2TNP2QcLn4jQZbdBHm P+8PAgdfh0H47ELhdCz06Z4tEWr/CTLWYkffhXElR2nj46bKbi9G59CgDlGGwKFM9HiS 8Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0m203mk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:31 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EmPvt005343; Thu, 5 Jan 2023 14:53:31 GMT Received: from ppma05fra.de.ibm.com (6c.4a.5195.ip4.static.sl-reverse.com [149.81.74.108]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0m203m6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:30 +0000 Received: from pps.filterd (ppma05fra.de.ibm.com [127.0.0.1]) by ppma05fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 30510Go0031909; Thu, 5 Jan 2023 14:53:28 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma05fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6n2yp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:28 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErPVB45875532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:25 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2AAD220040; Thu, 5 Jan 2023 14:53:25 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 38B5420043; Thu, 5 Jan 2023 14:53:24 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:24 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 09/11] qapi/s390/cpu topology: monitor query topology information Date: Thu, 5 Jan 2023 15:53:11 +0100 Message-Id: <20230105145313.168489-10-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6IHhwYZB6svt4j4rL7yFUwggOX7Xy38p X-Proofpoint-ORIG-GUID: pePEw0VYIkb8a7FFdQ556fsPjFoVO0ZS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 adultscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 impostorscore=0 clxscore=1015 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Reporting the current topology informations to the admin through the QEMU monitor. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 66 ++++++++++++++++++++++++++++++++++ include/monitor/hmp.h | 1 + hw/s390x/cpu-topology.c | 76 ++++++++++++++++++++++++++++++++++++++++ hmp-commands-info.hx | 16 +++++++++ 4 files changed, 159 insertions(+) diff --git a/qapi/machine-target.json b/qapi/machine-target.json index 75b0aa254d..927618a78f 100644 --- a/qapi/machine-target.json +++ b/qapi/machine-target.json @@ -371,3 +371,69 @@ }, 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @S390CpuTopology: +# +# CPU Topology information +# +# @drawer: the destination drawer where to move the vCPU +# +# @book: the destination book where to move the vCPU +# +# @socket: the destination socket where to move the vCPU +# +# @polarity: optional polarity, default is last polarity set by the guest +# +# @dedicated: optional, if the vCPU is dedicated to a real CPU +# +# @origin: offset of the first bit of the core mask +# +# @mask: mask of the cores sharing the same topology +# +# Since: 8.0 +## +{ 'struct': 'S390CpuTopology', + 'data': { + 'drawer': 'int', + 'book': 'int', + 'socket': 'int', + 'polarity': 'int', + 'dedicated': 'bool', + 'origin': 'int', + 'mask': 'str' + }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} + +## +# @query-topology: +# +# Return information about CPU Topology +# +# Returns a @CpuTopology instance describing the CPU Toplogy +# being currently used by QEMU. +# +# Since: 8.0 +# +# Example: +# +# -> { "execute": "cpu-topology" } +# <- {"return": [ +# { +# "drawer": 0, +# "book": 0, +# "socket": 0, +# "polarity": 0, +# "dedicated": true, +# "origin": 0, +# "mask": 0xc000000000000000, +# }, +# ] +# } +# +## +{ 'command': 'query-topology', + 'returns': ['S390CpuTopology'], + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h index 15c36bf549..0b3c758231 100644 --- a/include/monitor/hmp.h +++ b/include/monitor/hmp.h @@ -145,5 +145,6 @@ void hmp_human_readable_text_helper(Monitor *mon, void hmp_info_stats(Monitor *mon, const QDict *qdict); void hmp_pcie_aer_inject_error(Monitor *mon, const QDict *qdict); void hmp_change_topology(Monitor *mon, const QDict *qdict); +void hmp_query_topology(Monitor *mon, const QDict *qdict); #endif diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index 0faffe657e..c3748654ff 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -524,3 +524,79 @@ void hmp_change_topology(Monitor *mon, const QDict *qdict) return; } } + +static S390CpuTopologyList *s390_cpu_topology_list(void) +{ + S390CpuTopologyList *head = NULL; + S390TopologyEntry *entry; + + QTAILQ_FOREACH_REVERSE(entry, &s390_topology.list, next) { + S390CpuTopology *item = g_new0(typeof(*item), 1); + + item->drawer = entry->id.drawer; + item->book = entry->id.book; + item->socket = entry->id.socket; + item->polarity = entry->id.p; + if (entry->id.d) { + item->dedicated = true; + } + item->origin = entry->id.origin; + item->mask = g_strdup_printf("0x%016lx", entry->mask); + + QAPI_LIST_PREPEND(head, item); + } + return head; +} + +S390CpuTopologyList *qmp_query_topology(Error **errp) +{ + if (!s390_has_topology()) { + error_setg(errp, "This machine doesn't support topology"); + return NULL; + } + + return s390_cpu_topology_list(); +} + +void hmp_query_topology(Monitor *mon, const QDict *qdict) +{ + Error *err = NULL; + S390CpuTopologyList *l = qmp_query_topology(&err); + + if (hmp_handle_error(mon, err)) { + return; + } + + monitor_printf(mon, "CPU Topology:\n"); + while (l) { + uint64_t d = -1UL; + uint64_t b = -1UL; + uint64_t s = -1UL; + uint64_t p = -1UL; + uint64_t dd = -1UL; + + if (d != l->value->drawer) { + monitor_printf(mon, " drawer : \"%" PRIu64 "\"\n", + l->value->drawer); + } + if (b != l->value->book) { + monitor_printf(mon, " book : \"%" PRIu64 "\"\n", + l->value->book); + } + if (s != l->value->socket) { + monitor_printf(mon, " socket : \"%" PRIu64 "\"\n", + l->value->socket); + } + if (p != l->value->polarity) { + monitor_printf(mon, " polarity : \"%" PRIu64 "\"\n", + l->value->polarity); + } + if (dd != l->value->dedicated) { + monitor_printf(mon, " dedicated: \"%d\"\n", l->value->dedicated); + } + monitor_printf(mon, " mask : \"%s\"\n", l->value->mask); + + + l = l->next; + } +} diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx index 754b1e8408..5730a47f71 100644 --- a/hmp-commands-info.hx +++ b/hmp-commands-info.hx @@ -993,3 +993,19 @@ SRST ``info virtio-queue-element`` *path* *queue* [*index*] Display element of a given virtio queue ERST + +#if defined(TARGET_S390X) && defined(CONFIG_KVM) + { + .name = "query-topology", + .args_type = "", + .params = "", + .help = "Show information about CPU topology", + .cmd = hmp_query_topology, + .flags = "p", + }, + +SRST + ``info query-topology`` + Show information about CPU topology +ERST +#endif From patchwork Thu Jan 5 14:53:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85DCEC54EBE for ; Thu, 5 Jan 2023 14:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234381AbjAEOym (ORCPT ); Thu, 5 Jan 2023 09:54:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234311AbjAEOyF (ORCPT ); Thu, 5 Jan 2023 09:54:05 -0500 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E7F3A5B176 for ; Thu, 5 Jan 2023 06:54:04 -0800 (PST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305EokUr021279; Thu, 5 Jan 2023 14:53:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=tJNKQ6XLjc0XBA0Vpw8/hB1o+7HqorlTgkt4Qg6FCGk=; b=UkixJwpxsLWgXQXiBSFp9O1F4BU6W6QUP4bMBh1QsE444Pjl2BY7M31SBTiO466VyW0N EBNT7dFMz0hqiLHgwECykcbav6w6id9Ln4Z+Cv5OgX7exY8QTGZzIBDMExhh3qnuboqX V9jUkxrIEI0/ppGtQasb99+cX4vstl73yOnZ1e4SJRlizhKMjoYRplJ8bwtSnGOLWqbs q2ICFhYm+RfCVwWiZJzdhP+0dlWkh9u02vQw7pk+OqDlrH/TYtUTJnAodycgtOQfKc+z GcqT77hqFbCZFERFrSnpMnFDm0ExNziXFfMPnhAo1CDdkaibxbqcDqI6/E+LRWrrCDUA /Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0p101v4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:33 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305ErW25032765; Thu, 5 Jan 2023 14:53:32 GMT Received: from ppma02fra.de.ibm.com (47.49.7a9f.ip4.static.sl-reverse.com [159.122.73.71]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0p101uh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:32 +0000 Received: from pps.filterd (ppma02fra.de.ibm.com [127.0.0.1]) by ppma02fra.de.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3050abd3018137; Thu, 5 Jan 2023 14:53:29 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma02fra.de.ibm.com (PPS) with ESMTPS id 3mtcq6w1xb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:29 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErQD942992086 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:26 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 26C9A20040; Thu, 5 Jan 2023 14:53:26 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3C91720049; Thu, 5 Jan 2023 14:53:25 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:25 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 10/11] qapi/s390/cpu topology: POLARITY_CHANGE qapi event Date: Thu, 5 Jan 2023 15:53:12 +0100 Message-Id: <20230105145313.168489-11-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: bvS4Hef6DIb0ypJN4eff5Kb9XX84u_5M X-Proofpoint-ORIG-GUID: YSD0-adTzK4EwNlM93Xbdko_jbUkEv84 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 mlxscore=0 clxscore=1015 phishscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the guest asks to change the polarity this change is forwarded to the admin using QAPI. The admin is supposed to take according decisions concerning CPU provisioning. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 21 +++++++++++++++++++++ hw/s390x/cpu-topology.c | 2 ++ 2 files changed, 23 insertions(+) diff --git a/qapi/machine-target.json b/qapi/machine-target.json index 927618a78f..10235cfb45 100644 --- a/qapi/machine-target.json +++ b/qapi/machine-target.json @@ -437,3 +437,24 @@ 'returns': ['S390CpuTopology'], 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @POLARITY_CHANGE: +# +# Emitted when the guest asks to change the polarity. +# +# @polarity: polarity specified by the guest +# +# Since: 8.0 +# +# Example: +# +# <- { "event": "POLARITY_CHANGE", +# "data": { "polarity": 0 }, +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } } +# +## +{ 'event': 'POLARITY_CHANGE', + 'data': { 'polarity': 'int' }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM'] } +} diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c index c3748654ff..45621387d5 100644 --- a/hw/s390x/cpu-topology.c +++ b/hw/s390x/cpu-topology.c @@ -19,6 +19,7 @@ #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "qapi/qapi-commands-machine-target.h" +#include "qapi/qapi-events-machine-target.h" #include "qapi/qmp/qdict.h" #include "monitor/hmp.h" #include "monitor/monitor.h" @@ -128,6 +129,7 @@ void s390_topology_set_polarity(int polarity) } } s390_cpu_topology_set(); + qapi_event_send_polarity_change(polarity); } /* From patchwork Thu Jan 5 14:53:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pierre Morel X-Patchwork-Id: 13090061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A000C54EBF for ; Thu, 5 Jan 2023 14:54:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234313AbjAEOyF (ORCPT ); Thu, 5 Jan 2023 09:54:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234213AbjAEOxk (ORCPT ); Thu, 5 Jan 2023 09:53:40 -0500 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72DA41104 for ; Thu, 5 Jan 2023 06:53:39 -0800 (PST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 305Eknkl030038; Thu, 5 Jan 2023 14:53:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=3JHlvSVoOoD113y+YLcDLmKD3ivZV4W6BebEz367UL8=; b=kJm9KeV5pFVVDkJzurUFw7awn3zbtqOOx9fwMeq6CYtWwcZzlriTWXofxOqDx0oX4aYe s2qfldJhRaicLhQ77Dab6XnVlry4lbINbcfchH2FVpVIcAZ8rmuITPvLl3RKwclb8bR4 UWZijM8GJNdc4wrB+8exEZhCkL6i1wOHyrodlXS/zXlx/gP39pds+yoPTbd9O95K7aku jEV0NqYvBuRsPVULN9g9Kj9nYnmXWXXhgamUquRhig49eUnbdjWKQcZvuAQJ5cqVLDpf nX1iWkUKq/DI3C2K091WC5iqonxqBl5xvHlCPB44+2ezHQJCrdAKvcvNon2pm+2Y3qK/ kA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0m203n4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:33 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 305EougW015329; Thu, 5 Jan 2023 14:53:32 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3mx0m203mn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:32 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 3054fPjU001024; Thu, 5 Jan 2023 14:53:30 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma06ams.nl.ibm.com (PPS) with ESMTPS id 3mtcbfewh7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 Jan 2023 14:53:30 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 305ErRsU42664314 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 Jan 2023 14:53:27 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2CDC320040; Thu, 5 Jan 2023 14:53:27 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 39A9420043; Thu, 5 Jan 2023 14:53:26 +0000 (GMT) Received: from li-c6ac47cc-293c-11b2-a85c-d421c8e4747b.ibm.com.com (unknown [9.171.26.113]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 5 Jan 2023 14:53:26 +0000 (GMT) From: Pierre Morel To: qemu-s390x@nongnu.org Cc: qemu-devel@nongnu.org, borntraeger@de.ibm.com, pasic@linux.ibm.com, richard.henderson@linaro.org, david@redhat.com, thuth@redhat.com, cohuck@redhat.com, mst@redhat.com, pbonzini@redhat.com, kvm@vger.kernel.org, ehabkost@redhat.com, marcel.apfelbaum@gmail.com, eblake@redhat.com, armbru@redhat.com, seiden@linux.ibm.com, nrb@linux.ibm.com, scgl@linux.ibm.com, frankja@linux.ibm.com, berrange@redhat.com, clg@kaod.org Subject: [PATCH v14 11/11] docs/s390x/cpu topology: document s390x cpu topology Date: Thu, 5 Jan 2023 15:53:13 +0100 Message-Id: <20230105145313.168489-12-pmorel@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20230105145313.168489-1-pmorel@linux.ibm.com> References: <20230105145313.168489-1-pmorel@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 19vAfV-2JQ_c3lBA8hN6fOba60K3eEm9 X-Proofpoint-ORIG-GUID: tdgqR736KXCs19IBdNBvXKmEd1uZnnTP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2023-01-05_05,2023-01-05_01,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 suspectscore=0 adultscore=0 spamscore=0 lowpriorityscore=0 mlxlogscore=999 priorityscore=1501 phishscore=0 impostorscore=0 clxscore=1015 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2301050114 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add some basic examples for the definition of cpu topology in s390x. Signed-off-by: Pierre Morel --- docs/system/s390x/cpu-topology.rst | 292 +++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 + 2 files changed, 293 insertions(+) create mode 100644 docs/system/s390x/cpu-topology.rst diff --git a/docs/system/s390x/cpu-topology.rst b/docs/system/s390x/cpu-topology.rst new file mode 100644 index 0000000000..0020b70b50 --- /dev/null +++ b/docs/system/s390x/cpu-topology.rst @@ -0,0 +1,292 @@ +CPU Topology on s390x +===================== + +CPU Topology on S390x provides up to 5 levels of topology containers: +nodes, drawers, books, sockets and CPUs. +While the higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the parent container. + +Currently QEMU CPU topology uses a single level of container: the sockets. + +For backward compatibility, threads can be declared on the ``-smp`` command +line. They will be seen as CPUs by the guest as long as multithreading +is not really supported by QEMU for S390. + +Beside the topological tree, S390x provides 3 CPU attributes: +- CPU type +- polarity entitlement +- dedication + +Prerequisites +------------- + +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version +of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM +QEMU use of the CPU Topology. + +Enabling CPU topology +--------------------- + +Currently, CPU topology is only enabled in the host model. + +Enabling CPU topology in a CPU model is done by setting the CPU flag +``ctop`` to ``on`` like in: + +.. code-block:: bash + + -cpu gen16b,ctop=on + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags. + +Default topology usage +---------------------- + +The CPU Topology, can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments +without using any new attributes. +In this case, the topology will be calculated by simply adding +to the topology the cores based on the core-id starting with +core-0 at position 0 of socket-0, book-0, drawer-0 with default +modifier attributes: horizontal polarity and no dedication. + +In the following machine we define 8 sockets with 4 cores each. +Note that S390 QEMU machines do not implement multithreading. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ + -device host-s390x-cpu,core-id=14 \ + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 4 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will + be placed in socket 2, with core-id 9 + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be written in a big endian mask: + +* in socket 0: 0xf000000000000000 (core id 0,1,2,3) +* in socket 1: 0x0800000000000000 (core id 4) +* in socket 2: 0x0040000000000000 (core id 9) +* in socket 3: 0x0002000000000000 (core id 14) + +Defining the topology on command line +------------------------------------- + +The topology can be defined entirely during the CPU definition, +with the exception of CPU 0 which must be defined with the -smp +argument. + +For example, here we set the position of the cores 1,2,3 on +drawer 1, book 1, socket 2 and cores 0,9 and 14 on drawer 0, +book 0, socket 0 with all horizontal polarity and not dedicated. +The core 4, will be set on its default position on socket 1 +(since we have 4 core per socket) and we define it with dedication and +vertical high entitlement. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \ + \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \ + \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \ + \ + -device gen16b-s390x-cpu,core-id=4,dedicated=on,polarity=3 \ + +Polarity and dedication +----------------------- + +Polarity can be of two types: horizontal or vertical. + +The horizontal polarization specifies that all guest's vCPUs get +almost the same amount of provisioning of real CPU by the host. + +The vertical polarization specifies that guest's vCPU can get +different real CPU provisions: + +- a vCPU with Vertical high entitlement specifies that this + vCPU gets 100% of the real CPU provisioning. + +- a vCPU with Vertical medium entitlement specifies that this + vCPU shares the real CPU with other vCPU. + +- a vCPU with Vertical low entitlement specifies that this + vCPU only get real CPU provisioning when no other vCPU need it. + +In the case a vCPU with vertical high entitlement does not use +the real CPU, the unused "slack" can be dispatched to other vCPU +with medium or low entitlement. + +The host indicates to the guest how the real CPU resources are +provided to the vCPUs through the SYSIB with two polarity bits +inside the CPU TLE. + +Bits d - Polarization +0 0 Horizontal +0 1 Vertical low entitlement +1 0 Vertical medium entitlement +1 1 Vertical high entitlement + +A subsystem reset puts all vCPU of the configuration into the +horizontal polarization. + +The admin specifies the dedicated bit when the vCPU is dedicated +to a single real CPU. + +As for the Linux admin, the dedicated bit is an indication on the +affinity of a vCPU for a real CPU while the entitlement indicates the +sharing or exclusivity of use. + +QAPI interface for topology +--------------------------- + +Let's start QEMU with the following command: + +.. code-block:: bash + + sudo /usr/local/bin/qemu-system-s390x \ + -enable-kvm \ + -cpu z14,ctop=on \ + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \ + \ + -device z14-s390x-cpu,core-id=19,polarity=3 \ + -device z14-s390x-cpu,core-id=11,polarity=1 \ + -device z14-s390x-cpu,core-id=12,polarity=3 \ + ... + +and see the result when using of the QAPI interface. + +query-topology ++++++++++++++++ + +The command cpu-topology allows the admin to query the topology +tree and modifier for all configured vCPU. + +.. code-block:: QMP + + -> { "execute": "query-topology" } + {"return": + [ + { + "origin": 0, + "dedicated": false, + "book": 0, + "socket": 0, + "drawer": 0, + "polarity": 0, + "mask": "0x8000000000000000" + }, + { + "origin": 0, + "dedicated": false, + "book": 2, + "socket": 1, + "drawer": 0, + "polarity": 1, + "mask": "0x0010000000000000" + }, + { + "origin": 0, + "dedicated": false, + "book": 0, + "socket": 0, + "drawer": 1, + "polarity": 3, + "mask": "0x0008000000000000" + }, + { + "origin": 0, + "dedicated": false, + "book": 1, + "socket": 1, + "drawer": 1, + "polarity": 3, + "mask": "0x0000100000000000" + } + ] + } + +change-topology ++++++++++++++++ + +The command change-topology allows the admin to modify the topology +tree or the topology modifiers of a vCPU in the configuration. + +.. code-block:: QMP + + -> { "execute": "change-topology", + "arguments": { + "core": 11, + "socket": 0, + "book": 0, + "drawer": 0, + "polarity": 0, + "dedicated": false + } + } + <- {"return": {}} + + +event POLARITY_CHANGE ++++++++++++++++++++++ + +When a guest is requesting a modification of the polarity, +QEMU sends a POLARITY_CHANGE event. + +When requesting the change, the guest only specifies horizontal or +vertical polarity. +The dedication and fine grain vertical entitlement depends on admin +to set according to its response to this event. + +Note that a vertical polarized dedicated vCPU can only have a high +entitlement, this gives 6 possibilities for a vCPU polarity: + +- Horizontal +- Horizontal dedicated +- Vertical low +- Vertical medium +- Vertical high +- Vertical high dedicated + +Example of the event received when the guest issues PTF(0) to request +an horizontal polarity: + +.. code-block:: QMP + + <- { "event": "POLARITY_CHANGE", + "data": { "polarity": 0 }, + "timestamp": { "seconds": 1401385907, "microseconds": 422329 } } + + diff --git a/docs/system/target-s390x.rst b/docs/system/target-s390x.rst index c636f64113..ff0ffe04f3 100644 --- a/docs/system/target-s390x.rst +++ b/docs/system/target-s390x.rst @@ -33,3 +33,4 @@ Architectural features .. toctree:: s390x/bootdevices s390x/protvirt + s390x/cpu-topology