From patchwork Thu Feb 24 18:58:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 12759141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C4C4C433F5 for ; Thu, 24 Feb 2022 19:04:26 +0000 (UTC) Received: from localhost ([::1]:53418 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nNJPd-0006JP-5L for qemu-devel@archiver.kernel.org; Thu, 24 Feb 2022 14:04:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55998) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJK3-0005cu-ML; Thu, 24 Feb 2022 13:58:43 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:50666) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJK1-0006Q3-7p; Thu, 24 Feb 2022 13:58:39 -0500 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21OIGYUv010089; Thu, 24 Feb 2022 18:58:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=iTMRQi/iweOfP4GiDnHB9q6xHW8rySQdndGuWXp0vtQ=; b=ZCeXYk2lQgGepFGx3Ls32rbKwvRNgvN8iVquyPodnP+zqSYUsD/6jPKsmExUPcZ8NjQK 04kNzfZR7OHN5qsRdgcyDR9/t2qU+3PufWY77LaaYc8TV/9esl2iYrZCgV4YpGzlNAyI qWRKi1Bf7g3sgoZBld5qD+uq0g0h5zzIe8MKLeeZXtwCP88WitamMQb6ZRQVH3H3BMcN Eda/Trhc+NIM776bod5DVELpyEOPquR/T90NV0o2KUXAX/F0zNcC8W1RlIjffUaA5ik7 TPOPT4SXxURmQkaRpa+G1kubj0i8KRGyVU7cAMGCRnIjXTVmB7K1pnUx68WMDinJqIEn gg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edq0k17hb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:30 +0000 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 21OIkS0T002199; Thu, 24 Feb 2022 18:58:29 GMT Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edq0k17h1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:29 +0000 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 21OIcYVd017175; Thu, 24 Feb 2022 18:58:28 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma05wdc.us.ibm.com with ESMTP id 3edr92v7ws-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:28 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 21OIwRqu30277946 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Feb 2022 18:58:27 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7BF7CC6055; Thu, 24 Feb 2022 18:58:27 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9930AC6065; Thu, 24 Feb 2022 18:58:25 +0000 (GMT) Received: from farosas.linux.ibm.com.com (unknown [9.211.135.43]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 24 Feb 2022 18:58:25 +0000 (GMT) From: Fabiano Rosas To: qemu-devel@nongnu.org Subject: [RFC PATCH 1/4] target/ppc: TCG: Migrate tb_offset and decr Date: Thu, 24 Feb 2022 15:58:14 -0300 Message-Id: <20220224185817.2207228-2-farosas@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220224185817.2207228-1-farosas@linux.ibm.com> References: <20220224185817.2207228-1-farosas@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 8Rk_D-S0RRxv-yGdPSEAFjOrBhOMjan1 X-Proofpoint-GUID: d0uEoR33GiE0mVkOwmm-3QXo0mNYtsr3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-24_03,2022-02-24_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 priorityscore=1501 phishscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 bulkscore=0 clxscore=1015 mlxscore=0 impostorscore=0 spamscore=0 mlxlogscore=955 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202240104 Received-SPF: pass client-ip=148.163.156.1; envelope-from=farosas@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, danielhb413@gmail.com, npiggin@gmail.com, qemu-ppc@nongnu.org, clg@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" These two were not migrated so the remote end was starting with the decrementer expired. I am seeing less frequent crashes with this patch (tested with -smp 4 and -smp 32). It certainly doesn't fix all issues but it looks like it helps. Signed-off-by: Fabiano Rosas --- target/ppc/machine.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/target/ppc/machine.c b/target/ppc/machine.c index 1b63146ed1..7ee1984500 100644 --- a/target/ppc/machine.c +++ b/target/ppc/machine.c @@ -9,6 +9,7 @@ #include "qemu/main-loop.h" #include "kvm_ppc.h" #include "power8-pmu.h" +#include "hw/ppc/ppc.h" static void post_load_update_msr(CPUPPCState *env) { @@ -666,6 +667,18 @@ static const VMStateDescription vmstate_compat = { } }; +static const VMStateDescription vmstate_tb_env = { + .name = "cpu/env/tb_env", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_INT64(tb_offset, ppc_tb_t), + VMSTATE_UINT64(decr_next, ppc_tb_t), + VMSTATE_TIMER_PTR(decr_timer, ppc_tb_t), + VMSTATE_END_OF_LIST() + } +}; + const VMStateDescription vmstate_ppc_cpu = { .name = "cpu", .version_id = 5, @@ -696,12 +709,16 @@ const VMStateDescription vmstate_ppc_cpu = { /* Backward compatible internal state */ VMSTATE_UINTTL(env.hflags_compat_nmsr, PowerPCCPU), + VMSTATE_STRUCT_POINTER_V(env.tb_env, PowerPCCPU, 1, + vmstate_tb_env, ppc_tb_t), + /* Sanity checking */ VMSTATE_UINTTL_TEST(mig_msr_mask, PowerPCCPU, cpu_pre_2_8_migration), VMSTATE_UINT64_TEST(mig_insns_flags, PowerPCCPU, cpu_pre_2_8_migration), VMSTATE_UINT64_TEST(mig_insns_flags2, PowerPCCPU, cpu_pre_2_8_migration), VMSTATE_UINT32_TEST(mig_nb_BATs, PowerPCCPU, cpu_pre_2_8_migration), + VMSTATE_END_OF_LIST() }, .subsections = (const VMStateDescription*[]) { From patchwork Thu Feb 24 18:58:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 12759143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DE5AC433F5 for ; Thu, 24 Feb 2022 19:08:43 +0000 (UTC) Received: from localhost ([::1]:36496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nNJTm-0005ZY-1F for qemu-devel@archiver.kernel.org; Thu, 24 Feb 2022 14:08:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJKA-0005kv-UD; Thu, 24 Feb 2022 13:58:47 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:53640) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJK8-0006RY-Rp; Thu, 24 Feb 2022 13:58:46 -0500 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21OIa5a8016542; Thu, 24 Feb 2022 18:58:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=+OLMEFlc9ii3IEAQZh3JSePrf8F+Yh0cx/2Hzq4ols0=; b=JctTP9ec2Nm90hCF1JC8zeT/EpnKypTzwhh072yecf/Ewz5l23Pi7In2SUBX4wshXnp8 R2PYUPREL7+MebmDMKtKPo/x+AFsZL0QmJaf+MIWX6N2or4F0pJUxFJp2yevWYQgx2Kq w/KfFeEZRaM97vcSRJn++vqDwMzMQJGAnCv59+vTnY6OGkKEjhlsaKmqTm/ZSgF0dxhL p8SZNR/of3lPGwmtE0lpuOsizTFA0oBFtZi2FUYilu5cFbus/N7JgGd3BZEhq0mxHj4t rA8l6Y5Cct20rxwZBL9pPnzu0kc25qx10hg+9+KcgNs6MWONGggLCn5x/mQaz6+X2SRe KQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edxfe6x4b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:32 +0000 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 21OIwVeP029300; Thu, 24 Feb 2022 18:58:31 GMT Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 3edxfe6x43-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:31 +0000 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 21OIcWsK025980; Thu, 24 Feb 2022 18:58:31 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma03dal.us.ibm.com with ESMTP id 3ear6cn1fk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:30 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 21OIwT1e33095978 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Feb 2022 18:58:29 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BF275C605D; Thu, 24 Feb 2022 18:58:29 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF693C6057; Thu, 24 Feb 2022 18:58:27 +0000 (GMT) Received: from farosas.linux.ibm.com.com (unknown [9.211.135.43]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 24 Feb 2022 18:58:27 +0000 (GMT) From: Fabiano Rosas To: qemu-devel@nongnu.org Subject: [RFC PATCH 2/4] spapr: TCG: Migrate spapr_cpu->prod Date: Thu, 24 Feb 2022 15:58:15 -0300 Message-Id: <20220224185817.2207228-3-farosas@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220224185817.2207228-1-farosas@linux.ibm.com> References: <20220224185817.2207228-1-farosas@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: YpqVO1nrTLv2S5IFcy-ApujKvVlcmoM7 X-Proofpoint-GUID: PnM7HAdCJBrsUvygqat_MaT4pTO2UkeK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-24_04,2022-02-24_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 priorityscore=1501 phishscore=0 bulkscore=0 mlxscore=0 adultscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202240104 Received-SPF: pass client-ip=148.163.158.5; envelope-from=farosas@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, danielhb413@gmail.com, npiggin@gmail.com, qemu-ppc@nongnu.org, clg@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" I'm seeing some stack traces in the migrated guest going through cede and some hangs at the plpar_hcall_norets so let's make sure everything related to cede/prod is being migrated just in case. Signed-off-by: Fabiano Rosas --- hw/ppc/spapr_cpu_core.c | 1 + include/hw/ppc/spapr_cpu_core.h | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index ed84713960..efda7730f1 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -179,6 +179,7 @@ static const VMStateDescription vmstate_spapr_cpu_state = { .version_id = 1, .minimum_version_id = 1, .fields = (VMStateField[]) { + VMSTATE_BOOL(prod, SpaprCpuState), VMSTATE_END_OF_LIST() }, .subsections = (const VMStateDescription * []) { diff --git a/include/hw/ppc/spapr_cpu_core.h b/include/hw/ppc/spapr_cpu_core.h index b560514560..2772689c84 100644 --- a/include/hw/ppc/spapr_cpu_core.h +++ b/include/hw/ppc/spapr_cpu_core.h @@ -45,7 +45,7 @@ typedef struct SpaprCpuState { uint64_t vpa_addr; uint64_t slb_shadow_addr, slb_shadow_size; uint64_t dtl_addr, dtl_size; - bool prod; /* not migrated, only used to improve dispatch latencies */ + bool prod; struct ICPState *icp; struct XiveTCTX *tctx; From patchwork Thu Feb 24 18:58:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 12759148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 05B58C433F5 for ; Thu, 24 Feb 2022 19:14:26 +0000 (UTC) Received: from localhost ([::1]:43658 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nNJZI-0002Bw-RI for qemu-devel@archiver.kernel.org; Thu, 24 Feb 2022 14:14:24 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56036) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJKA-0005jE-E8; Thu, 24 Feb 2022 13:58:46 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:15666 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJK8-0006Rd-Rq; Thu, 24 Feb 2022 13:58:46 -0500 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21OHjhhc015271; Thu, 24 Feb 2022 18:58:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=c06R9rhorprusNcGnTAXki4aKSw3B+gv2z7d7ctL11A=; b=kAHO829l+Wtyh9/0gjbw/R+abG/GD8oNIXUXGynZbse+yNPxV4NXW7qKfRgK+CH7m0mu pUUCVkkz5MLKi/XNd37VRqM0p5lveGhEQDpipF5vRnQylNR7vLveVIECkYR0hp3crUFV fJCdAf/3joKEtxTPadPbCyMgibQpyX3sbFnw6UjIPZWlvXszSILUkvSPdcUACGUQnmoI kQ4D9zj4mSI+5sYachPz/QajyT9XOQVMxbRJcjdSowBTQPTcevG+vaZvL7rh19XeH819 ZjHSy2zY+iKndWwToNwjM5l91qdHm7PkyJFtPJeh36lQTwSAeNbexHDThyyzsIvuFyeA vQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3edw51qpy7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:33 +0000 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 21OIrF0h015429; Thu, 24 Feb 2022 18:58:33 GMT Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 3edw51qpy2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:33 +0000 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 21OIcY4j017180; Thu, 24 Feb 2022 18:58:33 GMT Received: from b03cxnp07027.gho.boulder.ibm.com (b03cxnp07027.gho.boulder.ibm.com [9.17.130.14]) by ppma05wdc.us.ibm.com with ESMTP id 3edr92v7xc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:32 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp07027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 21OIwWJk29491538 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Feb 2022 18:58:32 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1C4A0C605F; Thu, 24 Feb 2022 18:58:32 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2928EC6057; Thu, 24 Feb 2022 18:58:30 +0000 (GMT) Received: from farosas.linux.ibm.com.com (unknown [9.211.135.43]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 24 Feb 2022 18:58:29 +0000 (GMT) From: Fabiano Rosas To: qemu-devel@nongnu.org Subject: [RFC PATCH 3/4] hw/ppc: Take nested guest into account when saving timebase Date: Thu, 24 Feb 2022 15:58:16 -0300 Message-Id: <20220224185817.2207228-4-farosas@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220224185817.2207228-1-farosas@linux.ibm.com> References: <20220224185817.2207228-1-farosas@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: skjoQof9lLVBf7CDYVTp76dVhvVN3rru X-Proofpoint-ORIG-GUID: 5k4RBRVCMLsGKhwInqWUa3BbSSM6KU0i X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-24_04,2022-02-24_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 suspectscore=0 priorityscore=1501 lowpriorityscore=0 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 malwarescore=0 phishscore=0 impostorscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202240102 Received-SPF: pass client-ip=148.163.158.5; envelope-from=farosas@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, danielhb413@gmail.com, npiggin@gmail.com, qemu-ppc@nongnu.org, clg@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When saving the guest "timebase" we look to the first_cpu for its tb_offset. If that CPU happens to be running a nested guest at this time, the tb_offset will have the nested guest value. This was caught by code inspection. Signed-off-by: Fabiano Rosas --- hw/ppc/ppc.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/hw/ppc/ppc.c b/hw/ppc/ppc.c index 9e99625ea9..093cd87014 100644 --- a/hw/ppc/ppc.c +++ b/hw/ppc/ppc.c @@ -36,6 +36,7 @@ #include "kvm_ppc.h" #include "migration/vmstate.h" #include "trace.h" +#include "hw/ppc/spapr_cpu_core.h" static void cpu_ppc_tb_stop (CPUPPCState *env); static void cpu_ppc_tb_start (CPUPPCState *env); @@ -961,19 +962,33 @@ static void timebase_save(PPCTimebase *tb) { uint64_t ticks = cpu_get_host_ticks(); PowerPCCPU *first_ppc_cpu = POWERPC_CPU(first_cpu); + int64_t tb_offset; if (!first_ppc_cpu->env.tb_env) { error_report("No timebase object"); return; } + tb_offset = first_ppc_cpu->env.tb_env->tb_offset; + + if (first_ppc_cpu->vhyp && vhyp_cpu_in_nested(first_ppc_cpu)) { + SpaprCpuState *spapr_cpu = spapr_cpu_state(first_ppc_cpu); + + /* + * If the first_cpu happens to be running a nested guest at + * this time, tb_env->tb_offset will contain the nested guest + * offset. + */ + tb_offset -= spapr_cpu->nested_tb_offset; + } + /* not used anymore, we keep it for compatibility */ tb->time_of_the_day_ns = qemu_clock_get_ns(QEMU_CLOCK_HOST); /* * tb_offset is only expected to be changed by QEMU so * there is no need to update it from KVM here */ - tb->guest_timebase = ticks + first_ppc_cpu->env.tb_env->tb_offset; + tb->guest_timebase = ticks + tb_offset; tb->runstate_paused = runstate_check(RUN_STATE_PAUSED) || runstate_check(RUN_STATE_SAVE_VM); From patchwork Thu Feb 24 18:58:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabiano Rosas X-Patchwork-Id: 12759161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 623D2C433FE for ; Thu, 24 Feb 2022 19:19:35 +0000 (UTC) Received: from localhost ([::1]:52034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nNJeI-00086i-Gd for qemu-devel@archiver.kernel.org; Thu, 24 Feb 2022 14:19:34 -0500 Received: from eggs.gnu.org ([209.51.188.92]:56062) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJKF-0005v4-BA; Thu, 24 Feb 2022 13:58:51 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:59284) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nNJKD-0006SD-9M; Thu, 24 Feb 2022 13:58:51 -0500 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21OHq2ep030326; Thu, 24 Feb 2022 18:58:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=hB7YXwzh4Tx+sRdVNmT/eGf3ri1RtthXBaISHOCZab0=; b=iVEtodhpc7rPAaSghL3tHgtVSI8Cnetp+9PFDaYUlm9yuLwcNK5+zbZ1f3635UVy1ib0 cVSfATVA+2iu2Kn4gAkafLo3IwL0R5khFhtfrZAsoXFih5AhV8eguLvKQYzWXDUAlB+X uQlOIGZWVecTGG+RqKSMbEIp5RiA3zrs2G1n38OVRwDrIeIF6Qvy/AXTmtoXb3loAn6L sh/rmh5/e6Mav6xFOaEhwUqP5ZRh639HOqRVwO/L95Xm7PHftzkufHdIzuCaJpf8TSss C86wYqabnovLsGsb+R59C85eA6IM/8LN9NA3Ne7zqTpvCItgXzdjYSCdqZMDReQKko05 vw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3eds79wum3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:37 +0000 Received: from m0098417.ppops.net (m0098417.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 21OIfQLS008800; Thu, 24 Feb 2022 18:58:36 GMT Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com with ESMTP id 3eds79wukr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:36 +0000 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 21OIcaw6016649; Thu, 24 Feb 2022 18:58:36 GMT Received: from b03cxnp07028.gho.boulder.ibm.com (b03cxnp07028.gho.boulder.ibm.com [9.17.130.15]) by ppma02dal.us.ibm.com with ESMTP id 3ear6bw928-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Feb 2022 18:58:35 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 21OIwY6g29819292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 24 Feb 2022 18:58:34 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6A7A8C6065; Thu, 24 Feb 2022 18:58:34 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 804B3C605F; Thu, 24 Feb 2022 18:58:32 +0000 (GMT) Received: from farosas.linux.ibm.com.com (unknown [9.211.135.43]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 24 Feb 2022 18:58:32 +0000 (GMT) From: Fabiano Rosas To: qemu-devel@nongnu.org Subject: [RFC PATCH 4/4] spapr: Add KVM-on-TCG migration support Date: Thu, 24 Feb 2022 15:58:17 -0300 Message-Id: <20220224185817.2207228-5-farosas@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220224185817.2207228-1-farosas@linux.ibm.com> References: <20220224185817.2207228-1-farosas@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: n3GCtEqbhqQSRzhkmoAej0M0o3bgYk0K X-Proofpoint-ORIG-GUID: IW7OpOQKknPGTNBdPVcS8J_koaZMwxl3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.64.514 definitions=2022-02-24_03,2022-02-24_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 impostorscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202240104 Received-SPF: pass client-ip=148.163.158.5; envelope-from=farosas@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, danielhb413@gmail.com, npiggin@gmail.com, qemu-ppc@nongnu.org, clg@kaod.org, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This adds migration support for TCG pseries machines running a KVM-HV guest. The state that needs to be migrated is: - the nested PTCR value; - the in_nested flag; - the nested_tb_offset. - the saved host CPUPPCState structure; Signed-off-by: Fabiano Rosas --- (this migrates just fine with L2 running stress and 1 VCPU in L1. With 32 VCPUs in L1 there's crashes which I still don't understand. They might be related to L1 migration being flaky right now) --- hw/ppc/spapr.c | 19 +++++++++++ hw/ppc/spapr_cpu_core.c | 76 +++++++++++++++++++++++++++++++++++++++++ target/ppc/machine.c | 44 ++++++++++++++++++++++++ 3 files changed, 139 insertions(+) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index f0b75b22bb..6e87c515db 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1934,6 +1934,13 @@ static bool spapr_patb_entry_needed(void *opaque) return !!spapr->patb_entry; } +static bool spapr_nested_ptcr_needed(void *opaque) +{ + SpaprMachineState *spapr = opaque; + + return !!spapr->nested_ptcr; +} + static const VMStateDescription vmstate_spapr_patb_entry = { .name = "spapr_patb_entry", .version_id = 1, @@ -1945,6 +1952,17 @@ static const VMStateDescription vmstate_spapr_patb_entry = { }, }; +static const VMStateDescription vmstate_spapr_nested_ptcr = { + .name = "spapr_nested_ptcr", + .version_id = 1, + .minimum_version_id = 1, + .needed = spapr_nested_ptcr_needed, + .fields = (VMStateField[]) { + VMSTATE_UINT64(nested_ptcr, SpaprMachineState), + VMSTATE_END_OF_LIST() + }, +}; + static bool spapr_irq_map_needed(void *opaque) { SpaprMachineState *spapr = opaque; @@ -2069,6 +2087,7 @@ static const VMStateDescription vmstate_spapr = { &vmstate_spapr_cap_fwnmi, &vmstate_spapr_fwnmi, &vmstate_spapr_cap_rpt_invalidate, + &vmstate_spapr_nested_ptcr, NULL } }; diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c index efda7730f1..3ec13c0660 100644 --- a/hw/ppc/spapr_cpu_core.c +++ b/hw/ppc/spapr_cpu_core.c @@ -25,6 +25,7 @@ #include "sysemu/reset.h" #include "sysemu/hw_accel.h" #include "qemu/error-report.h" +#include "migration/cpu.h" static void spapr_reset_vcpu(PowerPCCPU *cpu) { @@ -174,6 +175,80 @@ static const VMStateDescription vmstate_spapr_cpu_vpa = { } }; +static bool nested_needed(void *opaque) +{ + SpaprCpuState *spapr_cpu = opaque; + + return spapr_cpu->in_nested; +} + +static int nested_state_pre_save(void *opaque) +{ + CPUPPCState *env = opaque; + + env->spr[SPR_LR] = env->lr; + env->spr[SPR_CTR] = env->ctr; + env->spr[SPR_XER] = cpu_read_xer(env); + env->spr[SPR_CFAR] = env->cfar; + + return 0; +} + +static int nested_state_post_load(void *opaque, int version_id) +{ + CPUPPCState *env = opaque; + + env->lr = env->spr[SPR_LR]; + env->ctr = env->spr[SPR_CTR]; + cpu_write_xer(env, env->spr[SPR_XER]); + env->cfar = env->spr[SPR_CFAR]; + + return 0; +} + +static const VMStateDescription vmstate_nested_host_state = { + .name = "spapr_nested_host_state", + .version_id = 1, + .minimum_version_id = 1, + .pre_save = nested_state_pre_save, + .post_load = nested_state_post_load, + .fields = (VMStateField[]) { + VMSTATE_UINTTL_ARRAY(gpr, CPUPPCState, 32), + VMSTATE_UINTTL_ARRAY(spr, CPUPPCState, 1024), + VMSTATE_UINT32_ARRAY(crf, CPUPPCState, 8), + VMSTATE_UINTTL(nip, CPUPPCState), + VMSTATE_UINTTL(msr, CPUPPCState), + VMSTATE_END_OF_LIST() + } +}; + +static int nested_cpu_pre_load(void *opaque) +{ + SpaprCpuState *spapr_cpu = opaque; + + spapr_cpu->nested_host_state = g_try_malloc(sizeof(CPUPPCState)); + if (!spapr_cpu->nested_host_state) { + return -1; + } + + return 0; +} + +static const VMStateDescription vmstate_spapr_cpu_nested = { + .name = "spapr_cpu/nested", + .version_id = 1, + .minimum_version_id = 1, + .needed = nested_needed, + .pre_load = nested_cpu_pre_load, + .fields = (VMStateField[]) { + VMSTATE_BOOL(in_nested, SpaprCpuState), + VMSTATE_INT64(nested_tb_offset, SpaprCpuState), + VMSTATE_STRUCT_POINTER_V(nested_host_state, SpaprCpuState, 1, + vmstate_nested_host_state, CPUPPCState), + VMSTATE_END_OF_LIST() + }, +}; + static const VMStateDescription vmstate_spapr_cpu_state = { .name = "spapr_cpu", .version_id = 1, @@ -184,6 +259,7 @@ static const VMStateDescription vmstate_spapr_cpu_state = { }, .subsections = (const VMStateDescription * []) { &vmstate_spapr_cpu_vpa, + &vmstate_spapr_cpu_nested, NULL } }; diff --git a/target/ppc/machine.c b/target/ppc/machine.c index 7ee1984500..ae09b1bcfe 100644 --- a/target/ppc/machine.c +++ b/target/ppc/machine.c @@ -10,6 +10,7 @@ #include "kvm_ppc.h" #include "power8-pmu.h" #include "hw/ppc/ppc.h" +#include "hw/ppc/spapr_cpu_core.h" static void post_load_update_msr(CPUPPCState *env) { @@ -679,6 +680,48 @@ static const VMStateDescription vmstate_tb_env = { } }; +static const VMStateDescription vmstate_hdecr = { + .name = "cpu/hdecr", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT64(hdecr_next, ppc_tb_t), + VMSTATE_TIMER_PTR(hdecr_timer, ppc_tb_t), + VMSTATE_END_OF_LIST() + } +}; + +static bool nested_needed(void *opaque) +{ + PowerPCCPU *cpu = opaque; + SpaprCpuState *spapr_cpu = spapr_cpu_state(cpu); + + return spapr_cpu->in_nested; +} + +static int nested_pre_load(void *opaque) +{ + PowerPCCPU *cpu = opaque; + CPUPPCState *env = &cpu->env; + + cpu_ppc_hdecr_init(env); + + return 0; +} + +static const VMStateDescription vmstate_nested = { + .name = "cpu/nested-guest", + .version_id = 1, + .minimum_version_id = 1, + .needed = nested_needed, + .pre_load = nested_pre_load, + .fields = (VMStateField[]) { + VMSTATE_STRUCT_POINTER_V(env.tb_env, PowerPCCPU, 1, + vmstate_hdecr, ppc_tb_t), + VMSTATE_END_OF_LIST() + } +}; + const VMStateDescription vmstate_ppc_cpu = { .name = "cpu", .version_id = 5, @@ -734,6 +777,7 @@ const VMStateDescription vmstate_ppc_cpu = { &vmstate_tlbemb, &vmstate_tlbmas, &vmstate_compat, + &vmstate_nested, NULL } };