From patchwork Tue Aug 22 12:23:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Henrique Barboza X-Patchwork-Id: 9915049 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E4A55603FA for ; Tue, 22 Aug 2017 12:26:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D374F28791 for ; Tue, 22 Aug 2017 12:26:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C7938287EC; Tue, 22 Aug 2017 12:26:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 09E8728791 for ; Tue, 22 Aug 2017 12:26:02 +0000 (UTC) Received: from localhost ([::1]:41417 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dk8FK-000777-CY for patchwork-qemu-devel@patchwork.kernel.org; Tue, 22 Aug 2017 08:25:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38737) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dk8E6-000740-Po for qemu-devel@nongnu.org; Tue, 22 Aug 2017 08:24:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dk8E5-0006gi-8Z for qemu-devel@nongnu.org; Tue, 22 Aug 2017 08:24:10 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:43795 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dk8E5-0006gV-3y for qemu-devel@nongnu.org; Tue, 22 Aug 2017 08:24:09 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v7MCO7j4083652 for ; Tue, 22 Aug 2017 08:24:08 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0a-001b2d01.pphosted.com with ESMTP id 2cgjmk8n2w-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 22 Aug 2017 08:24:07 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 22 Aug 2017 08:23:52 -0400 Received: from b01cxnp22033.gho.pok.ibm.com (9.57.198.23) by e12.ny.us.ibm.com (146.89.104.199) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 22 Aug 2017 08:23:49 -0400 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v7MCNncv24445030; Tue, 22 Aug 2017 12:23:49 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 794E7112051; Tue, 22 Aug 2017 08:23:37 -0400 (EDT) Received: from localhost.localdomain (unknown [9.80.204.221]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP id ECF7C112047; Tue, 22 Aug 2017 08:23:35 -0400 (EDT) From: Daniel Henrique Barboza To: qemu-devel@nongnu.org Date: Tue, 22 Aug 2017 09:23:27 -0300 X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170822122327.9567-1-danielhb@linux.vnet.ibm.com> References: <20170822122327.9567-1-danielhb@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17082212-0048-0000-0000-000001D7415B X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007591; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000223; SDB=6.00906003; UDB=6.00454055; IPR=6.00686206; BA=6.00005549; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016810; XFM=3.00000015; UTC=2017-08-22 12:23:51 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17082212-0049-0000-0000-000042503B8E Message-Id: <20170822122327.9567-2-danielhb@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-08-22_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1708220192 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v2] target/ppc: 'PVR != host PVR' in KVM_SET_SREGS workaround X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, david@gibson.dropbear.id.au Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Commit d5fc133eed ("ppc: Rework CPU compatibility testing across migration") changed the way cpu_post_load behaves with the PVR setting, causing an unexpected bug in KVM-HV migrations between hosts that are compatible (POWER8 and POWER8E, for example). Even with pvr_match() returning true, the guest freezes right after cpu_post_load. The reason is that the guest kernel can't handle a different PVR value other that the running host in KVM_SET_SREGS. In [1] it was discussed the possibility of a new KVM capability that would indicate that the guest kernel can handle a different PVR in KVM_SET_SREGS. Even if such feature is implemented, there is still the problem with older kernels that will not have this capability and will fail to migrate. This patch implements a workaround for that scenario. If running with KVM, check if the guest kernel does not have the capability (named here as 'cap_ppc_pvr_compat'). If it doesn't, calls kvmppc_is_pr() to see if the guest is running in KVM-HV. If all this happens, set env->spr[SPR_PVR] to the same value as the current host PVR. This ensures that we allow migrations with 'close enough' PVRs to still work in KVM-HV but also makes the code ready for this new KVM capability when it is done. A new function called 'kvmppc_pvr_workaround_required' was created to encapsulate the conditions said above and to avoid calling too many kvm.c internals inside cpu_post_load. [1] https://lists.gnu.org/archive/html/qemu-ppc/2017-06/msg00503.html Signed-off-by: Daniel Henrique Barboza --- target/ppc/kvm.c | 34 ++++++++++++++++++++++++++++++++++ target/ppc/kvm_ppc.h | 1 + target/ppc/machine.c | 22 ++++++++++++++++++++++ 3 files changed, 57 insertions(+) diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index 8571379..ab903e5 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -90,6 +90,7 @@ static int cap_htm; /* Hardware transactional memory support */ static int cap_mmu_radix; static int cap_mmu_hash_v3; static int cap_resize_hpt; +static int cap_ppc_pvr_compat; static uint32_t debug_inst_opcode; @@ -147,6 +148,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_mmu_radix = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_RADIX); cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3); cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT); + /* + * Note: setting it to false because there is not such capability + * in KVM at this moment. + * + * TODO: call kvm_vm_check_extension() with the right capability + * after the kernel starts implementing it.*/ + cap_ppc_pvr_compat = false; if (!cap_interrupt_level) { fprintf(stderr, "KVM: Couldn't find level irq capability. Expect the " @@ -2785,3 +2793,29 @@ void kvmppc_update_sdr1(target_ulong sdr1) run_on_cpu(cs, kvmppc_pivot_hpt_cpu, RUN_ON_CPU_TARGET_PTR(sdr1)); } } + +/* + * This is a helper function to detect a post migration scenario + * in which a guest, running as KVM-HV, freezes in cpu_post_load because + * the guest kernel can't handle a PVR value other than the actual host + * PVR in KVM_SET_SREGS, even if pvr_match() returns true. + * + * If we don't have cap_ppc_pvr_compat and we're not running in PR + * (so, we're HV), return true. The workaround itself is done in + * cpu_post_load. + * + * The order here is important: we'll only check for KVM PR as a + * fallback if the guest kernel can't handle the situation itself. + * We need to avoid as much as possible querying the running KVM type + * in QEMU level. + */ +bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu) +{ + CPUState *cs = CPU(cpu); + + if (cap_ppc_pvr_compat) { + return false; + } + + return kvm_enabled() && !kvmppc_is_pr(cs->kvm_state); +} diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h index 6bc6fb3..381afe6 100644 --- a/target/ppc/kvm_ppc.h +++ b/target/ppc/kvm_ppc.h @@ -67,6 +67,7 @@ void kvmppc_check_papr_resize_hpt(Error **errp); int kvmppc_resize_hpt_prepare(PowerPCCPU *cpu, target_ulong flags, int shift); int kvmppc_resize_hpt_commit(PowerPCCPU *cpu, target_ulong flags, int shift); void kvmppc_update_sdr1(target_ulong sdr1); +bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu); bool kvmppc_is_mem_backend_page_size_ok(const char *obj_path); diff --git a/target/ppc/machine.c b/target/ppc/machine.c index abe0a1c..e36b710 100644 --- a/target/ppc/machine.c +++ b/target/ppc/machine.c @@ -9,6 +9,7 @@ #include "mmu-hash64.h" #include "migration/cpu.h" #include "qapi/error.h" +#include "kvm_ppc.h" static int cpu_load_old(QEMUFile *f, void *opaque, int version_id) { @@ -249,6 +250,27 @@ static int cpu_post_load(void *opaque, int version_id) } } + /* + * If we're running with KVM HV, there is a chance that the guest + * is running with KVM HV and its kernel does not have the + * capability of dealing with a different PVR other than this + * exact host PVR in KVM_SET_SREGS. If that happens, the + * guest freezes after migration. + * + * The function kvmppc_pvr_workaround_required does this verification + * by first checking if the kernel has the cap, returning true immediately + * if that is the case. Otherwise, it checks if we're running in KVM PR. + * If the guest kernel does not have the cap and we're not running KVM-PR + * (so, it is running KVM-HV), we need to ensure that KVM_SET_SREGS will + * receive the PVR it expects as a workaround. + * + */ +#if defined(CONFIG_KVM) + if (kvmppc_pvr_workaround_required(cpu)) { + env->spr[SPR_PVR] = env->spr_cb[SPR_PVR].default_value; + } +#endif + env->lr = env->spr[SPR_LR]; env->ctr = env->spr[SPR_CTR]; cpu_write_xer(env, env->spr[SPR_XER]);