From patchwork Wed Aug 5 15:47:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Greg Kurz X-Patchwork-Id: 11701927 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82832913 for ; Wed, 5 Aug 2020 15:48:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 05E662312B for ; Wed, 5 Aug 2020 15:48:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05E662312B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:40616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k3Le6-0001IU-Pl for patchwork-qemu-devel@patchwork.kernel.org; Wed, 05 Aug 2020 11:48:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39892) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k3Ldc-0000mb-Kj for qemu-devel@nongnu.org; Wed, 05 Aug 2020 11:47:32 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:31293 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k3Lda-0004rV-HK for qemu-devel@nongnu.org; Wed, 05 Aug 2020 11:47:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-6Ex9XWPnMhmFtzCc59JbqQ-1; Wed, 05 Aug 2020 11:47:20 -0400 X-MC-Unique: 6Ex9XWPnMhmFtzCc59JbqQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C1DAD57; Wed, 5 Aug 2020 15:47:18 +0000 (UTC) Received: from bahia.lan (ovpn-112-38.ams2.redhat.com [10.36.112.38]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B66788D78; Wed, 5 Aug 2020 15:47:16 +0000 (UTC) Subject: [PATCH] spapr: Clarify error and documentation for broken KVM XICS From: Greg Kurz To: David Gibson Date: Wed, 05 Aug 2020 17:47:16 +0200 Message-ID: <159664243614.622889.18307368735989783528.stgit@bahia.lan> User-Agent: StGit/0.21 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kaod.org Received-SPF: softfail client-ip=205.139.110.120; envelope-from=groug@kaod.org; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/05 00:45:55 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -12 X-Spam_score: -1.3 X-Spam_bar: - X-Spam_report: (-1.3 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Satheesh Rajendran , qemu-ppc@nongnu.org, =?utf-8?q?C=C3=A9dric_Le_Goater?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When starting an L2 KVM guest with `ic-mode=dual,kernel-irqchip=on`, QEMU fails with: KVM is too old to support ic-mode=dual,kernel-irqchip=on This error message was introduced to detect older KVM versions that didn't allow destruction and re-creation of the XICS KVM device that we do at reboot. But it is actually the same issue that we get with nested guests : when running under pseries, KVM currently provides a genuine XICS device (not the XICS-on-XIVE device that we get under powernv) which doesn't support destruction/re-creation. This will eventually be fixed in KVM but in the meantime, update the error message and documentation to mention the nested case. While here, mention that in "No XIVE support in KVM" section that this can also happen with "guest OSes supporting XIVE" since we check this at init time before starting the guest. Reported-by: Satheesh Rajendran Buglink: https://bugs.launchpad.net/qemu/+bug/1890290 Signed-off-by: Greg Kurz Reviewed-by: Cédric Le Goater --- docs/specs/ppc-spapr-xive.rst | 5 ++++- hw/ppc/spapr_irq.c | 12 +++++++++--- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst index 7199db730b82..7144347560f1 100644 --- a/docs/specs/ppc-spapr-xive.rst +++ b/docs/specs/ppc-spapr-xive.rst @@ -126,6 +126,9 @@ xics XICS KVM XICS emul. XICS KVM (1) QEMU warns with ``warning: kernel_irqchip requested but unavailable: IRQ_XIVE capability must be present for KVM`` + In some cases (old host kernels or KVM nested guests), one may hit a + QEMU/KVM incompatibility due to device destruction in reset. QEMU fails + with ``KVM is incompatible with ic-mode=dual,kernel-irqchip=on`` (2) QEMU fails with ``kernel_irqchip requested but unavailable: IRQ_XIVE capability must be present for KVM`` @@ -148,7 +151,7 @@ xics XICS KVM XICS emul. XICS KVM mode (XICS), either don't set the ic-mode machine property or try ic-mode=xics or ic-mode=dual`` (4) QEMU/KVM incompatibility due to device destruction in reset. QEMU fails - with ``KVM is too old to support ic-mode=dual,kernel-irqchip=on`` + with ``KVM is incompatible with ic-mode=dual,kernel-irqchip=on`` XIVE Device tree properties diff --git a/hw/ppc/spapr_irq.c b/hw/ppc/spapr_irq.c index 2f8f7d62f875..72bb938375ef 100644 --- a/hw/ppc/spapr_irq.c +++ b/hw/ppc/spapr_irq.c @@ -139,6 +139,7 @@ SpaprIrq spapr_irq_dual = { static int spapr_irq_check(SpaprMachineState *spapr, Error **errp) { + ERRP_GUARD(); MachineState *machine = MACHINE(spapr); /* @@ -179,14 +180,19 @@ static int spapr_irq_check(SpaprMachineState *spapr, Error **errp) /* * On a POWER9 host, some older KVM XICS devices cannot be destroyed and - * re-created. Detect that early to avoid QEMU to exit later when the - * guest reboots. + * re-created. Same happens with KVM nested guests. Detect that early to + * avoid QEMU to exit later when the guest reboots. */ if (kvm_enabled() && spapr->irq == &spapr_irq_dual && kvm_kernel_irqchip_required() && xics_kvm_has_broken_disconnect(spapr)) { - error_setg(errp, "KVM is too old to support ic-mode=dual,kernel-irqchip=on"); + error_setg(errp, + "KVM is incompatible with ic-mode=dual,kernel-irqchip=on"); + error_append_hint(errp, + "This can happen with an old KVM or in a KVM nested guest.\n"); + error_append_hint(errp, + "Try without kernel-irqchip or with kernel-irqchip=off.\n"); return -1; }