From patchwork Fri Nov 20 17:46:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kurz X-Patchwork-Id: 11921457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7D97C56201 for ; Fri, 20 Nov 2020 17:53:07 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 669712240B for ; Fri, 20 Nov 2020 17:53:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 669712240B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:50212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kgAan-0006qj-Uq for qemu-devel@archiver.kernel.org; Fri, 20 Nov 2020 12:53:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:55286) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kgAV5-0008UT-Sa for qemu-devel@nongnu.org; Fri, 20 Nov 2020 12:47:14 -0500 Received: from us-smtp-delivery-44.mimecast.com ([207.211.30.44]:60686) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kgAV2-00057u-Ii for qemu-devel@nongnu.org; Fri, 20 Nov 2020 12:47:11 -0500 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-41-DMiUE2lSO1mAhJKNP9UXVg-1; Fri, 20 Nov 2020 12:46:49 -0500 X-MC-Unique: DMiUE2lSO1mAhJKNP9UXVg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 77C221DDE5; Fri, 20 Nov 2020 17:46:48 +0000 (UTC) Received: from bahia.redhat.com (ovpn-112-44.ams2.redhat.com [10.36.112.44]) by smtp.corp.redhat.com (Postfix) with ESMTP id F33825C1D5; Fri, 20 Nov 2020 17:46:46 +0000 (UTC) From: Greg Kurz To: qemu-devel@nongnu.org Subject: [PATCH for-6.0 0/8] spapr: Address the confusion between IPI numbers and vCPU ids Date: Fri, 20 Nov 2020 18:46:38 +0100 Message-Id: <20201120174646.619395-1-groug@kaod.org> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: kaod.org Received-SPF: softfail client-ip=207.211.30.44; envelope-from=groug@kaod.org; helo=us-smtp-delivery-44.mimecast.com X-Spam_score_int: -11 X-Spam_score: -1.2 X-Spam_bar: - X-Spam_report: (-1.2 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_SOFTFAIL=0.665 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Greg Kurz , qemu-ppc@nongnu.org, =?utf-8?q?C=C3=A9dric_L?= =?utf-8?q?e_Goater?= , David Gibson Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" A regression was recently fixed in the sPAPR XIVE code for QEMU 5.2 RC3 [1]. It boiled down to a confusion between IPI numbers and vCPU ids, which happen to be numerically equal in general, but are really different entities that can diverge in some setups. When this happens, we end up misconfiguring XIVE in a way that is almost fatal for the guest. The confusion comes from XICS which has historically assumed equality between interrupt server numbers and vCPU ids, as mentionned in a comment back from 2011 in the linux kernel icp_native_init_one_node() function: /* This code does the theorically broken assumption that the interrupt * server numbers are the same as the hard CPU numbers. * This happens to be the case so far but we are playing with fire... * should be fixed one of these days. -BenH. */ This assumption crept into QEMU through the "ibm,interrupt-server-ranges" property of the "interrupt-controller" node in the DT. This property contains ranges of consecutive vCPU ids defined as (first id, # of ids). In the case of QEMU, we define a single range starting from 0 up to the highest vCPU id, as returned by spapr_max_server_number(). This has always been associated to the "nr_servers" wording when naming variables or function arguments. When XIVE got added, we introduced an sPAPR IRQ abstraction to be able to control several interrupt controller backends. The sPAPR IRQ base class provides a dt() handler used to populate the "interrupt-controller" node in the DT. This handler takes an "nr_server" argument inherited from XICS and we ended up using it to populate the "ibm,xive-lisn-ranges" property used with XIVE, which has completely different semantics. It contain ranges of interrupt numbers that the guest can use for any purpose. Since one obvious purpose is IPI, its first range should just be able to accomodate all possible vCPUs. In the case of QEMU, we define a single range starting from 0 up to "nr_server" but we should rather size it to the number of vCPUs actually (ie. smp.max_cpus). This series aims at getting rid of the "nr_server" argument in the sPAPR IC handlers. Since both the highest possible vCPU id and the maximum number of vCPUs are invariants for XICS and XIVE respectively, let's make them device properties to be configured by the machine when it creates the interrupt controllers and use them where needed. This doesn't cause any visible change to setups using the default VSMT machine settings. This changes "ibm,xive-lisn-ranges" for setups that mess with VSMT, but this is acceptable since linux only allocates one interrupt per vCPU and the higher part of the range was never used. [1] https://git.qemu.org/?p=qemu.git;a=commit;h=6d24795ee7e3199d199d3c415312c93382ad1807 Greg Kurz (8): spapr/xive: Turn some sanity checks into assertions spapr/xive: Introduce spapr_xive_nr_ends() spapr/xive: Add "nr-servers" property spapr/xive: Add "nr-ipis" property spapr/xics: Drop unused argument to xics_kvm_has_broken_disconnect() spapr/xics: Add "nr-servers" property spapr: Drop "nr_servers" argument of the sPAPR IC activate() operation spapr: Drop "nr_servers" argument of the sPAPR IC dt() operation include/hw/ppc/spapr.h | 4 +-- include/hw/ppc/spapr_irq.h | 9 ++--- include/hw/ppc/spapr_xive.h | 25 ++++++++++++- include/hw/ppc/xics_spapr.h | 23 +++++++++--- hw/intc/spapr_xive.c | 72 ++++++++++++++++++++++--------------- hw/intc/spapr_xive_kvm.c | 4 +-- hw/intc/xics_kvm.c | 4 +-- hw/intc/xics_spapr.c | 45 ++++++++++++++--------- hw/ppc/spapr.c | 7 ++-- hw/ppc/spapr_irq.c | 27 +++++++------- 10 files changed, 141 insertions(+), 79 deletions(-)