From patchwork Tue Feb 4 17:31:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13959473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B3477C02194 for ; Tue, 4 Feb 2025 17:31:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.881749.1291919 (Exim 4.92) (envelope-from ) id 1tfMlj-0006Fc-NS; Tue, 04 Feb 2025 17:31:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 881749.1291919; Tue, 04 Feb 2025 17:31:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tfMlj-0006FV-Jn; Tue, 04 Feb 2025 17:31:27 +0000 Received: by outflank-mailman (input) for mailman id 881749; Tue, 04 Feb 2025 17:31:26 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tfMli-00069x-AX for xen-devel@lists.xenproject.org; Tue, 04 Feb 2025 17:31:26 +0000 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [2a00:1450:4864:20::32f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id db4225d1-e31d-11ef-99a4-01e77a169b0f; Tue, 04 Feb 2025 18:31:24 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-43675b1155bso67872215e9.2 for ; Tue, 04 Feb 2025 09:31:24 -0800 (PST) Received: from CSGPROD108744.Home (0545937c.skybroadband.com. [5.69.147.124]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c5c102ac9sm16137748f8f.29.2025.02.04.09.31.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 09:31:23 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: db4225d1-e31d-11ef-99a4-01e77a169b0f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1738690283; x=1739295083; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qgoSralsaqhirBRapBktqNUa0D2GysAhNdu/v1g4OCo=; b=B0thRuFoq8EUqAJbW/VEH720NchbmQe8JpkoXVa1/qHXIh3h39fRTgklmBPaFYAVJj 1bmwVYVE95zsiEaJGmy+fZQZ0sZ2IsIDbbrQLHFMp69OAB8YhHmSMbgzOVjYhvOF4RtZ p2b5DR8lLnOL3yC2Mg9LbTwFX/M687+yZpdIQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738690283; x=1739295083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qgoSralsaqhirBRapBktqNUa0D2GysAhNdu/v1g4OCo=; b=AFHAS/882PQoRsjit2xaTXnCw7ACCXmghXl4lX4RHYvgotdE8hYMY6T9dnSzIuUy3u sI/MARUzEaO5Svn+TaOs7dm3RsRDlA9edGbmQBO23xcrGbBKWQs7nOY6eDfTl3y4O7ik YVdxKGaDdhwUSc+CgKT++kkxiv/tzLJuruUlR3nnOtoQcEtqzUPJJIOzuYO9Q6rCDoT/ 7iLUW7c0myxNtU3WOKUYG12HItIHmfcePadjhfj9noK5XMeBv3qqAEXxXa6Coev87p2d dJ8ZLDOHHssFBJW8dxTsIhJTNokniV8YwDD8ni1iv73YKmvUvzraIP/01lU+od/GHxaQ 9klw== X-Gm-Message-State: AOJu0YwUW40IwhvonsNk1IbNjmZuI/vPG8KeqYP5xj33ToPcLbxjPpRA 3RoH051NkN76H3ikM9E9p5h8YrrQqErwkXPr6Ejs2cz7gGdx39uhnSd2VGYhZFAH8dlQVDEFu4v AFvk= X-Gm-Gg: ASbGncv4jHHj2K84iCoOk4Wp/Hhuwvbfv28yeME5CC3pQLzJx5/OzwywiAeK7xQnXQh +OL1h18gNS5/0i7ns+3ajMMWvS2ngZO/ym9ukSvD0BGkIvtL9C+oO3AY8xPJ9DU+hxwNtrlXLM8 gtvIh/A8mL6ac24t8z7aibqC6Yg/yZzfRWzWYwWyyb5VByvH4Q1QBNXO4TAvHgW6aMvRjxpx+fL goLnqLxptS4HJ8ejYXe3A1vuYpezS2kIAlXT+XhemY8HM8b0PHoyaWFX5O4GZ60mnKA2y5PlFJP LnEzNaMPusMy1aJwAhlgwpp27mAcXmlqgZ6IksVLq8+UFa35fAUGh3F3WQ== X-Google-Smtp-Source: AGHT+IH6X6R2AxQBlPRb618z7+KrnNLc6cr17S6oqlnZ0rgHuV7dz8ffkuIFABO4r01nmrMVYagVFQ== X-Received: by 2002:a05:600c:4ecb:b0:434:a815:2b5d with SMTP id 5b1f17b1804b1-438dc40db89mr207051105e9.24.1738690283620; Tue, 04 Feb 2025 09:31:23 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v3 1/2] tools/hvmloader: Retrieve APIC IDs from the APs themselves Date: Tue, 4 Feb 2025 17:31:19 +0000 Message-ID: <20250204173120.56598-2-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204173120.56598-1-alejandro.vallejo@cloud.com> References: <20250204173120.56598-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Make it so the APs expose their own APIC IDs in a lookup table (LUT). We can use that LUT to populate the MADT, decoupling the algorithm that relates CPU IDs and APIC IDs from hvmloader. Modified the printf to also print the APIC ID of each CPU, as well as fixing a (benign) wrong specifier being used for the vcpu id. Signed-off-by: Alejandro Vallejo --- v2->v3: * Moved extern from patch2 into patch1 for (transient) MISRA compliance. v1->v2: * Removed "(x2)" from the comment of cpu_to_apicid. * Added "APIC ID" to the printed string on AP boot up. Changes from the v7 version of this patch in the longer topology series: * s/cpu_to_x2apicid/cpu_to_apicid/ * Though, as I already stated, I don't think this is a good idea. * Dynamically size cpu_to_apicid rather than using HVM_MAX_VCPUS. * Got rid of the ap_callin removal. It's not as trivial if we don't want to assume cpu0 always has apicid=0. Part of the complaints on the previous versions involved the inability to do that. * For debugging sanity, I've added the apicid to the CPU boot printf. * Later on, toolstack will choose the APIC IDs and it's helpful to know the relationship in the logs. * While at it, fix the vcpu specifier s/%d/%u/ * Check for leaf 0xb while probing for x2apic support. --- tools/firmware/hvmloader/config.h | 2 ++ tools/firmware/hvmloader/smp.c | 43 ++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h index cd716bf39245..b32bcbf4a1f2 100644 --- a/tools/firmware/hvmloader/config.h +++ b/tools/firmware/hvmloader/config.h @@ -48,6 +48,8 @@ extern uint8_t ioapic_version; #define IOAPIC_ID 0x01 +extern uint32_t *cpu_to_apicid; + #define LAPIC_BASE_ADDRESS 0xfee00000 #define LAPIC_ID(vcpu_id) ((vcpu_id) * 2) diff --git a/tools/firmware/hvmloader/smp.c b/tools/firmware/hvmloader/smp.c index 1b940cefd071..341fd6e0b61a 100644 --- a/tools/firmware/hvmloader/smp.c +++ b/tools/firmware/hvmloader/smp.c @@ -31,9 +31,38 @@ static int ap_callin; +/** True if x2apic support is exposed to the guest. */ +static bool has_x2apic; + +/** + * Lookup table of APIC IDs. + * + * Each entry is populated for its respective CPU as they come online. This is + * required for generating the MADT with minimal assumptions about ID + * relationships. + */ +uint32_t *cpu_to_apicid; + +static uint32_t read_apic_id(void) +{ + uint32_t apic_id; + + if ( has_x2apic ) + cpuid(0xb, NULL, NULL, NULL, &apic_id); + else + { + cpuid(1, NULL, &apic_id, NULL, NULL); + apic_id >>= 24; + } + + return apic_id; +} + static void cpu_setup(unsigned int cpu) { - printf(" - CPU%d ... ", cpu); + uint32_t apicid = cpu_to_apicid[cpu] = read_apic_id(); + + printf(" - CPU%u APIC ID %u ... ", cpu, apicid); cacheattr_init(); printf("done.\n"); @@ -104,8 +133,20 @@ static void boot_cpu(unsigned int cpu) void smp_initialise(void) { unsigned int i, nr_cpus = hvm_info->nr_vcpus; + uint32_t ecx, max_leaf; + + cpuid(0, &max_leaf, NULL, NULL, NULL); + if ( max_leaf >= 0xb ) + { + cpuid(1, NULL, NULL, &ecx, NULL); + has_x2apic = (ecx >> 21) & 1; + if ( has_x2apic ) + printf("x2APIC supported\n"); + } printf("Multiprocessor initialisation:\n"); + cpu_to_apicid = scratch_alloc(sizeof(*cpu_to_apicid) * nr_cpus, + sizeof(*cpu_to_apicid)); cpu_setup(0); for ( i = 1; i < nr_cpus; i++ ) boot_cpu(i); From patchwork Tue Feb 4 17:31:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13959474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C0AD7C02198 for ; Tue, 4 Feb 2025 17:31:38 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.881750.1291925 (Exim 4.92) (envelope-from ) id 1tfMlk-0006Iy-2I; Tue, 04 Feb 2025 17:31:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 881750.1291925; Tue, 04 Feb 2025 17:31:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tfMlj-0006IN-Rd; Tue, 04 Feb 2025 17:31:27 +0000 Received: by outflank-mailman (input) for mailman id 881750; Tue, 04 Feb 2025 17:31:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tfMli-000612-R1 for xen-devel@lists.xenproject.org; Tue, 04 Feb 2025 17:31:26 +0000 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [2a00:1450:4864:20::436]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id dc3fb122-e31d-11ef-a0e7-8be0dac302b0; Tue, 04 Feb 2025 18:31:26 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id ffacd0b85a97d-38a25d4b9d4so3073283f8f.0 for ; Tue, 04 Feb 2025 09:31:26 -0800 (PST) Received: from CSGPROD108744.Home (0545937c.skybroadband.com. [5.69.147.124]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38c5c102ac9sm16137748f8f.29.2025.02.04.09.31.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 09:31:23 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: dc3fb122-e31d-11ef-a0e7-8be0dac302b0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1738690285; x=1739295085; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xgMGUVDS+iYC3WViwSsK8gEHt5WvmsVZpIvxDEu6Ypk=; b=AGyRW7KgqnmAkbWZhtv0S4vWaclmuciWRuTvJiJwLUTgl3fcbrfXlEoK5Jcvv99Fqq evyg814KXVv64S2AI5jE228oBwq2xa5B3h4+Q8muKfwcYrk/EOiTSBGiJA+oEadYnpBk iJr0nTzl+34V6v9ERRHZ9rNm3Y9eolv4p02jY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738690285; x=1739295085; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xgMGUVDS+iYC3WViwSsK8gEHt5WvmsVZpIvxDEu6Ypk=; b=RznGl5oTjmcvbwQjyubhXNr+HDozY0VV3AimZIhaOKccDLgcG4+HjLN7nWEpRzWnWj VmURYuDgXuL3lhxtxWZTBImnzQVZ3OlAOUlhUg/pCSj87AB9JLjntnUC89Os4KzAWaof tKD8AApeYq+N1HHFYD38HfjAxMXIse+n3kD5hdRlwW0NWQcFn3mK90bG5V9Nh/Jn876B xl65ZMFf13eUamM/hNr9wa4Sw4TAU6mooAubuXtbTguGPTJ/fFuAerh0XutMkWGaOP+X JJ/2M1oUUihcGRtoVLYr4wZvqNUyJZ+Z7PNZNdRhwIG52Lb9/hwAVNAelAskG6uCplfB q9fA== X-Gm-Message-State: AOJu0YwXH22r5VqYvq8uygeH/8CiLjE3SDKE89xwZEea+x6tUSBisK4x ylUf2aKJAuBONZ2x+IkmibHtXAhA6JWaZ2Aaloty7xEI/VoMr4efaKIFsIULv3Xxn+V05Zw92pR ocQQ= X-Gm-Gg: ASbGncvEYPl0cpBXPm16kK3SX8jLGVG9sAlVaZCx2pJ2EM4V1TeV+kOHE3Yfvtysqbm v4gImB5MnF0Le1qH8LZP0+Z5il93B+k+yPQImUSaOhyOFju9+YOItum5nTqyRslyEcGJA8P/NEL bkheHBF1ClSkHaDx4Hm1kvGZfKnpsMDFaA5t1MVRkq99krlrWaTKY3KrI/22kjS4Uey1XIwAGo4 cy3mPKh7035dmZz0GFe5iDmLNbJ9y1/5QDCZpQy8J4xSb4eK7QyxN/fKciE39KwHV+vGfNzPNhS YLMoId0kwpIJzdzCYSD2yi3nVE6xftTy5iFil7Wd+PUpYl66WxF9ko6Ssg== X-Google-Smtp-Source: AGHT+IEXrn3wc5aGCv18hOHhvl082oyjyCUvFY1nYJjuWj0HiME3c+dMjiFm1GSgaRBLjfKH+M2fPw== X-Received: by 2002:a5d:5988:0:b0:38b:ed1c:a70d with SMTP id ffacd0b85a97d-38c5167b477mr23844300f8f.0.1738690284442; Tue, 04 Feb 2025 09:31:24 -0800 (PST) From: Alejandro Vallejo To: xen-devel@lists.xenproject.org Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Anthony PERARD Subject: [PATCH v3 2/2] tools/hvmloader: Replace LAPIC_ID() with cpu_to_apicid[] Date: Tue, 4 Feb 2025 17:31:20 +0000 Message-ID: <20250204173120.56598-3-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250204173120.56598-1-alejandro.vallejo@cloud.com> References: <20250204173120.56598-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Replace uses of the LAPIC_ID() macro with accesses to the cpu_to_apicid[] lookup table. This table contains the APIC IDs of each vCPU as probed at runtime rather than assuming a predefined relation. Moved smp_initialise() ahead of apic_setup() in order to initialise cpu_to_apicid ASAP and avoid using it uninitialised. Note that bringing up the APs doesn't need the APIC in hvmloader becasue it always runs virtualized and uses the PV interface. Signed-off-by: Alejandro Vallejo --- v2->v3: * Moved extern in config.h to patch1 for (transient) MISRA compliance. v1->v2: * No changes Changes wrt original series * No changes (it was wrongly stated in v1 that something did. That was part of the following patch) --- tools/firmware/hvmloader/config.h | 1 - tools/firmware/hvmloader/hvmloader.c | 6 +++--- tools/firmware/hvmloader/mp_tables.c | 2 +- tools/firmware/hvmloader/util.c | 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a/tools/firmware/hvmloader/config.h b/tools/firmware/hvmloader/config.h index b32bcbf4a1f2..6e1da137d779 100644 --- a/tools/firmware/hvmloader/config.h +++ b/tools/firmware/hvmloader/config.h @@ -51,7 +51,6 @@ extern uint8_t ioapic_version; extern uint32_t *cpu_to_apicid; #define LAPIC_BASE_ADDRESS 0xfee00000 -#define LAPIC_ID(vcpu_id) ((vcpu_id) * 2) #define PCI_ISA_DEVFN 0x08 /* dev 1, fn 0 */ #define PCI_ISA_IRQ_MASK 0x0c20U /* ISA IRQs 5,10,11 are PCI connected */ diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c index f8af88fabf24..4e330fc1e241 100644 --- a/tools/firmware/hvmloader/hvmloader.c +++ b/tools/firmware/hvmloader/hvmloader.c @@ -224,7 +224,7 @@ static void apic_setup(void) /* 8259A ExtInts are delivered through IOAPIC pin 0 (Virtual Wire Mode). */ ioapic_write(0x10, APIC_DM_EXTINT); - ioapic_write(0x11, SET_APIC_ID(LAPIC_ID(0))); + ioapic_write(0x11, SET_APIC_ID(cpu_to_apicid[0])); } struct bios_info { @@ -341,11 +341,11 @@ int main(void) printf("CPU speed is %u MHz\n", get_cpu_mhz()); + smp_initialise(); + apic_setup(); pci_setup(); - smp_initialise(); - perform_tests(); if ( bios->bios_info_setup ) diff --git a/tools/firmware/hvmloader/mp_tables.c b/tools/firmware/hvmloader/mp_tables.c index 77d3010406d0..3c93a5c947d9 100644 --- a/tools/firmware/hvmloader/mp_tables.c +++ b/tools/firmware/hvmloader/mp_tables.c @@ -199,7 +199,7 @@ static void fill_mp_config_table(struct mp_config_table *mpct, int length) static void fill_mp_proc_entry(struct mp_proc_entry *mppe, int vcpu_id) { mppe->type = ENTRY_TYPE_PROCESSOR; - mppe->lapic_id = LAPIC_ID(vcpu_id); + mppe->lapic_id = cpu_to_apicid[vcpu_id]; mppe->lapic_version = 0x11; mppe->cpu_flags = CPU_FLAG_ENABLED; if ( vcpu_id == 0 ) diff --git a/tools/firmware/hvmloader/util.c b/tools/firmware/hvmloader/util.c index d3b3f9038e64..2d07ce129013 100644 --- a/tools/firmware/hvmloader/util.c +++ b/tools/firmware/hvmloader/util.c @@ -827,7 +827,7 @@ static void acpi_mem_free(struct acpi_ctxt *ctxt, static uint32_t acpi_lapic_id(unsigned cpu) { - return LAPIC_ID(cpu); + return cpu_to_apicid[cpu]; } void hvmloader_acpi_build_tables(struct acpi_config *config,