From patchwork Thu Jun 25 13:37:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Salil Mehta X-Patchwork-Id: 11625393 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7D5014E3 for ; Thu, 25 Jun 2020 13:45:53 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 904B920781 for ; Thu, 25 Jun 2020 13:45:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="wM14igu7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 904B920781 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=n1WewWBsyxDIgy+gMf/4PRcF6i/BvR4/emZUmMOjYMQ=; b=wM14igu78rexZMYZz2hc6XX6M obL5nbTlhacP6zeZoJPmr17fY1jbs7VNhc3QjGg/wfSV7ZW4ZNoFKNA4c2KVOzd2IoqoO3z6MSuAr 72VafbPPQiZjP5qHKUO0D2FjniOCW/sZwCDvrM4SNvCC74LWYnr18Wc4/kOgSHr+p9U864Kf/G2q5 S6SwtknLF4lcvoKZ8hENOv8GtdMPuyVe95tfd+sP9Y/Zf8TmxA994KXwPh3ocqCFQ7xtwFrNjszBz D0lrB0TGki0F8VQZzYARO4awhAlnuoZfQOjGN8kF/JcRECPNO3XC6PbM7fv8lXZgZrAMS047cq3NC nItBTGnBg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1joSAg-0004pE-IE; Thu, 25 Jun 2020 13:44:06 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190] helo=huawei.com) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1joSAc-0004mi-3O for linux-arm-kernel@lists.infradead.org; Thu, 25 Jun 2020 13:44:04 +0000 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 52A299654FC5BEE3B867; Thu, 25 Jun 2020 21:43:55 +0800 (CST) Received: from A190218597.china.huawei.com (10.47.76.118) by DGGEMS402-HUB.china.huawei.com (10.3.19.202) with Microsoft SMTP Server id 14.3.487.0; Thu, 25 Jun 2020 21:43:48 +0800 From: Salil Mehta To: Subject: [PATCH RFC 1/4] arm64: kernel: Handle disabled[(+)present] cpus in MADT/GICC during init Date: Thu, 25 Jun 2020 14:37:54 +0100 Message-ID: <20200625133757.22332-2-salil.mehta@huawei.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20200625133757.22332-1-salil.mehta@huawei.com> References: <20200625133757.22332-1-salil.mehta@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.47.76.118] X-CFilter-Loop: Reflected X-Spam-Note: CRM114 invocation failed X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [45.249.212.190 listed in list.dnswl.org] -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.0 SPF_PASS SPF: sender matches SPF record 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [45.249.212.190 listed in wl.mailspike.net] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, peter.maydell@linaro.org, gshan@redhat.com, kvm@vger.kernel.org, mst@redhat.com, catalin.marinas@arm.com, linuxarm@huawei.com, linux-kernel@vger.kernel.org, will@kernel.org, Xiongfeng Wang , lorenzo.pieralisi@arm.com, maz@kernel.org, david@redhat.com, drjones@redhat.com, andre.przywara@arm.com, mehta.salil.lnk@gmail.com, richard.henderson@linaro.org, eric.auger@redhat.com, qemu-arm@nongnu.org, imammedo@redhat.com, Salil Mehta , christoffer.dall@arm.com, james.morse@arm.com, sudeep.holla@arm.com, pbonzini@redhat.com Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With ACPI enabled, cpus get identified by the presence of the GICC entry in the MADT Table. Each GICC entry part of MADT presents cpu as enabled or disabled. As of now, the disabled cpus are skipped as physical cpu hotplug is not supported. These remain disabled even after the kernel has booted. To support virtual cpu hotplug(in which case disabled vcpus could be hotplugged even after kernel has booted), QEMU will populate MADT Table with appropriate details of GICC entry for each possible(present+disabled) vcpu. Now, during the init time vcpus will be identified as present or disabled. To achieve this, below changes have been made with respect to the present/possible vcpu handling along with the mentioned reasoning: 1. Identify all possible(present+disabled) vcpus at boot/init time and set their present mask and possible mask. In the existing code, cpus are being marked present quite late within smp_prepare_cpus() function, which gets called in context to the kernel thread. Since the cpu hotplug is not supported, present cpus are always equal to the possible cpus. But with cpu hotplug enabled, this assumption is not true. Hence, present cpus should be marked while MADT GICC entries are bring parsed for each vcpu. 2. Set possible cpus to include disabled. This needs to be done now while parsing MADT GICC entries corresponding to each vcpu as the disabled vcpu info is available only at this point as for hotplug case possible vcpus is not equal to present vcpus. 3. We will store the parsed madt/gicc entry even for the disabled vcpus during init time. This is needed as some modules like PMU registers IRQs for each possible vcpus during init time. Therefore, a valid entry of the MADT GICC should be present for all possible vcpus. 4. Refactoring related to DT/OF is also done to align it with the init changes to support vcpu hotplug. Signed-off-by: Salil Mehta Signed-off-by: Xiongfeng Wang --- arch/arm64/kernel/smp.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index e43a8ff19f0f..51a707928302 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -509,13 +509,12 @@ static int __init smp_cpu_setup(int cpu) if (ops->cpu_init(cpu)) return -ENODEV; - set_cpu_possible(cpu, true); - return 0; } static bool bootcpu_valid __initdata; static unsigned int cpu_count = 1; +static unsigned int disabled_cpu_count; #ifdef CONFIG_ACPI static struct acpi_madt_generic_interrupt cpu_madt_gicc[NR_CPUS]; @@ -534,10 +533,17 @@ struct acpi_madt_generic_interrupt *acpi_cpu_get_madt_gicc(int cpu) static void __init acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor) { + unsigned int total_cpu_count = disabled_cpu_count + cpu_count; u64 hwid = processor->arm_mpidr; if (!(processor->flags & ACPI_MADT_ENABLED)) { +#ifndef CONFIG_ACPI_HOTPLUG_CPU pr_debug("skipping disabled CPU entry with 0x%llx MPIDR\n", hwid); +#else + cpu_madt_gicc[total_cpu_count] = *processor; + set_cpu_possible(total_cpu_count, true); + disabled_cpu_count++; +#endif return; } @@ -546,7 +552,7 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor) return; } - if (is_mpidr_duplicate(cpu_count, hwid)) { + if (is_mpidr_duplicate(total_cpu_count, hwid)) { pr_err("duplicate CPU MPIDR 0x%llx in MADT\n", hwid); return; } @@ -567,9 +573,9 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor) return; /* map the logical cpu id to cpu MPIDR */ - cpu_logical_map(cpu_count) = hwid; + cpu_logical_map(total_cpu_count) = hwid; - cpu_madt_gicc[cpu_count] = *processor; + cpu_madt_gicc[total_cpu_count] = *processor; /* * Set-up the ACPI parking protocol cpu entries @@ -580,8 +586,10 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor) * initialize the cpu if the parking protocol is * the only available enable method). */ - acpi_set_mailbox_entry(cpu_count, processor); + acpi_set_mailbox_entry(total_cpu_count, processor); + set_cpu_possible(total_cpu_count, true); + set_cpu_present(total_cpu_count, true); cpu_count++; } @@ -614,6 +622,9 @@ static void __init acpi_parse_and_init_cpus(void) acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT, acpi_parse_gic_cpu_interface, 0); + pr_debug("possible cpus(%u) present cpus(%u) disabled cpus(%u)\n", + cpu_count+disabled_cpu_count, cpu_count, disabled_cpu_count); + /* * In ACPI, SMP and CPU NUMA information is provided in separate * static tables, namely the MADT and the SRAT. @@ -684,6 +695,9 @@ static void __init of_parse_and_init_cpus(void) cpu_logical_map(cpu_count) = hwid; early_map_cpu_to_node(cpu_count, of_node_to_nid(dn)); + + set_cpu_possible(cpu_count, true); + set_cpu_present(cpu_count, true); next: cpu_count++; } @@ -768,7 +782,6 @@ void __init smp_prepare_cpus(unsigned int max_cpus) if (err) continue; - set_cpu_present(cpu, true); numa_store_cpu_info(cpu); } }