From patchwork Wed Nov 20 23:05:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 11255371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CE8E14E5 for ; Thu, 21 Nov 2019 06:59:50 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5D882088F for ; Thu, 21 Nov 2019 06:59:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5D882088F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXgPp-0003Ni-0s; Thu, 21 Nov 2019 06:58:09 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXgPn-0003Nd-SY for xen-devel@lists.xenproject.org; Thu, 21 Nov 2019 06:58:07 +0000 X-Inumbo-ID: 43286697-0c2c-11ea-a323-12813bfff9fa Received: from mga05.intel.com (unknown [192.55.52.43]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 43286697-0c2c-11ea-a323-12813bfff9fa; Thu, 21 Nov 2019 06:58:04 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Nov 2019 22:58:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,224,1571727600"; d="scan'208";a="197132540" Received: from knl-rvp-beta.sh.intel.com ([10.239.48.66]) by orsmga007.jf.intel.com with ESMTP; 20 Nov 2019 22:57:59 -0800 From: Chao Gao To: xen-devel@lists.xenproject.org Date: Thu, 21 Nov 2019 07:05:54 +0800 Message-Id: <1574291155-26032-1-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 Subject: [Xen-devel] [PATCH v1 1/2] x86/cpu: maintain a parked CPU bitmap X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Jan Beulich , Chao Gao , Volodymyr Babchuk , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" It helps to distinguish parked CPUs from those are really offlined or hot-added. We need to know the parked CPUs in order to do a special check against them to ensure that all CPUs, except those are really offlined or hot-added, have the same ucode version. Signed-off-by: Chao Gao --- Note that changes on ARM side are untested. --- xen/arch/arm/smpboot.c | 1 + xen/arch/x86/cpu/common.c | 4 ++++ xen/arch/x86/smpboot.c | 1 + xen/common/cpu.c | 4 ++++ xen/include/asm-arm/smp.h | 1 + xen/include/asm-x86/smp.h | 1 + xen/include/xen/cpumask.h | 1 + 7 files changed, 13 insertions(+) diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c index 00b64c3..1b57ba4 100644 --- a/xen/arch/arm/smpboot.c +++ b/xen/arch/arm/smpboot.c @@ -39,6 +39,7 @@ cpumask_t cpu_online_map; cpumask_t cpu_present_map; cpumask_t cpu_possible_map; +cpumask_var_t cpu_parked_map; struct cpuinfo_arm cpu_data[NR_CPUS]; diff --git a/xen/arch/x86/cpu/common.c b/xen/arch/x86/cpu/common.c index 6c6bd63..fbb961d 100644 --- a/xen/arch/x86/cpu/common.c +++ b/xen/arch/x86/cpu/common.c @@ -337,7 +337,11 @@ void __init early_cpu_init(void) } if (!(c->x86_vendor & (X86_VENDOR_AMD | X86_VENDOR_HYGON))) + { park_offline_cpus = opt_mce; + if (park_offline_cpus && !zalloc_cpumask_var(&cpu_parked_map)) + panic("No memory for CPU parked map\n"); + } initialize_cpu_data(0); } diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index fa691b6..f66de8d 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -60,6 +60,7 @@ cpumask_t cpu_online_map __read_mostly; EXPORT_SYMBOL(cpu_online_map); bool __read_mostly park_offline_cpus; +cpumask_var_t cpu_parked_map; unsigned int __read_mostly nr_sockets; cpumask_t **__read_mostly socket_cpumask; diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 66c855c..0090a19 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -117,6 +117,8 @@ int cpu_down(unsigned int cpu) cpu_notifier_call_chain(cpu, CPU_DEAD, NULL, true); send_global_virq(VIRQ_PCPU_STATE); + if ( park_offline_cpus ) + cpumask_set_cpu(cpu, cpu_parked_map); cpu_hotplug_done(); return 0; @@ -154,6 +156,8 @@ int cpu_up(unsigned int cpu) cpu_notifier_call_chain(cpu, CPU_ONLINE, NULL, true); send_global_virq(VIRQ_PCPU_STATE); + if ( park_offline_cpus ) + cpumask_clear_cpu(cpu, cpu_parked_map); cpu_hotplug_done(); return 0; diff --git a/xen/include/asm-arm/smp.h b/xen/include/asm-arm/smp.h index fdbcefa..4b392fa 100644 --- a/xen/include/asm-arm/smp.h +++ b/xen/include/asm-arm/smp.h @@ -19,6 +19,7 @@ DECLARE_PER_CPU(cpumask_var_t, cpu_core_mask); * would otherwise prefer them to be off? */ #define park_offline_cpus false +extern cpumask_var_t cpu_parked_map; extern void noreturn stop_cpu(void); diff --git a/xen/include/asm-x86/smp.h b/xen/include/asm-x86/smp.h index dbeed2f..886737d 100644 --- a/xen/include/asm-x86/smp.h +++ b/xen/include/asm-x86/smp.h @@ -31,6 +31,7 @@ DECLARE_PER_CPU(cpumask_var_t, scratch_cpumask); * would otherwise prefer them to be off? */ extern bool park_offline_cpus; +extern cpumask_var_t cpu_parked_map; void smp_send_nmi_allbutself(void); diff --git a/xen/include/xen/cpumask.h b/xen/include/xen/cpumask.h index 256b60b..543cec5 100644 --- a/xen/include/xen/cpumask.h +++ b/xen/include/xen/cpumask.h @@ -457,6 +457,7 @@ extern cpumask_t cpu_present_map; #define for_each_possible_cpu(cpu) for_each_cpu(cpu, &cpu_possible_map) #define for_each_online_cpu(cpu) for_each_cpu(cpu, &cpu_online_map) #define for_each_present_cpu(cpu) for_each_cpu(cpu, &cpu_present_map) +#define for_each_parked_cpu(cpu) for_each_cpu(cpu, cpu_parked_map) /* Copy to/from cpumap provided by control tools. */ struct xenctl_bitmap; From patchwork Wed Nov 20 23:05:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chao Gao X-Patchwork-Id: 11255369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D115112B for ; Thu, 21 Nov 2019 06:59:15 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4FA9C2088F for ; Thu, 21 Nov 2019 06:59:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FA9C2088F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXgPu-0003OC-Cu; Thu, 21 Nov 2019 06:58:14 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iXgPs-0003O4-Nl for xen-devel@lists.xenproject.org; Thu, 21 Nov 2019 06:58:12 +0000 X-Inumbo-ID: 4785307a-0c2c-11ea-a323-12813bfff9fa Received: from mga17.intel.com (unknown [192.55.52.151]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4785307a-0c2c-11ea-a323-12813bfff9fa; Thu, 21 Nov 2019 06:58:11 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Nov 2019 22:58:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,224,1571727600"; d="scan'208";a="197132584" Received: from knl-rvp-beta.sh.intel.com ([10.239.48.66]) by orsmga007.jf.intel.com with ESMTP; 20 Nov 2019 22:58:07 -0800 From: Chao Gao To: xen-devel@lists.xenproject.org Date: Thu, 21 Nov 2019 07:05:55 +0800 Message-Id: <1574291155-26032-2-git-send-email-chao.gao@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1574291155-26032-1-git-send-email-chao.gao@intel.com> References: <1574291155-26032-1-git-send-email-chao.gao@intel.com> Subject: [Xen-devel] [PATCH v1 2/2] microcode: reject late ucode loading if any core is parked X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= , Jan Beulich , Wei Liu , Chao Gao MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" If a core with all of its thread being parked, late ucode loading which currently only loads ucode on online threads would lead to differing ucode revisions in the system. In general, keeping ucode revision consistent would be less error-prone. To this end, if there is a parked thread doesn't have an online sibling thread, late ucode loading is rejected. Two threads are on the same core or computing unit iff they have the same phys_proc_id and cpu_core_id/compute_unit_id. Based on phys_proc_id and cpu_core_id/compute_unit_id, an unique core id is generated for each thread. And use a bitmap to reduce the number of comparison. Signed-off-by: Chao Gao --- Changes: - traverse the new parked cpu bitmap to find a parked core. It avoids access uninitialized cpu_data of a hot-added CPU. - use bitmap_empty() rather than find_first_bit() to check whether a bitmap is empty. --- xen/arch/x86/microcode.c | 63 +++++++++++++++++++++++++++++++++++++++++ xen/include/asm-x86/processor.h | 1 + 2 files changed, 64 insertions(+) diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c index 65d1f41..dcc8e4b 100644 --- a/xen/arch/x86/microcode.c +++ b/xen/arch/x86/microcode.c @@ -584,6 +584,51 @@ static int do_microcode_update(void *patch) return ret; } +static unsigned int unique_core_id(unsigned int cpu, unsigned int socket_shift) +{ + unsigned int core_id = cpu_to_cu(cpu); + + if ( core_id == INVALID_CUID ) + core_id = cpu_to_core(cpu); + + return (cpu_to_socket(cpu) << socket_shift) + core_id; +} + +static int has_parked_core(void) +{ + int ret; + unsigned int cpu, max_bits, core_width; + unsigned int max_sockets = 1, max_cores = 1; + unsigned long *bitmap; + + if ( !park_offline_cpus ) + return 0; + + for_each_parked_cpu(cpu) + { + /* Note that cpu_to_socket() get an ID starting from 0. */ + max_sockets = max(max_sockets, cpu_to_socket(cpu) + 1); + max_cores = max(max_cores, cpu_data[cpu].x86_max_cores); + } + + core_width = fls(max_cores); + max_bits = max_sockets << core_width; + bitmap = xzalloc_array(unsigned long, BITS_TO_LONGS(max_bits)); + if ( !bitmap ) + return -ENOMEM; + + for_each_parked_cpu(cpu) + __set_bit(unique_core_id(cpu, core_width), bitmap); + + for_each_online_cpu(cpu) + __clear_bit(unique_core_id(cpu, core_width), bitmap); + + ret = !bitmap_empty(bitmap, max_bits); + xfree(bitmap); + + return ret; +} + int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len) { int ret; @@ -629,6 +674,24 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len) return -EPERM; } + /* + * If there is a core with all of its threads parked, late loading may + * cause differing ucode revisions in the system. Refuse this operation. + */ + ret = has_parked_core(); + if ( ret ) + { + if ( ret > 0 ) + { + printk(XENLOG_WARNING + "Aborted: found a parked core (parked CPU bitmap: %*pbl)\n", + CPUMASK_PR(cpu_parked_map)); + ret = -EPERM; + } + xfree(buffer); + goto put; + } + patch = parse_blob(buffer, len); xfree(buffer); if ( IS_ERR(patch) ) diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 557f9b6..f8a9e93 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -171,6 +171,7 @@ extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c); #define cpu_to_core(_cpu) (cpu_data[_cpu].cpu_core_id) #define cpu_to_socket(_cpu) (cpu_data[_cpu].phys_proc_id) +#define cpu_to_cu(_cpu) (cpu_data[_cpu].compute_unit_id) unsigned int apicid_to_socket(unsigned int);