From patchwork Thu Mar 21 12:24:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Dyasli X-Patchwork-Id: 10863421 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A4BEE13B5 for ; Thu, 21 Mar 2019 12:26:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8808C28B03 for ; Thu, 21 Mar 2019 12:26:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C5172A138; Thu, 21 Mar 2019 12:26:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 633562A143 for ; Thu, 21 Mar 2019 12:26:39 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1h6wkf-0002J4-6v; Thu, 21 Mar 2019 12:24:53 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1h6wke-0002Iu-Fr for xen-devel@lists.xenproject.org; Thu, 21 Mar 2019 12:24:52 +0000 X-Inumbo-ID: 530cf15c-4bd4-11e9-bc90-bc764e045a96 Received: from SMTP03.CITRIX.COM (unknown [162.221.156.55]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id 530cf15c-4bd4-11e9-bc90-bc764e045a96; Thu, 21 Mar 2019 12:24:51 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.60,252,1549929600"; d="scan'208";a="81228494" To: Chao Gao , References: <1552291056-20286-1-git-send-email-chao.gao@intel.com> <1552291056-20286-13-git-send-email-chao.gao@intel.com> From: Sergey Dyasli Openpgp: preference=signencrypt Autocrypt: addr=sergey.dyasli@citrix.com; keydata= mQINBFtMVHEBEADc/hZcLexrB6vGTdGqEUsYZkFGQh6Z1OO7bCtM1go1RugSMeq9tkFHQSOc 9c7W9NVQqLgn8eefikIHxgic6tGgKoIQKcPuSsnqGao2YabsTSSoeatvmO5HkR0xGaUd+M6j iqv3cD7/WL602NhphT4ucKXCz93w0TeoJ3gleLuILxmzg1gDhKtMdkZv6TngWpKgIMRfoyHQ jsVzPbTTjJl/a9Cw99vuhFuEJfzbLA80hCwhoPM+ZQGFDcG4c25GQGQFFatpbQUhNirWW5b1 r2yVOziSJsvfTLnyzEizCvU+r/Ek2Kh0eAsRFr35m2X+X3CfxKrZcePxzAf273p4nc3YIK9h cwa4ZpDksun0E2l0pIxg/pPBXTNbH+OX1I+BfWDZWlPiPxgkiKdgYPS2qv53dJ+k9x6HkuCy i61IcjXRtVgL5nPGakyOFQ+07S4HIJlw98a6NrptWOFkxDt38x87mSM7aSWp1kjyGqQTGoKB VEx5BdRS5gFdYGCQFc8KVGEWPPGdeYx9Pj2wTaweKV0qZT69lmf/P5149Pc81SRhuc0hUX9K DnYBa1iSHaDjifMsNXKzj8Y8zVm+J6DZo/D10IUxMuExvbPa/8nsertWxoDSbWcF1cyvZp9X tUEukuPoTKO4Vzg7xVNj9pbK9GPxSYcafJUgDeKEIlkn3iVIPwARAQABtChTZXJnZXkgRHlh c2xpIDxzZXJnZXkuZHlhc2xpQGNpdHJpeC5jb20+iQJOBBMBCgA4FiEEkI7HMI5EbM2FLA1L Aa+w5JvbyusFAltMVHECGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQAa+w5JvbyuuQ JBAAry/oRK6m0I+ck1Tarz9a1RrF73r1YoJUk5Bw+PSxsBJOPp3vDeAz3Kqw58qmBXeNlMU4 1cqAxFxCCKMtER1gpmrKWBA1/H1ZoBRtzhaHgPTQLyR7LB1OgdpgwEOjN1Q5gME8Pk21y/3N cG5YBgD/ZHbq8nWS/G3r001Ie3nX55uacGk/Ry175cS48+asrerShKMDNMT1cwimo9zH/3Lm RTpWloh2dG4jjwtCXqB7s+FEE5wQVCpPp9p55+9pPd+3DXmsQEcJ/28XHo/UJW663WjRlRc4 wgPwiC9Co1HqaMKSzdPpZmI5D4HizWH8jF7ppUjWoPapwk4dEA7Al0vx1Bz3gbJAL8DaRgQp H4j/16ifletfGUNbHJR2vWljZ5SEf2vMVcdubf9eFUfBF/9OOR1Kcj1PISP8sPhcP7oCfFtH RcxXh1OStrRFtltJt2VlloKXAUggdewwyyD4xl9UHCfI4lSexOK37wNSQYPQcVcOS1bl4NhQ em6pw2AC32NsnQE5PmczFADDIpWhO/+WtkTFeE2HHfAn++y3YDtKQd7xes9UJjQNiGziArST l6Zrx4/nShVLeYRVW76l27gI5a8BZLWwBVRsWniGM50OOJULvSag7kh+cjsrXXpNuA4rfEoB Bxr7pso9e5YghupDc8XftsYd7mlAgOTCAC8uZme5Ag0EW0xUcQEQAMKi97v3DwwPgYVPYIbQ JAvoMgubJllC9RcE0PQsE6nEKSrfOT6Gh5/LHOXLbQI9nzU/xdr6kMfwbYVTnZIY/SwsLrJa gSKm64t11MjC1Vf03/sncx1tgI7nwqMMIAYLsXnQ9X/Up5L/gLO2YDIPxrQ6g4glgRYPT53i r6/hTz3dlpqyPCorpuF+WY7P2ujhlFlXCAaD6btPPM/9LZSmI0xS4aCBLH+pZeCr0UGSMhsX JYN0QRLjfsIDGyqaXVH9gwV2Hgsq6z8fNPQlBc3IpDvfXa1rYtgldYBfG521L3wnsMcKoFSr R5dpH7Jtvv5YBuAk8r571qlMhyAmVKiEnc+RonWl503D5bAHqNmFNjV248J5scyRD/+BcYLI 2CFG28XZrCvjxq3ux5hpmg2fCu+y98h6/yuwB/JhbFlDOSoluEpysiEL3R5GTKbxOF664q5W fiSObxNONxs86UtghqNDRUJgyS0W6TfykGOnZDVYAC9Gg8SbQDta1ymA0q76S/NG2MrJEOIr 1GtOr/UjNv2x4vW56dzX/3yuhK1ilpgzh1q504ETC6EKXMaFT8cNgsMlk9dOvWPwlsIJ249+ PizMDFGITxGTIrQAaUBO+HRLSBYdHNrHJtytkBoTjykCt7M6pl7l+jFYjGSw4fwexVy0MqsD AZ2coH82RTPb6Q7JABEBAAGJAjYEGAEKACAWIQSQjscwjkRszYUsDUsBr7Dkm9vK6wUCW0xU cQIbDAAKCRABr7Dkm9vK6+9uD/9Ld3X5cvnrwrkFMddpjFKoJ4yphtX2s+EQfKT6vMq3A1dJ tI7zHTFm60uBhX6eRbQow8fkHPcjXGJEoCSJf8ktwx/HYcBcnUK/aulHpvHIIYEma7BHry4x L+Ap7oBbBNiraS3Wu1k+MaX07BWhYYkpu7akUEtaYsCceVc4vpYNITUzPYCHeMwc5pLICA+7 VdI1rrTSAwlCtLGBt7ttbvaAKN4dysiN+/66Hlxnn8n952lZdG4ThPPzafG50EgcTa+dASgm tc6HaQAmJiwb4iWUOoUoM+udLRHcN6cE0bQivyH1bqF4ROeFBRz00MUJKvzUynR9E50F9hmd DOBJkyM3Z5imQ0RayEkRHhlhj7uECaojnUeewq4zjpAg2HTSMkdEzKRbdMEyXCdQXFnSCmUB 5yMIULuDbOODWo3EufExLjAKzIRWEKQ/JidLzO6hrhlQffsJ7MPTU+Hg7WxqWfn4zhuUcIQB SlkiRMalSiJITC2jG7oQRRh9tyNaDMkKzTbeFtHKRmUUAuhE0LBXP8Wc+5W7b3WOf2SO8JMR 4TqDZ0K06s66S5fOTW0h56iCCxTsAnRvM/tA4SERyRoFs/iTqJzboskZY0yKeWV4/IQxfOyC YwdU3//zANM1ZpqeE/8lnW/kx+fyzVyEioLSwkjDvdG++4GQ5r6PHQ7BbdEWhA== Message-ID: <1d9184c6-38c4-dc85-9542-fb7397d6c43a@citrix.com> Date: Thu, 21 Mar 2019 12:24:46 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <1552291056-20286-13-git-send-email-chao.gao@intel.com> Content-Language: en-US Subject: [Xen-devel] [RFC PATCH v6 13/12] microcode: add sequential application policy X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: "sergey.dyasli@citrix.com >> Sergey Dyasli" , Wei Liu , Ashok Raj , Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Hi Chao, Updating microcode in parallel by default should be fine, but I think there are no guarantees that a parallel application will be fine for all future microcodes. To retain the ability to update microcode on cores sequentially and give some choice to a user, I developed the below patch. Could you consider including something like this into v7? (The patch is for Xen 4.11 but forward-porting should be trivial) --- tools/misc/xen-microcode.c | 17 +++++++++++++-- xen/arch/x86/microcode.c | 35 +++++++++++++++++++++++++++++-- xen/arch/x86/platform_hypercall.c | 3 ++- xen/include/asm-x86/processor.h | 3 ++- xen/include/public/platform.h | 3 +++ 5 files changed, 55 insertions(+), 6 deletions(-) diff --git a/tools/misc/xen-microcode.c b/tools/misc/xen-microcode.c index 2c1b2e362b..55d7d2f5df 100644 --- a/tools/misc/xen-microcode.c +++ b/tools/misc/xen-microcode.c @@ -46,7 +46,7 @@ void show_help(void) { fprintf(stderr, "xen-microcode: Xen microcode updating tool\n" - "Usage: xen-microcode \n"); + "Usage: xen-microcode \n"); get_cpu_family(cpuid_eax(1)); fprintf(stderr, @@ -62,13 +62,25 @@ int main(int argc, char *argv[]) struct xen_platform_op op; xc_interface *xch; DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc); + uint32_t strategy; - if (argc < 2) + if (argc < 3) { show_help(); return 0; } + if (!strcmp(argv[2], "parallel")) + strategy = XENPF_microcode_parallel; + else if (!strcmp(argv[2], "sequential")) + strategy = XENPF_microcode_sequential; + else + { + show_help(); + return 0; + } + + filename = argv[1]; fd = open(filename, O_RDONLY); if (fd < 0) { @@ -107,6 +119,7 @@ int main(int argc, char *argv[]) op.cmd = XENPF_microcode_update; op.interface_version = XENPF_INTERFACE_VERSION; op.u.microcode.length = len; + op.u.microcode.strategy = strategy; ret = xc_platform_op(xch, &op); if ( ret ) fprintf(stderr, "Failed to update microcode. (err: %d)\n", ret); diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c index 95e39c33bc..63b01b571f 100644 --- a/xen/arch/x86/microcode.c +++ b/xen/arch/x86/microcode.c @@ -37,6 +37,8 @@ #include #include +#include + #include #include #include @@ -216,6 +218,10 @@ DEFINE_PER_CPU(struct cpu_signature, cpu_sig); */ static atomic_t cpu_in, cpu_out; +static uint32_t application_strategy; +/* The next CPU to perform a ucode update */ +static int next_cpu; + /* * Save an ucode patch to the global cache list. * @@ -351,6 +357,16 @@ static int do_microcode_update(void *unused) if ( ret ) return ret; + while ( application_strategy == XENPF_microcode_sequential && + cpu != next_cpu ) + { + finished = atomic_read(&cpu_out); + if ( wait_for_cpus(&cpu_out, finished + 1, + MICROCODE_UPDATE_TIMEOUT_US) ) + panic("Timeout during sequential microcode update (finished %d/%d)", + finished, cpu_nr); + } + /* * Initiate an update on all processors which don't have an online sibling * thread with a lower thread id. Other sibling threads just await the @@ -358,6 +374,10 @@ static int do_microcode_update(void *unused) */ if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) ) ret = microcode_update_cpu(); + + if ( application_strategy == XENPF_microcode_sequential ) + next_cpu = cpumask_next(next_cpu, &cpu_online_map); + /* * Increase the wait timeout to a safe value here since we're serializing * the microcode update and that could take a while on a large number of @@ -393,7 +413,8 @@ static int do_microcode_update(void *unused) return ret; } -int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len) +int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len, + uint32_t strategy) { int ret; void *buffer; @@ -405,6 +426,10 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len) if ( microcode_ops == NULL ) return -EINVAL; + if ( strategy != XENPF_microcode_parallel && + strategy != XENPF_microcode_sequential ) + return -EINVAL; + buffer = xmalloc_bytes(len); if ( !buffer ) { @@ -449,13 +474,19 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len) if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) ) nr_cores++; - printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores); + printk(XENLOG_INFO "%d cores are to update their microcode %s\n", nr_cores, + strategy == XENPF_microcode_parallel ? "in parallel" : + "sequentially"); /* * We intend to disable interrupt for long time, which may lead to * watchdog timeout. */ watchdog_disable(); + + application_strategy = strategy; + if ( strategy == XENPF_microcode_sequential ) + next_cpu = cpumask_first(&cpu_online_map); /* * Late loading dance. Why the heavy-handed stop_machine effort? * diff --git a/xen/arch/x86/platform_hypercall.c b/xen/arch/x86/platform_hypercall.c index ea18c3215a..b5d2c5790b 100644 --- a/xen/arch/x86/platform_hypercall.c +++ b/xen/arch/x86/platform_hypercall.c @@ -296,7 +296,8 @@ ret_t do_platform_op(XEN_GUEST_HANDLE_PARAM(xen_platform_op_t) u_xenpf_op) ret = microcode_update( guest_handle_to_param(data, const_void), - op->u.microcode.length); + op->u.microcode.length, + op->u.microcode.strategy); spin_unlock(&vcpu_alloc_lock); } break; diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h index 0a62dfa0a3..612b786e81 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -573,7 +573,8 @@ int rdmsr_hypervisor_regs(uint32_t idx, uint64_t *val); int wrmsr_hypervisor_regs(uint32_t idx, uint64_t val); void microcode_set_module(unsigned int); -int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len); +int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len, + uint32_t strategy); int early_microcode_update_cpu(void); int early_microcode_init(void); int microcode_init_intel(void); diff --git a/xen/include/public/platform.h b/xen/include/public/platform.h index 94dbc3feb4..4dea94b84d 100644 --- a/xen/include/public/platform.h +++ b/xen/include/public/platform.h @@ -114,6 +114,9 @@ struct xenpf_microcode_update { /* IN variables. */ XEN_GUEST_HANDLE(const_void) data;/* Pointer to microcode data */ uint32_t length; /* Length of microcode data. */ +#define XENPF_microcode_parallel 0 +#define XENPF_microcode_sequential 1 + uint32_t strategy; /* Application strategy. */ }; typedef struct xenpf_microcode_update xenpf_microcode_update_t; DEFINE_XEN_GUEST_HANDLE(xenpf_microcode_update_t);