From patchwork Fri May 20 17:36:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13A80C433F5 for ; Fri, 20 May 2022 17:36:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352261AbiETRgt (ORCPT ); Fri, 20 May 2022 13:36:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352248AbiETRgq (ORCPT ); Fri, 20 May 2022 13:36:46 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C787D819AC for ; Fri, 20 May 2022 10:36:44 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id g63-20020a636b42000000b003db2a3daf30so4428387pgc.22 for ; Fri, 20 May 2022 10:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GdWBZFDhJiVgrbjToTXhMKwWYC+jS0aqPfp6IBtPjQU=; b=MeXm4aLrzF/JggIkcCr1PvoEpV6DMd8bww6676h5oDd2dcsV8p3zOG5LystUOsuCGl +PK5X74pQ7RrY9thlR1anCQ1iKunPHuXh6p0LSoV6fQXAFirPznf/6vTI+vZblqAN83Z UXkgTosFoxaddqiKXxYXfvgF9tZWkISLTJGGY/PZw9wI3nMByRT4Ls0s40EPdTKBTf6f +ur9xWSNfb7kk8gwo5lwWQLLiu8AzQAyXBaLpMnUfybCi74VQ84/z1G9y1E8YR2IRI/E ULn5dYSsQJj8N2iisfxOD6wdcS2AbCWy0rs0JMMwM3qMMsT6Jhw9utNdoyS2foJHH862 Bwvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GdWBZFDhJiVgrbjToTXhMKwWYC+jS0aqPfp6IBtPjQU=; b=bNmVACf4ytJmGYswvgM3zmyBiPwNCjiaQexc0+/UH0BlrCuRjR42AqG6DAfhZ1GgAN rv2J+ZxsWCHW+IBTGjzqx2uunCq4tawN89QJk9r0JfhwoAKIE0tLJmLbnpg3AodutU6m 9oAMwh9IVYOiy/BsBso58WKt8zAbUNezrw5yoJU1F1i7rYnjdvOAwcu6+iDodLfgI1wU AMjthwBnHgUGFX+bnVqENSS65t0RG7U194ZgoU9FEcJCaUUdQJ18np/xUOwtKzYPKmrt J+d8qxFCP3zSG74KDgFhs2Lr3/fJnQhp21bfZBt0mw2ZN5VNt9/ygrwQPV4GDcoNKoHY P/Tw== X-Gm-Message-State: AOAM533tyj6yeLJpyk/pDUis77LS5jl4YNSMofLkeXlrzixMzOrCp2/c M/NRw8lzv/ZptmnhUxHTHRfCeJSD X-Google-Smtp-Source: ABdhPJwNVlkpJ/TPfKNggxiFy6ka3cDtK7QG8lj+hLMU3nOs4j8N8YMHpgHu7wabKnt8E4PztWXcDYsQ X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a05:6a00:1393:b0:518:68fe:f036 with SMTP id t19-20020a056a00139300b0051868fef036mr5258388pfg.54.1653068204233; Fri, 20 May 2022 10:36:44 -0700 (PDT) Date: Fri, 20 May 2022 10:36:31 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-2-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 1/8] KVM: x86: Make APIC_VERSION capture only the magic 0x14UL. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To implement Corrected Machine Check Interrupt (CMCI) as another LVT vector, the APIC LVT logic needs to be able to handle an additional LVT vector conditioned on whether MCG_CMCI_P is enabled on the vCPU, this is because CMCI signaling can only be enabled when the CPU's MCG_CMCI_P bit is set (Intel SDM, section 15.3.1.1). This patch factors out the dependency on KVM_APIC_LVT_NUM from the APIC_VERSION macro. In later patches, KVM_APIC_LVT_NUM will be replaced with a helper kvm_apic_get_nr_lvt_entries that reports different LVT number conditioned on whether MCG_CMCI_P is enabled on the vCPU. Suggested-by: Sean Christopherson Signed-off-by: Jue Wang --- arch/x86/kvm/lapic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 66b0eb0bda94..a5caa77e279f 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -54,7 +54,7 @@ #define PRIo64 "o" /* 14 is the version for Xeon and Pentium 8.4.8*/ -#define APIC_VERSION (0x14UL | ((KVM_APIC_LVT_NUM - 1) << 16)) +#define APIC_VERSION 0x14UL #define LAPIC_MMIO_LENGTH (1 << 12) /* followed define is not in apicdef.h */ #define MAX_APIC_VECTOR 256 @@ -401,7 +401,7 @@ static inline int apic_lvt_nmi_mode(u32 lvt_val) void kvm_apic_set_version(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; - u32 v = APIC_VERSION; + u32 v = APIC_VERSION | ((KVM_APIC_LVT_NUM - 1) << 16); if (!lapic_in_kernel(vcpu)) return; From patchwork Fri May 20 17:36:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC0BAC433EF for ; Fri, 20 May 2022 17:36:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352294AbiETRgx (ORCPT ); Fri, 20 May 2022 13:36:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352211AbiETRgs (ORCPT ); Fri, 20 May 2022 13:36:48 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6924981999 for ; Fri, 20 May 2022 10:36:47 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id bj12-20020a056a02018c00b003a9eebaad34so4433773pgb.10 for ; Fri, 20 May 2022 10:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4cDWfCwmCOKbHFB0IhxZNuHaEjb2PhpPJoHO1NQ3cko=; b=IIwVZxJEQ0U2YAqdYGbV7g2xNvT6/zvfWJtEXu/pC2+yzK2pvCrs/TG3OoaLNazycE 5vOOZQMRqFtw8mGjegNItUTKayt3QBHtan4vbMWX4wUPzD+7ip7ldIerxGXE5nBzUK44 TMa/FDlr0j5499nJQLKmFMQfQcW5i4QHFJLwdzQ9Z/u32YuxZx4gHvI87x5pM4GMPTyn qyVT2Tc5MiIMck761QpfXWOLe1UZv7eirGwQvOInZbi9rGSZQSH06R/hZRq9bl4uqg2c 5IkayQ+fguKUYwRC2+e9aQikFAZJhpySxukHgSIppAxw7QPsdiMToq8v1QNVV9ihDPkq oTGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4cDWfCwmCOKbHFB0IhxZNuHaEjb2PhpPJoHO1NQ3cko=; b=P/b6Rok8IbK4SZsoR1iAsRV9DHLCAsEgl5dDrHbBljKivgpCdKqancNLObdIXjuVh8 W8zs1CjAf8eyPWnprpNMVpqa43NotP7VR9Adf7rxm54jOcd5mRi1UksKJpTFtJOVP5N5 A69UIub9pa2ZSNY9/JCIzcqgho7zFeTJwj7eAm4EMNR6B/AZ7AiLtYO4KSOkT+YOwFjh /K4DDujQfZPp17REFrIrkrMAk40oV9Ec3egOujBikzfW+aB11Y4uEoyYV94iIh5XutVH lVKSjvaI9KSwqK8azmeYrmYydwT4mgIjKpYG9NSvgsvcJL7S1aISa9S1dT4mCkUcSRRT VOCA== X-Gm-Message-State: AOAM532Jf2wUoXUFVnOXQjy9bTm/xgurtkPt3SXSjCdKpJu8bSiNW+53 ATwd/V0jgogpcrLnIKC4jIEp90/G X-Google-Smtp-Source: ABdhPJxZhI7upGz/mVmJfn0bCUNEPk5Z59JFVA9gIqkFVYzEdZ4S/CaHhLaIDwWhPbc9L2mr1XrjWF30 X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a17:902:6846:b0:161:5b6f:8995 with SMTP id f6-20020a170902684600b001615b6f8995mr10765465pln.66.1653068206752; Fri, 20 May 2022 10:36:46 -0700 (PDT) Date: Fri, 20 May 2022 10:36:32 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-3-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 2/8] KVM: x86: Fill apic_lvt_mask with enums / explicit entries. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To implement Corrected Machine Check Interrupt (CMCI) as an additional LVT vector, the apic_lvt_mask array needs to handle LVT_CMCI transparently when LVT_CMCI is added. This patch defines a lapic_lvt_entry enum and use its elements as explicit indices to the apic_lvt_mask array, naturally extensible to support an additional LVT_CMCI enum hence apic_lvt_mask element in the future. Suggested-by: Sean Christopherson Signed-off-by: Jue Wang --- arch/x86/kvm/lapic.c | 19 ++++++++++--------- arch/x86/kvm/lapic.h | 12 +++++++++++- 2 files changed, 21 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index a5caa77e279f..73f5cd248a63 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -401,7 +401,7 @@ static inline int apic_lvt_nmi_mode(u32 lvt_val) void kvm_apic_set_version(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; - u32 v = APIC_VERSION | ((KVM_APIC_LVT_NUM - 1) << 16); + u32 v = APIC_VERSION | ((KVM_APIC_MAX_NR_LVT_ENTRIES - 1) << 16); if (!lapic_in_kernel(vcpu)) return; @@ -419,12 +419,13 @@ void kvm_apic_set_version(struct kvm_vcpu *vcpu) kvm_lapic_set_reg(apic, APIC_LVR, v); } -static const unsigned int apic_lvt_mask[KVM_APIC_LVT_NUM] = { - LVT_MASK , /* part LVTT mask, timer mode mask added at runtime */ - LVT_MASK | APIC_MODE_MASK, /* LVTTHMR */ - LVT_MASK | APIC_MODE_MASK, /* LVTPC */ - LINT_MASK, LINT_MASK, /* LVT0-1 */ - LVT_MASK /* LVTERR */ +static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = { + [LVT_TIMER] = LVT_MASK, /* timer mode mask added at runtime */ + [LVT_THERMAL_MONITOR] = LVT_MASK | APIC_MODE_MASK, + [LVT_PERFORMANCE_COUNTER] = LVT_MASK | APIC_MODE_MASK, + [LVT_LINT0] = LINT_MASK, + [LVT_LINT1] = LINT_MASK, + [LVT_ERROR] = LVT_MASK }; static int find_highest_vector(void *bitmap) @@ -2084,7 +2085,7 @@ static int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) int i; u32 lvt_val; - for (i = 0; i < KVM_APIC_LVT_NUM; i++) { + for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) { lvt_val = kvm_lapic_get_reg(apic, APIC_LVTT + 0x10 * i); kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i, @@ -2383,7 +2384,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_apic_set_xapic_id(apic, vcpu->vcpu_id); kvm_apic_set_version(apic->vcpu); - for (i = 0; i < KVM_APIC_LVT_NUM; i++) + for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i, APIC_LVT_MASKED); apic_update_lvtt(apic); if (kvm_vcpu_is_reset_bsp(vcpu) && diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 4e4f8a22754f..4990793c2034 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -10,7 +10,6 @@ #define KVM_APIC_INIT 0 #define KVM_APIC_SIPI 1 -#define KVM_APIC_LVT_NUM 6 #define APIC_SHORT_MASK 0xc0000 #define APIC_DEST_NOSHORT 0x0 @@ -29,6 +28,17 @@ enum lapic_mode { LAPIC_MODE_X2APIC = MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE, }; +enum lapic_lvt_entry { + LVT_TIMER, + LVT_THERMAL_MONITOR, + LVT_PERFORMANCE_COUNTER, + LVT_LINT0, + LVT_LINT1, + LVT_ERROR, + + KVM_APIC_MAX_NR_LVT_ENTRIES, +}; + struct kvm_timer { struct hrtimer timer; s64 period; /* unit: ns */ From patchwork Fri May 20 17:36:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50FF8C433FE for ; Fri, 20 May 2022 17:36:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352289AbiETRg4 (ORCPT ); Fri, 20 May 2022 13:36:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345654AbiETRgu (ORCPT ); Fri, 20 May 2022 13:36:50 -0400 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FC1287202 for ; Fri, 20 May 2022 10:36:48 -0700 (PDT) Received: by mail-pg1-x54a.google.com with SMTP id x16-20020a63f710000000b003f6082673afso4095130pgh.15 for ; Fri, 20 May 2022 10:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=uuOd9ok94EP/uvmfz9OLiHMjZtd/xYxJGu2lVm3MtUM=; b=scril5l+ZemMu34/UcQk1QtKakyvUafIXzF5bQCcgC6Hv+43k/2YYsJCXpBxRHk7xZ uMdlnJ0Kc8ZTMXf07VVIiFTglqdvfZLO8htdo1LU2t2CgnaUUBpgkar7CJGgv3ksyfqp p4W81waHFvq44aDEwxqIrBKuRHErXy9xvuCDsFJPgZXMGObRFGleQQt2H6KBNHRxU+F2 6GcYKXOZFnQ8YBtI21ORZ4j9hxQJUaRTtVSIwPiaY2Y9Rh2YrEgQshkMWj6Wv9XdSyeF 7b8BBSlNkWyQK3Ke186n0MqYm+r7TUdFGlKqmSym5oG/7xkFBdpT7wURyKQGx4+GCXYR 30+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=uuOd9ok94EP/uvmfz9OLiHMjZtd/xYxJGu2lVm3MtUM=; b=nxJ5vjQGYYcpy3sKid3XYPfIPuocUwo0hSCn3qqoxJQ7p71hYfvBaEARqP1+g+YzQ2 uCnqAM/0Yx5v0MyJkeGik75FpTUvnfDgmcURubSwJjxSKhXdjudVysu0LwGuKfTGWUnS ZgpPvdHh83S+6XkFyPT/cYYo9vfAt1uCAu0ArUqiYkvX0GWugnZpKg6IHKri03yRk5/g 4rVABxpDPxb/W9nBNrGS6b0CvYGsXvDMldgv2eiNUZQnkOrTECxOFaP7c7oB4TbgBazg vDVC0Ue0N13ZeiT0IVMGXr2ckmUzXWW5N6PvvwRgf7+DS4PSBWr4v4OHEKxhC6YJPf+H 2P6A== X-Gm-Message-State: AOAM533dyCqA/lvTWN069WMltHccvvLmX7/XU1w7JbWoMH/FCGSOk3eq Z60TmfoZNwAYrM461PvbqSgOVeSL X-Google-Smtp-Source: ABdhPJztmp5xMuFrdfq5fX+gVIV59x2RLBsFj0VhBQK/E5O/8EEOJMg65srELGmOyYd6Yckcck4g+dfj X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a05:6a00:2310:b0:4fa:7eb1:e855 with SMTP id h16-20020a056a00231000b004fa7eb1e855mr11365191pfh.14.1653068208392; Fri, 20 May 2022 10:36:48 -0700 (PDT) Date: Fri, 20 May 2022 10:36:33 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-4-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 3/8] KVM: x86: Add APIC_LVTx() macro. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To implement Corrected Machine Check Interrupt (CMCI) as an additional LVT vector, the code needs to be able to calculate the APIC_LVTx register offset based on the register indices in the lapic_lvt_entry enum which will be used in all places looping through supported APIC_LVTx registers. APIC_LVTx macro is introduced for this purpose. Suggested-by: Sean Christopherson Signed-off-by: Jue Wang --- arch/x86/kvm/lapic.c | 7 +++---- arch/x86/kvm/lapic.h | 2 ++ 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 73f5cd248a63..db12d2ef1aef 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2086,9 +2086,8 @@ static int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) u32 lvt_val; for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) { - lvt_val = kvm_lapic_get_reg(apic, - APIC_LVTT + 0x10 * i); - kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i, + lvt_val = kvm_lapic_get_reg(apic, APIC_LVTx(i)); + kvm_lapic_set_reg(apic, APIC_LVTx(i), lvt_val | APIC_LVT_MASKED); } apic_update_lvtt(apic); @@ -2385,7 +2384,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_apic_set_version(apic->vcpu); for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) - kvm_lapic_set_reg(apic, APIC_LVTT + 0x10 * i, APIC_LVT_MASKED); + kvm_lapic_set_reg(apic, APIC_LVTx(i), APIC_LVT_MASKED); apic_update_lvtt(apic); if (kvm_vcpu_is_reset_bsp(vcpu) && kvm_check_has_quirk(vcpu->kvm, KVM_X86_QUIRK_LINT0_REENABLED)) diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 4990793c2034..2d197ed0b8ce 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -39,6 +39,8 @@ enum lapic_lvt_entry { KVM_APIC_MAX_NR_LVT_ENTRIES, }; +#define APIC_LVTx(x) (APIC_LVTT + 0x10 * (x)) + struct kvm_timer { struct hrtimer timer; s64 period; /* unit: ns */ From patchwork Fri May 20 17:36:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE1FAC433F5 for ; Fri, 20 May 2022 17:36:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352290AbiETRg5 (ORCPT ); Fri, 20 May 2022 13:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352292AbiETRgw (ORCPT ); Fri, 20 May 2022 13:36:52 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC5DB80221 for ; Fri, 20 May 2022 10:36:50 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id u1-20020a17090a2b8100b001d9325a862fso4558280pjd.6 for ; Fri, 20 May 2022 10:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=WfQVdpk3KC0ZrKW0uOOMpSnVFPF0oApLtFRNK/IW/54=; b=jl9x8sHu3l+6astnzShzJE5hVWme/+zuCxpnKR9kTiB7o9kJFwoNXZ1TDDXSAs1eik dURXZJfLXxPKswZXNFfJDFwqnSXl4bMhJvSd2cHNS/1hbW1LRJFI0US0VMqMvvXr6Ly0 lr34Ri7NZZMq9l9woAgG+Z5d928QUps/f1HOrcVy6GeQ6wfLTI3bYeJMQHMJj9tNOqvh rb4WBIKXSh2uQt7nsxsgQJPGYxs79JfSXgtr+uVLE6yfLmTZ4d+dB1kRTBw4/pUPRkPC eHhUjL800IxsPTs4GauYByCovtT/ZG0vYYs+p+b1VJ+KpBsGiZ5z7IafYRL7yfg8OP37 YuIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WfQVdpk3KC0ZrKW0uOOMpSnVFPF0oApLtFRNK/IW/54=; b=PL5QCy7c6Q8q0a+rNLilVMTZ4iwPgmYYepoc3Cey8uqrBq8xvoPJnxx9Sbrch8/CyS /mQ6A9N1uYM+ljS10guffs69o+lx2+hyRlZnixVPbaIotvyPvPRctUooiAghp/7+NMGB enlU1X5nCdrQY2Z/eAt7I8sim3/rgINHEy5wIjjGMOy81oLQRPY7ofAGYODxvyhkEsCx YKzOGBL1hQArzEzRMMh6RyGLsPC2YpApgI39YZENoC5OopgH3JNf6MwHmTQcA7ECnW8+ wYqCUuW+xIC3FAAZ6bie65LbM/0q/m52tQetuIHovwSaR1U2CkIn/i3Ajmku5gw4ki6T z5Ug== X-Gm-Message-State: AOAM530TKbrLJMp69kNB5QN2X9FOwip7gOIHrpCq0m/RUsozFItvcdee /uGMiyYtvEeLYicyQIeVPj4hBBBN X-Google-Smtp-Source: ABdhPJxbzbYd1i+dndeXTNOGvIP/tAZfeW4D00c79VNIkJGIY0714UqdPXvzZvpzjYurgczyn+o6ecoC X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a62:cd0b:0:b0:518:11b3:c9f with SMTP id o11-20020a62cd0b000000b0051811b30c9fmr11238387pfg.46.1653068210183; Fri, 20 May 2022 10:36:50 -0700 (PDT) Date: Fri, 20 May 2022 10:36:34 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-5-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 4/8] KVM: x86: Add Corrected Machine Check Interrupt (CMCI) emulation to lapic. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch adds the handling of APIC_LVTCMCI, conditioned on whether the vCPU has set MCG_CMCI_P in MCG_CAP register. Signed-off-by: Jue Wang --- arch/x86/kvm/lapic.c | 40 +++++++++++++++++++++++++++++++++------- arch/x86/kvm/lapic.h | 3 ++- 2 files changed, 35 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index db12d2ef1aef..e2186a7c0eed 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -398,14 +399,26 @@ static inline int apic_lvt_nmi_mode(u32 lvt_val) return (lvt_val & (APIC_MODE_MASK | APIC_LVT_MASKED)) == APIC_DM_NMI; } +static inline bool kvm_is_cmci_supported(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.mcg_cap & MCG_CMCI_P; +} + +static inline int kvm_apic_get_nr_lvt_entries(struct kvm_lapic *apic) +{ + return KVM_APIC_MAX_NR_LVT_ENTRIES - !kvm_is_cmci_supported(apic->vcpu); +} + void kvm_apic_set_version(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; - u32 v = APIC_VERSION | ((KVM_APIC_MAX_NR_LVT_ENTRIES - 1) << 16); + u32 v = 0; if (!lapic_in_kernel(vcpu)) return; + v = APIC_VERSION | ((kvm_apic_get_nr_lvt_entries(apic) - 1) << 16); + /* * KVM emulates 82093AA datasheet (with in-kernel IOAPIC implementation) * which doesn't have EOI register; Some buggy OSes (e.g. Windows with @@ -425,7 +438,8 @@ static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = { [LVT_PERFORMANCE_COUNTER] = LVT_MASK | APIC_MODE_MASK, [LVT_LINT0] = LINT_MASK, [LVT_LINT1] = LINT_MASK, - [LVT_ERROR] = LVT_MASK + [LVT_ERROR] = LVT_MASK, + [LVT_CMCI] = LVT_MASK | APIC_MODE_MASK }; static int find_highest_vector(void *bitmap) @@ -1445,6 +1459,9 @@ static int kvm_lapic_reg_read(struct kvm_lapic *apic, u32 offset, int len, APIC_REG_MASK(APIC_TMCCT) | APIC_REG_MASK(APIC_TDCR); + if (kvm_is_cmci_supported(apic->vcpu)) + valid_reg_mask |= APIC_REG_MASK(APIC_LVTCMCI); + /* * ARBPRI and ICR2 are not valid in x2APIC mode. WARN if KVM reads ICR * in x2APIC mode as it's an 8-byte register in x2APIC and needs to be @@ -2083,12 +2100,10 @@ static int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) apic_set_spiv(apic, val & mask); if (!(val & APIC_SPIV_APIC_ENABLED)) { int i; - u32 lvt_val; - for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) { - lvt_val = kvm_lapic_get_reg(apic, APIC_LVTx(i)); + for (i = 0; i < kvm_apic_get_nr_lvt_entries(apic); i++) { kvm_lapic_set_reg(apic, APIC_LVTx(i), - lvt_val | APIC_LVT_MASKED); + kvm_lapic_get_reg(apic, APIC_LVTx(i)) | APIC_LVT_MASKED); } apic_update_lvtt(apic); atomic_set(&apic->lapic_timer.pending, 0); @@ -2140,6 +2155,17 @@ static int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) apic_update_lvtt(apic); break; + case APIC_LVTCMCI: + if (!kvm_is_cmci_supported(apic->vcpu)) { + ret = 1; + break; + } + if (!kvm_apic_sw_enabled(apic)) + val |= APIC_LVT_MASKED; + val &= apic_lvt_mask[LVT_CMCI]; + kvm_lapic_set_reg(apic, APIC_LVTCMCI, val); + break; + case APIC_TMICT: if (apic_lvtt_tscdeadline(apic)) break; @@ -2383,7 +2409,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) kvm_apic_set_xapic_id(apic, vcpu->vcpu_id); kvm_apic_set_version(apic->vcpu); - for (i = 0; i < KVM_APIC_MAX_NR_LVT_ENTRIES; i++) + for (i = 0; i < kvm_apic_get_nr_lvt_entries(apic); i++) kvm_lapic_set_reg(apic, APIC_LVTx(i), APIC_LVT_MASKED); apic_update_lvtt(apic); if (kvm_vcpu_is_reset_bsp(vcpu) && diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 2d197ed0b8ce..16298bcb2abf 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -35,11 +35,12 @@ enum lapic_lvt_entry { LVT_LINT0, LVT_LINT1, LVT_ERROR, + LVT_CMCI, KVM_APIC_MAX_NR_LVT_ENTRIES, }; -#define APIC_LVTx(x) (APIC_LVTT + 0x10 * (x)) +#define APIC_LVTx(x) ((x) == LVT_CMCI ? APIC_LVTCMCI : APIC_LVTT + 0x10 * (x)) struct kvm_timer { struct hrtimer timer; From patchwork Fri May 20 17:36:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE4E2C433EF for ; Fri, 20 May 2022 17:37:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352267AbiETRhA (ORCPT ); Fri, 20 May 2022 13:37:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352300AbiETRgz (ORCPT ); Fri, 20 May 2022 13:36:55 -0400 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 021428CB3D for ; Fri, 20 May 2022 10:36:53 -0700 (PDT) Received: by mail-pg1-x549.google.com with SMTP id 185-20020a6304c2000000b003f5d1f7f49aso4451846pge.7 for ; Fri, 20 May 2022 10:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0vS5tkNj/9TlbMnAs+B1cjDPjcuZkRtuDIjGGmBhLLU=; b=PEmM1RGmVuLBu120z/RlrWPyZpLJYhPj+jDp22paKqbA/AW15hv5/0aK0v72q/e89H z8Ct1YOmwPpr+CX+7sNGXnn1e5P+zjwlLZvf/uuu1OeIe7khyBlkltTJPHDywwSyI9PY 7HYYg1JHOnR8Mmq0hY8xzTSngiCMWr5WMTVpDuO7OYY4+syqH3vCoEfwlsUwzuC12is9 7IFRDUTL3i95C5s9zR00oBMqY8T0QVO4FnbmX9VK32pEci18oxqSckrsWr+cVAURioQe di6DfTnjJkDs0c2fzNJzC2JOOVU0/R5DCScyVS+DcvM2YjSm+j1CSM5d94jlaRrGcMVl LkTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0vS5tkNj/9TlbMnAs+B1cjDPjcuZkRtuDIjGGmBhLLU=; b=TRumoWcRImyC5DnYjyJ6lVz4pjmOwJzesjzqFyHyk8FbS88MenbJ8DTbvS0zh1Err2 3Dh4JvW7gixYPvpbQ/GNupRQfiisI8wI2CYfqrWo7nAmPgWblZ2cDyAeWTqRhMuVQ0/y R7UwEC+8mzJuOecHJrD6BjwUT2ApkJCweee3Z8nFfwdSjdZoF5fnWkx/HC2NBJBKLqLN 9b9LTR+ktbdB9wdKTPV062trx0UvHuivSbx0GU6RI/GNOYnNyxY6d9hdu4oSWI7Rn0Om kbWALKmn6gBvsZrD5PgZRImA6kmeNQCt02dDtR8MZIpTwqq459qL9apBMtA4tWUq0QFd 0j7g== X-Gm-Message-State: AOAM530PQpb8TBlz5cNl2bU/FTs6wy6qkb8j3dW/bAtjR5MEgLsSRhnw zfP5Tc46I+FRXSDSXY47sJFIlULP X-Google-Smtp-Source: ABdhPJyhNjLRNc0GopP5PHCHktoHPui+KBu6dJbbM+pH4fINJdFd4yaSbUXeatsX2631xXX4JnzbxL/Q X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a05:6a00:2450:b0:4f7:bf07:c063 with SMTP id d16-20020a056a00245000b004f7bf07c063mr10926460pfj.51.1653068213138; Fri, 20 May 2022 10:36:53 -0700 (PDT) Date: Fri, 20 May 2022 10:36:35 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-6-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 5/8] KVM: x86: Use kcalloc to allocate the mce_banks array. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Corrected Machine Check Interrupt (CMCI) can be configured via the per Machine Check bank registers: IA32_MCI_CTL2. To emulate IA32_MCI_CTL2 registers, it's necessary to introduce another array mci_ctl2_banks in analogy to the mce_banks array under struct kvm_vcpu_arch. This patch updates the allocation of mce_banks with the array allocation API (kcalloc) as a precedent for the later mci_ctl2_banks. Suggested-by: Sean Christopherson Signed-off-by: Jue Wang --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4790f0d7d40b..0e839077ce52 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11224,7 +11224,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) goto fail_free_lapic; vcpu->arch.pio_data = page_address(page); - vcpu->arch.mce_banks = kzalloc(KVM_MAX_MCE_BANKS * sizeof(u64) * 4, + vcpu->arch.mce_banks = kcalloc(KVM_MAX_MCE_BANKS * 4, sizeof(u64), GFP_KERNEL_ACCOUNT); if (!vcpu->arch.mce_banks) goto fail_free_pio_data; From patchwork Fri May 20 17:36:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13BF0C433F5 for ; Fri, 20 May 2022 17:37:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352315AbiETRhC (ORCPT ); Fri, 20 May 2022 13:37:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352291AbiETRg6 (ORCPT ); Fri, 20 May 2022 13:36:58 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DAB28CCDE for ; Fri, 20 May 2022 10:36:56 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id j23-20020aa78017000000b005180c6e4ef2so4546507pfi.12 for ; Fri, 20 May 2022 10:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GkWWUJ5UHxNAMkMwmQ8pt3Dz4eac3cPZcPaxr3WNiBY=; b=YwyQ8S8284TzSQLhmWCpvf+otP9mk9dVBBrQuujKvlQKf7/VyhEZXz6a3o19skP7T6 BkS5704oyu7u+E5COhBFVgsoYI/eO7WE+7QVRfXR0y1KGMVfzrkVLFdImReuBYW0cloY +m3bQrySDDS87yiS9vahC95njn25+9htpVwnBxxjq59YsxNjy1DVPpIfOPy22I/W+tTj OSdb9Yu+KkFaJV9MzB8R2oq5EpSNNQyPXOxPofbvSkLmTuQUAsfUe4QorN41PgtYNu7g rmuKAfxCtimg0ACY12Vr+kc3dQsX/Ne5f1hhWhvagEiuE6wv9QkK/xOlqalKWT1Ka8Ky aKTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GkWWUJ5UHxNAMkMwmQ8pt3Dz4eac3cPZcPaxr3WNiBY=; b=eCeJNGzrRAC2UFtCmxn5XI2CI88usU/miuM7pvY1iLVgFFQ/98ddZRSyAHqsn20yoJ v5hSQzNtxIPqyDV94kn7D+Hxsjac/eLdOT3lHAsaoiWPXqr4Jw4Ad66Al4TkvFpp4HaW 3pmiLU1gC0bniYvHsCwpUilCa/b+tEeVDESOh4OqbALOkS7T5czfvAiYpPyTEu96Y9vM ZKVmuK7EdMwdRy8UqBMfuHPxvFgPbWkeorm6dXc2mHNIjXK7AjW4xwArLTObPsy9Zg3l 3mBP5VKKtgllCNzeip3p4sjUQLiy9UL/WUmMXPuEZwdn53TYqz7obfO54njG6Fr7rzn3 Q20A== X-Gm-Message-State: AOAM530vUuLpqrCMO+fSH1alIKRNrRDCfqXz79YG/ib3O4AZlUeMpi5m 64Z8fLOOMSzRQD8NfcIe2L8Xdfe8 X-Google-Smtp-Source: ABdhPJww4OQ7R2geI+UnDtj2h18YQ/m+7uHdW/mwiZ2s9dXuV13il9Z2dOyyZCKbo/hA/Bv4eHbzjzV7 X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a17:90b:3b81:b0:1dc:32ac:a66b with SMTP id pc1-20020a17090b3b8100b001dc32aca66bmr12135469pjb.49.1653068216331; Fri, 20 May 2022 10:36:56 -0700 (PDT) Date: Fri, 20 May 2022 10:36:36 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-7-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 6/8] KVM: x86: Add emulation for MSR_IA32_MCx_CTL2 MSRs. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Corrected Machine Check Interrupt (CMCI) can be configured via the per Machine Check bank registers: IA32_MCi_CTL2. This patch adds the emulation of IA32_MCi_CTL2 registers to KVM. A separate mci_ctl2_banks array is used to keep the existing mce_banks register layout intact. In Machine Check Architecture (MCA), MCG_CMCI_P (bit 10 of MCG_CAP) is the corrected MC error counting/signaling extension present flag. When this bit is set, it does not imply CMCI reported corrected error or UCNA error is supported across all MCA banks. Software should check on a bank by bank basis (i.e. if bit 30 in each IA32_MCi_CTL2 register is set). Signed-off-by: Jue Wang --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 130 ++++++++++++++++++++++---------- 2 files changed, 92 insertions(+), 39 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 4ff36610af6a..178b7e01bf8f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -806,6 +806,7 @@ struct kvm_vcpu_arch { u64 mcg_ctl; u64 mcg_ext_ctl; u64 *mce_banks; + u64 *mci_ctl2_banks; /* Cache MMIO info */ u64 mmio_gva; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0e839077ce52..f8ab592f519b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3174,6 +3174,16 @@ static void kvmclock_sync_fn(struct work_struct *work) KVMCLOCK_SYNC_PERIOD); } +/* These helpers are safe iff @msr is known to be an MCx bank MSR. */ +static bool is_mci_control_msr(u32 msr) +{ + return (msr & 3) == 0; +} +static bool is_mci_status_msr(u32 msr) +{ + return (msr & 3) == 1; +} + /* * On AMD, HWCR[McStatusWrEn] controls whether setting MCi_STATUS results in #GP. */ @@ -3192,6 +3202,7 @@ static int set_msr_mce(struct kvm_vcpu *vcpu, struct msr_data *msr_info) unsigned bank_num = mcg_cap & 0xff; u32 msr = msr_info->index; u64 data = msr_info->data; + u32 offset, last_msr; switch (msr) { case MSR_IA32_MCG_STATUS: @@ -3205,32 +3216,50 @@ static int set_msr_mce(struct kvm_vcpu *vcpu, struct msr_data *msr_info) return 1; vcpu->arch.mcg_ctl = data; break; - default: - if (msr >= MSR_IA32_MC0_CTL && - msr < MSR_IA32_MCx_CTL(bank_num)) { - u32 offset = array_index_nospec( - msr - MSR_IA32_MC0_CTL, - MSR_IA32_MCx_CTL(bank_num) - MSR_IA32_MC0_CTL); - - /* only 0 or all 1s can be written to IA32_MCi_CTL - * some Linux kernels though clear bit 10 in bank 4 to - * workaround a BIOS/GART TBL issue on AMD K8s, ignore - * this to avoid an uncatched #GP in the guest - */ - if ((offset & 0x3) == 0 && - data != 0 && (data | (1 << 10)) != ~(u64)0) - return -1; - - /* MCi_STATUS */ - if (!msr_info->host_initiated && - (offset & 0x3) == 1 && data != 0) { - if (!can_set_mci_status(vcpu)) - return -1; - } + case MSR_IA32_MC0_CTL2 ... MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) - 1: + last_msr = MSR_IA32_MCx_CTL2(bank_num) - 1; + if (msr > last_msr) + return 1; - vcpu->arch.mce_banks[offset] = data; - break; - } + if (!(mcg_cap & MCG_CMCI_P) && (data || !msr_info->host_initiated)) + return 1; + /* An attempt to write a 1 to a reserved bit raises #GP */ + if (data & ~(MCI_CTL2_CMCI_EN | MCI_CTL2_CMCI_THRESHOLD_MASK)) + return 1; + offset = array_index_nospec(msr - MSR_IA32_MC0_CTL2, + last_msr + 1 - MSR_IA32_MC0_CTL2); + vcpu->arch.mci_ctl2_banks[offset] = data; + break; + case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: + last_msr = MSR_IA32_MCx_CTL(bank_num) - 1; + if (msr > last_msr) + return 1; + + /* + * Only 0 or all 1s can be written to IA32_MCi_CTL, all other + * values are architecturally undefined. But, some Linux + * kernels clear bit 10 in bank 4 to workaround a BIOS/GART TLB + * issue on AMD K8s, allow bit 10 to be clear when setting all + * other bits in order to avoid an uncaught #GP in the guest. + */ + if (is_mci_control_msr(msr) && + data != 0 && (data | (1 << 10)) != ~(u64)0) + return 1; + + /* + * All CPUs allow writing 0 to MCi_STATUS MSRs to clear the MSR. + * AMD-based CPUs allow non-zero values, but if and only if + * HWCR[McStatusWrEn] is set. + */ + if (!msr_info->host_initiated && is_mci_status_msr(msr) && + data != 0 && !can_set_mci_status(vcpu)) + return 1; + + offset = array_index_nospec(msr - MSR_IA32_MC0_CTL, + last_msr + 1 - MSR_IA32_MC0_CTL); + vcpu->arch.mce_banks[offset] = data; + break; + default: return 1; } return 0; @@ -3514,7 +3543,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) return 1; } break; - case 0x200 ... 0x2ff: + case 0x200 ... MSR_IA32_MC0_CTL2 - 1: + case MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) ... 0x2ff: return kvm_mtrr_set_msr(vcpu, msr, data); case MSR_IA32_APICBASE: return kvm_set_apic_base(vcpu, msr_info); @@ -3671,6 +3701,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) case MSR_IA32_MCG_CTL: case MSR_IA32_MCG_STATUS: case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: + case MSR_IA32_MC0_CTL2 ... MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) - 1: return set_msr_mce(vcpu, msr_info); case MSR_K7_PERFCTR0 ... MSR_K7_PERFCTR3: @@ -3775,6 +3806,7 @@ static int get_msr_mce(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host) u64 data; u64 mcg_cap = vcpu->arch.mcg_cap; unsigned bank_num = mcg_cap & 0xff; + u32 offset, last_msr; switch (msr) { case MSR_IA32_P5_MC_ADDR: @@ -3792,16 +3824,27 @@ static int get_msr_mce(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, bool host) case MSR_IA32_MCG_STATUS: data = vcpu->arch.mcg_status; break; - default: - if (msr >= MSR_IA32_MC0_CTL && - msr < MSR_IA32_MCx_CTL(bank_num)) { - u32 offset = array_index_nospec( - msr - MSR_IA32_MC0_CTL, - MSR_IA32_MCx_CTL(bank_num) - MSR_IA32_MC0_CTL); + case MSR_IA32_MC0_CTL2 ... MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) - 1: + last_msr = MSR_IA32_MCx_CTL2(bank_num) - 1; + if (msr > last_msr) + return 1; - data = vcpu->arch.mce_banks[offset]; - break; - } + if (!(mcg_cap & MCG_CMCI_P) && !host) + return 1; + offset = array_index_nospec(msr - MSR_IA32_MC0_CTL2, + last_msr + 1 - MSR_IA32_MC0_CTL2); + data = vcpu->arch.mci_ctl2_banks[offset]; + break; + case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: + last_msr = MSR_IA32_MCx_CTL(bank_num) - 1; + if (msr > last_msr) + return 1; + + offset = array_index_nospec(msr - MSR_IA32_MC0_CTL, + last_msr + 1 - MSR_IA32_MC0_CTL); + data = vcpu->arch.mce_banks[offset]; + break; + default: return 1; } *pdata = data; @@ -3898,7 +3941,8 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) break; } case MSR_MTRRcap: - case 0x200 ... 0x2ff: + case 0x200 ... MSR_IA32_MC0_CTL2 - 1: + case MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) ... 0x2ff: return kvm_mtrr_get_msr(vcpu, msr_info->index, &msr_info->data); case 0xcd: /* fsb frequency */ msr_info->data = 3; @@ -4014,6 +4058,7 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) case MSR_IA32_MCG_CTL: case MSR_IA32_MCG_STATUS: case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: + case MSR_IA32_MC0_CTL2 ... MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) - 1: return get_msr_mce(vcpu, msr_info->index, &msr_info->data, msr_info->host_initiated); case MSR_IA32_XSS: @@ -4769,9 +4814,12 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu, /* Init IA32_MCG_CTL to all 1s */ if (mcg_cap & MCG_CTL_P) vcpu->arch.mcg_ctl = ~(u64)0; - /* Init IA32_MCi_CTL to all 1s */ - for (bank = 0; bank < bank_num; bank++) + /* Init IA32_MCi_CTL to all 1s, IA32_MCi_CTL2 to all 0s */ + for (bank = 0; bank < bank_num; bank++) { vcpu->arch.mce_banks[bank*4] = ~(u64)0; + if (mcg_cap & MCG_CMCI_P) + vcpu->arch.mci_ctl2_banks[bank] = 0; + } static_call(kvm_x86_setup_mce)(vcpu); out: @@ -11226,7 +11274,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) vcpu->arch.mce_banks = kcalloc(KVM_MAX_MCE_BANKS * 4, sizeof(u64), GFP_KERNEL_ACCOUNT); - if (!vcpu->arch.mce_banks) + vcpu->arch.mci_ctl2_banks = kcalloc(KVM_MAX_MCE_BANKS, sizeof(u64), + GFP_KERNEL_ACCOUNT); + if (!vcpu->arch.mce_banks || !vcpu->arch.mci_ctl2_banks) goto fail_free_pio_data; vcpu->arch.mcg_cap = KVM_MAX_MCE_BANKS; @@ -11279,6 +11329,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) free_cpumask_var(vcpu->arch.wbinvd_dirty_mask); fail_free_mce_banks: kfree(vcpu->arch.mce_banks); + kfree(vcpu->arch.mci_ctl2_banks); fail_free_pio_data: free_page((unsigned long)vcpu->arch.pio_data); fail_free_lapic: @@ -11323,6 +11374,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) kvm_hv_vcpu_uninit(vcpu); kvm_pmu_destroy(vcpu); kfree(vcpu->arch.mce_banks); + kfree(vcpu->arch.mci_ctl2_banks); kvm_free_lapic(vcpu); idx = srcu_read_lock(&vcpu->kvm->srcu); kvm_mmu_destroy(vcpu); From patchwork Fri May 20 17:36:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA064C433EF for ; Fri, 20 May 2022 17:37:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352326AbiETRhG (ORCPT ); Fri, 20 May 2022 13:37:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352311AbiETRhC (ORCPT ); Fri, 20 May 2022 13:37:02 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00F9B84A08 for ; Fri, 20 May 2022 10:36:59 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id me18-20020a17090b17d200b001dfa3d25c37so5890998pjb.8 for ; Fri, 20 May 2022 10:36:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DV2RMsryiu11Btf+O0wwsjjvh9trhw/nWQ7OXrbtao8=; b=Mp7p/79Xi+0P9FW+2vqGO4EMCA3GVyk0XdRpgRPMpzaYw/KWkaXmBVOufpTf/65ES5 ZIv74+4o9Mh4fvdl0gzjGZBPyQrQw/eKR13pPVSJqKVxIkakuHKMPWwqYjYYeROyh+7i Os9UThXKkSrS/erRbzl4ZmFLbWyn4DB/7Hz8ilze/lfNACwglaIS7WBNM7Y3swJtf6k6 SHF7Ky+jxWXaxiYtjw+Spq6BByv7nINJEegt4YqtFYvVDFwnJb/semYyqXG3VqM64Bcc W9y1B5XIbcfHypV8Hbwp+fIH4S6dPS2AVFK0nH2OwpewrmNeEgIdgSdOY9vyLe20FRim 6j7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DV2RMsryiu11Btf+O0wwsjjvh9trhw/nWQ7OXrbtao8=; b=3pww7k2B6JqdJpCJogKVwF/45sthJn+5/XViIBlR/Ovw0iVzHM0J5+E3HVisJwewef ti1eAbqhukfmmaoa9RC97flNReE/9qvz5pK2b6g7JFvnrzluyxOSmG/eitpkkqQnzntq dLa0O6L64ZpvgK6HzVquIomKieNQ3BCuFCCsSVWZTscijaDB+2FYGZ9BOBPgVP0EWZVl Xcw+QM8bdVkiQ4H4q70JpHREBHB4QzFZDxkyV5yCZMqOQx0NZHcdoG9mP2DmPPgKaZdi ajv/8pjFv+Pv+Ko2NyEKw7TUjCzSUjT3nanX4mTf3Y9m9PEWRU1rDrjmrDudaljYcbl/ MXpw== X-Gm-Message-State: AOAM531bD53D5VfS+JM0OD/tbkyZDvXoEpsrSP/tQriOhXUdEDAb6LHy Jmv1mzX7Uhsk37MFkRC+6fteP0Rg X-Google-Smtp-Source: ABdhPJw9TnfuptiNDRCpELFyZlALfBKb0SvjM8EcCkmc5pf4/llS/lf7n+hGP5SI1IoLSlFSMiO5YzDK X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a05:6a00:1348:b0:518:6ef5:ac3 with SMTP id k8-20020a056a00134800b005186ef50ac3mr4228587pfu.69.1653068218884; Fri, 20 May 2022 10:36:58 -0700 (PDT) Date: Fri, 20 May 2022 10:36:37 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-8-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [PATCH v4 7/8] KVM: x86: Enable CMCI capability by default and handle injected UCNA errors From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Make KVM support the CMCI capability by default by adding MCG_CMCI_P to kvm_mce_cap_supported. A vCPU can request for this capability via KVM_X86_SETUP_MCE. Uncorrectable Error No Action required (UCNA) injection reuses the MCE injection ioctl KVM_X86_SET_MCE. Neither of the CMCI and UCNA emulations depends on hardware. Signed-off-by: Jue Wang --- arch/x86/kvm/vmx/vmx.c | 1 + arch/x86/kvm/x86.c | 50 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 610355b9ccce..1aed964ee4ee 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8037,6 +8037,7 @@ static __init int hardware_setup(void) } kvm_mce_cap_supported |= MCG_LMCE_P; + kvm_mce_cap_supported |= MCG_CMCI_P; if (pt_mode != PT_MODE_SYSTEM && pt_mode != PT_MODE_HOST_GUEST) return -EINVAL; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index f8ab592f519b..d0b1bb6e5e4a 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4826,6 +4826,52 @@ static int kvm_vcpu_ioctl_x86_setup_mce(struct kvm_vcpu *vcpu, return r; } +/* + * Validate this is an UCNA error by checking the MCG_STATUS and MCi_STATUS + * registers that none of the bits for Machine Check Exceptions are set and + * both the VAL (valid) and UC (uncorrectable) bits are set. + * UCNA - UnCorrectable No Action required + * SRAR - Software Recoverable Action Required + * MCI_STATUS_PCC - Processor Context Corrupted + * MCI_STATUS_S - Signaled as a Machine Check Exception + * MCI_STATUS_AR - This MCE is "software recoverable action required" + */ +static bool is_ucna(struct kvm_x86_mce *mce) +{ + return !mce->mcg_status && + !(mce->status & (MCI_STATUS_PCC | MCI_STATUS_S | MCI_STATUS_AR)) && + (mce->status & MCI_STATUS_VAL) && + (mce->status & MCI_STATUS_UC); +} + +static int kvm_vcpu_x86_set_ucna(struct kvm_vcpu *vcpu, struct kvm_x86_mce *mce) +{ + u64 mcg_cap = vcpu->arch.mcg_cap; + unsigned int bank_num = mcg_cap & 0xff; + u64 *banks = vcpu->arch.mce_banks; + + if (mce->bank >= bank_num) + return -EINVAL; + + if (!is_ucna(mce)) + return -EINVAL; + + banks += 4 * mce->bank; + banks[1] = mce->status; + banks[2] = mce->addr; + banks[3] = mce->misc; + vcpu->arch.mcg_status = mce->mcg_status; + + if (!(mcg_cap & MCG_CMCI_P) || + !(vcpu->arch.mci_ctl2_banks[mce->bank] & MCI_CTL2_CMCI_EN)) + return 0; + + if (lapic_in_kernel(vcpu)) + kvm_apic_local_deliver(vcpu->arch.apic, APIC_LVTCMCI); + + return 0; +} + static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu, struct kvm_x86_mce *mce) { @@ -4835,6 +4881,10 @@ static int kvm_vcpu_ioctl_x86_set_mce(struct kvm_vcpu *vcpu, if (mce->bank >= bank_num || !(mce->status & MCI_STATUS_VAL)) return -EINVAL; + + if (is_ucna(mce)) + return kvm_vcpu_x86_set_ucna(vcpu, mce); + /* * if IA32_MCG_CTL is not all 1s, the uncorrected error * reporting is disabled From patchwork Fri May 20 17:36:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jue Wang X-Patchwork-Id: 12857055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6FEDC433FE for ; Fri, 20 May 2022 17:37:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352313AbiETRhH (ORCPT ); Fri, 20 May 2022 13:37:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36246 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352325AbiETRhE (ORCPT ); Fri, 20 May 2022 13:37:04 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2D52939FD for ; Fri, 20 May 2022 10:37:01 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id g3-20020a17090a128300b001df6b8706e3so5008794pja.9 for ; Fri, 20 May 2022 10:37:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=JlkDy6zr1BhiWoo/HTRnV5a5QX+CIhKUQxFVb56phFo=; b=jh6wg3UAnq+/6GQqO9PI8T8UBmoc0dX5GqEFmFACOQqmBm4sQndx3DUx1SMazlzyZT S8sPKTt6m8KBh2SHIN8GJxu3Pm/TDOlXLJfJBZFiofui9QGucarnHmqXYOdrE+E51lgC dR4ZjHqD861q6mJeOGgd0/c4K/zgNaMZXgbN7FZVcawVlvF+9QFxXQBNQQwp+Hj0+Kcp DvMZQDdkAUDXQOGfXMMmQB7dr1isy01/9hkQhT2ZYWitaD6e5cF8LsSrZ63f91XNsTRp 3PtKOWvEJAnMWPKZgCFG/hToe6om6Fw+1MSJvA4r318SV0sbQiaHYvFH7jfzpn1zthBI Enrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JlkDy6zr1BhiWoo/HTRnV5a5QX+CIhKUQxFVb56phFo=; b=S6aAGJwV9kiH5ha5o1LVtAfTYXDC90cg5Rwu0VAelOvXnyp/tfdrLW8qTBbot2ftQZ m+uZveOlLelAb2uRLxKDqfMeTNaJw3XnCz27+fuhhyXUxN3fqRtx2jqbItkdEwkz8Hkj +RO+DtpIrZBjFbe/45li979ML4dIetR95AZ0XaPyQuwzj3C+fuqwvH6fKEzXBvHDaD6i CuJ1Z8w70Kr+ly89R4S9vgfciTAN15KOBrFiqq2TWppJS+Tkc5sVqQ3SYCNbEibCxEuU VYBNX3ZeROmFsoRjXE3ohDgp7MIQg5buqNZ+nWYdBFOtQRniX/bblWSwteSV9X6jb2Z5 oBRQ== X-Gm-Message-State: AOAM532s+g63xNGyCDmTIm853Efkea5ZWQScekWVoab/oM64vDlvr0F0 OUKeDpLbzM9+5e1ObPpGNwoCuS/B X-Google-Smtp-Source: ABdhPJxP1Rk7Ndw8Y3O9qn2iAKuc6aQDrNbC4AYjpzPkhe1VO8M0cRjiWolEy9TTAQd4wtr4Dxt6mMrc X-Received: from juew-desktop.sea.corp.google.com ([2620:15c:100:202:4c5:ddc5:8182:560f]) (user=juew job=sendgmr) by 2002:a05:6a00:234f:b0:4f6:f0c0:ec68 with SMTP id j15-20020a056a00234f00b004f6f0c0ec68mr11160263pfj.14.1653068221318; Fri, 20 May 2022 10:37:01 -0700 (PDT) Date: Fri, 20 May 2022 10:36:38 -0700 In-Reply-To: <20220520173638.94324-1-juew@google.com> Message-Id: <20220520173638.94324-9-juew@google.com> Mime-Version: 1.0 References: <20220520173638.94324-1-juew@google.com> X-Mailer: git-send-email 2.36.1.124.g0e6072fb45-goog Subject: [RFC v4 8/8] KVM: selftests: Add a self test for UCNA injection. From: Jue Wang To: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , David Matlack Cc: Tony Luck , kvm@vger.kernel.org, Greg Thelen , Jiaqi Yan , Jue Wang Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch add a self test that verifies user space can inject UnCorrectable No Action required (UCNA) memory errors to the guest. Signed-off-by: Jue Wang --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/apic.h | 1 + .../selftests/kvm/include/x86_64/mce.h | 25 ++ .../selftests/kvm/include/x86_64/processor.h | 1 + .../selftests/kvm/lib/x86_64/processor.c | 2 +- .../kvm/x86_64/ucna_injection_test.c | 287 ++++++++++++++++++ 7 files changed, 317 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/kvm/include/x86_64/mce.h create mode 100644 tools/testing/selftests/kvm/x86_64/ucna_injection_test.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 0b0e4402bba6..a8eb9d3f082c 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -37,6 +37,7 @@ /x86_64/tsc_scaling_sync /x86_64/sync_regs_test /x86_64/tsc_msrs_test +/x86_64/ucna_injection_test /x86_64/userspace_io_test /x86_64/userspace_msr_exit_test /x86_64/vmx_apic_access_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 681b173aa87c..8d9858b54b98 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -66,6 +66,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_preemption_timer_test TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test TEST_GEN_PROGS_x86_64 += x86_64/svm_int_ctl_test TEST_GEN_PROGS_x86_64 += x86_64/sync_regs_test +TEST_GEN_PROGS_x86_64 += x86_64/ucna_injection_test TEST_GEN_PROGS_x86_64 += x86_64/userspace_io_test TEST_GEN_PROGS_x86_64 += x86_64/userspace_msr_exit_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_apic_access_test diff --git a/tools/testing/selftests/kvm/include/x86_64/apic.h b/tools/testing/selftests/kvm/include/x86_64/apic.h index ac88557dcc9a..bed316fdecd5 100644 --- a/tools/testing/selftests/kvm/include/x86_64/apic.h +++ b/tools/testing/selftests/kvm/include/x86_64/apic.h @@ -35,6 +35,7 @@ #define APIC_SPIV_APIC_ENABLED (1 << 8) #define APIC_IRR 0x200 #define APIC_ICR 0x300 +#define APIC_LVTCMCI 0x2f0 #define APIC_DEST_SELF 0x40000 #define APIC_DEST_ALLINC 0x80000 #define APIC_DEST_ALLBUT 0xC0000 diff --git a/tools/testing/selftests/kvm/include/x86_64/mce.h b/tools/testing/selftests/kvm/include/x86_64/mce.h new file mode 100644 index 000000000000..6119321f3f5d --- /dev/null +++ b/tools/testing/selftests/kvm/include/x86_64/mce.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * tools/testing/selftests/kvm/include/x86_64/mce.h + * + * Copyright (C) 2022, Google LLC. + */ + +#ifndef SELFTEST_KVM_MCE_H +#define SELFTEST_KVM_MCE_H + +#define MCG_CTL_P BIT_ULL(8) /* MCG_CTL register available */ +#define MCG_SER_P BIT_ULL(24) /* MCA recovery/new status bits */ +#define MCG_LMCE_P BIT_ULL(27) /* Local machine check supported */ +#define MCG_CMCI_P BIT_ULL(10) /* CMCI supported */ +#define KVM_MAX_MCE_BANKS 32 +#define MCG_CAP_BANKS_MASK 0xff /* Bit 0-7 of the MCG_CAP register are #banks */ +#define MCI_STATUS_VAL (1ULL << 63) /* valid error */ +#define MCI_STATUS_UC (1ULL << 61) /* uncorrected error */ +#define MCI_STATUS_EN (1ULL << 60) /* error enabled */ +#define MCI_STATUS_MISCV (1ULL << 59) /* misc error reg. valid */ +#define MCI_STATUS_ADDRV (1ULL << 58) /* addr reg. valid */ +#define MCM_ADDR_PHYS 2 /* physical address */ +#define MCI_CTL2_CMCI_EN BIT_ULL(30) + +#endif /* SELFTEST_KVM_MCE_H */ diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h index d0d51adec76e..fa316c3b7725 100644 --- a/tools/testing/selftests/kvm/include/x86_64/processor.h +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h @@ -481,6 +481,7 @@ struct kvm_cpuid2 *kvm_get_supported_hv_cpuid(void); void vcpu_set_hv_cpuid(struct kvm_vm *vm, uint32_t vcpuid); struct kvm_cpuid2 *vcpu_get_supported_hv_cpuid(struct kvm_vm *vm, uint32_t vcpuid); void vm_xsave_req_perm(int bit); +void vcpu_setup(struct kvm_vm *vm, int vcpuid); enum x86_page_size { X86_PAGE_SIZE_4K = 0, diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c index 33ea5e9955d9..bb1ef665cd10 100644 --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c @@ -580,7 +580,7 @@ static void kvm_setup_tss_64bit(struct kvm_vm *vm, struct kvm_segment *segp, kvm_seg_fill_gdt_64bit(vm, segp); } -static void vcpu_setup(struct kvm_vm *vm, int vcpuid) +void vcpu_setup(struct kvm_vm *vm, int vcpuid) { struct kvm_sregs sregs; diff --git a/tools/testing/selftests/kvm/x86_64/ucna_injection_test.c b/tools/testing/selftests/kvm/x86_64/ucna_injection_test.c new file mode 100644 index 000000000000..104a9f116b23 --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/ucna_injection_test.c @@ -0,0 +1,287 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * ucna_injection_test + * + * Copyright (C) 2022, Google LLC. + * + * This work is licensed under the terms of the GNU GPL, version 2. + * + * Test that user space can inject UnCorrectable No Action required (UCNA) + * memory errors to the guest. + * + * The test starts one vCPU with the MCG_CMCI_P enabled. It verifies that + * proper UCNA errors can be injected to a vCPU with MCG_CMCI_P and + * corresponding per-bank control register (MCI_CTL2) bit enabled. + * The test also checks that the UCNA errors get recorded in the + * Machine Check bank registers no matter the error signal interrupts get + * delivered into the guest or not. + * + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include +#include +#include +#include + +#include "kvm_util_base.h" +#include "kvm_util.h" +#include "mce.h" +#include "processor.h" +#include "test_util.h" +#include "apic.h" + +#define UCNA_VCPU_ID 0 +#define SYNC_FIRST_UCNA 9 +#define SYNC_SECOND_UCNA 10 +#define FIRST_UCNA_ADDR 0xdeadbeef +#define SECOND_UCNA_ADDR 0xcafeb0ba + +/* + * Vector for the CMCI interrupt. + * Value is arbitrary. Any value in 0x20-0xFF should work: + * https://wiki.osdev.org/Interrupt_Vector_Table + */ +#define CMCI_VECTOR 0xa9 + +#define UCNA_BANK 0x7 // IMC0 bank + +/* + * Record states about the injected UCNA. + * The variables started with the 'i_' prefixes are recorded in interrupt + * handler. Variables without the 'i_' prefixes are recorded in guest main + * execution thread. + */ +static volatile uint64_t i_ucna_rcvd; +static volatile uint64_t i_ucna_addr; +static volatile uint64_t ucna_addr; +static volatile uint64_t ucna_addr2; + +struct thread_params { + struct kvm_vm *vm; + uint32_t vcpu_id; +}; + +static void verify_apic_base_addr(void) +{ + uint64_t msr = rdmsr(MSR_IA32_APICBASE); + uint64_t base = GET_APIC_BASE(msr); + + GUEST_ASSERT(base == APIC_DEFAULT_GPA); +} + +static void guest_code(void) +{ + uint64_t ctl2; + verify_apic_base_addr(); + xapic_enable(); + + /* Sets up the interrupt vector and enables per-bank CMCI sigaling. */ + xapic_write_reg(APIC_LVTCMCI, CMCI_VECTOR | APIC_DM_FIXED); + ctl2 = rdmsr(MSR_IA32_MCx_CTL2(UCNA_BANK)); + wrmsr(MSR_IA32_MCx_CTL2(UCNA_BANK), ctl2 | MCI_CTL2_CMCI_EN); + + /* Enables interrupt in guest. */ + asm volatile("sti"); + + /* Let user space inject the first UCNA */ + GUEST_SYNC(SYNC_FIRST_UCNA); + + ucna_addr = rdmsr(MSR_IA32_MCx_ADDR(UCNA_BANK)); + + /* Disables the per-bank CMCI signaling. */ + ctl2 = rdmsr(MSR_IA32_MCx_CTL2(UCNA_BANK)); + wrmsr(MSR_IA32_MCx_CTL2(UCNA_BANK), ctl2 & ~MCI_CTL2_CMCI_EN); + + /* Let the user space inject the second UCNA */ + GUEST_SYNC(SYNC_SECOND_UCNA); + + ucna_addr2 = rdmsr(MSR_IA32_MCx_ADDR(UCNA_BANK)); + GUEST_DONE(); +} + +static void guest_cmci_handler(struct ex_regs *regs) +{ + i_ucna_rcvd++; + i_ucna_addr = rdmsr(MSR_IA32_MCx_ADDR(UCNA_BANK)); + xapic_write_reg(APIC_EOI, 0); +} + +static void inject_ucna(struct kvm_vm *vm, uint32_t vcpu_id, uint64_t addr) { + /* + * A UCNA error is indicated with VAL=1, UC=1, PCC=0, S=0 and AR=0 in + * the IA32_MCi_STATUS register. + * MSCOD=1 (BIT[16] - MscodDataRdErr). + * MCACOD=0x0090 (Memory controller error format, channel 0) + */ + uint64_t status = MCI_STATUS_VAL | MCI_STATUS_UC | MCI_STATUS_EN | + MCI_STATUS_MISCV | MCI_STATUS_ADDRV | 0x10090; + struct kvm_x86_mce mce = {}; + mce.status = status; + mce.mcg_status = 0; + /* + * MCM_ADDR_PHYS indicates the reported address is a physical address. + * Lowest 6 bits is the recoverable address LSB, i.e., the injected MCE + * is at 4KB granularity. + */ + mce.misc = (MCM_ADDR_PHYS << 6) | 0xc; + mce.addr = addr; + mce.bank = UCNA_BANK; + + TEST_ASSERT(_vcpu_ioctl(vm, vcpu_id, KVM_X86_SET_MCE, &mce) != -1, + "Inject UCNA"); +} + +static void *vcpu_thread(void *arg) +{ + struct thread_params *params = (struct thread_params *)arg; + struct ucall uc; + int old; + int r; + unsigned int exit_reason; + + r = pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old); + TEST_ASSERT(r == 0, + "pthread_setcanceltype failed on vcpu_id=%u with errno=%d", + params->vcpu_id, r); + + fprintf(stderr, "vCPU thread running vCPU %u\n", params->vcpu_id); + vcpu_run(params->vm, params->vcpu_id); + + exit_reason = vcpu_state(params->vm, params->vcpu_id)->exit_reason; + TEST_ASSERT(exit_reason == KVM_EXIT_IO, + "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO", + params->vcpu_id, exit_reason, exit_reason_str(exit_reason)); + TEST_ASSERT(get_ucall(params->vm, params->vcpu_id, &uc) == UCALL_SYNC, + "Expect UCALL_SYNC\n"); + TEST_ASSERT(uc.args[1] == SYNC_FIRST_UCNA, "Injecting first UCNA."); + + fprintf(stderr, "Injecting first UCNA at %#x.\n", FIRST_UCNA_ADDR); + + inject_ucna(params->vm, params->vcpu_id, FIRST_UCNA_ADDR); + vcpu_run(params->vm, params->vcpu_id); + + exit_reason = vcpu_state(params->vm, params->vcpu_id)->exit_reason; + TEST_ASSERT(exit_reason == KVM_EXIT_IO, + "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO", + params->vcpu_id, exit_reason, exit_reason_str(exit_reason)); + TEST_ASSERT(get_ucall(params->vm, params->vcpu_id, &uc) == UCALL_SYNC, + "Expect UCALL_SYNC\n"); + TEST_ASSERT(uc.args[1] == SYNC_SECOND_UCNA, "Injecting second UCNA."); + + fprintf(stderr, "Injecting second UCNA at %#x.\n", SECOND_UCNA_ADDR); + + inject_ucna(params->vm, params->vcpu_id, SECOND_UCNA_ADDR); + vcpu_run(params->vm, params->vcpu_id); + + exit_reason = vcpu_state(params->vm, params->vcpu_id)->exit_reason; + TEST_ASSERT(exit_reason == KVM_EXIT_IO, + "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO", + params->vcpu_id, exit_reason, exit_reason_str(exit_reason)); + if (get_ucall(params->vm, params->vcpu_id, &uc) == UCALL_ABORT) { + TEST_ASSERT(false, + "vCPU %u exited with error: %s.\n", + params->vcpu_id, (const char *)uc.args[0]); + } + + return NULL; +} + +static void setup_mce_cap(struct kvm_vm *vm, uint32_t vcpuid) +{ + uint64_t supported_mcg_caps = 0; + uint64_t mcg_caps = MCG_CMCI_P | MCG_CTL_P | MCG_SER_P | MCG_LMCE_P + | KVM_MAX_MCE_BANKS; + + TEST_ASSERT(_kvm_ioctl(vm, KVM_X86_GET_MCE_CAP_SUPPORTED, + &supported_mcg_caps) != -1, + "KVM_GET_MCE_CAP_SUPPORTED"); + fprintf(stderr, "KVM supported MCG_CAP: %#lx\n", supported_mcg_caps); + + TEST_ASSERT(supported_mcg_caps & MCG_CMCI_P, "MCG_CMCI_P is not supported"); + + mcg_caps &= supported_mcg_caps | MCG_CAP_BANKS_MASK; + TEST_ASSERT(_vcpu_ioctl(vm, vcpuid, KVM_X86_SETUP_MCE, &mcg_caps) != -1, + "KVM_X86_SETUP_MCE"); +} + +static void create_vcpu_with_mce_cap(struct kvm_vm *vm, uint32_t vcpuid, void *guest_code) +{ + struct kvm_mp_state mp_state; + struct kvm_regs regs; + vm_vaddr_t stack_vaddr; + stack_vaddr = vm_vaddr_alloc(vm, DEFAULT_STACK_PGS * getpagesize(), + DEFAULT_GUEST_STACK_VADDR_MIN); + + vm_vcpu_add(vm, vcpuid); + setup_mce_cap(vm, vcpuid); + + vcpu_set_cpuid(vm, vcpuid, kvm_get_supported_cpuid()); + vcpu_setup(vm, vcpuid); + + /* Setup guest general purpose registers */ + vcpu_regs_get(vm, vcpuid, ®s); + regs.rflags = regs.rflags | 0x2; + regs.rsp = stack_vaddr + (DEFAULT_STACK_PGS * getpagesize()); + regs.rip = (unsigned long) guest_code; + vcpu_regs_set(vm, vcpuid, ®s); + + /* Setup the MP state */ + mp_state.mp_state = 0; + vcpu_set_mp_state(vm, vcpuid, &mp_state); +} + +int main(int argc, char *argv[]) +{ + int r; + void *retval; + int run_secs = 3; + pthread_t threads[2]; + struct thread_params params[2]; + struct kvm_vm *vm; + uint64_t *p_i_ucna_rcvd, *p_i_ucna_addr; + uint64_t *p_ucna_addr, *p_ucna_addr2; + + kvm_check_cap(KVM_CAP_MCE); + + vm = vm_create_without_vcpus(VM_MODE_DEFAULT, DEFAULT_GUEST_PHY_PAGES); + + create_vcpu_with_mce_cap(vm, UCNA_VCPU_ID, guest_code); + + params[0].vm = vm; + + vm_init_descriptor_tables(vm); + vcpu_init_descriptor_tables(vm, UCNA_VCPU_ID); + vm_install_exception_handler(vm, CMCI_VECTOR, guest_cmci_handler); + + virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA); + + p_i_ucna_rcvd = (uint64_t *)addr_gva2hva(vm, (uint64_t)&i_ucna_rcvd); + p_i_ucna_addr = (uint64_t *)addr_gva2hva(vm, (uint64_t)&i_ucna_addr); + p_ucna_addr = (uint64_t *)addr_gva2hva(vm, (uint64_t)&ucna_addr); + p_ucna_addr2 = (uint64_t *)addr_gva2hva(vm, (uint64_t)&ucna_addr2); + + /* Start vCPU thread and wait for it to execute first HLT. */ + params[0].vcpu_id = UCNA_VCPU_ID; + r = pthread_create(&threads[0], NULL, vcpu_thread, ¶ms[0]); + TEST_ASSERT(r == 0, + "pthread_create vcpu thread failed errno=%d", errno); + fprintf(stderr, "vCPU thread started\n"); + + sleep(run_secs); + + r = pthread_join(threads[0], &retval); + TEST_ASSERT(r == 0, + "pthread_join on vcpu_id=%d failed with errno=%d", + UCNA_VCPU_ID, r); + + fprintf(stderr, + "Test successful after running for %d seconds.\n" + "UCNA CMCI interrupts received: %ld\n" + "Last UCNA address received via CMCI: %lx\n" + "First UCNA address in vCPU thread: %lx\n" + "Second UCNA address in vCPU thread: %lx\n", + run_secs, *p_i_ucna_rcvd, *p_i_ucna_addr, *p_ucna_addr, *p_ucna_addr2); + + kvm_vm_free(vm); +}