From patchwork Wed Jul 13 12:25:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EED3CC433EF for ; Wed, 13 Jul 2022 12:25:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236362AbiGMMZd (ORCPT ); Wed, 13 Jul 2022 08:25:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236340AbiGMMZa (ORCPT ); Wed, 13 Jul 2022 08:25:30 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6378AE395; Wed, 13 Jul 2022 05:25:29 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id z12-20020a17090a7b8c00b001ef84000b8bso3406536pjc.1; Wed, 13 Jul 2022 05:25:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sGw98pQzA/OzIltzhNgUUf9pAcra/Fm3uI9aTke5FqA=; b=TyIVPchdiF/GzdH7oRyJr0rn4OyBgjY4caeb5LS4BVSg1H5ztWt2j2lkBoAZ7CosUu f1wrAcxsLxpb1xaegh3GneIq97RBve3yn13DzDr04QxamYcSLt+mwRSA9k3a89mgELaK WSuMk6S3uxgO3Mopdiw7cGKxPRLbMLZNwEYfV99NiEgOSv26ckAK4r0/RpXM8PAlKlLP VTRQscBdWXw5XGJoPH/pwSynvl5om1KmFJwAi0acwS5v9C2tTUdJNdCwRzGSBjNJd0Sm QR0UWUKop8ukCbTxghqtiV7ow0OJYouZms8om0uCf24eozvx3RD7IZy1WZNNZa0srdoR KPKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sGw98pQzA/OzIltzhNgUUf9pAcra/Fm3uI9aTke5FqA=; b=mppshnSiZt4RuIgaI6VAH3J/AR1J4MuJhP/YmXyiRSbKze2UTO/h3LnAtv6w1gjDaf NphcUBoC9FL/nWi8QW4czPClewjFlFMbw4wqj1NIsilAKG/njAjRgfcM9suq9H8TkWzq AUtDmX+Yv/jaq2Lu1OHUgO0gl6QewqfgNiUv4/FpWyqG5Qc6v924772rZYLSk08k5lxT /h3p1/52dzJCqMrOnmscQVAudb63191VkoanRN6di2E3mpVuOGxBecVbzmGWlBgrXODQ 5b9aCU3liXpw1lpirRbCYzf1jVeafiAknK3CSJ4+I+yNbuwHy5BicUA/cA067moJEzUf xYoA== X-Gm-Message-State: AJIora/ntVBrNn+3q1n9Ah1k1wNGitaGiuQSd+aPDUqoIs2gBtW7j9TA j89sKSKw4aw25JX1ybaREHg= X-Google-Smtp-Source: AGRyM1u5TQOQ4BD2wrwBRekcnatgoY2J1u9IDj2UQNp826QlH4Mx6whVQK93J2HtMk03ULlkCmvBaQ== X-Received: by 2002:a17:903:228c:b0:16c:2f61:9158 with SMTP id b12-20020a170903228c00b0016c2f619158mr3160488plh.140.1657715129461; Wed, 13 Jul 2022 05:25:29 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:28 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu , Kan Liang Subject: [PATCH 1/7] perf/x86/core: Update x86_pmu.pebs_capable for ICELAKE_{X,D} Date: Wed, 13 Jul 2022 20:25:00 +0800 Message-Id: <20220713122507.29236-2-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu Ice Lake microarchitecture with EPT-Friendly PEBS capability also support the Extended feature, which means that all counters (both fixed function and general purpose counters) can be used for PEBS events. Update x86_pmu.pebs_capable like SPR to apply PEBS_ALL semantics. Cc: Kan Liang Fixes: fb358e0b811e ("perf/x86/intel: Add EPT-Friendly PEBS for Ice Lake Server") Signed-off-by: Like Xu --- arch/x86/events/intel/core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 4e9b7af9cc45..e46fd496187b 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -6239,6 +6239,7 @@ __init int intel_pmu_init(void) case INTEL_FAM6_ICELAKE_X: case INTEL_FAM6_ICELAKE_D: x86_pmu.pebs_ept = 1; + x86_pmu.pebs_capable = ~0ULL; pmem = true; fallthrough; case INTEL_FAM6_ICELAKE_L: From patchwork Wed Jul 13 12:25:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0187CCA479 for ; Wed, 13 Jul 2022 12:25:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236374AbiGMMZg (ORCPT ); Wed, 13 Jul 2022 08:25:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236359AbiGMMZd (ORCPT ); Wed, 13 Jul 2022 08:25:33 -0400 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BC43BAAAE; Wed, 13 Jul 2022 05:25:32 -0700 (PDT) Received: by mail-pg1-x529.google.com with SMTP id 73so10270234pgb.10; Wed, 13 Jul 2022 05:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TaYxKDKezkeJqxjAzRgb2Y2a2mg+MWcpNMKzdnJpiyQ=; b=OWtM6Kd1CxThMkelwGOXdvT0MbLgJ2lngvcGR3sxmKVPqgG6dnYenJStdePoSX95NZ 1q5tcEGuzIJqNs6MiirgdI/5Jm2THMsvBD7acYsXKsCSJmgXFk0VGTUsh9VgjRWgSuzQ wl9ZzBWbRLCcY57/Ud2rZfrkaD7p5kY7yCLtXtZSm1kXcxWFVSEnS8bUzsPiLAox0Y4z 9c49iCP7ERS3MEy+1Zo3R+aLllxEPesE1WKOANTNbIQkg7m04SSy7o96SVup2m2A96P8 0k/mHRFzCtYHCLkdg46h8n+5nCLsm1xCiQ2h3DqkYLNH2bs1XWlGMHqjK8L2lQIzkYpF UMwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TaYxKDKezkeJqxjAzRgb2Y2a2mg+MWcpNMKzdnJpiyQ=; b=2cEv8dv/ftn593URpMJU7yHm1tee/+JEJEiVm5qiktov48gf931OUgjSbYCqbJoPwh 5dUIBSEUDjVJ/vMdywfC88XFcVKsfVh3YpjOlY24AEPr465mgM1IUBjLFA0m0v3IUwLt b0YLaYBqPAe2NzpCBW6H6KXGKLF+izoHqqvv7L+oWvLZa3SHY8Owq3T+UJiPSNuXuQj2 EFfU2Il2cNRtsCLKsh52hjJAzcudYjxxDJqlR+jI5eM8Bl5K4Suk3VGl5XdN8hMReU3u Px/0Sg3Mb6FcUMNTlYKCHm7gNt9G+Po8xpFJY9uRx+qEuemm+XIVXj4I13wiWa5evQL3 T5TQ== X-Gm-Message-State: AJIora+IEMoBh6sqVcx5aeYvS4v7GPtefxQVsvmiCEVmqXCs6eo4NAYq psRRZjtoFGZ0XZ/aoezCwFErhr8DXH4= X-Google-Smtp-Source: AGRyM1uxgBT+0EwaomlXbLBJGUhNRBB5BPqvi/YuDSDzAts81IV3eAeOpSvpHwPdpVMdwEu9DHJ5Yw== X-Received: by 2002:a63:6c81:0:b0:3fd:4be3:8ee9 with SMTP id h123-20020a636c81000000b003fd4be38ee9mr2682106pgc.188.1657715131789; Wed, 13 Jul 2022 05:25:31 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:31 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu Subject: [PATCH 2/7] perf/x86/core: Completely disable guest PEBS via guest's global_ctrl Date: Wed, 13 Jul 2022 20:25:01 +0800 Message-Id: <20220713122507.29236-3-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu When a guest PEBS counter is cross-mapped by a host counter, software will remove the corresponding bit in the arr[global_ctrl].guest and expect hardware to perform a change of state "from enable to disable" via the msr_slot[] switch during the vmx transaction. The real world is that if user adjust the counter overflow value small enough, it still opens a tiny race window for the previously PEBS-enabled counter to write cross-mapped PEBS records into the guest's PEBS buffer, when arr[global_ctrl].guest has been prioritised (switch_msr_special stuff) to switch into the enabled state, while the arr[pebs_enable].guest has not. Close this window by clearing invalid bits in the arr[global_ctrl].guest. Fixes: 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") Signed-off-by: Like Xu --- arch/x86/events/intel/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index e46fd496187b..495ac447bb3a 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -4052,8 +4052,9 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) /* Disable guest PEBS if host PEBS is enabled. */ arr[pebs_enable].guest = 0; } else { - /* Disable guest PEBS for cross-mapped PEBS counters. */ + /* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */ arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask; + arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask; /* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */ arr[global_ctrl].guest |= arr[pebs_enable].guest; } From patchwork Wed Jul 13 12:25:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1E94C43334 for ; Wed, 13 Jul 2022 12:25:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236397AbiGMMZn (ORCPT ); Wed, 13 Jul 2022 08:25:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236373AbiGMMZg (ORCPT ); Wed, 13 Jul 2022 08:25:36 -0400 Received: from mail-pg1-x529.google.com (mail-pg1-x529.google.com [IPv6:2607:f8b0:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C8D8C1FED; Wed, 13 Jul 2022 05:25:34 -0700 (PDT) Received: by mail-pg1-x529.google.com with SMTP id 73so10270312pgb.10; Wed, 13 Jul 2022 05:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LXqovdDSl/MXsNZWjftmy81y+mPSnXdzim3KrCIDlE0=; b=LLUGPEi/RmbpZwk9P9m5HPEB9sm6XJcxnSoyd8dxAk1RNNLJ4kiniTUwgPSb0SOGsM kP+pNwDVu/sTpyz6I3D/+OMo6wkcGTfbDIRFNQ691ukX50uy0TldoRp8f4t0i9vvxOzH 80ChLc51D1OvQtifYaQiYdhlY71ZzVH4iI0PPvhDUdqEMt49vaXvOzPsk+nlGDBD8GEe SJmwD0W3bG9akHMxtA7QQzlCpwE4MGD2Z64Xa527qvNuflNkm74pUzI7MYDf0YeXcycX 6Yw8pLPAeUttos0Pqjkze0stGX3eqjEUcqHG3fv//3b6A5eCuYtiOmloxFrTqD6Z+69U S9nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LXqovdDSl/MXsNZWjftmy81y+mPSnXdzim3KrCIDlE0=; b=HOVViA/Y5RW1/ssLS2ONKdAJwYxQR4Wr+sGQNogb8Z3xOtfLg5u63pvB4JENer/qDN 7ZbPkPsBvNFf8wgzM54BFRDwFIB10sqb5BfRSuvktUzSz/+WcyHgHZ2NqbB3xgsKO8L+ PJBw7ydjr5rS2ErUtC1kyfedoc/kLtytcoLFQErYFWpkNw0mZ4FgxfUOAe7FC9IPpnlg VEk/IYPa9sg29ZbVsXK2IhxQ2GiBKB4BxHhMr/DzqYTqgsgk/OKYuKgTyrKIIdB0d/91 9eXL0aUVSwnkZF8kyXrQ4bxgMweNdXWxrNm60/iu0RCmemuBMv77+cyDdK9kuVaHnjh1 Z9JQ== X-Gm-Message-State: AJIora+G6OsmxC3mUktNwv5byABEqLj8osQqo+4KK2NnKiRjBPY0ZDpX M0KY3MoZ/5qKGQDgRWNcUIc= X-Google-Smtp-Source: AGRyM1sk8JQ44ful7LGxlOnlp8i/NXsMnI/Tct++/5RfdnANJujj/ifyPCW1uswJ2UUvzKZRMWYw3w== X-Received: by 2002:a63:df49:0:b0:412:58fe:2332 with SMTP id h9-20020a63df49000000b0041258fe2332mr2603882pgj.505.1657715134057; Wed, 13 Jul 2022 05:25:34 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:33 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu Subject: [PATCH 3/7] KVM: x86/pmu: Avoid setting BIT_ULL(-1) to pmu->host_cross_mapped_mask Date: Wed, 13 Jul 2022 20:25:02 +0800 Message-Id: <20220713122507.29236-4-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu In the extreme case of host counters multiplexing and contention, the perf_event requested by the guest's pebs counter is not allocated to any actual physical counter, in which case hw.idx is bookkept as -1, resulting in an out-of-bounds access to host_cross_mapped_mask. Fixes: 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") Signed-off-by: Like Xu --- arch/x86/kvm/vmx/pmu_intel.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 53ccba896e77..1588627974fa 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -783,20 +783,19 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu) void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu) { struct kvm_pmc *pmc = NULL; - int bit; + int bit, hw_idx; for_each_set_bit(bit, (unsigned long *)&pmu->global_ctrl, X86_PMC_IDX_MAX) { pmc = intel_pmc_idx_to_pmc(pmu, bit); if (!pmc || !pmc_speculative_in_use(pmc) || - !intel_pmc_is_enabled(pmc)) + !intel_pmc_is_enabled(pmc) || !pmc->perf_event) continue; - if (pmc->perf_event && pmc->idx != pmc->perf_event->hw.idx) { - pmu->host_cross_mapped_mask |= - BIT_ULL(pmc->perf_event->hw.idx); - } + hw_idx = pmc->perf_event->hw.idx; + if (hw_idx != pmc->idx && hw_idx != -1) + pmu->host_cross_mapped_mask |= BIT_ULL(hw_idx); } } From patchwork Wed Jul 13 12:25:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F4B2CCA480 for ; Wed, 13 Jul 2022 12:25:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236411AbiGMMZp (ORCPT ); Wed, 13 Jul 2022 08:25:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236387AbiGMMZn (ORCPT ); Wed, 13 Jul 2022 08:25:43 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AA5FCC7BF; Wed, 13 Jul 2022 05:25:37 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id x184so10102106pfx.2; Wed, 13 Jul 2022 05:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vMzQD9O55lJzNyZFNJ1xoHaqPwdV3msqkHZ+m49+h34=; b=WydBXzVEYu/cVe7J6XyAmfLhj+dAt0yf5xvx+xbKudRn2JX+4II6KY09dQg67oIUj/ 9T01SiKabL7tUSW/s1ymM1a5mrBIJDpbjDKCHJOYtAGdTwdmCGMfMU96c9OZfdb1k4gd r2abQ4UZnmkdQv800qzAcvG1NEyqWIVoCt+1KWYhGVD0sanZivzkJjDGTLjDc6zGUpDX 1jQbR7CMXxBXsp0MduVqg9D/exfIA6EfJQyVOL03aBqELQZSgmSfueu/RxzfgPK5RDlI nWO8fjH2tXhbbA0oA6DVl78qt4j+GDWG1Sx8HeRl58rcjCba9naJ5qMoCOdU5PA9tYPo Scfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vMzQD9O55lJzNyZFNJ1xoHaqPwdV3msqkHZ+m49+h34=; b=lCw3KRe5YuMYLU9ByJxH15AMvkj7uh1UZiu+cJKEXPRzyRd4Rtt+g1zmWYgN7QMn4w +p1ANOJHkC7pi8AN2U60XmtThqW7SG4UfPC6T/aQWdLXFGLp6BEUb2IBhWRyC2tjVBhc gcrsmBsaftfaJUeR01L7daqc9/9+dLdmnA7M5I1zChJiai6Bs1cJytZceFZduSrCmFQl n92uEfqmii25CmPaVizB5zv4UjOtblzBS/4w7ITWTg8CyOUtVKoetfX16e3lNiUwnu+9 ItjXgcuIDDZbEluNB2TASOYgaHpzMPorE78Ml9FAbA7XWeuyFaKv9BuprLcGiw4h/xjn yCkA== X-Gm-Message-State: AJIora8Y2sLcn63H8xzEpLWdjqhrRN4Hd+iJy4JQNbZfR8LG0Fp4gUly Zs9AsgURhXxJ9iYn7hd9jtazcLxYFno= X-Google-Smtp-Source: AGRyM1tY8so1ixNTckOPXhzC2LOBUE2mscDc3wZqmnE/E+iykXN/TgGwaDMgSj9CKy2VCVzGNiD/2Q== X-Received: by 2002:a05:6a00:1589:b0:52a:eb00:71d8 with SMTP id u9-20020a056a00158900b0052aeb0071d8mr2850797pfk.38.1657715136251; Wed, 13 Jul 2022 05:25:36 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:35 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu Subject: [PATCH 4/7] KVM: x86/pmu: Not to generate PEBS records for emulated instructions Date: Wed, 13 Jul 2022 20:25:03 +0800 Message-Id: <20220713122507.29236-5-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu The KVM accumulate an enabeld counter for at least INSTRUCTIONS or BRANCH_INSTRUCTION hw event from any KVM emulated instructions, generating emulated overflow interrupt on counter overflow, which in theory should also happen when the PEBS counter overflows but it currently lacks this part of the underlying support (e.g. through software injection of records in the irq context or a lazy approach). In this case, KVM skips the injection of this BUFFER_OVF PMI (effectively dropping one PEBS record) and let the overflow counter move on. The loss of a single sample does not introduce a loss of accuracy, but is easily noticeable for certain specific instructions. This issue is expected to be addressed along with the issue of PEBS cross-mapped counters with a slow-path proposal. Fixes: 79f3e3b58386 ("KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter") Signed-off-by: Like Xu --- arch/x86/kvm/pmu.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 02f9e4f245bd..08ee0fed63d5 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -106,9 +106,14 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) return; if (pmc->perf_event && pmc->perf_event->attr.precise_ip) { - /* Indicate PEBS overflow PMI to guest. */ - skip_pmi = __test_and_set_bit(GLOBAL_STATUS_BUFFER_OVF_BIT, - (unsigned long *)&pmu->global_status); + if (!in_pmi) { + /* The emulated instructions does not generate PEBS records. */ + skip_pmi = true; + } else { + /* Indicate PEBS overflow PMI to guest. */ + skip_pmi = __test_and_set_bit(GLOBAL_STATUS_BUFFER_OVF_BIT, + (unsigned long *)&pmu->global_status); + } } else { __set_bit(pmc->idx, (unsigned long *)&pmu->global_status); } From patchwork Wed Jul 13 12:25:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4471AC433EF for ; Wed, 13 Jul 2022 12:25:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236440AbiGMMZz (ORCPT ); Wed, 13 Jul 2022 08:25:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236404AbiGMMZo (ORCPT ); Wed, 13 Jul 2022 08:25:44 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1B08D64E0; Wed, 13 Jul 2022 05:25:39 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id g16-20020a17090a7d1000b001ea9f820449so3354399pjl.5; Wed, 13 Jul 2022 05:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=udbgFzljPHJNNNSaz06lLoXszbhtSudUqIf9IGRqT7U=; b=mj2n5SgIT+F2QDRM3BpZnyXs6Yh6Ygj9gh7+fWHqMJZwnuknJVNN0bl1YRotFIDQsJ GsI9i7jQBgJidSmd11GN2PHTgRt3i4ZZJtHfS9OMeNaaXgJUNrfH4uqoc06kdbZB8Hbh UvnWadpBCV21ZxwfGdQ/+0aIcYsVoZpCoXkJx2QwhynlVdlrJpeMTuGFch+eft4BXJXL p4Irowzfv5HKDiNXwzDyB7MwSJmhxZ/RdofEREmLk++E8zwypQpIASsTvJwp3l+UFcB4 gG4cxujLUtV0zK5l+G98VnnkkiOxy7hDRwe7+d0WZMhGj42iVCc9SnbrVmYbtlbzudyK Rjig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=udbgFzljPHJNNNSaz06lLoXszbhtSudUqIf9IGRqT7U=; b=uPjdZXfSU93ht0zjGLz3H73vB/bgZ/b0z+eFRRTq6P6dtFe5wyiefqgylElyfK+SKq HT1Ctk7RJWuImC/hmXOIx7SafGhbpJSNLuzG3SdMPmYG0V2ArhBlfAhDVP+6JfgCfYp3 97o52tVIPvVaKNkaO35cQEzlscUbhpDN9asdLBZ8hoBx5XAf8zyHQ/lHHWqEMS++LkJF OJo57sJsvBfh0YZV0tnPwe+oPMgX6qEKoPQtCdnIA5suRpe2twsSCKlQaEOqui684+rq nwcvrJIbcBxvxWx1x/fi9E/sc+6QC97Rf1Gr0J4+Uy9UF49CL6D0iP7SPl1rOs6E/T7X fTQA== X-Gm-Message-State: AJIora8c3NV18I3DuflXJo5WE/MyF4QuhV/Ez4svHcu266OtDAXgbojm GxvMJ0vE3Ozklk1/iB4z2/A= X-Google-Smtp-Source: AGRyM1vqSK3+LrdHAXn4Vtr3vZJPIpGgMg4bHBPX5fq9kvsvDnzTOPYcrKYznNm8XEY9vXrIMziutQ== X-Received: by 2002:a17:90b:2384:b0:1ef:8506:374e with SMTP id mr4-20020a17090b238400b001ef8506374emr9991320pjb.99.1657715138574; Wed, 13 Jul 2022 05:25:38 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:38 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu Subject: [PATCH 5/7] KVM: x86/pmu: Avoid using PEBS perf_events for normal counters Date: Wed, 13 Jul 2022 20:25:04 +0800 Message-Id: <20220713122507.29236-6-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu The check logic in the pmc_resume_counter() to determine whether a perf_event is reusable is partial and flawed, especially when it comes to a pseudocode sequence (not correct but clearly valid) like: - enabling a counter and its PEBS bit - enable global_ctrl - run workload - disable only the PEBS bit, leaving the global_ctrl bit enabled In this corner case, a perf_event created for PEBS can be reused by a normal counter before it has been released and recreated, and when this normal counter overflows, it triggers a PEBS interrupt (precise_ip != 0). To address this issue, the reuse check has been revamped and KVM will go back to do reprogram_counter() when any bit of guest PEBS_ENABLE msr has changed, which is similar to what global_ctrl_changed() does. Fixes: 79f3e3b58386 ("KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter") Signed-off-by: Like Xu --- arch/x86/kvm/pmu.c | 4 ++-- arch/x86/kvm/vmx/pmu_intel.c | 14 +++++++------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 08ee0fed63d5..2c03fe208093 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -232,8 +232,8 @@ static bool pmc_resume_counter(struct kvm_pmc *pmc) get_sample_period(pmc, pmc->counter))) return false; - if (!test_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->pebs_enable) && - pmc->perf_event->attr.precise_ip) + if (test_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->pebs_enable) != + (!!pmc->perf_event->attr.precise_ip)) return false; /* reuse perf_event to serve as pmc_reprogram_counter() does*/ diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 1588627974fa..5f6b9f596f16 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -68,15 +68,11 @@ static struct kvm_pmc *intel_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx) } } -/* function is called when global control register has been updated. */ -static void global_ctrl_changed(struct kvm_pmu *pmu, u64 data) +static void reprogram_counters(struct kvm_pmu *pmu, u64 diff) { int bit; - u64 diff = pmu->global_ctrl ^ data; struct kvm_pmc *pmc; - pmu->global_ctrl = data; - for_each_set_bit(bit, (unsigned long *)&diff, X86_PMC_IDX_MAX) { pmc = intel_pmc_idx_to_pmc(pmu, bit); if (pmc) @@ -404,7 +400,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) struct kvm_pmc *pmc; u32 msr = msr_info->index; u64 data = msr_info->data; - u64 reserved_bits; + u64 reserved_bits, diff; switch (msr) { case MSR_CORE_PERF_FIXED_CTR_CTRL: @@ -425,7 +421,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (pmu->global_ctrl == data) return 0; if (kvm_valid_perf_global_ctrl(pmu, data)) { - global_ctrl_changed(pmu, data); + diff = pmu->global_ctrl ^ data; + pmu->global_ctrl = data; + reprogram_counters(pmu, diff); return 0; } break; @@ -440,7 +438,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (pmu->pebs_enable == data) return 0; if (!(data & pmu->pebs_enable_mask)) { + diff = pmu->pebs_enable ^ data; pmu->pebs_enable = data; + reprogram_counters(pmu, diff); return 0; } break; From patchwork Wed Jul 13 12:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38197CCA480 for ; Wed, 13 Jul 2022 12:25:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236457AbiGMMZ5 (ORCPT ); Wed, 13 Jul 2022 08:25:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236418AbiGMMZq (ORCPT ); Wed, 13 Jul 2022 08:25:46 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A9A7C3AF4; Wed, 13 Jul 2022 05:25:41 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id s206so10288943pgs.3; Wed, 13 Jul 2022 05:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vDEqN9x4jRvJS83n9tjgBw0tzXpZmoWu6o7+jYdFVOY=; b=K2Ep1JD9ExPzt6o06X+uL7DcvXrvOUxWB5TOD0FraSIK/10HBUp2sLWFTlQZs2jJr+ gTYJd0Y1SBwKqa4iT4og+CA/UlPzFyAJZPA8MEDwMC+0uCFkTng2LLrXSpjSQtZ/dJlS PBozM9skxjRkryWbWL5C6oArRL/L4nuzRJ0oTj4tP/WUKiriASFGr15ELyWlr0jZQp/q Jh2Ymdu66QvMmEKL5Rb3pJQBRr2na8fV5ZVdKqDLYWXnWULIxWaZBfxOgIpdxh3mFZZf JGZmN419Mf1A9Htpey+T137ffkYmBiriVNm5775IOIPFs/iZV0XfndABWYpxoqziGWRm F3pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vDEqN9x4jRvJS83n9tjgBw0tzXpZmoWu6o7+jYdFVOY=; b=KrD/2FC7dP5BIJhXMuudbWqAFOuzvA4AgfhnUE6NYYDhiN4bCnhEsRh52OErY7lVIx aCS5WZypVzr4PRx+jxyt0m0F2/xa5I/zr/lOCL6abfUE7GKY1Li+49u9thXqiGveyrWl JJh8kQuh/bfySZeGcewS5OC21iDSLDEy1FbTDCZkfMeeGbhEcoVEKyxhgiytoigW+wZM mtWeHraeTQRaEzPXlddYNbC0F50MwQamyt0yGdngXpu+P5NfFhbxohh+L0fhMtKkA5fc otDysHot4ObW5w+0Ra5P5TSG6ipUOol0prAfgC+mO7QfdJIZ0VT3IpqJHNEgqee4/s3f u9TQ== X-Gm-Message-State: AJIora/JzrOYF4qFuzTfFoY4jVIt/HOm8qUBuHiKQi9ymrI6um4QiCn4 xv+5M3Chcxlj/UBF9iLjoXI= X-Google-Smtp-Source: AGRyM1tfgKzg6Zl3D6EHKV1RAJXtNWseDWAQ3ME2LWwiUktteGmqgwm3gL85FzcQsXzUjBoNPE835A== X-Received: by 2002:a63:4d61:0:b0:412:6081:4bbf with SMTP id n33-20020a634d61000000b0041260814bbfmr2743515pgl.109.1657715140717; Wed, 13 Jul 2022 05:25:40 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:40 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu Subject: [PATCH 6/7] KVM: x86/pmu: Defer reprogram_counter() to kvm_pmu_handle_event() Date: Wed, 13 Jul 2022 20:25:05 +0800 Message-Id: <20220713122507.29236-7-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu During a KVM-trap from vm-exit to vm-entry, requests from different sources will try to create one or more perf_events via reprogram_counter(), which will allow some predecessor actions to be undone posteriorly, especially repeated calls to some perf subsystem interfaces. These repetitive calls can be omitted because only the final state of the perf_event and the hardware resources it occupies will take effect for the guest right before the vm-entry. To realize this optimization, KVM marks the creation requirements via reprogram_pmi, and then defers the actual execution with the help of vcpu KVM_REQ_PMU request. Opportunistically update a comment for pmu->reprogram_pmi. Signed-off-by: Like Xu --- arch/x86/kvm/pmu.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 2c03fe208093..681d3ac8d75c 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -101,7 +101,7 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) struct kvm_pmu *pmu = pmc_to_pmu(pmc); bool skip_pmi = false; - /* Ignore counters that have been reprogrammed already. */ + /* Ignore counters that have not been reprogrammed. */ if (test_and_set_bit(pmc->idx, pmu->reprogram_pmi)) return; @@ -289,6 +289,13 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc) } void reprogram_counter(struct kvm_pmc *pmc) +{ + __set_bit(pmc->idx, pmc_to_pmu(pmc)->reprogram_pmi); + kvm_make_request(KVM_REQ_PMU, pmc->vcpu); +} +EXPORT_SYMBOL_GPL(reprogram_counter); + +static void __reprogram_counter(struct kvm_pmc *pmc) { struct kvm_pmu *pmu = pmc_to_pmu(pmc); u64 eventsel = pmc->eventsel; @@ -330,7 +337,6 @@ void reprogram_counter(struct kvm_pmc *pmc) !(eventsel & ARCH_PERFMON_EVENTSEL_OS), eventsel & ARCH_PERFMON_EVENTSEL_INT); } -EXPORT_SYMBOL_GPL(reprogram_counter); void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) { @@ -340,11 +346,12 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) for_each_set_bit(bit, pmu->reprogram_pmi, X86_PMC_IDX_MAX) { struct kvm_pmc *pmc = static_call(kvm_x86_pmu_pmc_idx_to_pmc)(pmu, bit); - if (unlikely(!pmc || !pmc->perf_event)) { + if (unlikely(!pmc)) { clear_bit(bit, pmu->reprogram_pmi); continue; } - reprogram_counter(pmc); + + __reprogram_counter(pmc); } /* @@ -522,7 +529,7 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc) prev_count = pmc->counter; pmc->counter = (pmc->counter + 1) & pmc_bitmask(pmc); - reprogram_counter(pmc); + __reprogram_counter(pmc); if (pmc->counter < prev_count) __kvm_perf_overflow(pmc, false); } From patchwork Wed Jul 13 12:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12916646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28DA1C433EF for ; Wed, 13 Jul 2022 12:26:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235849AbiGMM0U (ORCPT ); Wed, 13 Jul 2022 08:26:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44228 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236432AbiGMMZy (ORCPT ); Wed, 13 Jul 2022 08:25:54 -0400 Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DA41D1EF7; Wed, 13 Jul 2022 05:25:44 -0700 (PDT) Received: by mail-pg1-x535.google.com with SMTP id f11so9417034pgj.7; Wed, 13 Jul 2022 05:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XsanBgHWAtIbJ++WDURz73dEqP2UMuYGIlKVLnPoouk=; b=Y5hdn3XjAkLtN0bs7hF/F+rLM8h7uAY3iIqtdnZTtnmpLwDa1I0L/T57cs+IgRuUdk +ZfRnyXvezchBsjiVlEbTYL9JeMsEpNNUHLVhcCHjuf5fwfySPyzsSMbDjSGH4vpoyBG Y/qC1fT7kaFG5aN1DpErYK4ZEzPJN8NzdX3SnvFrZ2bXQg71ZPAplpsdjQqV0Zm6aFX+ AYAikFHsCAH617zxsJKiweYcpy9vpCkjfeiwMkzZ+nwNBoyHAr0R0Q8qw4FbG0W5Sz1q Xx4v4htHtQpgC4zIxvC4sSQhADYe2koLNMhq7JiddNIX8VVv8nOvmlkvV1X4tEL7V2na ZWXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XsanBgHWAtIbJ++WDURz73dEqP2UMuYGIlKVLnPoouk=; b=BmeQ7yCQiwRYW1eDhsO4LaswhM75trkcZaaHulf6q41n5bMkmvgVfEFb/VNP4nXe7o EtB5Z9SBZzL7A96UFCM02TKskbq2PWBFy9OggLKjX4hddfgnhVEBTQ+iF5mqxDWNKnnf rUxVTdb8yL75bbxU0GWO8x7mmD4B9Sng4jek1ic5FzOHl8fg2enpwDFMPb+RG0a9Tp7N fMYMe1TApnXHzZ42LaLMQM4KyGRej1Jt2+/EcB2fWNhsmRWgbQ+rZn70Z3foWAMuBzXG DnO3pypQI/oQlmo/oRSV6gXvS2daDR+SVogupVXApDvpLwAtZ4f1Ertsvb0deQqAFpmg gBUQ== X-Gm-Message-State: AJIora8DQmvLMF6TmcvANm81WnYSUkbkeN5CtVdgIir7Lq3MyefPst4c Xx6ASeAq8RrMvhZ+L7yWd8U= X-Google-Smtp-Source: AGRyM1sm7IiaOrlOxLUkwk4IaSse3JCDRb0MdFl+laNBUvVoaf/bsCXrFGoueA2xDWSOwmDHME4nvg== X-Received: by 2002:a63:100d:0:b0:411:8781:121a with SMTP id f13-20020a63100d000000b004118781121amr2681022pgl.583.1657715143218; Wed, 13 Jul 2022 05:25:43 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id m7-20020a170902bb8700b0016bf1ed3489sm8719233pls.143.2022.07.13.05.25.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Jul 2022 05:25:42 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Sean Christopherson , Paolo Bonzini Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Like Xu , Wanpeng Li Subject: [PATCH 7/7] KVM: x86/pmu: Defer counter emulated overflow via pmc->stale_counter Date: Wed, 13 Jul 2022 20:25:06 +0800 Message-Id: <20220713122507.29236-8-likexu@tencent.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220713122507.29236-1-likexu@tencent.com> References: <20220713122507.29236-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu There are contextual restrictions on the functions that can be called in the *_exit_handlers_fastpath path, for example calling pmc_reprogram_counter() brings up a host complaint like: [*] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580 [*] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 2981888, name: CPU 15/KVM [*] preempt_count: 1, expected: 0 [*] RCU nest depth: 0, expected: 0 [*] INFO: lockdep is turned off. [*] irq event stamp: 0 [*] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [*] hardirqs last disabled at (0): [] copy_process+0x146a/0x62d0 [*] softirqs last enabled at (0): [] copy_process+0x14a9/0x62d0 [*] softirqs last disabled at (0): [<0000000000000000>] 0x0 [*] Preemption disabled at: [*] [] vcpu_enter_guest+0x1001/0x3dc0 [kvm] [*] CPU: 17 PID: 2981888 Comm: CPU 15/KVM Kdump: 5.19.0-rc1-g239111db364c-dirty #2 [*] Call Trace: [*] [*] dump_stack_lvl+0x6c/0x9b [*] __might_resched.cold+0x22e/0x297 [*] __mutex_lock+0xc0/0x23b0 [*] perf_event_ctx_lock_nested+0x18f/0x340 [*] perf_event_pause+0x1a/0x110 [*] reprogram_counter+0x2af/0x1490 [kvm] [*] kvm_pmu_trigger_event+0x429/0x950 [kvm] [*] kvm_skip_emulated_instruction+0x48/0x90 [kvm] [*] handle_fastpath_set_msr_irqoff+0x349/0x3b0 [kvm] [*] vmx_vcpu_run+0x268e/0x3b80 [kvm_intel] [*] vcpu_enter_guest+0x1d22/0x3dc0 [kvm] A new stale_counter field is introduced to keep this part of the semantics invariant. It records the current counter value and it's used to determine whether to inject an emulated overflow interrupt in the later kvm_pmu_handle_event(), given that the internal count value from its perf_event has not been added to pmc->counter in time, or the guest will update the value of a running counter directly. Opportunistically shrink sizeof(struct kvm_pmc) a bit. Suggested-by: Wanpeng Li Fixes: 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") Signed-off-by: Like Xu --- arch/x86/include/asm/kvm_host.h | 5 +++-- arch/x86/kvm/pmu.c | 15 ++++++++------- arch/x86/kvm/svm/pmu.c | 2 +- arch/x86/kvm/vmx/pmu_intel.c | 4 ++-- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index de5a149d0971..4d85d189d082 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -489,7 +489,10 @@ enum pmc_type { struct kvm_pmc { enum pmc_type type; u8 idx; + bool is_paused; + bool intr; u64 counter; + u64 stale_counter; u64 eventsel; struct perf_event *perf_event; struct kvm_vcpu *vcpu; @@ -498,8 +501,6 @@ struct kvm_pmc { * ctrl value for fixed counters. */ u64 current_config; - bool is_paused; - bool intr; }; #define KVM_PMC_MAX_FIXED 3 diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 681d3ac8d75c..b43561bd8daf 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -352,6 +352,12 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) } __reprogram_counter(pmc); + + if (pmc->stale_counter) { + if (pmc->counter < pmc->stale_counter) + __kvm_perf_overflow(pmc, false); + pmc->stale_counter = 0; + } } /* @@ -524,14 +530,9 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu) static void kvm_pmu_incr_counter(struct kvm_pmc *pmc) { - u64 prev_count; - - prev_count = pmc->counter; + pmc->stale_counter = pmc->counter; pmc->counter = (pmc->counter + 1) & pmc_bitmask(pmc); - - __reprogram_counter(pmc); - if (pmc->counter < prev_count) - __kvm_perf_overflow(pmc, false); + reprogram_counter(pmc); } static inline bool eventsel_match_perf_hw_id(struct kvm_pmc *pmc, diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index f24613a108c5..e9c66dd659a6 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -290,7 +290,7 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu) struct kvm_pmc *pmc = &pmu->gp_counters[i]; pmc_stop_counter(pmc); - pmc->counter = pmc->eventsel = 0; + pmc->counter = pmc->stale_counter = pmc->eventsel = 0; } } diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 5f6b9f596f16..ca4b55085c87 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -654,14 +654,14 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu) pmc = &pmu->gp_counters[i]; pmc_stop_counter(pmc); - pmc->counter = pmc->eventsel = 0; + pmc->counter = pmc->stale_counter = pmc->eventsel = 0; } for (i = 0; i < KVM_PMC_MAX_FIXED; i++) { pmc = &pmu->fixed_counters[i]; pmc_stop_counter(pmc); - pmc->counter = 0; + pmc->counter = pmc->stale_counter = 0; } pmu->fixed_ctr_ctrl = pmu->global_ctrl = pmu->global_status = 0;