From patchwork Thu Jul 21 10:35:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40A79C433EF for ; Thu, 21 Jul 2022 10:36:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232969AbiGUKgC (ORCPT ); Thu, 21 Jul 2022 06:36:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233272AbiGUKgB (ORCPT ); Thu, 21 Jul 2022 06:36:01 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EAB128E1C; Thu, 21 Jul 2022 03:36:00 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id t3-20020a17090a3b4300b001f21eb7e8b0so4227049pjf.1; Thu, 21 Jul 2022 03:36:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ojTdOvekZFxnrXtD4DiHIEP3F03nTJIQqCh8/9hHa4k=; b=Ri9b82ZtYT7TeTf8FyMtOLfy7LgXbRy8Pg5DodWL9WG+QG3isp+1cOfz3ZxbU1aA9/ AVmqOO5TEGYah7SohIfcWJnAo1jiqIxJU96meKkoJnv4gJOanz28svdhhPEJzgaoJDw1 G7KP8heB0Wjwa2ID3JPy8lOpIdkrBFMLzMd/COvVv6HG5gQOtXfFVn8ge11W07FRarRs ayk/gwFuXl7T8zR0zJzb3iA1SC0pTsZe99xu2vtW0X5jfgQxMmfZwLLrfz1dfdHhnzZo Gat5HKev0LK5Y4oA2ZLoIkJNv5CT44OMUYosNZQ61M1jwfmqJLdKmqXtWHpOQKQ/2q+m Fk1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ojTdOvekZFxnrXtD4DiHIEP3F03nTJIQqCh8/9hHa4k=; b=PWEf7UxG4s5p46uNX82Kve5c4R3Q10jDCr5A+y6o+0kk53PEG3QLQxT00EmQ5s8deX yU6bkEu3tmz1gyctZoEGVRTi+ITScygV0CH/eZ9Q2vr6UHnmO4Wj2gYK34MUHlmkbrot TwnyjmzjEjjodrybTAdvNk+J6sANfCO2Ro4pD+yQnfYOiW8hcWliRzH77s9r20pLz+lO 1wKX6Kwc1UM1w0ZfKFwUT6UAXGwgoQPqxEieWu/gxkjYRkuKNNsjpcCxRM9eve+i6FKa O8WSsc1kcB+o2safn/ENf462/8kHrSBaWSyah+0JWprN5GDJVpolPbq5Ut78WsFHon5A SvTA== X-Gm-Message-State: AJIora9eQtzRZ2E7Lc0HqAJP9XCnYYjWtI86N43vW8MzNn/fBOU4KTRY FBnNS42ff0DDdOsraJrotoA= X-Google-Smtp-Source: AGRyM1vPplN6H7FMHen1Lt6GXz+XBq7RgCnVoZksODNCncTaXOXIGYsJFGEKg8EbIMwaIgmFGYl3Qw== X-Received: by 2002:a17:902:a616:b0:16c:d74e:4654 with SMTP id u22-20020a170902a61600b0016cd74e4654mr24025363plq.4.1658399759960; Thu, 21 Jul 2022 03:35:59 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.35.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:35:59 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Kan Liang Subject: [PATCH v2 1/7] perf/x86/core: Update x86_pmu.pebs_capable for ICELAKE_{X,D} Date: Thu, 21 Jul 2022 18:35:42 +0800 Message-Id: <20220721103549.49543-2-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu Ice Lake microarchitecture with EPT-Friendly PEBS capability also support the Extended feature, which means that all counters (both fixed function and general purpose counters) can be used for PEBS events. Update x86_pmu.pebs_capable like SPR to apply PEBS_ALL semantics. Cc: Kan Liang Fixes: fb358e0b811e ("perf/x86/intel: Add EPT-Friendly PEBS for Ice Lake Server") Signed-off-by: Like Xu --- arch/x86/events/intel/core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index 4e9b7af9cc45..e46fd496187b 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -6239,6 +6239,7 @@ __init int intel_pmu_init(void) case INTEL_FAM6_ICELAKE_X: case INTEL_FAM6_ICELAKE_D: x86_pmu.pebs_ept = 1; + x86_pmu.pebs_capable = ~0ULL; pmem = true; fallthrough; case INTEL_FAM6_ICELAKE_L: From patchwork Thu Jul 21 10:35:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2937C43334 for ; Thu, 21 Jul 2022 10:36:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233284AbiGUKgF (ORCPT ); Thu, 21 Jul 2022 06:36:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233277AbiGUKgD (ORCPT ); Thu, 21 Jul 2022 06:36:03 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 633812D1CE; Thu, 21 Jul 2022 03:36:02 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id y24so1411798plh.7; Thu, 21 Jul 2022 03:36:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gDRQtRi/q/kMC+1Mbr451BbZ8n+5KHSY3o4YQBH6BZ4=; b=RZiQ4nF0eC6iDy4YnxcewVc4TO8dRSPbHqdWpQBYl/gJmswoium8hRY/32dzFjpr/F wwG+73HueMEig0tn4e4/RFEU9tK81/iMMVNbJ+um0DliNmkhdN8xzJPf0WQypRkhLvYX ZNEiqcBhpdyUi5t+E1r7sder7W+4SQgcCjayfmAgElNOtrpzOvvmoF3ACMRFFEeQDD7d IArAUR27ntGU9UtFA8UAvb4kzz+FcxavU5p6BsM7p2lNRb9lqeRJMZkpWImAH0MsyxlT sZuVqDnx66NQiOv9ZGfvNNJIcD9OlSBF9dj3s90fbRTM9drfN0o9j+t6adUi4Rs1FA1N WbIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gDRQtRi/q/kMC+1Mbr451BbZ8n+5KHSY3o4YQBH6BZ4=; b=b7xk/aigyCANhqs8vz7wQHple/o23bC5PDRRv1K52MqC1nIZIBhGfSIgYQQmhRXuF1 f/lJrW85YfT8SphLMNaFEcipuqFgte8DK00G42xzP7J4So0KTPM0W873d1ClFBNlQz7E mSgba7d3aJdngujOpngay4Wi3HN+aoD1OQS5CYbhdIuLe2N+x2JJQJ6zjyJG8ses6aAv QPXcF+NNqxovGENAp9Q76LmpFVxMTWs/QjUZ1W6Mg6LN+X6ic7b+jYIZZHEdYY53NAJh 5r3oKM25BRGeRhEJQf3kiJPMOutfFD0ApYtBODLfcOD+ChhM5hGiV4PfqfEB5QrUSrJ7 3IyQ== X-Gm-Message-State: AJIora/HimFkQ/hY69G15GAFs3i3Hki/LqEPHWlwb9mG7mNwLM6hoT4Q vIqguzEZorMqoVL3+2ZxSEwQn5+QzkENaQ== X-Google-Smtp-Source: AGRyM1uEQlbFVsHB+3J2X/Jj7VjG+qIEVzZxyu7U+D4MzljkwEKH501d7gRSg3NjBl4RzRpI2F99pg== X-Received: by 2002:a17:902:e742:b0:16c:44b7:c8b6 with SMTP id p2-20020a170902e74200b0016c44b7c8b6mr42947535plf.140.1658399761871; Thu, 21 Jul 2022 03:36:01 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:01 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v2 2/7] perf/x86/core: Completely disable guest PEBS via guest's global_ctrl Date: Thu, 21 Jul 2022 18:35:43 +0800 Message-Id: <20220721103549.49543-3-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu When a guest PEBS counter is cross-mapped by a host counter, software will remove the corresponding bit in the arr[global_ctrl].guest and expect hardware to perform a change of state "from enable to disable" via the msr_slot[] switch during the vmx transaction. The real world is that if user adjust the counter overflow value small enough, it still opens a tiny race window for the previously PEBS-enabled counter to write cross-mapped PEBS records into the guest's PEBS buffer, when arr[global_ctrl].guest has been prioritised (switch_msr_special stuff) to switch into the enabled state, while the arr[pebs_enable].guest has not. Close this window by clearing invalid bits in the arr[global_ctrl].guest. Fixes: 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") Signed-off-by: Like Xu --- arch/x86/events/intel/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c index e46fd496187b..495ac447bb3a 100644 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -4052,8 +4052,9 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr, void *data) /* Disable guest PEBS if host PEBS is enabled. */ arr[pebs_enable].guest = 0; } else { - /* Disable guest PEBS for cross-mapped PEBS counters. */ + /* Disable guest PEBS thoroughly for cross-mapped PEBS counters. */ arr[pebs_enable].guest &= ~kvm_pmu->host_cross_mapped_mask; + arr[global_ctrl].guest &= ~kvm_pmu->host_cross_mapped_mask; /* Set hw GLOBAL_CTRL bits for PEBS counter when it runs for guest */ arr[global_ctrl].guest |= arr[pebs_enable].guest; } From patchwork Thu Jul 21 10:35:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B45BCC433EF for ; Thu, 21 Jul 2022 10:36:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233303AbiGUKgK (ORCPT ); Thu, 21 Jul 2022 06:36:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233280AbiGUKgF (ORCPT ); Thu, 21 Jul 2022 06:36:05 -0400 Received: from mail-pf1-x435.google.com (mail-pf1-x435.google.com [IPv6:2607:f8b0:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 193CE10F7; Thu, 21 Jul 2022 03:36:04 -0700 (PDT) Received: by mail-pf1-x435.google.com with SMTP id 70so1393814pfx.1; Thu, 21 Jul 2022 03:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eFWQxgmoOmb0OXJQ/pZFCZuTJ4aZfLkbiYFQ43Ipza4=; b=phLpZKgJUCca2MNXFSj7jUoVdqQrWwulesLvEoe7ojliOoavpkNnI5EvlaZKu2XHi8 w7JTFKlHfgQox1tmYzRK4fS6CBH3hqfOBnmx/u7I4DktajZoL560ap1IswAp/LHURXUj b6U1SCCPI2HKOg81+ziNbd+IopWjvTCZZLfGMtcNh164odPg18iK0bbZwW0a94vylAwH NYCehxkioPtYKeBTnO5P+ppYWrN6LQ8R+pTC31J2m4kIE9sBAqmHjFSSyGS659VDPAVv 1DWJzeWI0Ry3MG6mK63JppRubrFZ0/CD/i1yYL+sjWS67RzlX3ikvVl1ploXIqEW/D3o jDrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eFWQxgmoOmb0OXJQ/pZFCZuTJ4aZfLkbiYFQ43Ipza4=; b=GiNgHy5O/1syaZzlKmSjwNKFnpU2GBu/8LCyUX69EIbd61wkSdr6IFCl1Jl7M6r67K XS5u3omEOS7w7iH/5s7FdtjJsEOSYBpeXTpUmxRfwBwmZ8rACtmsVOTRdseAAebnOLko ff7FUgtjEG1k0jhMgzSYC5H/Ayo/ANp6FDIweOKWA3M82fQPINZfTkom4y/IV+3rX8B5 wz/qM6qcp814It8nHDMt1qSgSMpIc2SKRMTsdmHaLtKVWMwAn/3Dn8TXYAlZxEwvSAvC G68OiMT08u+BQs3gEgxWQhAMxRSK3PFlED+Z7mWvQml1lcqs3yos2ytMhB5Rxb5jKKmu D0kQ== X-Gm-Message-State: AJIora8/IPuUjXubFvV7z6sr88bSDpAhvTVBA8GfRvYQMB5lI1JiUixK /RBe+b51EFVSmKng731+Zow= X-Google-Smtp-Source: AGRyM1v9f/CPoSpHsCfRwb3fbhYN2WhLVkyAvAo38NyFTK8Ye5td8wRcdZNqi+b7pMLQuo2rYYHa5Q== X-Received: by 2002:a63:1648:0:b0:41a:49f9:77ae with SMTP id 8-20020a631648000000b0041a49f977aemr11979681pgw.377.1658399763573; Thu, 21 Jul 2022 03:36:03 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:03 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v2 3/7] KVM: x86/pmu: Avoid setting BIT_ULL(-1) to pmu->host_cross_mapped_mask Date: Thu, 21 Jul 2022 18:35:44 +0800 Message-Id: <20220721103549.49543-4-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu In the extreme case of host counters multiplexing and contention, the perf_event requested by the guest's pebs counter is not allocated to any actual physical counter, in which case hw.idx is bookkept as -1, resulting in an out-of-bounds access to host_cross_mapped_mask. Fixes: 854250329c02 ("KVM: x86/pmu: Disable guest PEBS temporarily in two rare situations") Signed-off-by: Like Xu --- arch/x86/kvm/vmx/pmu_intel.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 4bc098fbec31..22793348aa14 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -783,20 +783,20 @@ static void intel_pmu_cleanup(struct kvm_vcpu *vcpu) void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu) { struct kvm_pmc *pmc = NULL; - int bit; + int bit, hw_idx; for_each_set_bit(bit, (unsigned long *)&pmu->global_ctrl, X86_PMC_IDX_MAX) { pmc = intel_pmc_idx_to_pmc(pmu, bit); if (!pmc || !pmc_speculative_in_use(pmc) || - !intel_pmc_is_enabled(pmc)) + !intel_pmc_is_enabled(pmc) || !pmc->perf_event) continue; - if (pmc->perf_event && pmc->idx != pmc->perf_event->hw.idx) { - pmu->host_cross_mapped_mask |= - BIT_ULL(pmc->perf_event->hw.idx); - } + hw_idx = pmc->perf_event->hw.idx; + /* make it a little less dependent on perf's exact behavior */ + if (hw_idx != pmc->idx && hw_idx > -1) + pmu->host_cross_mapped_mask |= BIT_ULL(hw_idx); } } From patchwork Thu Jul 21 10:35:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D80C43334 for ; Thu, 21 Jul 2022 10:36:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233287AbiGUKgR (ORCPT ); Thu, 21 Jul 2022 06:36:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233296AbiGUKgJ (ORCPT ); Thu, 21 Jul 2022 06:36:09 -0400 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC18867155; Thu, 21 Jul 2022 03:36:05 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id b133so1376772pfb.6; Thu, 21 Jul 2022 03:36:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5Ym1FB+/JJQNox/gaxXHdeqZtRcvY1gpDzJL3fRppc8=; b=QBkmWhfEU8U0o5GNBUv8rEQRlOYAJQ+vTkBO40BEjnLDJlROYBfKQJ2fzGQBlj8a8w a06a3ov8Maworp4uiXMcyEcfWOGJM6uGOQ+HoJEiEluHOkxBml3JgLCC5wWzuQ7CDbyp biT/kMvdUywg98YTHoetMxYer+lkN7YX8ZJG+7zmesfqFILkOXYs/V2weUwDSJJoW0Yb 3sXZu2Ry6NSB0FnrrAYkZBi5O0rjmhU1JRY/qvu5g//Z9twsgOBkFVb1JTUZLmX+Isoi Gfej0IrjXTIOEFUso7pI8mk380IS+rWpbP4ymtY2AiWZV8MdKaFt74ICHnWsu98hEvDN O5ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5Ym1FB+/JJQNox/gaxXHdeqZtRcvY1gpDzJL3fRppc8=; b=I+iIGGk6z5HDWRUhDQsOZlb3B9SmaeNpHLaeiuYUIqgthuiEChz7+CDqZO8A+cExJw x28c5NYZUC2V46P7ZSYXG3UYoVTA9hHMY3jHJaKbHNw9zWF2JVbDNAJpA0tzagPkZzPL 5CsBxUXpmRuFYh0TtcwDfsTO1Aa9pBxfViJTmlPefEvZaKuOTSSiqTVZizCHdZblIP6K KULBOC+W2IN9fId4nH61Et29kilD2gb/FmNCXB8nfC9ao5g7Zt1R9iA5azPuIE2T47Im WKhXBBtu7zx6Qi30CL0OOSx9ythqwmhoPBFCBFhe3O2gd46DWfhBaKc3n9zw5pVms85K q5Qg== X-Gm-Message-State: AJIora9UCg557qseC+ykrhXBwmTuB/Cr0A1r9nxEEVxghcgpqcCPhJb6 lyjPYA1MB0fwYfcGV65QRRA= X-Google-Smtp-Source: AGRyM1vB9s6zBC9K5Is/MIZjEzvQtPMT3Wb6hEsyvFEVyXHFhrnT+i4vcHnFWgMOj4wj1uZ18D1/IA== X-Received: by 2002:a62:5bc5:0:b0:528:c346:9632 with SMTP id p188-20020a625bc5000000b00528c3469632mr43099409pfb.48.1658399765341; Thu, 21 Jul 2022 03:36:05 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:05 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v2 4/7] KVM: x86/pmu: Don't generate PEBS records for emulated instructions Date: Thu, 21 Jul 2022 18:35:45 +0800 Message-Id: <20220721103549.49543-5-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu KVM will accumulate an enabled counter for at least INSTRUCTIONS or BRANCH_INSTRUCTION hw event from any KVM emulated instructions, generating emulated overflow interrupt on counter overflow, which in theory should also happen when the PEBS counter overflows but it currently lacks this part of the underlying support (e.g. through software injection of records in the irq context or a lazy approach). In this case, KVM skips the injection of this BUFFER_OVF PMI (effectively dropping one PEBS record) and let the overflow counter move on. The loss of a single sample does not introduce a loss of accuracy, but is easily noticeable for certain specific instructions. This issue is expected to be addressed along with the issue of PEBS cross-mapped counters with a slow-path proposal. Fixes: 79f3e3b58386 ("KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter") Signed-off-by: Like Xu --- arch/x86/kvm/pmu.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 02f9e4f245bd..390d697efde1 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -106,9 +106,19 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) return; if (pmc->perf_event && pmc->perf_event->attr.precise_ip) { - /* Indicate PEBS overflow PMI to guest. */ - skip_pmi = __test_and_set_bit(GLOBAL_STATUS_BUFFER_OVF_BIT, - (unsigned long *)&pmu->global_status); + if (!in_pmi) { + /* + * TODO: KVM is currently _choosing_ to not generate records + * for emulated instructions, avoiding BUFFER_OVF PMI when + * there are no records. Strictly speaking, it should be done + * as well in the right context to improve sampling accuracy. + */ + skip_pmi = true; + } else { + /* Indicate PEBS overflow PMI to guest. */ + skip_pmi = __test_and_set_bit(GLOBAL_STATUS_BUFFER_OVF_BIT, + (unsigned long *)&pmu->global_status); + } } else { __set_bit(pmc->idx, (unsigned long *)&pmu->global_status); } From patchwork Thu Jul 21 10:35:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1730DC433EF for ; Thu, 21 Jul 2022 10:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232488AbiGUKgS (ORCPT ); Thu, 21 Jul 2022 06:36:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33992 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233297AbiGUKgJ (ORCPT ); Thu, 21 Jul 2022 06:36:09 -0400 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE4936A9D3; Thu, 21 Jul 2022 03:36:07 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id s206so1294050pgs.3; Thu, 21 Jul 2022 03:36:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rk4YQEFZ9q3XkvsAPJzCLl3rTK6rSHOeH4GBUtkHDEc=; b=mj4kSgA8HGlu9Nw+Ii1qbzC6c5a71cbDpWq+8HbTfKQMJt6q53h7dzGfExghUxqZ0L VNjYxMAhM8QpDjpLuGA0Sjjio/1ip231Ov2k9hJMOsV59GtiWlvwna5GMLwR1Dl5hc0Z srdJJTXsyMe3EofYXsCSMS4lLKxNNg1SBlpvSL5V5jyknYGxa9M/HpExC4Ai0FMTqor8 vom/aUynFWpt0ZUGeIK0yIf8BAy+6tZjWHWTsTIBomiCkIzMqPub8ePIcomAySKwED66 VD/3yZ7un4bl5ZnDjmRbzYTAioo5Rf/uw66GHpNN4a1JtFsZ2DylxTwZwWqQDbXOajCp mclA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rk4YQEFZ9q3XkvsAPJzCLl3rTK6rSHOeH4GBUtkHDEc=; b=WjTJTNjLnp7LRjWeunY4WnxR34qsvz1x/9mjilr4m5/ElF/X0OZeya/IrlpPjwBFiD griIcUQ3eqEzIMm7T1yHyiESZrNhlEXPuV6JDFWUfJ/EzGC5U1FnAi2FX0bD5ez5mAj+ sytRlY+3/EHxp6U/7Mf7RbxJSfSAxL2POlhV3dDPTyCPAbvd8kw4ACAWQj1l33V+qE5c 7ZlWharKQrylOHwQWKCs46CHIx0doHHmM4EflQDU3RnLCAtlsuYaUydZIWvQ7gxEc/03 eRz8gqHDa3ym5PrqMGWnLZOYeynSKAKThAS4iP/jerNvTj4sp9wE+0bVwMm7yl/4LA1M osrg== X-Gm-Message-State: AJIora9pLmgHBnBE5t6d0B+ppbDdLO7yJ7KcN5C161GBXJBHyjQg0aqm dskgeuRLdt77WMwKHUUAN6s= X-Google-Smtp-Source: AGRyM1txJN0FEip0oGXy99I4KACumLEOC/+1s1HhOdVlBU3UjbAWgK+YkWGXz4gokpajNc1K6s7Ipw== X-Received: by 2002:a62:4e04:0:b0:52b:30f5:59b8 with SMTP id c4-20020a624e04000000b0052b30f559b8mr34941989pfb.37.1658399767349; Thu, 21 Jul 2022 03:36:07 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:07 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v2 5/7] KVM: x86/pmu: Avoid using PEBS perf_events for normal counters Date: Thu, 21 Jul 2022 18:35:46 +0800 Message-Id: <20220721103549.49543-6-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu The check logic in the pmc_resume_counter() to determine whether a perf_event is reusable is partial and flawed, especially when it comes to a pseudocode sequence (not correct but clearly valid) like: - enabling a counter and its PEBS bit - enable global_ctrl - run workload - disable only the PEBS bit, leaving the global_ctrl bit enabled In this corner case, a perf_event created for PEBS can be reused by a normal counter before it has been released and recreated, and when this normal counter overflows, it triggers a PEBS interrupt (precise_ip != 0). To address this issue, the reuse check has been revamped and KVM will go back to do reprogram_counter() when any bit of guest PEBS_ENABLE msr has changed, which is similar to what global_ctrl_changed() does. Fixes: 79f3e3b58386 ("KVM: x86/pmu: Reprogram PEBS event to emulate guest PEBS counter") Signed-off-by: Like Xu --- arch/x86/kvm/pmu.c | 4 ++-- arch/x86/kvm/vmx/pmu_intel.c | 14 +++++++------- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 390d697efde1..d9b9a0f0db17 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -237,8 +237,8 @@ static bool pmc_resume_counter(struct kvm_pmc *pmc) get_sample_period(pmc, pmc->counter))) return false; - if (!test_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->pebs_enable) && - pmc->perf_event->attr.precise_ip) + if (test_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->pebs_enable) != + (!!pmc->perf_event->attr.precise_ip)) return false; /* reuse perf_event to serve as pmc_reprogram_counter() does*/ diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 22793348aa14..97236b6cbe04 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -68,15 +68,11 @@ static struct kvm_pmc *intel_pmc_idx_to_pmc(struct kvm_pmu *pmu, int pmc_idx) } } -/* function is called when global control register has been updated. */ -static void global_ctrl_changed(struct kvm_pmu *pmu, u64 data) +static void reprogram_counters(struct kvm_pmu *pmu, u64 diff) { int bit; - u64 diff = pmu->global_ctrl ^ data; struct kvm_pmc *pmc; - pmu->global_ctrl = data; - for_each_set_bit(bit, (unsigned long *)&diff, X86_PMC_IDX_MAX) { pmc = intel_pmc_idx_to_pmc(pmu, bit); if (pmc) @@ -404,7 +400,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) struct kvm_pmc *pmc; u32 msr = msr_info->index; u64 data = msr_info->data; - u64 reserved_bits; + u64 reserved_bits, diff; switch (msr) { case MSR_CORE_PERF_FIXED_CTR_CTRL: @@ -425,7 +421,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (pmu->global_ctrl == data) return 0; if (kvm_valid_perf_global_ctrl(pmu, data)) { - global_ctrl_changed(pmu, data); + diff = pmu->global_ctrl ^ data; + pmu->global_ctrl = data; + reprogram_counters(pmu, diff); return 0; } break; @@ -440,7 +438,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (pmu->pebs_enable == data) return 0; if (!(data & pmu->pebs_enable_mask)) { + diff = pmu->pebs_enable ^ data; pmu->pebs_enable = data; + reprogram_counters(pmu, diff); return 0; } break; From patchwork Thu Jul 21 10:35:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68E0ECCA479 for ; Thu, 21 Jul 2022 10:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232453AbiGUKgU (ORCPT ); Thu, 21 Jul 2022 06:36:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233314AbiGUKgP (ORCPT ); Thu, 21 Jul 2022 06:36:15 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C922823BC; Thu, 21 Jul 2022 03:36:09 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id c139so1390795pfc.2; Thu, 21 Jul 2022 03:36:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7a5h0BPFo1w7O1Ws8TVqQESIJ7aIAVxGDx6UDb0IKxQ=; b=d4Oanrw6RiRXIeDbtRD4Ip/L3Nx7w+hEo5R0RXxc6R0D5cO4fbz1Jk/1FdoURwS0jF +uBSl5rtnMlpJ5+6JyTxv7mrKR1AjBllph1RogF46Hag/ptLKoRQNOTk0sTGsnFroFoU 4RepTrJtTNENrLZj0a6v1DG4H7XX3tMFXyh35dv6fnbb8FEuRwtL+0GmvP6o3kLkU6US yRp2UMlFh2NGedC+/0SdARASxxw9AqbOlUKA5C6bbXC4BG+YWBmYequXQmNOFCy6I5e/ 8JVfamgkqsBf60Pkb2xv7zBTCbhWv9lmCiTvHdVEN5SnCoRLZEzJFH7x0PtPt4jHm0dF wyRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7a5h0BPFo1w7O1Ws8TVqQESIJ7aIAVxGDx6UDb0IKxQ=; b=3Sl0RBwGLqvpSWRqEnsdby3zBBX/8ixPK0DI1UkOSK8TbUudzcSAnuSgNz4wDUZmBW E8k/Do0aS7omAidcnQzpKqlyDQ7IKbKcMmJQBfu1B7GgZCIaIB9Gb9GGM4FdtVQ5+sz+ 5rM10CbaurLbFj2Ntq72s4TC0ZllegdLFlQg9Ju5FEyjMicTvpxKFlUtHR2VDBGUOQ0a C3nfLlI9kpTLA2b9nAhkt9CR82AlYx7QE8/iVxII5qMiz5UIHJt3kaU6GdPVD6G+dFNa UudaqO3j3+LhCGvkZ5JdAm18CiakNGIg23kkaGA+l6v/CPnXHksRF812aGsbwvRLSs6O aMKg== X-Gm-Message-State: AJIora96cpJ3PN3ZmYCTQVE01Q+kNn5IF6XPvSq86nUkC0/XEd1PsR/t 2b4nS0woyHpZhXPeOdub07T7EU00DEZbIQ== X-Google-Smtp-Source: AGRyM1t/4JwWePBWG9eWg4ZwAierSsn+zFEqr4AEIKZFipYyUjRBrQBfi4enl7fmrsXSCn9ThHHwSw== X-Received: by 2002:a63:6a45:0:b0:419:cb1b:891b with SMTP id f66-20020a636a45000000b00419cb1b891bmr31165425pgc.135.1658399769087; Thu, 21 Jul 2022 03:36:09 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:08 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: [PATCH v2 6/7] KVM: x86/pmu: Defer reprogram_counter() to kvm_pmu_handle_event() Date: Thu, 21 Jul 2022 18:35:47 +0800 Message-Id: <20220721103549.49543-7-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu During a KVM-trap from vm-exit to vm-entry, requests from different sources will try to create one or more perf_events via reprogram_counter(), which will allow some predecessor actions to be undone posteriorly, especially repeated calls to some perf subsystem interfaces. These repetitive calls can be omitted because only the final state of the perf_event and the hardware resources it occupies will take effect for the guest right before the vm-entry. To realize this optimization, KVM marks the creation requirements via an inline version of reprogram_counter(), and then defers the actual execution with the help of vcpu KVM_REQ_PMU request. Opportunistically update related comments to avoid misunderstandings. Signed-off-by: Like Xu --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/pmu.c | 16 +++++++++------- arch/x86/kvm/pmu.h | 6 +++++- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e8281d64a431..0295b763bd14 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -493,6 +493,7 @@ struct kvm_pmc { struct perf_event *perf_event; struct kvm_vcpu *vcpu; /* + * only for creating or reusing perf_event, * eventsel value for general purpose counters, * ctrl value for fixed counters. */ diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index d9b9a0f0db17..6940cbeee54d 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -101,7 +101,7 @@ static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi) struct kvm_pmu *pmu = pmc_to_pmu(pmc); bool skip_pmi = false; - /* Ignore counters that have been reprogrammed already. */ + /* Ignore counters that have not been reprogrammed. */ if (test_and_set_bit(pmc->idx, pmu->reprogram_pmi)) return; @@ -293,7 +293,7 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc) return allow_event; } -void reprogram_counter(struct kvm_pmc *pmc) +static void __reprogram_counter(struct kvm_pmc *pmc) { struct kvm_pmu *pmu = pmc_to_pmu(pmc); u64 eventsel = pmc->eventsel; @@ -335,7 +335,6 @@ void reprogram_counter(struct kvm_pmc *pmc) !(eventsel & ARCH_PERFMON_EVENTSEL_OS), eventsel & ARCH_PERFMON_EVENTSEL_INT); } -EXPORT_SYMBOL_GPL(reprogram_counter); void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) { @@ -345,11 +344,12 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) for_each_set_bit(bit, pmu->reprogram_pmi, X86_PMC_IDX_MAX) { struct kvm_pmc *pmc = static_call(kvm_x86_pmu_pmc_idx_to_pmc)(pmu, bit); - if (unlikely(!pmc || !pmc->perf_event)) { + if (unlikely(!pmc)) { clear_bit(bit, pmu->reprogram_pmi); continue; } - reprogram_counter(pmc); + + __reprogram_counter(pmc); } /* @@ -527,7 +527,7 @@ static void kvm_pmu_incr_counter(struct kvm_pmc *pmc) prev_count = pmc->counter; pmc->counter = (pmc->counter + 1) & pmc_bitmask(pmc); - reprogram_counter(pmc); + __reprogram_counter(pmc); if (pmc->counter < prev_count) __kvm_perf_overflow(pmc, false); } @@ -542,7 +542,9 @@ static inline bool eventsel_match_perf_hw_id(struct kvm_pmc *pmc, static inline bool cpl_is_matched(struct kvm_pmc *pmc) { bool select_os, select_user; - u64 config = pmc->current_config; + u64 config = pmc_is_gp(pmc) ? pmc->eventsel : + (u64)fixed_ctrl_field(pmc_to_pmu(pmc)->fixed_ctr_ctrl, + pmc->idx - INTEL_PMC_IDX_FIXED); if (pmc_is_gp(pmc)) { select_os = config & ARCH_PERFMON_EVENTSEL_OS; diff --git a/arch/x86/kvm/pmu.h b/arch/x86/kvm/pmu.h index 5cc5721f260b..d193d1dc6de0 100644 --- a/arch/x86/kvm/pmu.h +++ b/arch/x86/kvm/pmu.h @@ -183,7 +183,11 @@ static inline void kvm_init_pmu_capability(void) KVM_PMC_MAX_FIXED); } -void reprogram_counter(struct kvm_pmc *pmc); +static inline void reprogram_counter(struct kvm_pmc *pmc) +{ + __set_bit(pmc->idx, pmc_to_pmu(pmc)->reprogram_pmi); + kvm_make_request(KVM_REQ_PMU, pmc->vcpu); +} void kvm_pmu_deliver_pmi(struct kvm_vcpu *vcpu); void kvm_pmu_handle_event(struct kvm_vcpu *vcpu); From patchwork Thu Jul 21 10:35:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Like Xu X-Patchwork-Id: 12924978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BB0DC433EF for ; Thu, 21 Jul 2022 10:36:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233278AbiGUKgW (ORCPT ); Thu, 21 Jul 2022 06:36:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233315AbiGUKgP (ORCPT ); Thu, 21 Jul 2022 06:36:15 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A06382ED5B; Thu, 21 Jul 2022 03:36:11 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id l124so1366869pfl.8; Thu, 21 Jul 2022 03:36:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=X5qNMyDQo4A34X5plEKgK+lj9B4G4Lhqi9J4w62YEBQ=; b=VQgJk5LP1KoQKRSQ7nZK5snNyQzFtjY+59zuQGsgmtxJ7YSYAhq8wa6PAzyZhygGgT 9Ri+R5jFLjcOvulBb3jQeAF1DMO4En1sm08q/hzxOSrYg2xXvvq3xk5YLBC52a3nLy61 jVjYEu8U/WxBWurcLderJ0eIntNHvADuliKexijtlYG8rMgz9lDBY/XxT28fQ5kX4LsL eTMAeCkpbWj1cqFgEjBpjLOWei0lN/Rwm2dNZ34w0udB1KDKbeWDZIj3BvpIwn4CtsBt KCA6jVG+89d71UqXwXYjzam68Tam4efJnfozeKwoX1WEULQApeHtFpM7P83kAlfssyhZ LwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=X5qNMyDQo4A34X5plEKgK+lj9B4G4Lhqi9J4w62YEBQ=; b=M6htiC8I6+soFlCHGWdxl5j7a6VEssARL/HICAO6bxGQCXie9HX/YKi8zPhfmn5jpB 6p2MLE68kpAqlRaUtBQ6eff/N8Mhi4mj0iyHAhZglUoLdXuUcxL8ZPDQ7OGcO4glNpcI isFbczr9KJYoergwvzd54HGypdhdN21TPWczYMl0WKzVREAw59MaoplVvwc9BjO7CKFm eiZcrqDIShrLRG7Tl9sbTnCPO3ItuxfbwF4mSvZojnCBVa8I9wDhGzizeGiMm52IlDlX FVxqyBoe9/Ck25yapjemNXbYFq9v4l6wKJk2+JZQscmV+Ao03V8veUJCo9nRWWURIIbE PfnA== X-Gm-Message-State: AJIora9owhJaPCRBvyJVmsvvWgGOd+UF2sWU5Ih4ZCA0KQ43qXl71xwM nht75PHOmBTGkhOEJsKkLgg= X-Google-Smtp-Source: AGRyM1vfdvl5vmayBovxnh4gAwflYBbqP4uXh++h+aebb2/OsssnTGPlG6s6MHI+2smXuKPMJVvgjQ== X-Received: by 2002:a05:6a00:a12:b0:527:dba9:c416 with SMTP id p18-20020a056a000a1200b00527dba9c416mr43148134pfh.33.1658399771032; Thu, 21 Jul 2022 03:36:11 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.32]) by smtp.gmail.com with ESMTPSA id q12-20020a65494c000000b00419aa0d9a2esm1161887pgs.28.2022.07.21.03.36.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Jul 2022 03:36:10 -0700 (PDT) From: Like Xu X-Google-Original-From: Like Xu To: Paolo Bonzini , Sean Christopherson Cc: Jim Mattson , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Wanpeng Li Subject: [PATCH v2 7/7] KVM: x86/pmu: Defer counter emulated overflow via pmc->stale_counter Date: Thu, 21 Jul 2022 18:35:48 +0800 Message-Id: <20220721103549.49543-8-likexu@tencent.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220721103549.49543-1-likexu@tencent.com> References: <20220721103549.49543-1-likexu@tencent.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Like Xu There are contextual restrictions on the functions that can be called in the *_exit_handlers_fastpath path, for example calling pmc_reprogram_counter() brings up a host complaint like: [*] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:580 [*] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 2981888, name: CPU 15/KVM [*] preempt_count: 1, expected: 0 [*] RCU nest depth: 0, expected: 0 [*] INFO: lockdep is turned off. [*] irq event stamp: 0 [*] hardirqs last enabled at (0): [<0000000000000000>] 0x0 [*] hardirqs last disabled at (0): [] copy_process+0x146a/0x62d0 [*] softirqs last enabled at (0): [] copy_process+0x14a9/0x62d0 [*] softirqs last disabled at (0): [<0000000000000000>] 0x0 [*] Preemption disabled at: [*] [] vcpu_enter_guest+0x1001/0x3dc0 [kvm] [*] CPU: 17 PID: 2981888 Comm: CPU 15/KVM Kdump: 5.19.0-rc1-g239111db364c-dirty #2 [*] Call Trace: [*] [*] dump_stack_lvl+0x6c/0x9b [*] __might_resched.cold+0x22e/0x297 [*] __mutex_lock+0xc0/0x23b0 [*] perf_event_ctx_lock_nested+0x18f/0x340 [*] perf_event_pause+0x1a/0x110 [*] reprogram_counter+0x2af/0x1490 [kvm] [*] kvm_pmu_trigger_event+0x429/0x950 [kvm] [*] kvm_skip_emulated_instruction+0x48/0x90 [kvm] [*] handle_fastpath_set_msr_irqoff+0x349/0x3b0 [kvm] [*] vmx_vcpu_run+0x268e/0x3b80 [kvm_intel] [*] vcpu_enter_guest+0x1d22/0x3dc0 [kvm] A new stale_counter field is introduced to keep this part of the semantics invariant. It records the current counter value and it's used to determine whether to inject an emulated overflow interrupt in the later kvm_pmu_handle_event(), given that the internal count value from its perf_event has not been added to pmc->counter in time, or the guest will update the value of a running counter directly. Opportunistically shrink sizeof(struct kvm_pmc) a bit. Suggested-by: Wanpeng Li Fixes: 9cd803d496e7 ("KVM: x86: Update vPMCs when retiring instructions") Signed-off-by: Like Xu --- arch/x86/include/asm/kvm_host.h | 5 +++-- arch/x86/kvm/pmu.c | 15 ++++++++------- arch/x86/kvm/svm/pmu.c | 2 +- arch/x86/kvm/vmx/pmu_intel.c | 4 ++-- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0295b763bd14..92f397361e3f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -488,7 +488,10 @@ enum pmc_type { struct kvm_pmc { enum pmc_type type; u8 idx; + bool is_paused; + bool intr; u64 counter; + u64 stale_counter; u64 eventsel; struct perf_event *perf_event; struct kvm_vcpu *vcpu; @@ -498,8 +501,6 @@ struct kvm_pmc { * ctrl value for fixed counters. */ u64 current_config; - bool is_paused; - bool intr; }; #define KVM_PMC_MAX_FIXED 3 diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c index 6940cbeee54d..45d062cb1dd5 100644 --- a/arch/x86/kvm/pmu.c +++ b/arch/x86/kvm/pmu.c @@ -350,6 +350,12 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu) } __reprogram_counter(pmc); + + if (pmc->stale_counter) { + if (pmc->counter < pmc->stale_counter) + __kvm_perf_overflow(pmc, false); + pmc->stale_counter = 0; + } } /* @@ -522,14 +528,9 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu) static void kvm_pmu_incr_counter(struct kvm_pmc *pmc) { - u64 prev_count; - - prev_count = pmc->counter; + pmc->stale_counter = pmc->counter; pmc->counter = (pmc->counter + 1) & pmc_bitmask(pmc); - - __reprogram_counter(pmc); - if (pmc->counter < prev_count) - __kvm_perf_overflow(pmc, false); + reprogram_counter(pmc); } static inline bool eventsel_match_perf_hw_id(struct kvm_pmc *pmc, diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c index f24613a108c5..e9c66dd659a6 100644 --- a/arch/x86/kvm/svm/pmu.c +++ b/arch/x86/kvm/svm/pmu.c @@ -290,7 +290,7 @@ static void amd_pmu_reset(struct kvm_vcpu *vcpu) struct kvm_pmc *pmc = &pmu->gp_counters[i]; pmc_stop_counter(pmc); - pmc->counter = pmc->eventsel = 0; + pmc->counter = pmc->stale_counter = pmc->eventsel = 0; } } diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 97236b6cbe04..e3d9f2876081 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -654,14 +654,14 @@ static void intel_pmu_reset(struct kvm_vcpu *vcpu) pmc = &pmu->gp_counters[i]; pmc_stop_counter(pmc); - pmc->counter = pmc->eventsel = 0; + pmc->counter = pmc->stale_counter = pmc->eventsel = 0; } for (i = 0; i < KVM_PMC_MAX_FIXED; i++) { pmc = &pmu->fixed_counters[i]; pmc_stop_counter(pmc); - pmc->counter = 0; + pmc->counter = pmc->stale_counter = 0; } pmu->fixed_ctr_ctrl = pmu->global_ctrl = pmu->global_status = 0;