From patchwork Fri Jun 25 15:32:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D74BC2B9F4 for ; Fri, 25 Jun 2021 15:32:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CA8A61945 for ; Fri, 25 Jun 2021 15:32:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230044AbhFYPeu (ORCPT ); Fri, 25 Jun 2021 11:34:50 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44575 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229884AbhFYPen (ORCPT ); Fri, 25 Jun 2021 11:34:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=INi7/NQmQ1q6t1hySvPoVbZhD1CFSoMp7ejAC0hIDQg=; b=E8yedwCWXGaK7urseubcbLtmOwcVZFSdvPLpKieVrfm+m0F6YMRF7SFNNOoueIUBZa1eVK 8VJkfP16d45y/etDvXJzwYJptPHjSPEo5+pDENuNN3n9K2luxTDyx7O5t14iGtJW8SOVwa b3kZ5jI6HA1MGOJOQCi2Sg+lfE7qWTM= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-533-_KzNI9EgP7yzx1TliiGntQ-1; Fri, 25 Jun 2021 11:32:18 -0400 X-MC-Unique: _KzNI9EgP7yzx1TliiGntQ-1 Received: by mail-io1-f72.google.com with SMTP id l15-20020a5e820f0000b02904bd1794d00eso7273573iom.7 for ; Fri, 25 Jun 2021 08:32:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=INi7/NQmQ1q6t1hySvPoVbZhD1CFSoMp7ejAC0hIDQg=; b=cpgup/AitTpHl+hVgm3i1TP6jk4zNmxosOeFpjLVs/pi7GhmuUkIlpagNAl2gxPrl/ VIHTw5MXBNEBB+OAYWhpyYpNTT3KVqN7HGfYYbigCTBys4hBmFTpYRsQUiWL4Y/fxCmH q7yn74HdFhaQYYjFaFk9pavEeo42EpWSZjeyQs7LCOp5MVda0ddm4jPRODJAimibsMJ5 vUKNDKi8ffZiu8BIx/tExBu2Rq/Je+Y1GniMj1/vVkz8d9Xq/030EeUsu5Adwx25phzS Zw0GAJtKWtvVsGRIFwMTxELxO+IVhJ/TLj78noczSx+FfirRRz/hr94oSnpmSq+9ZkKc 1I2A== X-Gm-Message-State: AOAM533SZ65jroEtCO3nzuN6Ge/jy/GeEmfvtBB60dZNY2uEuADeI6N0 J86bKt71lvMW7Oel14JlY4NIuSdbcTksTn298NcX/m1ke5XCbWnRFhNsizM2Wq4wfCT6l4A+CC7 275CDSJjPu8LM X-Received: by 2002:a05:6638:3fa:: with SMTP id s26mr10138670jaq.16.1624635138228; Fri, 25 Jun 2021 08:32:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzGLqINrVZS5FKrdcEe4KyJJCkXQZcmXAZYbOxtJXj3K1ST9f/AyKv8aClT53i5y2FrxTEjdA== X-Received: by 2002:a05:6638:3fa:: with SMTP id s26mr10138658jaq.16.1624635138108; Fri, 25 Jun 2021 08:32:18 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:17 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 1/9] KVM: X86: Add per-vm stat for max rmap list size Date: Fri, 25 Jun 2021 11:32:06 -0400 Message-Id: <20210625153214.43106-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a new statistic max_mmu_rmap_size, which stores the maximum size of rmap for the vm. Signed-off-by: Peter Xu --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/x86.c | 1 + 3 files changed, 4 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 974cbfb1eefe..d798650ad793 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1209,6 +1209,7 @@ struct kvm_vm_stat { u64 lpages; u64 nx_lpage_splits; u64 max_mmu_page_hash_collisions; + u64 max_mmu_rmap_size; }; struct kvm_vcpu_stat { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b888385d1933..eb16c1dbbb32 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2696,6 +2696,8 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, u64 *sptep, if (is_shadow_present_pte(*sptep)) { if (!was_rmapped) { rmap_count = rmap_add(vcpu, sptep, gfn); + if (rmap_count > vcpu->kvm->stat.max_mmu_rmap_size) + vcpu->kvm->stat.max_mmu_rmap_size = rmap_count; if (rmap_count > RMAP_RECYCLE_THRESHOLD) rmap_recycle(vcpu, sptep, gfn); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 8166ad113fb2..d83ccf35ce99 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -237,6 +237,7 @@ const struct _kvm_stats_desc kvm_vm_stats_desc[] = { STATS_DESC_ICOUNTER(VM, mmu_unsync), STATS_DESC_ICOUNTER(VM, lpages), STATS_DESC_ICOUNTER(VM, nx_lpage_splits), + STATS_DESC_PCOUNTER(VM, max_mmu_rmap_size), STATS_DESC_PCOUNTER(VM, max_mmu_page_hash_collisions) }; static_assert(ARRAY_SIZE(kvm_vm_stats_desc) == From patchwork Fri Jun 25 15:32:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D9FC49EA7 for ; Fri, 25 Jun 2021 15:32:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 649346193F for ; Fri, 25 Jun 2021 15:32:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229949AbhFYPet (ORCPT ); Fri, 25 Jun 2021 11:34:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48333 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229952AbhFYPen (ORCPT ); Fri, 25 Jun 2021 11:34:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tm+fvqcK0gMUF4HiB0nRTnFiIWjA8jvrus1t7f0ofxA=; b=XMv4slkdysVkdpBngxodZb2/88tqa+YRdPhOgNwY8ukYiEhvtx87m2XYVUvgchbcDR1HmV K8smNx5+vl0D2+n3WNO4r/zHrFQWLRaQYZ2MrzxgbPQWeQcXh4c1Ypwfv02m+ZwcHK9NGl y7j2zavdonEXDvGcCyFSZSrxS76XdK0= Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-262-PKyzM-ngPFaItvxK3kO4Wg-1; Fri, 25 Jun 2021 11:32:20 -0400 X-MC-Unique: PKyzM-ngPFaItvxK3kO4Wg-1 Received: by mail-io1-f69.google.com with SMTP id s14-20020a5eaa0e0000b02904abce57cb24so7188964ioe.21 for ; Fri, 25 Jun 2021 08:32:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Tm+fvqcK0gMUF4HiB0nRTnFiIWjA8jvrus1t7f0ofxA=; b=Q8GFKWbGV+92YhNKAEwfSSRrwe6N0u9XAGiwEVnl/qpOJQK8P7TTog1Ul52r2N0xEp P84PANpjP3QY8dxMBsW476j+qJyHCLfMfMsuLi//jWHemlzukH8GMo6EQ+W0J8K3Q7cs rXhNOps4qQwVdjbByYd+KQcAYoP2pIbkzKUHseyBXvcC4yMWrR5iwYIl72sT7jcDuzL3 Le/HD55MQMGv+kOBuEcsAtpj0YXL2sn2teBc+gMG4KRfc/E3qGB8AqFYDgthjHp4Jdqr tBDrglrdHEL8C9nCuisojgXKlYg2EW72S6TdK2fcpBr0K6IF9lRNkODQrXQ3XnQ36YJ/ S32A== X-Gm-Message-State: AOAM5321TOWHV5n4WVds3zNqWBJhRv+d7vMO1pxU/o0ZIR0d/NmOOQNP VMkpwmtd7zZgwTXKBXmALfAMDKwIlkHmP44+NrK/T8Q/fCWh/nDexj9Yfud4RNxz33Upedje4su BxYqklPCvk1C4 X-Received: by 2002:a02:942e:: with SMTP id a43mr10393264jai.74.1624635140314; Fri, 25 Jun 2021 08:32:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwoOys8GtsgCTOdC7HU1OSYsX1Y5M+rHwqiqWpxEhcIMrFYq9NjwR0UauN7xWhdPtQoSa3ovg== X-Received: by 2002:a02:942e:: with SMTP id a43mr10393241jai.74.1624635140109; Fri, 25 Jun 2021 08:32:20 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:19 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 2/9] KVM: Introduce kvm_get_kvm_safe() Date: Fri, 25 Jun 2021 11:32:07 -0400 Message-Id: <20210625153214.43106-3-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce this safe version of kvm_get_kvm() so that it can be called even during vm destruction. Use it in kvm_debugfs_open() and remove the verbose comment. Prepare to be used elsewhere. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 17 +++++++++++------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ae7735b490b4..c6fcd75dd8b9 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -720,6 +720,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, void kvm_exit(void); void kvm_get_kvm(struct kvm *kvm); +bool kvm_get_kvm_safe(struct kvm *kvm); void kvm_put_kvm(struct kvm *kvm); bool file_is_kvm(struct file *file); void kvm_put_kvm_no_destroy(struct kvm *kvm); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 3dcc2abbfc60..79b0c1b7b284 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1120,6 +1120,16 @@ void kvm_get_kvm(struct kvm *kvm) } EXPORT_SYMBOL_GPL(kvm_get_kvm); +/* + * Make sure the vm is not during destruction, which is a safe version of + * kvm_get_kvm(). Return true if kvm referenced successfully, false otherwise. + */ +bool kvm_get_kvm_safe(struct kvm *kvm) +{ + return refcount_inc_not_zero(&kvm->users_count); +} +EXPORT_SYMBOL_GPL(kvm_get_kvm_safe); + void kvm_put_kvm(struct kvm *kvm) { if (refcount_dec_and_test(&kvm->users_count)) @@ -4925,12 +4935,7 @@ static int kvm_debugfs_open(struct inode *inode, struct file *file, struct kvm_stat_data *stat_data = (struct kvm_stat_data *) inode->i_private; - /* The debugfs files are a reference to the kvm struct which - * is still valid when kvm_destroy_vm is called. - * To avoid the race between open and the removal of the debugfs - * directory we test against the users count. - */ - if (!refcount_inc_not_zero(&stat_data->kvm->users_count)) + if (!kvm_get_kvm_safe(stat_data->kvm)) return -ENOENT; if (simple_attr_open(inode, file, get, From patchwork Fri Jun 25 15:32:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5665FC49EAB for ; Fri, 25 Jun 2021 15:32:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C75A6193F for ; Fri, 25 Jun 2021 15:32:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229944AbhFYPev (ORCPT ); Fri, 25 Jun 2021 11:34:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53372 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229955AbhFYPep (ORCPT ); Fri, 25 Jun 2021 11:34:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CyUuFYaAoVveXOt5smrSbQaSHQszxWEWMgmcLEywPtc=; b=i4EDn/0QxuAIE6Q93U31zVi7EYCu7p8Sa/yNgW3d+Gb18OreVUuiicEJwt/GWJFj1hP6CB psZsnKULnNc8pz3Leds871Xqp3q8uzchd2vr6JG9V8ZoIZ0m0RVl3G9kOMwQ4KftAspAAZ uU9JTSNRhpkw52Kq9weaUzH7Oo5puaM= Received: from mail-il1-f198.google.com (mail-il1-f198.google.com [209.85.166.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-404-cmsDhvwAP2-JUqTaQ3r-MQ-1; Fri, 25 Jun 2021 11:32:22 -0400 X-MC-Unique: cmsDhvwAP2-JUqTaQ3r-MQ-1 Received: by mail-il1-f198.google.com with SMTP id x2-20020a056e021bc2b02901ee78f516b4so631234ilv.11 for ; Fri, 25 Jun 2021 08:32:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CyUuFYaAoVveXOt5smrSbQaSHQszxWEWMgmcLEywPtc=; b=WYjDmlr1BvQ+211L7VVrTVuW0cVujFhbB37VovRbr96AwVO2CBdodkyciUiwVGWxV2 VK0CkSCOAJfpi+Xkq41R1m6UATeG9cKhZH3nIsWtVV+Eo3p2o8h5NKD9z6zPwdo6ZMEG jTgWQBUxv/I3NPdDglwQiOJFiut/FKHhJ0dJw7GrhfxNTIuqpKsQr2j3NV4wO8XG7q2k 4nnUHI0IHJA9cakQjvFw3CVUgJG2pv8FkVprclybeda3+WmAwkBP3Rk9eZyIFAfgCzR7 Q8//VeP1P/i80/SyyI6tj3LpDQeHX1J1pHkyPfEdBHVopoHkI9RNxYpixuCqk/MDC+zZ 7TCw== X-Gm-Message-State: AOAM532Ql+8aZUr6dUyTyG83zjEJwIGWppySZAjLLVR1UtA+W/DYo11u jAmZkb2CFplybkfyPiJr5KoL6BKehhbhgCYn04w70+trMu9GAvOCdDdkGF5/mKEXY4kKzHu106B MOpgymEoB/wRK X-Received: by 2002:a02:628b:: with SMTP id d133mr9920050jac.27.1624635141966; Fri, 25 Jun 2021 08:32:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy9YNK4wl19El0hz9j0y8hwec2iaaS6VyEe/RlX8sikVF2hwYG4LnksdMqlMXL8QScakdtIwQ== X-Received: by 2002:a02:628b:: with SMTP id d133mr9920020jac.27.1624635141672; Fri, 25 Jun 2021 08:32:21 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:20 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 3/9] KVM: Allow to have arch-specific per-vm debugfs files Date: Fri, 25 Jun 2021 11:32:08 -0400 Message-Id: <20210625153214.43106-4-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Allow archs to create arch-specific nodes under kvm->debugfs_dentry directory besides the stats fields. The new interface kvm_arch_create_vm_debugfs() is defined but not yet used. It's called after kvm->debugfs_dentry is created, so it can be referenced directly in kvm_arch_create_vm_debugfs(). Arch should define their own versions when they want to create extra debugfs nodes. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 20 +++++++++++++++++++- 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index c6fcd75dd8b9..8521d3492eb2 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1035,6 +1035,7 @@ bool kvm_arch_dy_runnable(struct kvm_vcpu *vcpu); bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu); int kvm_arch_post_init_vm(struct kvm *kvm); void kvm_arch_pre_destroy_vm(struct kvm *kvm); +int kvm_arch_create_vm_debugfs(struct kvm *kvm); #ifndef __KVM_HAVE_ARCH_VM_ALLOC /* diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 79b0c1b7b284..516ba8d25bda 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -895,7 +895,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, int fd) char dir_name[ITOA_MAX_LEN * 2]; struct kvm_stat_data *stat_data; const struct _kvm_stats_desc *pdesc; - int i; + int i, ret; int kvm_debugfs_num_entries = kvm_vm_stats_header.num_desc + kvm_vcpu_stats_header.num_desc; @@ -940,6 +940,13 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, int fd) kvm->debugfs_dentry, stat_data, &stat_fops_per_vm); } + + ret = kvm_arch_create_vm_debugfs(kvm); + if (ret) { + kvm_destroy_vm_debugfs(kvm); + return i; + } + return 0; } @@ -960,6 +967,17 @@ void __weak kvm_arch_pre_destroy_vm(struct kvm *kvm) { } +/* + * Called after per-vm debugfs created. When called kvm->debugfs_dentry should + * be setup already, so we can create arch-specific debugfs entries under it. + * Cleanup should be automatic done in kvm_destroy_vm_debugfs() recursively, so + * a per-arch destroy interface is not needed. + */ +int __weak kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ + return 0; +} + static struct kvm *kvm_create_vm(unsigned long type) { struct kvm *kvm = kvm_arch_alloc_vm(); From patchwork Fri Jun 25 15:32:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B16EC48BC2 for ; Fri, 25 Jun 2021 15:32:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 446D961969 for ; Fri, 25 Jun 2021 15:32:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230126AbhFYPe7 (ORCPT ); Fri, 25 Jun 2021 11:34:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41896 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230006AbhFYPer (ORCPT ); Fri, 25 Jun 2021 11:34:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R6vqFh3euA4lluHZrjH7C0VB/8jbPFxvQZcd2X8+z8A=; b=gOg0EzZ/PO+pmdV7p/lvXOkYihwQyyn+imYWLEP6OAE36HqVGSZJZGM6TFLoZcvQg3RzYR Tk4GOnHFN1dlXfM26swAuX/WtBXuWuRVyOuptL+bmMVN13p8Acb+uloxfNlxlQ8XPVn+It 1cxC+0o3itqf+3DLnFV4iAkune3MqlU= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-155-eQaVRf95NLO2x_rnyNpxLw-1; Fri, 25 Jun 2021 11:32:24 -0400 X-MC-Unique: eQaVRf95NLO2x_rnyNpxLw-1 Received: by mail-io1-f72.google.com with SMTP id x21-20020a5d99150000b02904e00bb129f0so7237300iol.18 for ; Fri, 25 Jun 2021 08:32:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R6vqFh3euA4lluHZrjH7C0VB/8jbPFxvQZcd2X8+z8A=; b=XvtH+HKzwI4XL5ESYQXdv3uPe7HVh2izTQV9aoQOv0yJhs4LQgDAOVjsxU06gN8RVN BtHmYBJmC6pE4TnFvmILd+35OJD9QcehKFFtlZ0OffuLLV9oHlzKAN6D2vWQSXqJwO9x 0B3a9b2Z+Ub0sZTm4KmpxvldtzKuKNbUcNGEmp5mK+icdQ6jRwncFiTfKGTG49LWyl83 Ksg9DUtdLWkdKbA1H8w2QVlSDpG4fMmkl+hkvgCt3hz91JBZyhTPy4TAzo+QYJm/pQUs jW73L8nYYW93dZtsuFVC9oUopP5XqsyYL162vdSpeoznRrEPZn4gzWLoqxaJImk8l1Rq v3pA== X-Gm-Message-State: AOAM531kwJ9PWHxRm2GfTMcztXSAU+ZKmJ2rbNjko4+fTizMHiT6hU9A ILmHElPj/Rqw0p/h3C5YTlhAVsNA1rswDir7cXMGZc0YW60IehbhiBYhK+5RhQ2p02TpDyKylES Cmm2J+KNjL2KX X-Received: by 2002:a92:ddce:: with SMTP id d14mr8010926ilr.279.1624635143828; Fri, 25 Jun 2021 08:32:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxiQrN5psgZSp7PJvYOwSrLNB9dXZdC9/rSyism8PuiJvzWVvPfNV/3BaOd5802wgskCbtFHg== X-Received: by 2002:a92:ddce:: with SMTP id d14mr8010912ilr.279.1624635143672; Fri, 25 Jun 2021 08:32:23 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:23 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 4/9] KVM: X86: Introduce pte_list_count() helper Date: Fri, 25 Jun 2021 11:32:09 -0400 Message-Id: <20210625153214.43106-5-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This helper is used to count the number of rmap entries in the rmap list pointed by the kvm_rmap_head. Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 21 +++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 22 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index eb16c1dbbb32..b3f738a7c05e 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -990,6 +990,27 @@ static void pte_list_remove(struct kvm_rmap_head *rmap_head, u64 *sptep) __pte_list_remove(sptep, rmap_head); } +unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) +{ + struct pte_list_desc *desc; + unsigned int i, count = 0; + + if (!rmap_head->val) + return 0; + else if (!(rmap_head->val & 1)) + return 1; + + desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); + + while (desc) { + for (i = 0; (i < PTE_LIST_EXT) && desc->sptes[i]; i++) + count++; + desc = desc->more; + } + + return count; +} + static struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, struct kvm_memory_slot *slot) { diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 35567293c1fd..325b4242deed 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -131,6 +131,7 @@ bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, int min_level); void kvm_flush_remote_tlbs_with_address(struct kvm *kvm, u64 start_gfn, u64 pages); +unsigned int pte_list_count(struct kvm_rmap_head *rmap_head); /* * Return values of handle_mmio_page_fault, mmu.page_fault, and fast_page_fault(). From patchwork Fri Jun 25 15:32:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF4B6C49EAB for ; Fri, 25 Jun 2021 15:32:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 99F2C61948 for ; Fri, 25 Jun 2021 15:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230150AbhFYPfA (ORCPT ); Fri, 25 Jun 2021 11:35:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:33884 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230025AbhFYPes (ORCPT ); Fri, 25 Jun 2021 11:34:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pptgt+0cg64XdyXM9V3421uGBBdkZsSuTXz3mJfy9p8=; b=hkcCjioxaqPpDG/Rmp76C1djB1LWo+MHv241j3LvIHqQWmDDSTdw4lsn9PKHA3sn/nOzFZ Xk6fgVP2JHBHKEcyOeQPBjYGPM9Au7Fa1ICgHhLyDn/7RdidE/pz+OxIm3rqEJSu2JmA4w PW+fkBy3dSYm9ab0K6Ey0OSVYwGbkto= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-o80-R5B3OrigQP3UYN5kwA-1; Fri, 25 Jun 2021 11:32:26 -0400 X-MC-Unique: o80-R5B3OrigQP3UYN5kwA-1 Received: by mail-io1-f70.google.com with SMTP id q15-20020a6b710f0000b02904e2f00a469fso7279961iog.4 for ; Fri, 25 Jun 2021 08:32:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pptgt+0cg64XdyXM9V3421uGBBdkZsSuTXz3mJfy9p8=; b=SY+iI35oPVj7ZFnXY+kR6mpzJaDlLy4RZvMAm2z9yT1OIeK4aZT/5BNYCY3y88gUNx MxFNhl9xE2xxKzTHzsjSSwUPhA9o3hQEW8Eyh4ouNxKR7/GHRUdTcc66rd2DWIrCt+mv JVA1vdEQdeWh2oL93wo6dFHmv8jv9jR05U8DIDe828bADFbkmQXeI+t1N4YFKk05sc9m qwbKfNUYl3TWMi90FqJ4opCRxc6XBK7KdATKBtEwVvad07Pf+uvzCqKUGMTLjVzvJF5+ Q3TmnlB9V38GHW1L2FKRDmZt8xK470odP+qNdnKGOOaxcM9+qCvIkGhdLM10591MjbiX fL4w== X-Gm-Message-State: AOAM532ML0iNNEy3M/+n5ixkbNYfqmYtupOfnNjdHW44AZGsuE707mo4 Ej+hgAUMG/u1PIEqHHPrRjE9fOl9/QGqabVypLhUxE4s0hQJbgw+YEutrg17gprKmlYRs8NIAlt JEcH3enmXCWI1 X-Received: by 2002:a6b:8ec2:: with SMTP id q185mr9021647iod.22.1624635145376; Fri, 25 Jun 2021 08:32:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7vAtxUZdoN7t8hc5kr55b1s/kxAsmrBzk8buzgMj/tXFmxxm0ycrEYreLFaAKnJgoOKf43w== X-Received: by 2002:a6b:8ec2:: with SMTP id q185mr9021634iod.22.1624635145225; Fri, 25 Jun 2021 08:32:25 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:24 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 5/9] KVM: X86: Introduce kvm_mmu_slot_lpages() helpers Date: Fri, 25 Jun 2021 11:32:10 -0400 Message-Id: <20210625153214.43106-6-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce kvm_mmu_slot_lpages() to calculcate lpage_info and rmap array size. The other __kvm_mmu_slot_lpages() can take an extra parameter of npages rather than fetching from the memslot pointer. Start to use the latter one in kvm_alloc_memslot_metadata(). Signed-off-by: Peter Xu --- arch/x86/kvm/x86.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d83ccf35ce99..b64708f9f27d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -301,6 +301,20 @@ static struct kmem_cache *x86_fpu_cache; static struct kmem_cache *x86_emulator_cache; +static inline unsigned long +__kvm_mmu_slot_lpages(struct kvm_memory_slot *slot, unsigned long npages, + int level) +{ + return gfn_to_index(slot->base_gfn + npages - 1, + slot->base_gfn, level) + 1; +} + +static inline unsigned long +kvm_mmu_slot_lpages(struct kvm_memory_slot *slot, int level) +{ + return __kvm_mmu_slot_lpages(slot, slot->npages, level); +} + /* * When called, it means the previous get/set msr reached an invalid msr. * Return true if we want to ignore/silent this failed msr access. @@ -11388,8 +11402,7 @@ static int kvm_alloc_memslot_metadata(struct kvm *kvm, int lpages; int level = i + 1; - lpages = gfn_to_index(slot->base_gfn + npages - 1, - slot->base_gfn, level) + 1; + lpages = __kvm_mmu_slot_lpages(slot, npages, level); linfo = kvcalloc(lpages, sizeof(*linfo), GFP_KERNEL_ACCOUNT); if (!linfo) From patchwork Fri Jun 25 15:32:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D36AC49EA7 for ; Fri, 25 Jun 2021 15:32:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0D61361985 for ; Fri, 25 Jun 2021 15:32:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230192AbhFYPfD (ORCPT ); Fri, 25 Jun 2021 11:35:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:30573 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229974AbhFYPeu (ORCPT ); Fri, 25 Jun 2021 11:34:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5L+y5wxukijhnMqO8uY2oYn+ZxljnPTrsvoXiVvQBkI=; b=P6/ogbGKFMDGkYtxroXXL3dMZUC3SHxttqmzz+GQc9NyFXNGVyIt44WRRNDa9bm0qAtDBi pR1+gqa9SNVNnYcx0QsZWzoMCgHRAQqobmf+cDAUwUYQqRzj+vXfSfsufBECP5IBhZLxdv sOD7Ahueyf2yPAM9bWAxRUTaII+3E5Y= Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-221-33Ps384pP0Ojo0x-YBp0jw-1; Fri, 25 Jun 2021 11:32:28 -0400 X-MC-Unique: 33Ps384pP0Ojo0x-YBp0jw-1 Received: by mail-io1-f72.google.com with SMTP id d24-20020a5d95180000b02904f1970a0af0so1106660iom.6 for ; Fri, 25 Jun 2021 08:32:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5L+y5wxukijhnMqO8uY2oYn+ZxljnPTrsvoXiVvQBkI=; b=E5JdceTnhBSqkUdaeRZRzoB1s2cRj04xI6lmGI4/9vvzQp2MEu3RQOjfXbAmoRVIff 7VIEbivPGNipzJm4k94+jwJa6s/8xENUKPsmr1GOivsOxo4sH3dTR914IqDbXJ4tm7yd OEXTvoQWhWoQuQ4SSOqoysV2Nsncfa3I0+4nOqMYJuYL55u1Mqxnv0M6fMGXlWnbb9m/ WBEYwP4R++xGZE0dutkZL0UCjH/M1L97BnHyqpGyMFqUHKy2qHQnSUJxfAT2RMQOnUxP M9hsLaqz01eEncBV/bjLGV82nWeTP6Wsor0rNNxETYp/s1RGVQxQ+HnNc6zNttB1fCNp 3Dzw== X-Gm-Message-State: AOAM5308G35vu0a+LHsws4KWlKucVkNx1ODjSPW3DU1hCjdD9vHDQtB7 2xh7BYWjfpzgdqux7RqB9WnIjJtHZkYqAz5DPrwNoYgS/gar9uP7jHrj8YKiH7nrGeRDdqtV3Vj 4h/HbwzcmZPnP X-Received: by 2002:a6b:4905:: with SMTP id u5mr8857144iob.55.1624635146772; Fri, 25 Jun 2021 08:32:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyTfhbYTxu+uLNMWEhDkuuhs/pLwJAGvcKmYGmdYiFeDS8YErT8ZbVBAACcbTtCofcPlVsnpA== X-Received: by 2002:a6b:4905:: with SMTP id u5mr8857130iob.55.1624635146616; Fri, 25 Jun 2021 08:32:26 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id s8sm3668772ilj.51.2021.06.25.08.32.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:32:26 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Vitaly Kuznetsov , Paolo Bonzini , Sean Christopherson , peterx@redhat.com, Maxim Levitsky Subject: [PATCH v2 6/9] KVM: X86: Introduce mmu_rmaps_stat per-vm debugfs file Date: Fri, 25 Jun 2021 11:32:11 -0400 Message-Id: <20210625153214.43106-7-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use this file to dump rmap statistic information. The statistic is done by calculating the rmap count and the result is log-2-based. An example output of this looks like (idle 6GB guest, right after boot linux): Rmap_Count: 0 1 2-3 4-7 8-15 16-31 32-63 64-127 128-255 256-511 512-1023 Level=4K: 3086676 53045 12330 1272 502 121 76 2 0 0 0 Level=2M: 5947 231 0 0 0 0 0 0 0 0 0 Level=1G: 32 0 0 0 0 0 0 0 0 0 0 Signed-off-by: Peter Xu --- arch/x86/kvm/x86.c | 113 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b64708f9f27d..c3c51361cf6c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -30,6 +30,7 @@ #include "hyperv.h" #include "lapic.h" #include "xen.h" +#include "mmu/mmu_internal.h" #include #include @@ -59,6 +60,7 @@ #include #include #include +#include #include @@ -11138,6 +11140,117 @@ int kvm_arch_post_init_vm(struct kvm *kvm) return kvm_mmu_post_init_vm(kvm); } +/* + * This covers statistics <1024 (11=log(1024)+1), which should be enough to + * cover RMAP_RECYCLE_THRESHOLD. + */ +#define RMAP_LOG_SIZE 11 + +static const char *kvm_lpage_str[KVM_NR_PAGE_SIZES] = { "4K", "2M", "1G" }; + +static int kvm_mmu_rmaps_stat_show(struct seq_file *m, void *v) +{ + struct kvm_rmap_head *rmap; + struct kvm *kvm = m->private; + struct kvm_memory_slot *slot; + struct kvm_memslots *slots; + unsigned int lpage_size, index; + /* Still small enough to be on the stack */ + unsigned int *log[KVM_NR_PAGE_SIZES], *cur; + int i, j, k, l, ret; + + memset(log, 0, sizeof(log)); + + ret = -ENOMEM; + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) { + log[i] = kzalloc(RMAP_LOG_SIZE * sizeof(unsigned int), GFP_KERNEL); + if (!log[i]) + goto out; + } + + mutex_lock(&kvm->slots_lock); + write_lock(&kvm->mmu_lock); + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { + slots = __kvm_memslots(kvm, i); + for (j = 0; j < slots->used_slots; j++) { + slot = &slots->memslots[j]; + for (k = 0; k < KVM_NR_PAGE_SIZES; k++) { + rmap = slot->arch.rmap[k]; + lpage_size = kvm_mmu_slot_lpages(slot, k + 1); + cur = log[k]; + for (l = 0; l < lpage_size; l++) { + index = ffs(pte_list_count(&rmap[l])); + if (WARN_ON_ONCE(index >= RMAP_LOG_SIZE)) + index = RMAP_LOG_SIZE - 1; + cur[index]++; + } + } + } + } + + write_unlock(&kvm->mmu_lock); + mutex_unlock(&kvm->slots_lock); + + /* index=0 counts no rmap; index=1 counts 1 rmap */ + seq_printf(m, "Rmap_Count:\t0\t1\t"); + for (i = 2; i < RMAP_LOG_SIZE; i++) { + j = 1 << (i - 1); + k = (1 << i) - 1; + seq_printf(m, "%d-%d\t", j, k); + } + seq_printf(m, "\n"); + + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) { + seq_printf(m, "Level=%s:\t", kvm_lpage_str[i]); + cur = log[i]; + for (j = 0; j < RMAP_LOG_SIZE; j++) + seq_printf(m, "%d\t", cur[j]); + seq_printf(m, "\n"); + } + + ret = 0; +out: + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) + if (log[i]) + kfree(log[i]); + + return ret; +} + +static int kvm_mmu_rmaps_stat_open(struct inode *inode, struct file *file) +{ + struct kvm *kvm = inode->i_private; + + if (!kvm_get_kvm_safe(kvm)) + return -ENOENT; + + return single_open(file, kvm_mmu_rmaps_stat_show, kvm); +} + +static int kvm_mmu_rmaps_stat_release(struct inode *inode, struct file *file) +{ + struct kvm *kvm = inode->i_private; + + kvm_put_kvm(kvm); + + return single_release(inode, file); +} + +static const struct file_operations mmu_rmaps_stat_fops = { + .open = kvm_mmu_rmaps_stat_open, + .read = seq_read, + .llseek = seq_lseek, + .release = kvm_mmu_rmaps_stat_release, +}; + +int kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ + debugfs_create_file("mmu_rmaps_stat", 0644, kvm->debugfs_dentry, kvm, + &mmu_rmaps_stat_fops); + return 0; +} + static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu) { vcpu_load(vcpu); From patchwork Fri Jun 25 15:34:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50886C2B9F4 for ; Fri, 25 Jun 2021 15:34:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2BC9061945 for ; Fri, 25 Jun 2021 15:34:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229741AbhFYPgj (ORCPT ); Fri, 25 Jun 2021 11:36:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56717 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229630AbhFYPgj (ORCPT ); Fri, 25 Jun 2021 11:36:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635258; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P+irYxL4B6g9k/ospIqhO666waG3AOzkyA4aW0tTUSw=; b=GClHkAYfMmEioJE6BY4FC9ETTQ4cguM+mR1Sow9Cffa6ALLSIKPWkewxjdkQFIabyxZ3vQ QbHJpPwUY5edjsropCDLS6mqAj2Q0cDwthbiz0tu10b+uMKUCAdg9dfKRg9pDgNkfCUKRW W07yA6XQz7Tut61C0a6xr5AcjV6yb2w= Received: from mail-il1-f200.google.com (mail-il1-f200.google.com [209.85.166.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-212-9KMmvFwsMfqMe54oUXBygQ-1; Fri, 25 Jun 2021 11:34:16 -0400 X-MC-Unique: 9KMmvFwsMfqMe54oUXBygQ-1 Received: by mail-il1-f200.google.com with SMTP id m15-20020a923f0f0000b02901ee102ac952so6412449ila.8 for ; Fri, 25 Jun 2021 08:34:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=P+irYxL4B6g9k/ospIqhO666waG3AOzkyA4aW0tTUSw=; b=lJN72W7hYlzIjiCJe034MtW9gLKgT4R3ip+gRWP9q0NE+dG0vtIDvFtzMXq8EPhuj1 mxhVn4JDpvQqSSdn970VLCZ7pctSXXEyvnPA4kwrZ7sJu4qEByGc9E9IqITK/JVfmsY9 QpX309RbhJa8trAfK7Nyy+qzWZEP91og5b6535Xm/Pw/PK5VLxS4kQWW0tyV07h2X9fv aaZf090LqVqviVl8Shv6P7Ab9poEnJJEkGd3P8J8KGuLrGTrdX02hsX7Y3oA+RWXFwNF BTL9Slu9SD04EQlrpwlDVY9CabSa76y/Qd4qvT+9gnO06mbTAS3IHFncGuTfQ5OPc1KU qn/A== X-Gm-Message-State: AOAM530wVwvggvWa8+HgXYCSp7z1O2zWQ9RIn7pq7nudzmmWxjPH2iLS kRW4Ycj5mLXSyPVMfrM9RsjW6LQ1Umr185s3NP1fzdGExHSqdDUlbmN8e0/wySAqwu/TYy+NpG2 T9iB0nlm7QkjI X-Received: by 2002:a6b:6205:: with SMTP id f5mr9020427iog.60.1624635255422; Fri, 25 Jun 2021 08:34:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz6tjZFUGP/NqbgvCCXLVIQHOsoTMddP4SdvnmyH8q0xAkmsSrB5eK5GxDG+LLmaPOasLYXpw== X-Received: by 2002:a6b:6205:: with SMTP id f5mr9020417iog.60.1624635255257; Fri, 25 Jun 2021 08:34:15 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id l12sm3679281iln.20.2021.06.25.08.34.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:34:14 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Maxim Levitsky , Vitaly Kuznetsov , Sean Christopherson , Paolo Bonzini Subject: [PATCH v2 7/9] KVM: X86: MMU: Tune PTE_LIST_EXT to be bigger Date: Fri, 25 Jun 2021 11:34:13 -0400 Message-Id: <20210625153413.43570-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Currently rmap array element only contains 3 entries. However for EPT=N there could have a lot of guest pages that got tens of even hundreds of rmap entry. A normal distribution of a 6G guest (even if idle) shows this with rmap count statistics: Rmap_Count: 0 1 2-3 4-7 8-15 16-31 32-63 64-127 128-255 256-511 512-1023 Level=4K: 3089171 49005 14016 1363 235 212 15 7 0 0 0 Level=2M: 5951 227 0 0 0 0 0 0 0 0 0 Level=1G: 32 0 0 0 0 0 0 0 0 0 0 If we do some more fork some pages will grow even larger rmap counts. This patch makes PTE_LIST_EXT bigger so it'll be more efficient for the general use case of EPT=N as we do list reference less and the loops over PTE_LIST_EXT will be slightly more efficient; but still not too large so less waste when array not full. It should not affecting EPT=Y since EPT normally only has zero or one rmap entry for each page, so no array is even allocated. With a test case to fork 500 child and recycle them ("./rmap_fork 500" [1]), this patch speeds up fork time of about 22%. Before: 367.20 (+-4.58%) After: 302.00 (+-5.30%) [1] https://github.com/xzpeter/clibs/commit/825436f825453de2ea5aaee4bdb1c92281efe5b3 Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b3f738a7c05e..9b093985a2ef 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -137,8 +137,8 @@ module_param(dbg, bool, 0644); #include -/* make pte_list_desc fit well in cache line */ -#define PTE_LIST_EXT 3 +/* make pte_list_desc fit well in cache lines */ +#define PTE_LIST_EXT 15 struct pte_list_desc { u64 *sptes[PTE_LIST_EXT]; From patchwork Fri Jun 25 15:34:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15206C48BC2 for ; Fri, 25 Jun 2021 15:34:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EBC8E6145C for ; Fri, 25 Jun 2021 15:34:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229894AbhFYPgn (ORCPT ); Fri, 25 Jun 2021 11:36:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25942 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229873AbhFYPgl (ORCPT ); Fri, 25 Jun 2021 11:36:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635260; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GAefR5jsv2aB5x5SE+QJ86SlCiYChLGRfggmJlDrXNo=; b=FE0AIKLwVwVZoti+GERVLQlivLLsbGysy6w4O1fTXuRu+ZoTmwZFQU70xDrFDQErU4LhKh s5v2foMUIq5MXX3vKyEw3HMlfekKfveZMWRbnIjBlej9SJ16bN+Rg1aQxgdbvwY39BQNnH ExsIHtzGVXzMFgQJ/UL4qVCCa+m0CqY= Received: from mail-il1-f199.google.com (mail-il1-f199.google.com [209.85.166.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-570-d9qejar_O3aFtgrDrULiiA-1; Fri, 25 Jun 2021 11:34:19 -0400 X-MC-Unique: d9qejar_O3aFtgrDrULiiA-1 Received: by mail-il1-f199.google.com with SMTP id g14-20020a926b0e0000b02901bb2deb9d71so6380432ilc.6 for ; Fri, 25 Jun 2021 08:34:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GAefR5jsv2aB5x5SE+QJ86SlCiYChLGRfggmJlDrXNo=; b=hi5tUhV+2FmyT4HxWcJ0dxG8PoSFst7CY2eVubzBOw0dLc1YqXqynX4cF0m1+x5Ss4 pvMV8pYLaUiBO0UH63SquhttCUOr4eOgKgtpzx9lX8onR1xW3uzAmI/QiBaLG4bKvmau 2nNZa/8W7U6MBZ8+uRYQ/8M9PBD4DJTk3FINZOgPOtfemqIqosIXbgbCSk4XjxSnLv0Y dhqT+JyvuZ+hfJGGTBsLUu7VEl+nYg4aMmw9Iw2BKTYn5hU2YFjMAHFz2ypoTufymOn+ vJ/UNxomBOB4huaHzYHvz3G2DwdLkgpKfa5RAdjRD21ViI7FafCk6hA8xZXSDeUDwxp6 Yc4A== X-Gm-Message-State: AOAM533ibppQSyMbPiy1X+C/hrbJIHg1hPUC/OYPf1JS8W9FevtsYL/A 1d54aM5bKkH+FnYm5bPkeXVhXnUhFDMqfIrI7259p5wAxIOPIKDClQ2kdo8URHSjXZdD+eHLUkd Lx2Lu3snx4Tt5 X-Received: by 2002:a02:84a3:: with SMTP id f32mr10034952jai.63.1624635258713; Fri, 25 Jun 2021 08:34:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxq6qUS4K3DVs2P9SSjKenQ7FRjhcE3FWO4HRIgVDFbE6V1/5UvoM18/9G/9bY7GwfogG12eA== X-Received: by 2002:a02:84a3:: with SMTP id f32mr10034937jai.63.1624635258511; Fri, 25 Jun 2021 08:34:18 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id x11sm3863687ilg.59.2021.06.25.08.34.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:34:17 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Maxim Levitsky , Vitaly Kuznetsov , Sean Christopherson , Paolo Bonzini Subject: [PATCH v2 8/9] KVM: X86: Optimize pte_list_desc with per-array counter Date: Fri, 25 Jun 2021 11:34:15 -0400 Message-Id: <20210625153415.43620-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a counter field into pte_list_desc, so as to simplify the add/remove/loop logic. E.g., we don't need to loop over the array any more for most reasons. This will make more sense after we've switched the array size to be larger otherwise the counter will be a waste. Initially I wanted to store a tail pointer at the head of the array list so we don't need to traverse the list at least for pushing new ones (if without the counter we traverse both the list and the array). However that'll need slightly more change without a huge lot benefit, e.g., after we grow entry numbers per array the list traversing is not so expensive. So let's be simple but still try to get as much benefit as we can with just these extra few lines of changes (not to mention the code looks easier too without looping over arrays). I used the same a test case to fork 500 child and recycle them ("./rmap_fork 500" [1]), this patch further speeds up the total fork time of about 14%, which is a total of 38% of vanilla kernel: Vanilla: 367.20 (+-4.58%) 3->15 slots: 302.00 (+-5.30%) Add counter: 265.20 (+-9.88%) [1] https://github.com/xzpeter/clibs/commit/825436f825453de2ea5aaee4bdb1c92281efe5b3 Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 33 ++++++++++++++++++--------------- 1 file changed, 18 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 9b093985a2ef..ba0258bdebc4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -138,10 +138,15 @@ module_param(dbg, bool, 0644); #include /* make pte_list_desc fit well in cache lines */ -#define PTE_LIST_EXT 15 +#define PTE_LIST_EXT 14 struct pte_list_desc { u64 *sptes[PTE_LIST_EXT]; + /* + * Stores number of entries stored in the pte_list_desc. No need to be + * u64 but just for easier alignment. When PTE_LIST_EXT, means full. + */ + u64 spte_count; struct pte_list_desc *more; }; @@ -893,7 +898,7 @@ static int pte_list_add(struct kvm_vcpu *vcpu, u64 *spte, struct kvm_rmap_head *rmap_head) { struct pte_list_desc *desc; - int i, count = 0; + int count = 0; if (!rmap_head->val) { rmap_printk("%p %llx 0->1\n", spte, *spte); @@ -903,24 +908,24 @@ static int pte_list_add(struct kvm_vcpu *vcpu, u64 *spte, desc = mmu_alloc_pte_list_desc(vcpu); desc->sptes[0] = (u64 *)rmap_head->val; desc->sptes[1] = spte; + desc->spte_count = 2; rmap_head->val = (unsigned long)desc | 1; ++count; } else { rmap_printk("%p %llx many->many\n", spte, *spte); desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); - while (desc->sptes[PTE_LIST_EXT-1]) { + while (desc->spte_count == PTE_LIST_EXT) { count += PTE_LIST_EXT; - if (!desc->more) { desc->more = mmu_alloc_pte_list_desc(vcpu); desc = desc->more; + desc->spte_count = 0; break; } desc = desc->more; } - for (i = 0; desc->sptes[i]; ++i) - ++count; - desc->sptes[i] = spte; + count += desc->spte_count; + desc->sptes[desc->spte_count++] = spte; } return count; } @@ -930,13 +935,12 @@ pte_list_desc_remove_entry(struct kvm_rmap_head *rmap_head, struct pte_list_desc *desc, int i, struct pte_list_desc *prev_desc) { - int j; + int j = desc->spte_count - 1; - for (j = PTE_LIST_EXT - 1; !desc->sptes[j] && j > i; --j) - ; desc->sptes[i] = desc->sptes[j]; desc->sptes[j] = NULL; - if (j != 0) + desc->spte_count--; + if (desc->spte_count) return; if (!prev_desc && !desc->more) rmap_head->val = 0; @@ -969,7 +973,7 @@ static void __pte_list_remove(u64 *spte, struct kvm_rmap_head *rmap_head) desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); prev_desc = NULL; while (desc) { - for (i = 0; i < PTE_LIST_EXT && desc->sptes[i]; ++i) { + for (i = 0; i < desc->spte_count; ++i) { if (desc->sptes[i] == spte) { pte_list_desc_remove_entry(rmap_head, desc, i, prev_desc); @@ -993,7 +997,7 @@ static void pte_list_remove(struct kvm_rmap_head *rmap_head, u64 *sptep) unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) { struct pte_list_desc *desc; - unsigned int i, count = 0; + unsigned int count = 0; if (!rmap_head->val) return 0; @@ -1003,8 +1007,7 @@ unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); while (desc) { - for (i = 0; (i < PTE_LIST_EXT) && desc->sptes[i]; i++) - count++; + count += desc->spte_count; desc = desc->more; } From patchwork Fri Jun 25 15:34:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12345517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81583C2B9F4 for ; Fri, 25 Jun 2021 15:34:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A1266161E for ; Fri, 25 Jun 2021 15:34:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230064AbhFYPgt (ORCPT ); Fri, 25 Jun 2021 11:36:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28350 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229884AbhFYPgq (ORCPT ); Fri, 25 Jun 2021 11:36:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624635265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YjcyYxrINnzSou8d5xoDOd+Q0j+eHgX3Un7gw5UoKHs=; b=SLDnNfrFLu25XkZgpdqr93BaJ+kgGnqekCtt3nK0Dd1t6Y4KQxAsohZSqOWfdvS7oK0w8Q 5DBhU9+1MIveailE07PgPKxNm2Hxk+btGIAEeq0r2tYOurIBi4ahCV230KEqND2p6Zprvo h8oiuHOMliefeCN/Kl9I4UvMeJitfJ4= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-154-AQkvYzYHM7iFj7fhtMlNpQ-1; Fri, 25 Jun 2021 11:34:21 -0400 X-MC-Unique: AQkvYzYHM7iFj7fhtMlNpQ-1 Received: by mail-io1-f70.google.com with SMTP id w22-20020a5ed6160000b02904f28b1d759dso206708iom.8 for ; Fri, 25 Jun 2021 08:34:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YjcyYxrINnzSou8d5xoDOd+Q0j+eHgX3Un7gw5UoKHs=; b=P2+peqSPn9Uc3HlhF5W+cIgIr8cocVgDnCA5y3DyKKmbieLnUHuvFpV0IG1lwIKFum neJ8KYYr4qcjN/GpXJPlwBLNXQ9xlrC1paWm5TnvOKL5xrbbeQ+DDFZj/kpxkjcMdq/F 31oz3VZAhfanqp1f/m6kyF9tnGuhBNA2AR6UPOwJgtoIyaZeRVy+1MSRR62dZvwd/tqX H4v/QdE7DNXkkvEwdAoQ2Ji4/1591DeugXYxEVoXTnCB+ufb+5Rv0BZ11ZQFc/j3yxSD Y8IVS5z+xRVU9r0lZfRjNNDsLRJ5IQfd9zs9QYKirgMmByiedrrkCo/xDPVEvJA7tLdG 9ceg== X-Gm-Message-State: AOAM530pHDo1kp/1l0yI75x6o/GriVOUy9/E1uzZV8K36OdVOg8uc/e4 ZYtOsJt8e1GB4hDGAvWyperWsjHq0h9avCPDnIObjHk+MwdjCksBFz05Z71cUu2ZkvE/m90aOKj iv6b9yekgogEG X-Received: by 2002:a6b:2b44:: with SMTP id r65mr8803099ior.99.1624635261303; Fri, 25 Jun 2021 08:34:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzDrcMjN0grhcN1xdY2Kguub8DNog6SwRyubFGgLe1lkSEeK2CSuD5j9rsyWmXrReWuXadcJg== X-Received: by 2002:a6b:2b44:: with SMTP id r65mr8803084ior.99.1624635261102; Fri, 25 Jun 2021 08:34:21 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id 67sm3126275iob.15.2021.06.25.08.34.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Jun 2021 08:34:20 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterx@redhat.com, Maxim Levitsky , Vitaly Kuznetsov , Sean Christopherson , Paolo Bonzini Subject: [PATCH v2 9/9] KVM: X86: Optimize zapping rmap Date: Fri, 25 Jun 2021 11:34:19 -0400 Message-Id: <20210625153419.43671-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210625153214.43106-1-peterx@redhat.com> References: <20210625153214.43106-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Using rmap_get_first() and rmap_remove() for zapping a huge rmap list could be slow. The easy way is to travers the rmap list, collecting the a/d bits and free the slots along the way. Provide a pte_list_destroy() and do exactly that. Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 45 +++++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ba0258bdebc4..45aac78dcabc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1014,6 +1014,38 @@ unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) return count; } +/* Return true if rmap existed and callback called, false otherwise */ +static bool pte_list_destroy(struct kvm_rmap_head *rmap_head, + int (*callback)(u64 *sptep)) +{ + struct pte_list_desc *desc, *next; + int i; + + if (!rmap_head->val) + return false; + + if (!(rmap_head->val & 1)) { + if (callback) + callback((u64 *)rmap_head->val); + goto out; + } + + desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); + + while (desc) { + if (callback) + for (i = 0; i < desc->spte_count; i++) + callback(desc->sptes[i]); + next = desc->more; + mmu_free_pte_list_desc(desc); + desc = next; + } +out: + /* rmap_head is meaningless now, remember to reset it */ + rmap_head->val = 0; + return true; +} + static struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, struct kvm_memory_slot *slot) { @@ -1403,18 +1435,7 @@ static bool rmap_write_protect(struct kvm_vcpu *vcpu, u64 gfn) static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head, struct kvm_memory_slot *slot) { - u64 *sptep; - struct rmap_iterator iter; - bool flush = false; - - while ((sptep = rmap_get_first(rmap_head, &iter))) { - rmap_printk("spte %p %llx.\n", sptep, *sptep); - - pte_list_remove(rmap_head, sptep); - flush = true; - } - - return flush; + return pte_list_destroy(rmap_head, mmu_spte_clear_track_bits); } static bool kvm_unmap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,