From patchwork Thu Jun 24 18:13:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92D45C49EA5 for ; Thu, 24 Jun 2021 18:14:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7870D611CB for ; Thu, 24 Jun 2021 18:14:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232575AbhFXSQX (ORCPT ); Thu, 24 Jun 2021 14:16:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:37721 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232533AbhFXSQV (ORCPT ); Thu, 24 Jun 2021 14:16:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558442; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qCTE+G8btEQabyXMZ8pmtQF1JICGOfEU2RElhMZ1+8k=; b=awq9vvNm6mB561KjdmbgR0hB3ZIFFYbIvwXxMZeRPtq5WjYwPF3+L5AbOZ6vzqBtJjHn1b ac/9psa7oSMG2JRodv+6tO7PELIShOi5DxDkTtiPuohDoClOS9WTHF75zl+aFRjxCbzjKB mI3Ok9F/RcE+kLcYwIn/V0hxEPeBLXY= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-16-djXlzPYGPkSRDTOvKZnZCA-1; Thu, 24 Jun 2021 14:14:01 -0400 X-MC-Unique: djXlzPYGPkSRDTOvKZnZCA-1 Received: by mail-qv1-f71.google.com with SMTP id q10-20020a056214018ab029027751ec9742so7910488qvr.7 for ; Thu, 24 Jun 2021 11:14:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qCTE+G8btEQabyXMZ8pmtQF1JICGOfEU2RElhMZ1+8k=; b=M3OEmhjRkqjl81BMn8QYkHcIPgN/FDuuh/0X2IzcbJ2fX2ym7NJcL1z6/IzkzD5pW0 2e8YuuRYE2cpLq2qheck95s2SyRasnTByrNFT1lA86y5SUzYWTiiEhcvgkWoC2dbJVXl LxMD2CANzrcN4axAiPrVyvklfp8GoFQtYNFRZy+oskjFHDrj2h5uvOzPkGBWRRkJRVbF 6e9ms2gFRDSl91RAv+w21lujAmnv5s0pufVmXuWynW4eOtlJipV7BtCdB5XuWqww4xx2 TE5sUXADQyhUbicnJcwjdKtSBZmzhJn+rUgK1xFAGEHCxImv6lsWp1S6mhRj4yNCzX4i 2wZA== X-Gm-Message-State: AOAM532FpCxRVWcz81GJwdRYiPwesuaOvWDmLhc7yOvHQvZB3lcSLts4 1c42HII+YIYjirVOJRBiqE3osHqv6skthR5omIGBoGfWCQwN2oo9VfJxkGHy08A0zoVE+4tCkOJ ua3PZWUTKUaJW5SndRuZgoScrxy22yhT/Xonp7m0Um9SMVP9JPjKcp5cCJT/Nug== X-Received: by 2002:a05:620a:484:: with SMTP id 4mr6994638qkr.437.1624558440054; Thu, 24 Jun 2021 11:14:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx0LKAndulLpSV+fntL9ldHDDtswfF5CXYy8ioMeUBK1qfG/kGRqlgzLDVV9hZuBaru2K3QPw== X-Received: by 2002:a05:620a:484:: with SMTP id 4mr6994614qkr.437.1624558439818; Thu, 24 Jun 2021 11:13:59 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:13:59 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 1/9] KVM: X86: Add per-vm stat for max rmap list size Date: Thu, 24 Jun 2021 14:13:48 -0400 Message-Id: <20210624181356.10235-2-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a new statistic max_mmu_rmap_size, which stores the maximum size of rmap for the vm. Signed-off-by: Peter Xu --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu/mmu.c | 2 ++ arch/x86/kvm/x86.c | 1 + 3 files changed, 4 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55efbacfc244..937bc8021f7a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1138,6 +1138,7 @@ struct kvm_vm_stat { ulong lpages; ulong nx_lpage_splits; ulong max_mmu_page_hash_collisions; + ulong max_mmu_rmap_size; }; struct kvm_vcpu_stat { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5e60b00e8e50..6dd338738118 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2602,6 +2602,8 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, u64 *sptep, if (is_shadow_present_pte(*sptep)) { if (!was_rmapped) { rmap_count = rmap_add(vcpu, sptep, gfn); + if (rmap_count > vcpu->kvm->stat.max_mmu_rmap_size) + vcpu->kvm->stat.max_mmu_rmap_size = rmap_count; if (rmap_count > RMAP_RECYCLE_THRESHOLD) rmap_recycle(vcpu, sptep, gfn); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bbc4e04e67ad..4aa3cc6ae5d4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -257,6 +257,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { VM_STAT("largepages", lpages, .mode = 0444), VM_STAT("nx_largepages_splitted", nx_lpage_splits, .mode = 0444), VM_STAT("max_mmu_page_hash_collisions", max_mmu_page_hash_collisions), + VM_STAT("max_mmu_rmap_size", max_mmu_rmap_size), { NULL } }; From patchwork Thu Jun 24 18:13:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88651C49EA6 for ; Thu, 24 Jun 2021 18:14:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69463611CB for ; Thu, 24 Jun 2021 18:14:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232599AbhFXSQZ (ORCPT ); Thu, 24 Jun 2021 14:16:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:29524 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230450AbhFXSQX (ORCPT ); Thu, 24 Jun 2021 14:16:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a/z5ekGyyTIFDc5iC+PDsxm8TweH5l/mhGx4xsPY5QA=; b=RLASARGZMdeG+wRfOsZYQGrm+MeqH6JxGPAABoBRcy7oKUVpnns0fbcbmzgyjCYhHJjba7 3rX96/NqeWHnSh08vuxAJ32+tvuj2632O/B1YbDLb2Qc8l2Hd4MlOb87zuHuVo1BurElsm rSnsG1omzzleIVHn6Pj/pw89+8WnNxE= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-426-P20lZ0m3Pw6uX0G6e2R01Q-1; Thu, 24 Jun 2021 14:14:02 -0400 X-MC-Unique: P20lZ0m3Pw6uX0G6e2R01Q-1 Received: by mail-qt1-f197.google.com with SMTP id s20-20020ac85cd40000b029024ebedff900so7132380qta.10 for ; Thu, 24 Jun 2021 11:14:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a/z5ekGyyTIFDc5iC+PDsxm8TweH5l/mhGx4xsPY5QA=; b=gT77ds2nDmzCzee2cbWiEwl8acSca0lv5ycjCTsU0ig+pTIsJu2yW5BtOWsv8UGKIS EUQxedivLog/xndd0R1aEkfUcYDX9QSxmdz0STb2CYuW6aGZGhAli6sN2C/LMtXTMV+x LQKImsg5b4vwTJg16vtZO+fl7erHSkrrI7QdJYa0/A/xB7ol364qgrMHg+yT6tgOHA51 TkxexSahf4fmH291Y2ivRE0qqzEefc5IghOL0FNhdzq2GW9qM1ssffrYKwlGmxkxrpDL JuSo7A+vmCN6JLo1HP324grffyxyEh30ffrQwrdksWM9o04GcSv0tfcGPsA6FGD28hQs tDdQ== X-Gm-Message-State: AOAM533Sq5zQHrPBmZEDqXSWyR/m+ixgusvrhMOvLASQslIhO8D1XJ5J jMCkyHrOqnLfpJPfBT/ilrowaJVKEFsg+f1fengyNsWAeZ2iFQs7i55R+AS/LYiL4IXX/HJCsCd n/8m47rVHnuqSSxAnmqL9/1qQMIuvl2ihsPXqu6E82yUEagoKnM9T2oSxzOpdVw== X-Received: by 2002:a37:9d90:: with SMTP id g138mr3201335qke.212.1624558441963; Thu, 24 Jun 2021 11:14:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw81+DaAwL/DpqMKlW8EsBoQO4jQNMsJoAjiZVBV2+Lx9VL6USjuQp2YkIyZMNrp4T6w5dGMg== X-Received: by 2002:a37:9d90:: with SMTP id g138mr3201313qke.212.1624558441718; Thu, 24 Jun 2021 11:14:01 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:01 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 2/9] KVM: Introduce kvm_get_kvm_safe() Date: Thu, 24 Jun 2021 14:13:49 -0400 Message-Id: <20210624181356.10235-3-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce this safe version of kvm_get_kvm() so that it can be called even during vm destruction. Use it in kvm_debugfs_open() and remove the verbose comment. Prepare to be used elsewhere. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 17 +++++++++++------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2f34487e21f2..53d7d09eebd7 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -698,6 +698,7 @@ int kvm_init(void *opaque, unsigned vcpu_size, unsigned vcpu_align, void kvm_exit(void); void kvm_get_kvm(struct kvm *kvm); +bool kvm_get_kvm_safe(struct kvm *kvm); void kvm_put_kvm(struct kvm *kvm); bool file_is_kvm(struct file *file); void kvm_put_kvm_no_destroy(struct kvm *kvm); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d170f65e15b0..0b4f55370b18 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1049,6 +1049,16 @@ void kvm_get_kvm(struct kvm *kvm) } EXPORT_SYMBOL_GPL(kvm_get_kvm); +/* + * Make sure the vm is not during destruction, which is a safe version of + * kvm_get_kvm(). Return true if kvm referenced successfully, false otherwise. + */ +bool kvm_get_kvm_safe(struct kvm *kvm) +{ + return refcount_inc_not_zero(&kvm->users_count); +} +EXPORT_SYMBOL_GPL(kvm_get_kvm_safe); + void kvm_put_kvm(struct kvm *kvm) { if (refcount_dec_and_test(&kvm->users_count)) @@ -4713,12 +4723,7 @@ static int kvm_debugfs_open(struct inode *inode, struct file *file, struct kvm_stat_data *stat_data = (struct kvm_stat_data *) inode->i_private; - /* The debugfs files are a reference to the kvm struct which - * is still valid when kvm_destroy_vm is called. - * To avoid the race between open and the removal of the debugfs - * directory we test against the users count. - */ - if (!refcount_inc_not_zero(&stat_data->kvm->users_count)) + if (!kvm_get_kvm_safe(stat_data->kvm)) return -ENOENT; if (simple_attr_open(inode, file, get, From patchwork Thu Jun 24 18:13:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342789 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7C12C49EAB for ; Thu, 24 Jun 2021 18:14:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D301F613B1 for ; Thu, 24 Jun 2021 18:14:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232460AbhFXSQ0 (ORCPT ); Thu, 24 Jun 2021 14:16:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55733 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232596AbhFXSQZ (ORCPT ); Thu, 24 Jun 2021 14:16:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558445; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Nog/14DIgnUZ4IUiuS7R5o+tAccEjG5oUAeY24gLNU=; b=cfeWZYQeGbZOc9HD6XL336UVT7xuTVQgmZHe6nHVGGtl0vt2qC4MZAAZCIN28mI6cH9OsN zJQRUwKmurz3jVWfDOevmsR2n4jWw4Dz8mj8csCmwMXGAu0vWKTYOOIfY9xXwaBLoN8n2G fsN8p+4WC0W/KWto5q72vRXILfIYWBk= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-140-8u1DJxtSO7qk-8mLUrANfg-1; Thu, 24 Jun 2021 14:14:04 -0400 X-MC-Unique: 8u1DJxtSO7qk-8mLUrANfg-1 Received: by mail-qk1-f200.google.com with SMTP id c3-20020a37b3030000b02903ad0001a2e8so7924228qkf.3 for ; Thu, 24 Jun 2021 11:14:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2Nog/14DIgnUZ4IUiuS7R5o+tAccEjG5oUAeY24gLNU=; b=ZdhcOijSStVrQGNH7pGQX5H7yUxc0jr90SXnJMZ2IygVdQX4WXPM0c8YgqVD3NiWC0 avL34KKYgsQ8cGjW1ax2txAvDoOCohMLQ+GO0V9vzIOHX9lU0u8ovlFpDynY+PaQMY5m mFUgwHOjECsQG6Oe7dK3tsmiCelTfLgiDjJonsSgesi935sX6Kl1Z1fVsD7GngjjG4Zv PQjj0Faw9b6D11N4qwvEp7GpAx+2MRYwjD1ZDjCBMPIFI6krjH8SN0qP7ag2rGkjL3RK gjWuCaLMSIf8CCPJ64ZAeKm4/YU3uFqmlmQsRuvNMn1NifvRoPqdXHsFKZzELICMamA9 x3Fg== X-Gm-Message-State: AOAM5323tJX0i/HszSZdkE0pC2c3fIxm8/FGbhzx0QG4c0nd6vaJ+5Ug 5FPlQ2ZQRosc/Ad3PzGk8cge7y7TppAMzOo42h1ywDQ+ZB5yRl0jAyt72n4tAE7nj6iWgYnP81z B7X3X8vrriw5Hw+qSnqXl1LTAd16za4fMn78BRSYT4N6sJMYIH56KCRuOFmyUpQ== X-Received: by 2002:ac8:5784:: with SMTP id v4mr1635165qta.29.1624558443714; Thu, 24 Jun 2021 11:14:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyYmWyDHbFW8nYB7xKKg3q0sHZXRjTyTswPy+lanRmvJq7eYT+PD7e4EYtu6p98y3KOK2o0GA== X-Received: by 2002:ac8:5784:: with SMTP id v4mr1635136qta.29.1624558443350; Thu, 24 Jun 2021 11:14:03 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.14.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:02 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 3/9] KVM: Allow to have arch-specific per-vm debugfs files Date: Thu, 24 Jun 2021 14:13:50 -0400 Message-Id: <20210624181356.10235-4-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Allow archs to create arch-specific nodes under kvm->debugfs_dentry directory besides the stats fields. The new interface kvm_arch_create_vm_debugfs() is defined but not yet used. It's called after kvm->debugfs_dentry is created, so it can be referenced directly in kvm_arch_create_vm_debugfs(). Arch should define their own versions when they want to create extra debugfs nodes. Signed-off-by: Peter Xu --- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 53d7d09eebd7..480baa55d93f 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1009,6 +1009,7 @@ bool kvm_arch_dy_runnable(struct kvm_vcpu *vcpu); bool kvm_arch_dy_has_pending_interrupt(struct kvm_vcpu *vcpu); int kvm_arch_post_init_vm(struct kvm *kvm); void kvm_arch_pre_destroy_vm(struct kvm *kvm); +int kvm_arch_create_vm_debugfs(struct kvm *kvm); #ifndef __KVM_HAVE_ARCH_VM_ALLOC /* diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0b4f55370b18..6648743f4dcf 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -847,6 +847,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, int fd) char dir_name[ITOA_MAX_LEN * 2]; struct kvm_stat_data *stat_data; struct kvm_stats_debugfs_item *p; + int ret; if (!debugfs_initialized()) return 0; @@ -872,6 +873,13 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, int fd) kvm->debugfs_dentry, stat_data, &stat_fops_per_vm); } + + ret = kvm_arch_create_vm_debugfs(kvm); + if (ret) { + kvm_destroy_vm_debugfs(kvm); + return ret; + } + return 0; } @@ -892,6 +900,17 @@ void __weak kvm_arch_pre_destroy_vm(struct kvm *kvm) { } +/* + * Called after per-vm debugfs created. When called kvm->debugfs_dentry should + * be setup already, so we can create arch-specific debugfs entries under it. + * Cleanup should be automatic done in kvm_destroy_vm_debugfs() recursively, so + * a per-arch destroy interface is not needed. + */ +int __weak kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ + return 0; +} + static struct kvm *kvm_create_vm(unsigned long type) { struct kvm *kvm = kvm_arch_alloc_vm(); From patchwork Thu Jun 24 18:13:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F103C49EA5 for ; Thu, 24 Jun 2021 18:14:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1369A613B1 for ; Thu, 24 Jun 2021 18:14:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232630AbhFXSQ2 (ORCPT ); Thu, 24 Jun 2021 14:16:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42508 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232602AbhFXSQ0 (ORCPT ); Thu, 24 Jun 2021 14:16:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ny4N/dv5g7DO3oaKKsT3k5Lo/RLroZYI+1qWA8iS2Y=; b=i4hQ4JNrDXb07Ey5sILHPCkpkCzZR3IFYcMcDJ/Fcfss7e3wcjhInZarDw4mfZuvbH5sc1 NF2SWijnKUlmjf9KqMOGye7cjwDNnseM+c7dJhmiMRCOgiwkECXwc0k8nDDlADntilZs8U Lh9mSHThnwztZwRdy6adu5ES0rke++A= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-165-IOOgXmztP4OrDJWaZNQB6A-1; Thu, 24 Jun 2021 14:14:05 -0400 X-MC-Unique: IOOgXmztP4OrDJWaZNQB6A-1 Received: by mail-qk1-f197.google.com with SMTP id y206-20020a3764d70000b02903b2ff4c81b1so6543402qkb.21 for ; Thu, 24 Jun 2021 11:14:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2ny4N/dv5g7DO3oaKKsT3k5Lo/RLroZYI+1qWA8iS2Y=; b=i3ZudLv4I4pbWn4wj5AuIVm0Q4wiGXXxuRX8udwRkxRO71pYasQAYDxJieDHSNBdh9 WVVxdCqS1HwBdduc4vo/cOq+yQYk8pd1E93OfmgPnEKeSJRTfm4B/zrxRtMkl70t3kWk qF5xKCIl6/I9r0IzseoamYM40bMdDyaTb6SRljavXNpyNdfytihC2vMGZRIS0/7ffUzz PNAjDrKeDCFpHF1qcGk2ZFv5miRx6YBOAdV9mNjNOOBMBw9LL7uNKaummeGOZbJ/pJJl 7FJC9bcOaAUFSE1CfEkGcZm/WhBy1XfR1eEi6oOMcmC4ias8goIvR26fzwXBDpgZ5o6y L+Mg== X-Gm-Message-State: AOAM533nkfzHauu5U+JJDsJvtaBwcW1A2fGUaCTO8QILwuOCibAb93IA BQNA+ls3hKVn2agWhndDyrGv1ggtZdBmbGNj0xGMQH1mgpGq8RSBBRoHlfJmYphzx4FFthe7hv6 QjqhaQYZsGnXzmYeB/jzk50hOMtxWEvfdL/jfkj2u3xA507XVILRmzjwADn+gWg== X-Received: by 2002:a37:f510:: with SMTP id l16mr7017498qkk.205.1624558444897; Thu, 24 Jun 2021 11:14:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw5xeg8fkVzCvznQIXVQuhoVQJFpwIzM0cNNpma6+xzwHDzRQWv3casplWNqC4T8UNyOmhZfQ== X-Received: by 2002:a37:f510:: with SMTP id l16mr7017476qkk.205.1624558444679; Thu, 24 Jun 2021 11:14:04 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:04 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 4/9] KVM: X86: Introduce pte_list_count() helper Date: Thu, 24 Jun 2021 14:13:51 -0400 Message-Id: <20210624181356.10235-5-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This helper is used to count the number of rmap entries in the rmap list pointed by the kvm_rmap_head. Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 21 +++++++++++++++++++++ arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 22 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 6dd338738118..80263ecb1de3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -927,6 +927,27 @@ static void pte_list_remove(struct kvm_rmap_head *rmap_head, u64 *sptep) __pte_list_remove(sptep, rmap_head); } +unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) +{ + struct pte_list_desc *desc; + unsigned int i, count = 0; + + if (!rmap_head->val) + return 0; + else if (!(rmap_head->val & 1)) + return 1; + + desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); + + while (desc) { + for (i = 0; (i < PTE_LIST_EXT) && desc->sptes[i]; i++) + count++; + desc = desc->more; + } + + return count; +} + static struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, struct kvm_memory_slot *slot) { diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index d64ccb417c60..3cd1a878ffeb 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -126,6 +126,7 @@ bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, struct kvm_memory_slot *slot, u64 gfn); void kvm_flush_remote_tlbs_with_address(struct kvm *kvm, u64 start_gfn, u64 pages); +unsigned int pte_list_count(struct kvm_rmap_head *rmap_head); /* * Return values of handle_mmio_page_fault, mmu.page_fault, and fast_page_fault(). From patchwork Thu Jun 24 18:13:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E175FC49EA7 for ; Thu, 24 Jun 2021 18:14:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C78C66120D for ; Thu, 24 Jun 2021 18:14:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232655AbhFXSQb (ORCPT ); Thu, 24 Jun 2021 14:16:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54843 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232618AbhFXSQ2 (ORCPT ); Thu, 24 Jun 2021 14:16:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nhjhdzRJGuZzb8Uq4F9J7OQePoxIE2i+1r2VEiCpXU8=; b=DYT1DXMznO/hHwYtbY5UN5sa+Qoc/SstcTqB5NiPHqCWDxnGq2Zi1ouV/xJZqBvDwEBiol Wth08y5mtwpSESirbhfOqmcyemrjzUCsdlXT/4sR5JshdxsE/sDQ6BGEnj4hxw57W+VGjJ JxBPJocbNWuCl4+4zY5zoS07PYFFv9c= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-188-vAZyZzScOPSBKuCua49sQg-1; Thu, 24 Jun 2021 14:14:07 -0400 X-MC-Unique: vAZyZzScOPSBKuCua49sQg-1 Received: by mail-qv1-f72.google.com with SMTP id es19-20020a0562141933b029023930e98a57so7875995qvb.18 for ; Thu, 24 Jun 2021 11:14:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nhjhdzRJGuZzb8Uq4F9J7OQePoxIE2i+1r2VEiCpXU8=; b=UllBbehs7pxB8xQ+XiSl0SBAHS7FMaVuOpzvrp5NK8l9OJ9C6KM28CHEBwZ26v2ulR XwbjrueZPe0mAhgv/8ekCujZzjc1NrN2cbgnP02LbiaAA59lZpIQSs07pJcAyPDmLWsA FOafRH/z7dIFkKE8JYgAsAkIKfEQdCzGmBRkCjf8DqMSj1OQGiXL72tWWMmIX8uj93Oo 36gXkV/w7CLtx4vqHYsWm85vUvJfNfalcXU7nPH8gS1pLzV+4QUHguRNQO6uasBS+dBW v8wrBgW4J7M6FNRSSxTf3jWX02G5ydXdqgnW7DbW6FRHlNmdFfP+yeL+Lau57f8rtYbi 2F4A== X-Gm-Message-State: AOAM531XdgOGYnPzwK2uJvJgaooFWQ0mWkMOdzeWoPJDsO9qLyQDtXiz DH0EBnIXQjxCYc7nxCU380Hj1fo4DoVqxuL4KFVjWQbGECcb1FiCC7/SgOeGZLgOaGVsX6gIIBh 3OqJbPU/cfYZfpiAuKJy3SaJsAQgDoGqREjkv35F4meX77UPI6Z0X+P8bJg3jFw== X-Received: by 2002:a37:b947:: with SMTP id j68mr7058574qkf.486.1624558446728; Thu, 24 Jun 2021 11:14:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzyi77u9hQpo3cu5comp042O3ws2nw3qgLSs4sLThHjFTBtBf8S3k7Or4hGpuETo3oZS4Dx5w== X-Received: by 2002:a37:b947:: with SMTP id j68mr7058543qkf.486.1624558446472; Thu, 24 Jun 2021 11:14:06 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.14.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:05 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 5/9] KVM: X86: Introduce kvm_mmu_slot_lpages() helpers Date: Thu, 24 Jun 2021 14:13:52 -0400 Message-Id: <20210624181356.10235-6-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Introduce kvm_mmu_slot_lpages() to calculcate lpage_info and rmap array size. The other __kvm_mmu_slot_lpages() can take an extra parameter of npages rather than fetching from the memslot pointer. Start to use the latter one in kvm_alloc_memslot_metadata(). Signed-off-by: Peter Xu --- arch/x86/kvm/x86.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4aa3cc6ae5d4..d2acbea2f3b5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -269,6 +269,20 @@ static struct kmem_cache *x86_fpu_cache; static struct kmem_cache *x86_emulator_cache; +static inline unsigned long +__kvm_mmu_slot_lpages(struct kvm_memory_slot *slot, unsigned long npages, + int level) +{ + return gfn_to_index(slot->base_gfn + npages - 1, + slot->base_gfn, level) + 1; +} + +static inline unsigned long +kvm_mmu_slot_lpages(struct kvm_memory_slot *slot, int level) +{ + return __kvm_mmu_slot_lpages(slot, slot->npages, level); +} + /* * When called, it means the previous get/set msr reached an invalid msr. * Return true if we want to ignore/silent this failed msr access. @@ -10933,8 +10947,7 @@ static int kvm_alloc_memslot_metadata(struct kvm_memory_slot *slot, int lpages; int level = i + 1; - lpages = gfn_to_index(slot->base_gfn + npages - 1, - slot->base_gfn, level) + 1; + lpages = __kvm_mmu_slot_lpages(slot, npages, level); slot->arch.rmap[i] = kvcalloc(lpages, sizeof(*slot->arch.rmap[i]), From patchwork Thu Jun 24 18:13:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F87BC49EA5 for ; Thu, 24 Jun 2021 18:14:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8DB6261408 for ; Thu, 24 Jun 2021 18:14:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232715AbhFXSQh (ORCPT ); Thu, 24 Jun 2021 14:16:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46574 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232604AbhFXSQa (ORCPT ); Thu, 24 Jun 2021 14:16:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558450; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/yZ5ari2GxI81SYwIm60/E7Xbba1sdIVbXhz1cSIMcs=; b=bDWTHSPNVeOBWoxdPOqsgg8D3C0GgOkb6lIaTVkR07AQ6XZau2FlGzSRHruFLutTz6w+0c u28n555o52VaO+cno/cjm38j0ud2HYHks5WyrEo0YuJLUiA6Wu3HQE2OZtL3a/YMCwUUHS GyUvIP/+wfTzu1lQ0h6edoIkwSOR1yA= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-525-TgAUrnLQMJGhB9rv7ckxQg-1; Thu, 24 Jun 2021 14:14:09 -0400 X-MC-Unique: TgAUrnLQMJGhB9rv7ckxQg-1 Received: by mail-qt1-f199.google.com with SMTP id t6-20020ac80dc60000b029024e988e8277so7101092qti.23 for ; Thu, 24 Jun 2021 11:14:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/yZ5ari2GxI81SYwIm60/E7Xbba1sdIVbXhz1cSIMcs=; b=GHzuIbhSP4X6pqrupcr79nWTQ2OjeekcYrXT8+xJjLht5bUl1J2NPu/268a+bYiZ4b VXBoY9jTwodRwEchLYGV9elnE0X1IGSEbwQxpIaiC/fPU+RWiKBCqkN6pBQT+wUHfC4c vM/MpWKEe/XNSobNVhYsgaZ5cDXY9kZ7SH2q+YC5INmIeLgpTx5IbO9NGZBoAXeH+ptB 5zwefbcem7+sQFHbltMBknorNihUHFTurfZYo8NptLI4hdKpn6Tz+I0RPgfXc+kVL3yS Rlo2Y9urNQzblRXdPeBsor3Cwnq5LDFVmzqVXbtexSr8DVMQ2EMxn6PAwn6khLWgLPk+ qNmQ== X-Gm-Message-State: AOAM532KXNyXZhHRYJpJU+9BDKuqS4DlqOcN3twMsecYZmdYdBabjj8z 8UEdb1wo7CouGQUqdRzdu+Fu15LCjOKQEAf8u1hcmKIpx+m0FCcddv1ilQDQoZeJzY3vOGHb/+x 1JnlIIAvpNdgNrgcUz2pBcSe8TwNMylk0E3Wn8aq7QhAu7tKJDTzME/tNQqx8/A== X-Received: by 2002:a37:9a51:: with SMTP id c78mr6998659qke.51.1624558448163; Thu, 24 Jun 2021 11:14:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzqgpBf2cWmlIRdlKjq1uHOYzaPGIHk6EiH4zSJaOEnFdCdUw6l+gCzal6Txx9g68afbsz6bA== X-Received: by 2002:a37:9a51:: with SMTP id c78mr6998629qke.51.1624558447861; Thu, 24 Jun 2021 11:14:07 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:07 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 6/9] KVM: X86: Introduce mmu_rmaps_stat per-vm debugfs file Date: Thu, 24 Jun 2021 14:13:53 -0400 Message-Id: <20210624181356.10235-7-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use this file to dump rmap statistic information. Signed-off-by: Peter Xu --- arch/x86/kvm/x86.c | 113 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d2acbea2f3b5..6dfae8375c44 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -30,6 +30,7 @@ #include "hyperv.h" #include "lapic.h" #include "xen.h" +#include "mmu/mmu_internal.h" #include #include @@ -58,6 +59,7 @@ #include #include #include +#include #include @@ -10763,6 +10765,117 @@ int kvm_arch_post_init_vm(struct kvm *kvm) return kvm_mmu_post_init_vm(kvm); } +/* + * This covers statistics <1024 (11=log(1024)+1), which should be enough to + * cover RMAP_RECYCLE_THRESHOLD. + */ +#define RMAP_LOG_SIZE 11 + +static const char *kvm_lpage_str[KVM_NR_PAGE_SIZES] = { "4K", "2M", "1G" }; + +static int kvm_mmu_rmaps_stat_show(struct seq_file *m, void *v) +{ + struct kvm_rmap_head *rmap; + struct kvm *kvm = m->private; + struct kvm_memory_slot *slot; + struct kvm_memslots *slots; + unsigned int lpage_size, index; + /* Still small enough to be on the stack */ + unsigned int *log[KVM_NR_PAGE_SIZES], *cur; + int i, j, k, l, ret; + + memset(log, 0, sizeof(log)); + + ret = -ENOMEM; + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) { + log[i] = kzalloc(RMAP_LOG_SIZE * sizeof(unsigned int), GFP_KERNEL); + if (!log[i]) + goto out; + } + + mutex_lock(&kvm->slots_lock); + write_lock(&kvm->mmu_lock); + + for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) { + slots = __kvm_memslots(kvm, i); + for (j = 0; j < slots->used_slots; j++) { + slot = &slots->memslots[j]; + for (k = 0; k < KVM_NR_PAGE_SIZES; k++) { + rmap = slot->arch.rmap[k]; + lpage_size = kvm_mmu_slot_lpages(slot, k + 1); + cur = log[k]; + for (l = 0; l < lpage_size; l++) { + index = ffs(pte_list_count(&rmap[l])); + if (WARN_ON_ONCE(index >= RMAP_LOG_SIZE)) + index = RMAP_LOG_SIZE - 1; + cur[index]++; + } + } + } + } + + write_unlock(&kvm->mmu_lock); + mutex_unlock(&kvm->slots_lock); + + /* index=0 counts no rmap; index=1 counts 1 rmap */ + seq_printf(m, "Rmap_Count:\t0\t1\t"); + for (i = 2; i < RMAP_LOG_SIZE; i++) { + j = 1 << (i - 1); + k = (1 << i) - 1; + seq_printf(m, "%d-%d\t", j, k); + } + seq_printf(m, "\n"); + + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) { + seq_printf(m, "Level=%s:\t", kvm_lpage_str[i]); + cur = log[i]; + for (j = 0; j < RMAP_LOG_SIZE; j++) + seq_printf(m, "%d\t", cur[j]); + seq_printf(m, "\n"); + } + + ret = 0; +out: + for (i = 0; i < KVM_NR_PAGE_SIZES; i++) + if (log[i]) + kfree(log[i]); + + return ret; +} + +static int kvm_mmu_rmaps_stat_open(struct inode *inode, struct file *file) +{ + struct kvm *kvm = inode->i_private; + + if (!kvm_get_kvm_safe(kvm)) + return -ENOENT; + + return single_open(file, kvm_mmu_rmaps_stat_show, kvm); +} + +static int kvm_mmu_rmaps_stat_release(struct inode *inode, struct file *file) +{ + struct kvm *kvm = inode->i_private; + + kvm_put_kvm(kvm); + + return single_release(inode, file); +} + +static const struct file_operations mmu_rmaps_stat_fops = { + .open = kvm_mmu_rmaps_stat_open, + .read = seq_read, + .llseek = seq_lseek, + .release = kvm_mmu_rmaps_stat_release, +}; + +int kvm_arch_create_vm_debugfs(struct kvm *kvm) +{ + debugfs_create_file("mmu_rmaps_stat", 0644, kvm->debugfs_dentry, kvm, + &mmu_rmaps_stat_fops); + return 0; +} + static void kvm_unload_vcpu_mmu(struct kvm_vcpu *vcpu) { vcpu_load(vcpu); From patchwork Thu Jun 24 18:13:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F4B6C49EA6 for ; Thu, 24 Jun 2021 18:14:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47F1D6120D for ; Thu, 24 Jun 2021 18:14:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232585AbhFXSQk (ORCPT ); Thu, 24 Jun 2021 14:16:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22649 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232632AbhFXSQc (ORCPT ); Thu, 24 Jun 2021 14:16:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558452; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RK3peuD++IDsdruZsIC3KLTQe4djW8+2SfPfi+tnFco=; b=FCnXEK4DX+BLCqWs3NjQ4FgwsIOjcnXwNgkmlpvDxL6xsTqkt7psnwDzrhV6ju6ydZPGgk eK0D1GADDyFJjQx6ejIDZdDxiKKugxKotIzfMGkzpjBSQb4yGGlOK13L/KkFAFSjB0iFx2 ZNnOVuuUl+ivubhvkS06LNUIPestxGA= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-9KcBEAa2OjuyOiZHXM3cXA-1; Thu, 24 Jun 2021 14:14:11 -0400 X-MC-Unique: 9KcBEAa2OjuyOiZHXM3cXA-1 Received: by mail-qv1-f72.google.com with SMTP id cj11-20020a056214056bb029026a99960c7aso7844318qvb.22 for ; Thu, 24 Jun 2021 11:14:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RK3peuD++IDsdruZsIC3KLTQe4djW8+2SfPfi+tnFco=; b=aaZ8ndA6AeRtvkrJzh/Z1cAv7uV0x1tkVVpvmm1hC9lpw8ofWrCRE3lE58uxBUelsJ l2EGlQOCUvIOtmVLc/r/IYfaQ9+ytLJaz3RSFtkthpVDcNSfxVVkBVtSgZLY72Rd3iwb y8FI7B43i+jrLq+PZPsXUJZc+otZIQIXPhwcN8DKdII0vxFiC4vHQFOuhMX5bTRqnVtV M3FCMfeVJET2Q4kV0C4EXN2Cy5EjRt0QLzlDRSIqt6sCktmYAuz82W3F54BjS3qM6v5o YIEA/ceTyzftZrrnaeGDykZuQv8iOB34jbdUVjarlM50eDSWAzxBbpJbn7LugVZ3dSqW MKNA== X-Gm-Message-State: AOAM532hI3yLBoxnw1PrXHZVwZTvXL0VqqnTurZsc2HXeDXJ2SM1dFYm yOEdq98nJotnUHA7OFWgwnvAvaMZ1tETrCDN4SKuLKZW6f6OSCA0AHNxxJNFUcR3EEy6ciDqJhE 9hNebQIrcaRMitRLUtXveHRrQE0I17+b4B+s9X4kUQFHqI0XVHOQXKbgyxw5INQ== X-Received: by 2002:a05:620a:414a:: with SMTP id k10mr6879302qko.37.1624558449966; Thu, 24 Jun 2021 11:14:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyPfudshlkULqYiFpi8oeG5UyKIdtfiMvQjwTiVa7KxcRjj7roYy30ZxisLFXWhxkYMz9eKBA== X-Received: by 2002:a05:620a:414a:: with SMTP id k10mr6879273qko.37.1624558449667; Thu, 24 Jun 2021 11:14:09 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id b7sm2529301qti.21.2021.06.24.11.14.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:14:09 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , peterx@redhat.com, Sean Christopherson Subject: [PATCH 7/9] KVM: X86: MMU: Tune PTE_LIST_EXT to be bigger Date: Thu, 24 Jun 2021 14:13:54 -0400 Message-Id: <20210624181356.10235-8-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Currently rmap array element only contains 3 entries. However for EPT=N there could have a lot of guest pages that got tens of even hundreds of rmap entry. A normal distribution of a 6G guest (even if idle) shows this with rmap count statistics: Rmap_Count: 0 1 2-3 4-7 8-15 16-31 32-63 64-127 128-255 256-511 512-1023 Level=4K: 3089171 49005 14016 1363 235 212 15 7 0 0 0 Level=2M: 5951 227 0 0 0 0 0 0 0 0 0 Level=1G: 32 0 0 0 0 0 0 0 0 0 0 If we do some more fork some pages will grow even larger rmap counts. This patch makes PTE_LIST_EXT bigger so it'll be more efficient for the general use case of EPT=N as we do list reference less and the loops over PTE_LIST_EXT will be slightly more efficient; but still not too large so less waste when array not full. It should not affecting EPT=Y since EPT normally only has zero or one rmap entry for each page, so no array is even allocated. With a test case to fork 500 child and recycle them ("./rmap_fork 500" [1]), this patch speeds up fork time of about 22%. Before: 367.20 (+-4.58%) After: 302.00 (+-5.30%) [1] https://github.com/xzpeter/clibs/commit/825436f825453de2ea5aaee4bdb1c92281efe5b3 Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 80263ecb1de3..8888ae291cb9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -135,8 +135,8 @@ module_param(dbg, bool, 0644); #include -/* make pte_list_desc fit well in cache line */ -#define PTE_LIST_EXT 3 +/* make pte_list_desc fit well in cache lines */ +#define PTE_LIST_EXT 15 struct pte_list_desc { u64 *sptes[PTE_LIST_EXT]; From patchwork Thu Jun 24 18:15:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-24.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C713C49EA5 for ; Thu, 24 Jun 2021 18:15:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1516F60FEB for ; Thu, 24 Jun 2021 18:15:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232641AbhFXSRp (ORCPT ); Thu, 24 Jun 2021 14:17:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36754 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229464AbhFXSRo (ORCPT ); Thu, 24 Jun 2021 14:17:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558525; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EPRr+4E54rGtlJz/D76wpfJoXo3ikBD8ho8wpSiGbNg=; b=bG8G7MTPaCcdjWLLTKwzBedmlBHHrS1e967CmMJMOzPSA0RfIjPJZGexk+3Y9W0YBHzhqX 5PiDz4fRFDZEanQJvDMNFsP283qNfqEv/vQyc3gVZiaCOw23zrFSr3+l4HOzFmtc+fGne5 hfX9WjRJoKAp3Co4kTBxBjPaV8/vZfM= Received: from mail-il1-f197.google.com (mail-il1-f197.google.com [209.85.166.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-51-eGG1C3ZYN_as_kHqh5Bnsg-1; Thu, 24 Jun 2021 14:15:23 -0400 X-MC-Unique: eGG1C3ZYN_as_kHqh5Bnsg-1 Received: by mail-il1-f197.google.com with SMTP id f4-20020a056e020c64b02901ee69c9b838so973242ilj.20 for ; Thu, 24 Jun 2021 11:15:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EPRr+4E54rGtlJz/D76wpfJoXo3ikBD8ho8wpSiGbNg=; b=gNK+EdM+/OJaAWygLtJJAa1DBKy17TydeDtf0eTidsB5PKkhI2we5wsVPTgaNKdaiI 5VTmMhiyfdT+41DKPQTkPQ2zeyjxsTFrPZTER/GzPv7yLt4JH9CQtFP5fCZ4Rx76R/Yn k+dxishrvpYnUQNi6o2ji1RhdsJ1SsQmsqSKOdJbWCAc99QLbvczF4pEbe1JkKfiXpEh HgOThG0t6UL+7CJpf1mz/P8TW0Fgy2ijAcs0U5x8SjWKiIiuFNLm7xHOR4dRKd0z/JCV Pej03r4oAsYCoKy+gpfbDXL6ZX/XV4DzYDaIpjAQUr8ilJO9hPIxUEewSizkhrfCOq68 sORA== X-Gm-Message-State: AOAM532+890u7+WMPbZiCGXzNXyHNPchM1ZNBppb4EuVT7atXEnTzfLZ G7TC6qDpY0skKPbLqcNJq8SK4ZLEROeCUFi3MgGSqFkEpBr/GuFouKuvVcrNkGx2LNcIJIlLDQY TedZILr1v0gLUppvx+7MzYAXz0okZNMg6KRr5koVqEul+cpZSllAvFAG7QMxWXw== X-Received: by 2002:a92:c152:: with SMTP id b18mr4691986ilh.282.1624558522889; Thu, 24 Jun 2021 11:15:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyAzQfQXv5fiXNI8yt2uL42s+Rc32fWvJNBpFm0hlwRUETW7eXYH/w3gt1HorFD+WZTas4Kyg== X-Received: by 2002:a92:c152:: with SMTP id b18mr4691963ilh.282.1624558522583; Thu, 24 Jun 2021 11:15:22 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id e1sm2340254ilm.7.2021.06.24.11.15.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:15:22 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: peterx@redhat.com, Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , Sean Christopherson Subject: [PATCH 8/9] KVM: X86: Optimize pte_list_desc with per-array counter Date: Thu, 24 Jun 2021 14:15:20 -0400 Message-Id: <20210624181520.11012-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a counter field into pte_list_desc, so as to simplify the add/remove/loop logic. E.g., we don't need to loop over the array any more for most reasons. This will make more sense after we've switched the array size to be larger otherwise the counter will be a waste. Initially I wanted to store a tail pointer at the head of the array list so we don't need to traverse the list at least for pushing new ones (if without the counter we traverse both the list and the array). However that'll need slightly more change without a huge lot benefit, e.g., after we grow entry numbers per array the list traversing is not so expensive. So let's be simple but still try to get as much benefit as we can with just these extra few lines of changes (not to mention the code looks easier too without looping over arrays). I used the same a test case to fork 500 child and recycle them ("./rmap_fork 500" [1]), this patch further speeds up the total fork time of about 14%, which is a total of 38% of vanilla kernel: Vanilla: 367.20 (+-4.58%) 3->15 slots: 302.00 (+-5.30%) Add counter: 265.20 (+-9.88%) [1] https://github.com/xzpeter/clibs/commit/825436f825453de2ea5aaee4bdb1c92281efe5b3 Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 8888ae291cb9..b21e52dfc27b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -136,10 +136,15 @@ module_param(dbg, bool, 0644); #include /* make pte_list_desc fit well in cache lines */ -#define PTE_LIST_EXT 15 +#define PTE_LIST_EXT 14 struct pte_list_desc { u64 *sptes[PTE_LIST_EXT]; + /* + * Stores number of entries stored in the pte_list_desc. No need to be + * u64 but just for easier alignment. When PTE_LIST_EXT, means full. + */ + u64 spte_count; struct pte_list_desc *more; }; @@ -830,7 +835,7 @@ static int pte_list_add(struct kvm_vcpu *vcpu, u64 *spte, struct kvm_rmap_head *rmap_head) { struct pte_list_desc *desc; - int i, count = 0; + int count = 0; if (!rmap_head->val) { rmap_printk("%p %llx 0->1\n", spte, *spte); @@ -840,24 +845,24 @@ static int pte_list_add(struct kvm_vcpu *vcpu, u64 *spte, desc = mmu_alloc_pte_list_desc(vcpu); desc->sptes[0] = (u64 *)rmap_head->val; desc->sptes[1] = spte; + desc->spte_count = 2; rmap_head->val = (unsigned long)desc | 1; ++count; } else { rmap_printk("%p %llx many->many\n", spte, *spte); desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); - while (desc->sptes[PTE_LIST_EXT-1]) { + while (desc->spte_count == PTE_LIST_EXT) { count += PTE_LIST_EXT; - if (!desc->more) { desc->more = mmu_alloc_pte_list_desc(vcpu); desc = desc->more; + desc->spte_count = 0; break; } desc = desc->more; } - for (i = 0; desc->sptes[i]; ++i) - ++count; - desc->sptes[i] = spte; + count += desc->spte_count; + desc->sptes[desc->spte_count++] = spte; } return count; } @@ -873,8 +878,10 @@ pte_list_desc_remove_entry(struct kvm_rmap_head *rmap_head, ; desc->sptes[i] = desc->sptes[j]; desc->sptes[j] = NULL; + desc->spte_count--; if (j != 0) return; + WARN_ON_ONCE(desc->spte_count); if (!prev_desc && !desc->more) rmap_head->val = 0; else @@ -930,7 +937,7 @@ static void pte_list_remove(struct kvm_rmap_head *rmap_head, u64 *sptep) unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) { struct pte_list_desc *desc; - unsigned int i, count = 0; + unsigned int count = 0; if (!rmap_head->val) return 0; @@ -940,8 +947,7 @@ unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); while (desc) { - for (i = 0; (i < PTE_LIST_EXT) && desc->sptes[i]; i++) - count++; + count += desc->spte_count; desc = desc->more; } From patchwork Thu Jun 24 18:15:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12342801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EADC3C49EA5 for ; Thu, 24 Jun 2021 18:15:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC863611CB for ; Thu, 24 Jun 2021 18:15:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232665AbhFXSRt (ORCPT ); Thu, 24 Jun 2021 14:17:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:20476 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232533AbhFXSRr (ORCPT ); Thu, 24 Jun 2021 14:17:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624558527; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6oYmAaxF1m0ypshAXrxOXpFe7grcqzSbU76jZq5pQc4=; b=PNcu9gTMi97F66d0LiYo67JVbAUYFKlr/MIK6OYUE6fBl7krxNXcQtVkXON77CAW9eE/Mj Ax2N1i+xtBjgYar7XWzG8Lc/l97jOx3UXCNDkXErePCiT25zO+4PIqmB8k7ACHepChQJ06 mMy50XhwfR0mHYEBsU5BNFxjVosLqHE= Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-26-PaEIH-S6NtqyZszhsHBcEw-1; Thu, 24 Jun 2021 14:15:26 -0400 X-MC-Unique: PaEIH-S6NtqyZszhsHBcEw-1 Received: by mail-io1-f70.google.com with SMTP id 13-20020a05660220cdb02904e9d997e803so5075083ioz.20 for ; Thu, 24 Jun 2021 11:15:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6oYmAaxF1m0ypshAXrxOXpFe7grcqzSbU76jZq5pQc4=; b=lExFwyCplUxXvXfPEIP6FOAIOfQljELhWLYBdyv3MlvOSz6CuywxvKv872mvIdvzE5 x3CVUUDesO9CMvtkgIBAaf2L9LtgOCXdVKsLXPFZXSWUfh9v5piQt4zx/zCkVdQnxBjS bph9kQIOQMPchOGm6id9ACKEed/2wvsZDisLRF7sfqvhIz005yUScoJsAA+y6WUp4Jeo V6lVPZ4g6j/QxyQYS/bd23R5Y/vsLR6WgtKlztZNob2BfBQKGWgijrEwOMcYK3HfmVWR vdxIw+8VB+luxwqql44r6obP4+3yaCpxEkblmfEoZooW8Q/TCVI5WHMTjImA7CTvxUyC z8oA== X-Gm-Message-State: AOAM533G1oeGqI4ZF6BXqDjzkOT2ZqHr91IiJcBpJqOFo06HJFV6aBpc i6tPaQGA3W8kisnjzu9F/PAhVlOz8PYlTQuGz6vhaD2OdoU3oO7D8ch5gIYXohhrvwVBiYi4A2N 416b4KI51odj2lllCgTymrLvxpbt6tI18rVVKztc5J6orldKTwCMr8ZmOV6xp/Q== X-Received: by 2002:a92:2004:: with SMTP id j4mr4627827ile.53.1624558525373; Thu, 24 Jun 2021 11:15:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJaP8C3FxzWVbVxDDMkYVrrRulTZoc5GrgopLwqVo4msNj6pOvE/OkExDtHEmU59FudWjoFg== X-Received: by 2002:a92:2004:: with SMTP id j4mr4627786ile.53.1624558524837; Thu, 24 Jun 2021 11:15:24 -0700 (PDT) Received: from t490s.redhat.com (bras-base-toroon474qw-grc-65-184-144-111-238.dsl.bell.ca. [184.144.111.238]) by smtp.gmail.com with ESMTPSA id m19sm2170100ilj.52.2021.06.24.11.15.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jun 2021 11:15:24 -0700 (PDT) From: Peter Xu To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: peterx@redhat.com, Paolo Bonzini , Vitaly Kuznetsov , Maxim Levitsky , Sean Christopherson Subject: [PATCH 9/9] KVM: X86: Optimize zapping rmap Date: Thu, 24 Jun 2021 14:15:23 -0400 Message-Id: <20210624181523.11065-1-peterx@redhat.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210624181356.10235-1-peterx@redhat.com> References: <20210624181356.10235-1-peterx@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Using rmap_get_first() and rmap_remove() for zapping a huge rmap list could be slow. The easy way is to travers the rmap list, collecting the a/d bits and free the slots along the way. Provide a pte_list_destroy() and do exactly that. Signed-off-by: Peter Xu --- arch/x86/kvm/mmu/mmu.c | 45 +++++++++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index b21e52dfc27b..719fb6fd0aa0 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -954,6 +954,38 @@ unsigned int pte_list_count(struct kvm_rmap_head *rmap_head) return count; } +/* Return true if rmap existed and callback called, false otherwise */ +static bool pte_list_destroy(struct kvm_rmap_head *rmap_head, + int (*callback)(u64 *sptep)) +{ + struct pte_list_desc *desc, *next; + int i; + + if (!rmap_head->val) + return false; + + if (!(rmap_head->val & 1)) { + if (callback) + callback((u64 *)rmap_head->val); + goto out; + } + + desc = (struct pte_list_desc *)(rmap_head->val & ~1ul); + + while (desc) { + if (callback) + for (i = 0; i < desc->spte_count; i++) + callback(desc->sptes[i]); + next = desc->more; + mmu_free_pte_list_desc(desc); + desc = next; + } +out: + /* rmap_head is meaningless now, remember to reset it */ + rmap_head->val = 0; + return true; +} + static struct kvm_rmap_head *__gfn_to_rmap(gfn_t gfn, int level, struct kvm_memory_slot *slot) { @@ -1310,18 +1342,7 @@ static bool rmap_write_protect(struct kvm_vcpu *vcpu, u64 gfn) static bool kvm_zap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head, struct kvm_memory_slot *slot) { - u64 *sptep; - struct rmap_iterator iter; - bool flush = false; - - while ((sptep = rmap_get_first(rmap_head, &iter))) { - rmap_printk("spte %p %llx.\n", sptep, *sptep); - - pte_list_remove(rmap_head, sptep); - flush = true; - } - - return flush; + return pte_list_destroy(rmap_head, mmu_spte_clear_track_bits); } static bool kvm_unmap_rmapp(struct kvm *kvm, struct kvm_rmap_head *rmap_head,