From patchwork Wed Sep 18 15:28:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806986 Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCA1018B470 for ; Wed, 18 Sep 2024 15:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.119.213.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; cv=none; b=EC5B1t32MDn/vZYajv8V6xs1jLzixGDP79WqknUAz0z0d6DYvo1/81JMhAJMj3iohPEQaoD5lMJWfPdlOBVJljYT2bOhxeviqsoRQZXavTXsOPOvonSd6BKogCopzPJ8nJkt8e/P/mOyIVMQPxvT+LGDquF4T9vkO8QTEPJve7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; c=relaxed/simple; bh=ItAa8fO3kkj4DyZk3TKrr+9iURCiwBGC9y6QfuRBEkw=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FXC853vqnSBo9WVCND7ySFSKNOuiZK2iDjFlInX/K/aKncA6CG4FLklLn9wcgSZtG4iQCTEpPNkOIeUoKDID3abnwaPnscNsYUynox8hEFQA6xMy7s2DdzZYhMmPnb0Iu4zK2O3YrsqZhWkCW0uhCRSP44b9GcLveh1mJKJd6WY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=dfMqR+zz; arc=none smtp.client-ip=52.119.213.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="dfMqR+zz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673300; x=1758209300; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=R+cb/dbFY2OXe13tFaBYjWlfNZRYT3F0Ag124qVSVDc=; b=dfMqR+zzTPI4atsx39cdmx53aC/bmMHEqYw6lr90iTsGWm5Z/XQna7Ji 2gsoG1V2krbjI5s1o8WIO2gxrILM/fCYsFh8IqdSQw7Bn3/2dt7R6ll3K MVGZGZtaUPHc2GwP/aelT/F+9lkna1g1j0FahRLOin/izmxUsg8Ae6Qhp 8=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="232771159" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:17 +0000 Received: from EX19MTAEUA001.ant.amazon.com [10.0.10.100:18849] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.12:2525] with esmtp (Farcaster) id e7d64bef-93c6-4e89-8986-8a05a3aaf0da; Wed, 18 Sep 2024 15:28:15 +0000 (UTC) X-Farcaster-Flow-ID: e7d64bef-93c6-4e89-8986-8a05a3aaf0da Received: from EX19D018EUC003.ant.amazon.com (10.252.51.231) by EX19MTAEUA001.ant.amazon.com (10.252.50.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:15 +0000 Received: from EX19MTAUWC001.ant.amazon.com (10.250.64.145) by EX19D018EUC003.ant.amazon.com (10.252.51.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:15 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.210) by mail-relay.amazon.com (10.250.64.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:14 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id 5BA1E40421; Wed, 18 Sep 2024 15:28:13 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 1/8] arm64: add an interface for stage-2 page tracking Date: Wed, 18 Sep 2024 15:28:00 +0000 Message-ID: <20240918152807.25135-2-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add an interface for tracking stage-2 page accesses. The interface can be implemented by a driver for a device that has the capabilities e.g. AWS Graviton Page Tracking Agent accelerator. When a device implementing page_tracking_device interface is available, KVM will use it to accelerate dirty logging. The initial version of the interface supports dirty logging only, but the interface can be extended to other use cases, such as a WSS calculation. page_tracking_device supports tracking stage-2 translations by VMID and by CPU ID. While VMID filter is required, CPU ID is optional. CPU ID == -1 denotes any CPU. Similarly, page_tracking_device allows getting pages logged for either a particular CPU or for all. KVM can use CPU ID of -1 to populate dirty bitmaps and a specific CPU ID for per vCPU dirty rings. Signed-off-by: Lilit Janpoladyan --- arch/arm64/include/asm/page_tracking.h | 79 +++++++++++++ arch/arm64/kvm/Kconfig | 12 ++ arch/arm64/kvm/Makefile | 1 + arch/arm64/kvm/page_tracking.c | 158 +++++++++++++++++++++++++ 4 files changed, 250 insertions(+) create mode 100644 arch/arm64/include/asm/page_tracking.h create mode 100644 arch/arm64/kvm/page_tracking.c diff --git a/arch/arm64/include/asm/page_tracking.h b/arch/arm64/include/asm/page_tracking.h new file mode 100644 index 000000000000..5162fb5b648e --- /dev/null +++ b/arch/arm64/include/asm/page_tracking.h @@ -0,0 +1,79 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ARM64_PAGE_TRACKING_DEVICE_H +#define _ARM64_PAGE_TRACKING_DEVICE_H + +#include +#include + +/* Page tracking mode */ +enum pt_mode { + dirty_pages, +}; + +/* Configuration of a per-VM page tracker */ +struct pt_config { + enum pt_mode mode; /* Tracking mode */ + u32 vmid; /* VMID to track */ +}; + +/* Interface provided by the page tracking device */ +struct page_tracking_device { + + /* Allocates a per-VM tracker, returns tracking context */ + void* (*allocate_tracker)(struct pt_config config); + + /* Releases a per-VM tracker */ + int (*release_tracker)(void *ctx); + + /* + * Enables tracking for the specified @ctx and the specified @cpu, + * @cpu = -1 enables tracking for all cpus + * + * The function may be called for the same @ctx and @cpu multiple + * times and the implementation has to do reference counting to + * correctly disable the tracking. + * @returns 0 on success, negative errno in case of a failure + */ + int (*enable_tracking)(void *ctx, int cpu); + + /* + * Disables tracking for the @ctx + * + * Does actually disable the tracking of the @ctx and the @cpu only + * when the number of disable and enable calls matches, i.e. when the + * reference counter is at 0. @returns 0 in this case, -EBUSY while + * reference counter > 0 and negative errno in case of a failure + */ + int (*disable_tracking)(void *ctx, int cpu); + + /* + * Flushes any tracking data available for the @ctx, + * @returns 0 on success, negative errno in case of a failure + */ + int (*flush)(void *ctx); + + /* + * Reads up to @max dirty pages available for the @ctx + * In case @cpu id is not -1, reads only pages dirtied by the specified cpu + * @returns number of read pages and -errno in case of a failure + */ + int (*read_dirty_pages)(void *ctx, + int cpu, + gpa_t *pages, + u32 max); +}; + +/* Page tracking device tear-down, bring-up and existence checks */ +void page_tracking_device_unregister(struct page_tracking_device *pt_dev); +int page_tracking_device_register(struct page_tracking_device *pt_dev); +int page_tracking_device_registered(void); + +/* Page tracking device wrappers */ +void *page_tracking_allocate(struct pt_config config); +int page_tracking_release(void *ctx); +int page_tracking_enable(void *ctx, int cpu); +int page_tracking_disable(void *ctx, int cpu); +int page_tracking_flush(void *ctx); +int page_tracking_read_dirty_pages(void *ctx, int cpu, gpa_t *pages, u32 max); + +#endif /*_ARM64_PAGE_TRACKNG_DEVICE_H */ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 8304eb342be9..33844658279b 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -66,4 +66,16 @@ config PROTECTED_NVHE_STACKTRACE If unsure, or not using protected nVHE (pKVM), say N. +config HAVE_KVM_PAGE_TRACKING_DEVICE + bool "Support for hardware accelerated dirty tracking" + default n + help + Say Y to enable hardware accelerated dirty tracking + + Adds support for hardware accelerated dirty tracking during live + migration of a virtual machine. Requires a hardware accelerator. + + If there is no required hardware, say N. + + endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 86a629aaf0a1..4e4f5c63baf2 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -18,6 +18,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ guest.o debug.o reset.o sys_regs.o stacktrace.o \ vgic-sys-reg-v3.o fpsimd.o pkvm.o \ arch_timer.o trng.o vmid.o emulate-nested.o nested.o \ + page_tracking.o \ vgic/vgic.o vgic/vgic-init.o \ vgic/vgic-irqfd.o vgic/vgic-v2.o \ vgic/vgic-v3.o vgic/vgic-v4.o \ diff --git a/arch/arm64/kvm/page_tracking.c b/arch/arm64/kvm/page_tracking.c new file mode 100644 index 000000000000..a81c917d4faa --- /dev/null +++ b/arch/arm64/kvm/page_tracking.c @@ -0,0 +1,158 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include + +#ifndef CONFIG_HAVE_KVM_PAGE_TRACKING_DEVICE + +int page_tracking_device_register(struct page_tracking_device *dev) { return 0; } +void page_tracking_device_unregister(struct page_tracking_device *dev) {} +int page_tracking_device_registered(void) { return 0; } +void *page_tracking_allocate(struct pt_config config) { return NULL; } +int page_tracking_release(void *ctx) { return 0; } +int page_tracking_enable(void *ctx, int cpu) { return 0; } +int page_tracking_disable(void *ctx, int cpu) { return 0; } +int page_tracking_flush(void *ctx) { return 0; } +int page_tracking_read_dirty_pages(void *ctx, int cpu, gpa_t *pages, u32 max) { return 0; } + +#else + +static DEFINE_MUTEX(page_tracking_device_mutex); +static struct page_tracking_device __rcu *pt_dev __read_mostly; + +int page_tracking_device_register(struct page_tracking_device *dev) +{ + int rc = 0; + + mutex_lock(&page_tracking_device_mutex); + + if (rcu_dereference_protected(pt_dev, lockdep_is_held(&page_tracking_device_mutex))) { + rc = -EBUSY; + goto out; + } + rcu_assign_pointer(pt_dev, dev); +out: + mutex_unlock(&page_tracking_device_mutex); + return rc; +} +EXPORT_SYMBOL_GPL(page_tracking_device_register); + +void page_tracking_device_unregister(struct page_tracking_device *dev) +{ + mutex_lock(&page_tracking_device_mutex); + + if (dev == rcu_dereference_protected(pt_dev, + lockdep_is_held(&page_tracking_device_mutex))) { + /* Disable page tracking device */ + RCU_INIT_POINTER(pt_dev, NULL); + synchronize_rcu(); + } + mutex_unlock(&page_tracking_device_mutex); +} +EXPORT_SYMBOL_GPL(page_tracking_device_unregister); + +int page_tracking_device_registered(void) +{ + bool registered; + + rcu_read_lock(); + registered = (rcu_dereference(pt_dev) != NULL); + rcu_read_unlock(); + return registered; +} +EXPORT_SYMBOL_GPL(page_tracking_device_registered); + +/* Allocates a per-VM tracker, returns tracking context */ +void *page_tracking_allocate(struct pt_config config) +{ + struct page_tracking_device *dev; + void *ctx = NULL; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + ctx = dev->allocate_tracker(config); + rcu_read_unlock(); + return ctx; +} +EXPORT_SYMBOL_GPL(page_tracking_allocate); + +/* Releases a per-VM tracker */ +int page_tracking_release(void *ctx) +{ + int r; + struct page_tracking_device *dev; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + r = dev->release_tracker(ctx); + rcu_read_unlock(); + return r; +} +EXPORT_SYMBOL_GPL(page_tracking_release); + +/* Enables tracking for the specified @ctx and @cpu (-1 for all cpus) */ +int page_tracking_enable(void *ctx, int cpu) +{ + int r; + struct page_tracking_device *dev; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + r = dev->enable_tracking(ctx, cpu); + rcu_read_unlock(); + return r; +} +EXPORT_SYMBOL_GPL(page_tracking_enable); + +/* Disables tracking for the @ctx and @cpu */ +int page_tracking_disable(void *ctx, int cpu) +{ + int r; + struct page_tracking_device *dev; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + r = dev->disable_tracking(ctx, cpu); + rcu_read_unlock(); + return r; +} +EXPORT_SYMBOL_GPL(page_tracking_disable); + +/* Flushes any available data */ +int page_tracking_flush(void *ctx) +{ + int r; + struct page_tracking_device *dev; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + r = dev->flush(ctx); + rcu_read_unlock(); + return r; +} +EXPORT_SYMBOL_GPL(page_tracking_flush); + +/* + * Reads up to @max dirty pages available for the @ctx and @cpu (-1 for all cpus) + * @returns number of read pages and -errno in case of error + */ +int page_tracking_read_dirty_pages(void *ctx, int cpu, gpa_t *pages, u32 max) +{ + int r; + struct page_tracking_device *dev; + + rcu_read_lock(); + dev = rcu_dereference(pt_dev); + if (likely(dev)) + r = dev->read_dirty_pages(ctx, cpu, pages, max); + rcu_read_unlock(); + return r; +} +EXPORT_SYMBOL_GPL(page_tracking_read_dirty_pages); + +#endif From patchwork Wed Sep 18 15:28:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806985 Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com [52.119.213.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC42C79FD for ; Wed, 18 Sep 2024 15:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.119.213.150 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; cv=none; b=soZpPnf8gbjnNdWnD611J0Dj/WjNBj22k+UmhkpKIgbm/RjhVPEN2/LGfR+p6202L81v4rRyyUsW4kaN60cqu0jdZHo6JA9TYde8a0BcOLrgIAJIaO/LRK8ExRO8n37nfSFPo41VHuHJ7J0QKH6oT8ID111UZpRabCMymm3lXtY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; c=relaxed/simple; bh=kh2YDaGShz5thW3QPhYpY9fgec3JXqODlV0AI1myx1A=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DrLa6jwC7P0Rep1rNcJ9mi/DZVlz0lpD95QHXaxUomp0rxpHmH8mrchzi54wSqVtxJCJB4C1LumlX0BATSwySfDuB0cxxQ2KP1BQR3X8KIHLtg473R+B6EmeYYCl6Nig+urr0Mz3gXlN7hFgunWpCaX+rtnOOpCAwLnSqdOUn5c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=u4r8IaIV; arc=none smtp.client-ip=52.119.213.150 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="u4r8IaIV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673300; x=1758209300; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=0Jyj0savNbJCLUK2pqQ1ijmSGedpVV92FizpmekSTfE=; b=u4r8IaIVyRCR8RAxJLQH3hZWu1HIcISEpEjRoFssVqPo2qo3zouN4uhZ OYvMYuoP2fJCyuo+3cjm5hcgePvqi6FTUxNvqDXOm7OVmaacyGItGwTbF GSzEhj0mvzB2ojav0HqJilKD6lMr7sHonKPieUTW+f6fw1UWC3/RsdQPp I=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="659813185" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-52002.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:18 +0000 Received: from EX19MTAEUB002.ant.amazon.com [10.0.10.100:13771] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.46.202:2525] with esmtp (Farcaster) id eb63749e-64ec-4474-a98c-96448a4608db; Wed, 18 Sep 2024 15:28:16 +0000 (UTC) X-Farcaster-Flow-ID: eb63749e-64ec-4474-a98c-96448a4608db Received: from EX19D018EUC004.ant.amazon.com (10.252.51.172) by EX19MTAEUB002.ant.amazon.com (10.252.51.79) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:16 +0000 Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by EX19D018EUC004.ant.amazon.com (10.252.51.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:15 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:15 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id 487D94043E; Wed, 18 Sep 2024 15:28:14 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 2/8] KVM: arm64: add page tracking device as a capability Date: Wed, 18 Sep 2024 15:28:01 +0000 Message-ID: <20240918152807.25135-3-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add new capability KVM_CAP_ARM_PAGE_TRACKING_DEVICE to use page tracking device for dirty logging. The capability can be used only if platform supports such a device i.e. when KVM_CAP_ARM_PAGE_TRACKING_DEVICE extension is supported. Until there is dirty ring support, make new capability incompatible with the use of dirty ring. When page tracking device is in use, instead of logging dirty pages on faults KVM will collect a list of dirty pages from the device when userspace reads dirty bitmap. Signed-off-by: Lilit Janpoladyan --- Documentation/virt/kvm/api.rst | 17 +++++++++++++++++ arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/kvm/arm.c | 17 +++++++++++++++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 37 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index b3be87489108..989d5dd886fb 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -8950,6 +8950,23 @@ Do not use KVM_X86_SW_PROTECTED_VM for "real" VMs, and especially not in production. The behavior and effective ABI for software-protected VMs is unstable. +8.42 KVM_CAP_ARM_PAGE_TRACKING_DEVICE +_____________________________________ + +:Capability: KVM_CAP_ARM_PAGE_TRACKING_DEVICE +:Architecture: arm64 +:Type: vm +:Parameters: arg[0] whether feature should be enabled or not +:Returns 0 on success, -errno on failure + +This capability enables or disables hardware assistance for dirty page logging. + +In case page tracking device is available (i.e. if the host supports the +KVM_CAP_ARM_PAGE_TRACKING_DEVICE extension), the device can be used to accelerate +dirty logging. This capability turns the acceleration on and off. + +Not compatible with KVM_CAP_DIRTY_LOG_RING/KVM_CAP_DIRTY_LOG_RING_ACQ_REL. + 9. Known KVM API problems ========================= diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index a33f5996ca9f..5b5e3647fbda 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -326,6 +326,8 @@ struct kvm_arch { #define KVM_ARCH_FLAG_ID_REGS_INITIALIZED 7 /* Fine-Grained UNDEF initialised */ #define KVM_ARCH_FLAG_FGU_INITIALIZED 8 + /* Page tracking device enabled */ +#define KVM_ARCH_FLAG_PAGE_TRACKING_DEVICE_ENABLED 9 unsigned long flags; /* VM-wide vCPU feature set */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9bef7638342e..aea56df8ac04 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -40,6 +40,7 @@ #include #include #include +#include #include #include @@ -149,6 +150,19 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, } mutex_unlock(&kvm->slots_lock); break; + case KVM_CAP_ARM_PAGE_TRACKING_DEVICE: + if (page_tracking_device_registered() && + !kvm->dirty_ring_size /* Does not support dirty ring yet */) { + + r = 0; + if (cap->args[0]) + set_bit(KVM_ARCH_FLAG_PAGE_TRACKING_DEVICE_ENABLED, + &kvm->arch.flags); + else + clear_bit(KVM_ARCH_FLAG_PAGE_TRACKING_DEVICE_ENABLED, + &kvm->arch.flags); + } + break; default: break; } @@ -416,6 +430,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES: r = BIT(0); break; + case KVM_CAP_ARM_PAGE_TRACKING_DEVICE: + r = page_tracking_device_registered(); + break; default: r = 0; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 637efc055145..552ebede3f9d 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -933,6 +933,7 @@ struct kvm_enable_cap { #define KVM_CAP_PRE_FAULT_MEMORY 236 #define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237 #define KVM_CAP_X86_GUEST_MODE 238 +#define KVM_CAP_ARM_PAGE_TRACKING_DEVICE 239 struct kvm_irq_routing_irqchip { __u32 irqchip; From patchwork Wed Sep 18 15:28:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806993 Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com [207.171.188.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1ED01C986A for ; Wed, 18 Sep 2024 15:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=207.171.188.206 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673307; cv=none; b=Zd1TfEe4n0S25fAm7iLJEiJYu/n24FSz0bzyC+tqcjw6bc1PnU9PKMak41qefIDQbqgRC4hGdnQI7LTK6ayWLpUuBdU+XzuT1JaCDC7/cGvDX20GKpkCzaE3YDLOHcFJigthMcsZ9p6Uq6P+4sd27DgeoGgoU/k48QuemxiLI54= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673307; c=relaxed/simple; bh=Dnu2rnTshkRWcEVY9gMfGWqdViekhOAZM7F8/BSlsLs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pGwUCEEAT2NiOylbdaxDgaOjJgBoS4umDjG8fAYlKLEcVrTlSamCMNFom4B/OzNndGvemfChAGffL6xhNoOeAiXSktS5TN5qlwEj9ekd0yrt+APyB7NUvTnA2YqX1d7qcov8xcvTjMuei89+NvP7RjPF9dKoP9CGIop+lMpj8yI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=NHt6AG5D; arc=none smtp.client-ip=207.171.188.206 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="NHt6AG5D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673306; x=1758209306; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=m9B4SiInU9rOI1ITAfRDRhI4QCBJrIfBHpaGOqadohU=; b=NHt6AG5DVBYS/VNHUUyoWJhQRzr4ZEXsEP9lW7IsiMtmZtGL0wT+z88e zOuXRqBjfn0tWi/WQipoai5vUhXLLnvyeq2bQz9rp/nTcb6cZWgQC4Vee 4uMpEOmY/rdEC5Q7LzrCKs0VYwpSU2XNozrVJedCE4BjvHqNzF2oqLn98 E=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="761234704" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-9106.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:18 +0000 Received: from EX19MTAEUA001.ant.amazon.com [10.0.10.100:30946] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.12:2525] with esmtp (Farcaster) id 7ba2c8b8-2c15-4fc6-9db5-2b8ff092fee0; Wed, 18 Sep 2024 15:28:17 +0000 (UTC) X-Farcaster-Flow-ID: 7ba2c8b8-2c15-4fc6-9db5-2b8ff092fee0 Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by EX19MTAEUA001.ant.amazon.com (10.252.50.223) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:17 +0000 Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by EX19D018EUA002.ant.amazon.com (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:16 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:16 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id 243CA4063B; Wed, 18 Sep 2024 15:28:15 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 3/8] KVM: arm64: use page tracking interface to enable dirty logging Date: Wed, 18 Sep 2024 15:28:02 +0000 Message-ID: <20240918152807.25135-4-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If page tracking device is available, use it to enable and disable hardware dirty tracking. Allocate a tracking context on the first dirty logging enablement (for the first memslot) and deallocate the context when dirty logging is off for the VMID. Allocation and use of the context is not synchronized as they are done from the VM ioctls. Signed-off-by: Lilit Janpoladyan --- arch/arm64/include/asm/kvm_host.h | 5 +++ arch/arm64/kvm/arm.c | 54 +++++++++++++++++++++++++++++++ arch/mips/kvm/mips.c | 10 ++++++ arch/powerpc/kvm/book3s.c | 10 ++++++ arch/powerpc/kvm/booke.c | 10 ++++++ arch/s390/kvm/kvm-s390.c | 10 ++++++ arch/x86/kvm/x86.c | 10 ++++++ include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 19 +++++++---- 9 files changed, 123 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 5b5e3647fbda..db9bf42123e1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -377,6 +377,11 @@ struct kvm_arch { * the associated pKVM instance in the hypervisor. */ struct kvm_protected_vm pkvm; + + /* + * Stores page tracking context if page tracking device is in use + */ + void *page_tracking_ctx; }; struct kvm_vcpu_fault_info { diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index aea56df8ac04..c8dcf719ee99 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -267,6 +267,9 @@ static void kvm_destroy_mpidr_data(struct kvm *kvm) */ void kvm_arch_destroy_vm(struct kvm *kvm) { + if (kvm->arch.page_tracking_ctx) + page_tracking_release(kvm->arch.page_tracking_ctx); + bitmap_free(kvm->arch.pmu_filter); free_cpumask_var(kvm->arch.supported_cpus); @@ -1816,6 +1819,57 @@ long kvm_arch_vcpu_ioctl(struct file *filp, return r; } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + void *ctx = NULL; + struct pt_config config; + int r; + + if (!page_tracking_device_registered()) + return 0; + + if (!kvm->arch.page_tracking_ctx) { + config.vmid = (u32)kvm->arch.mmu.vmid.id.counter; + config.mode = dirty_pages; + ctx = page_tracking_allocate(config); + if (!ctx) + return -ENOENT; + + kvm->arch.page_tracking_ctx = ctx; + } + + r = page_tracking_enable(kvm->arch.page_tracking_ctx, -1); + + if (r) { + if (ctx) { + page_tracking_release(ctx); + kvm->arch.page_tracking_ctx = NULL; + } + } + return r; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + int r = 0; + + if (!page_tracking_device_registered()) + return 0; + + if (!kvm->arch.page_tracking_ctx) + return -ENOENT; + + r = page_tracking_disable(kvm->arch.page_tracking_ctx, -1); + + if (r == -EBUSY) { + r = 0; + } else { + page_tracking_release(kvm->arch.page_tracking_ctx); + kvm->arch.page_tracking_ctx = NULL; + } + return r; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index b5de770b092e..edc6f473af4e 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -974,6 +974,16 @@ long kvm_arch_vcpu_ioctl(struct file *filp, unsigned int ioctl, return r; } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index ff6c38373957..4c4a3ecc301c 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -844,6 +844,16 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu) return vcpu->kvm->arch.kvm_ops->check_requests(vcpu); } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 6a5be025a8af..f263ebc8fa49 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -1814,6 +1814,16 @@ int kvm_arch_vcpu_ioctl_translate(struct kvm_vcpu *vcpu, return r; } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return -EOPNOTSUPP; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return -EOPNOTSUPP; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 0fd96860fc45..d6a8f7dbc644 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -667,6 +667,16 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) return r; } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { int i; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c983c8e434b8..1be8bacfe2bd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6488,6 +6488,16 @@ static int kvm_vm_ioctl_reinject(struct kvm *kvm, return 0; } +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) +{ + return 0; +} + void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 0d5125a3e31a..ae905f54ec47 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1475,6 +1475,8 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn_offset, unsigned long mask); +int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot); +int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot); void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot); #ifndef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index cb2b78e92910..1fd5e234c188 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -1689,11 +1689,12 @@ static int kvm_prepare_memory_region(struct kvm *kvm, return r; } -static void kvm_commit_memory_region(struct kvm *kvm, - struct kvm_memory_slot *old, - const struct kvm_memory_slot *new, - enum kvm_mr_change change) +static int kvm_commit_memory_region(struct kvm *kvm, + struct kvm_memory_slot *old, + const struct kvm_memory_slot *new, + enum kvm_mr_change change) { + int r; int old_flags = old ? old->flags : 0; int new_flags = new ? new->flags : 0; /* @@ -1709,6 +1710,10 @@ static void kvm_commit_memory_region(struct kvm *kvm, int change = (new_flags & KVM_MEM_LOG_DIRTY_PAGES) ? 1 : -1; atomic_set(&kvm->nr_memslots_dirty_logging, atomic_read(&kvm->nr_memslots_dirty_logging) + change); + if (change > 0) + r = kvm_arch_enable_dirty_logging(kvm, new); + else + r = kvm_arch_disable_dirty_logging(kvm, new); } kvm_arch_commit_memory_region(kvm, old, new, change); @@ -1740,6 +1745,8 @@ static void kvm_commit_memory_region(struct kvm *kvm, default: BUG(); } + + return r; } /* @@ -1954,9 +1961,7 @@ static int kvm_set_memslot(struct kvm *kvm, * will directly hit the final, active memslot. Architectures are * responsible for knowing that new->arch may be stale. */ - kvm_commit_memory_region(kvm, old, new, change); - - return 0; + return kvm_commit_memory_region(kvm, old, new, change); } static bool kvm_check_memslot_overlap(struct kvm_memslots *slots, int id, From patchwork Wed Sep 18 15:28:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806987 Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com [99.78.197.219]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40E741BAEF4 for ; Wed, 18 Sep 2024 15:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.219 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; cv=none; b=AnTe110FQh5RGtl6OTKYq8uhbZLAM0Pma1WZhKOISSAjwD8MZqRQiYBbW4aOg/SfeOHXiVn0wV67lS1wR1k6r/ppLLBPko9DQJ75V4JRf6ikJeJZ0kJ0fmPM0svFiyn1Ijwdt2G7kfuKGa0jRLoGCqgklaQ4dPU1oEMPcFmDYAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673301; c=relaxed/simple; bh=GfPly/xfwNMZhmZd7O6Pa9rAtpA7qbVNaVYH8vNIrew=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aZZ1E6IeDui2EBZ88fZoFd5Fm5oVO9/CMPeOA7Q/znTuMJlBPdSM1/ovv64saMwUWlXFNVUeZlmtTTb80iDLhXDpY4lbf/uZiQlpiIcG9ZHKJYL4QvZUAJTq8YUqbFbtXEP9B0eAsg89/N5kk4wJNEXK0dDR/GKKe/aRCrO61kA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=QUFpfKnG; arc=none smtp.client-ip=99.78.197.219 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="QUFpfKnG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673300; x=1758209300; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=xNhiMTqaGcTO02GCyyfzFGUNlD6mif6/4fY2DqZbTcA=; b=QUFpfKnGtcsJMcmhqqyqynonpxW1iDQYicAfEmlr/tnqOfpWUDHrrBkr 4uqeiB+jxSUO4FQ7HTS3BDQ7BGJ0nvCJgWIppXRGAhh3Om2MYXZ0bqqiN 0RSxBq4Cn3Af+ItncTZSY+opAabBUhgh0lXg2VBrP9poFulKjM1vtAl+j c=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="127584763" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-80008.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:18 +0000 Received: from EX19MTAEUA002.ant.amazon.com [10.0.43.254:29472] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.12:2525] with esmtp (Farcaster) id 0af88632-bff0-4ba6-af68-ea5498bad667; Wed, 18 Sep 2024 15:28:17 +0000 (UTC) X-Farcaster-Flow-ID: 0af88632-bff0-4ba6-af68-ea5498bad667 Received: from EX19D018EUA004.ant.amazon.com (10.252.50.85) by EX19MTAEUA002.ant.amazon.com (10.252.50.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:17 +0000 Received: from EX19MTAUEC001.ant.amazon.com (10.252.135.222) by EX19D018EUA004.ant.amazon.com (10.252.50.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:16 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.43.8.6) by mail-relay.amazon.com (10.252.135.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:16 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id F407F4041D; Wed, 18 Sep 2024 15:28:15 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 4/8] KVM: return value from kvm_arch_sync_dirty_log Date: Wed, 18 Sep 2024 15:28:03 +0000 Message-ID: <20240918152807.25135-5-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Make kvm_arch_sync_dirty_log return a value, which is needed to propagate errors that could happen when getting dirty pages via a page tracking device. Signed-off-by: Lilit Janpoladyan --- arch/loongarch/kvm/mmu.c | 3 ++- arch/mips/kvm/mips.c | 4 ++-- arch/powerpc/kvm/book3s.c | 4 ++-- arch/powerpc/kvm/booke.c | 4 ++-- arch/riscv/kvm/mmu.c | 3 ++- arch/s390/kvm/kvm-s390.c | 3 ++- arch/x86/kvm/x86.c | 11 ++++++----- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 15 ++++++++++++--- 9 files changed, 31 insertions(+), 18 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..825c60d35529 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -943,8 +943,9 @@ int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) return 0; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { + return 0; } void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index edc6f473af4e..4326b8c721e9 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -984,9 +984,9 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return 0; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { - + return 0; } int kvm_arch_flush_remote_tlbs(struct kvm *kvm) diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index 4c4a3ecc301c..aab6f5c62aee 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -854,9 +854,9 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return 0; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { - + return 0; } int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index f263ebc8fa49..60629a320222 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -1824,9 +1824,9 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return -EOPNOTSUPP; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { - + return -EOPNOTSUPP; } int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index b63650f9b966..53ad23432b31 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -402,8 +402,9 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, gstage_wp_range(kvm, start, end); } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { + return 0; } void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free) diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index d6a8f7dbc644..5f1bb4bd4121 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -677,7 +677,7 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return 0; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { int i; gfn_t cur_gfn, last_gfn; @@ -705,6 +705,7 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) return; cond_resched(); } + return 0; } /* Section: vm related */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1be8bacfe2bd..e95e070c9bf3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6498,7 +6498,7 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return 0; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { /* @@ -6510,11 +6510,12 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) struct kvm_vcpu *vcpu; unsigned long i; - if (!kvm_x86_ops.cpu_dirty_log_size) - return; + if (kvm_x86_ops.cpu_dirty_log_size) { + kvm_for_each_vcpu(i, vcpu, kvm) + kvm_vcpu_kick(vcpu); + } - kvm_for_each_vcpu(i, vcpu, kvm) - kvm_vcpu_kick(vcpu); + return 0; } int kvm_vm_ioctl_irq_line(struct kvm *kvm, struct kvm_irq_level *irq_event, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ae905f54ec47..245b4172a7fb 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1477,7 +1477,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, unsigned long mask); int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot); int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot); -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot); +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot); #ifndef CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1fd5e234c188..d55d92f599b0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2145,6 +2145,7 @@ int kvm_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log, int i, as_id, id; unsigned long n; unsigned long any = 0; + int r; /* Dirty ring tracking may be exclusive to dirty log tracking */ if (!kvm_use_dirty_bitmap(kvm)) @@ -2163,7 +2164,9 @@ int kvm_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log, if (!(*memslot) || !(*memslot)->dirty_bitmap) return -ENOENT; - kvm_arch_sync_dirty_log(kvm, *memslot); + r = kvm_arch_sync_dirty_log(kvm, *memslot); + if (r) + return r; n = kvm_dirty_bitmap_bytes(*memslot); @@ -2210,6 +2213,7 @@ static int kvm_get_dirty_log_protect(struct kvm *kvm, struct kvm_dirty_log *log) unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; bool flush; + int r; /* Dirty ring tracking may be exclusive to dirty log tracking */ if (!kvm_use_dirty_bitmap(kvm)) @@ -2227,7 +2231,9 @@ static int kvm_get_dirty_log_protect(struct kvm *kvm, struct kvm_dirty_log *log) dirty_bitmap = memslot->dirty_bitmap; - kvm_arch_sync_dirty_log(kvm, memslot); + r = kvm_arch_sync_dirty_log(kvm, memslot); + if (r) + return r; n = kvm_dirty_bitmap_bytes(memslot); flush = false; @@ -2322,6 +2328,7 @@ static int kvm_clear_dirty_log_protect(struct kvm *kvm, unsigned long *dirty_bitmap; unsigned long *dirty_bitmap_buffer; bool flush; + int r; /* Dirty ring tracking may be exclusive to dirty log tracking */ if (!kvm_use_dirty_bitmap(kvm)) @@ -2349,7 +2356,9 @@ static int kvm_clear_dirty_log_protect(struct kvm *kvm, (log->num_pages < memslot->npages - log->first_page && (log->num_pages & 63))) return -EINVAL; - kvm_arch_sync_dirty_log(kvm, memslot); + r = kvm_arch_sync_dirty_log(kvm, memslot); + if (r) + return r; flush = false; dirty_bitmap_buffer = kvm_second_dirty_bitmap(memslot); From patchwork Wed Sep 18 15:28:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806992 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8784F1C9863 for ; Wed, 18 Sep 2024 15:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.95.49.90 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673307; cv=none; b=YKEzWv6BtpiF9brANRORd6UhqaQVVzrnwVr17sDLRtz+5AazbDm9gjJnFBQdGf+oIUeqcvGxcgw0HrrYBApDJlbp5ucGf+dJMjEu4wF+FwFmyApv/quzFvJ2y5wAUFxTl12ae/sri1e+FkEQHTw7LvXFZXMPyUOye8DuFDSvk/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673307; c=relaxed/simple; bh=nXvFGyAITAu4ekprdmS+sLhABiY4x0FQ60YvS6VUg5k=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HVr/HsSNtywZLbWhy1pSpqy6VW46KYZRk4nS+WZnRUYZWb1TbSLB5rJ6ow9nDcb0+Pv6MvZ48mYp0ImtXBMThPTJzk9/Ch3CqjKeCErw7aZGzx1pdk1oX4N9mY5Gec4AEEkBRQhGPTHsJWtV6OoH4WlghR43jkseHrNguiuHygo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=gJBO30TQ; arc=none smtp.client-ip=52.95.49.90 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="gJBO30TQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673306; x=1758209306; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=8XYY163qVrhuzbZHidEl0QQXvKIZLmTJ/Kwsl2aJs0Y=; b=gJBO30TQCceMiMst61VbcI77TgU0/8xOsnmWuElX+MqxPn81q49sojVv RWCFpFdjrtAyuKRe/JD9BUBNUWsL2FnRlVdG6wAHHxcECN1h+yFOC0S6J fTlGV2PnBh340WmxnlGWvRR2jw1WvaOOtf9v2RuA4QZ28en/wtYrFUwZw c=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="434305187" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:22 +0000 Received: from EX19MTAEUB001.ant.amazon.com [10.0.43.254:30066] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.41.159:2525] with esmtp (Farcaster) id fe3dd968-f541-43d1-af70-bc02ad3d8ab8; Wed, 18 Sep 2024 15:28:20 +0000 (UTC) X-Farcaster-Flow-ID: fe3dd968-f541-43d1-af70-bc02ad3d8ab8 Received: from EX19D018EUC002.ant.amazon.com (10.252.51.143) by EX19MTAEUB001.ant.amazon.com (10.252.51.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:19 +0000 Received: from EX19MTAUWC001.ant.amazon.com (10.250.64.145) by EX19D018EUC002.ant.amazon.com (10.252.51.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:19 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.210) by mail-relay.amazon.com (10.250.64.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:18 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id CFCFC40421; Wed, 18 Sep 2024 15:28:16 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 5/8] KVM: arm64: get dirty pages from the page tracking device Date: Wed, 18 Sep 2024 15:28:04 +0000 Message-ID: <20240918152807.25135-6-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If a page tracking device is available, use it in kvm_arch_sync_dirty_log to read device dirty log and mark logged pages as dirty. Allocate a page to use as a buffer for reading device dirty log; the allocation and access to the page are not synchronized - assumes that userspace won't try to enable/disable dirty logging and read dirty log at the same time. Signed-off-by: Lilit Janpoladyan --- arch/arm64/include/asm/kvm_host.h | 3 +- arch/arm64/kvm/arm.c | 46 ++++++++++++++++++++++++++++++- 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index db9bf42123e1..a76f25d4d2bc 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -379,9 +379,10 @@ struct kvm_arch { struct kvm_protected_vm pkvm; /* - * Stores page tracking context if page tracking device is in use + * Stores page tracking context and buffer if page tracking device is in use */ void *page_tracking_ctx; + gpa_t *page_tracking_pg; }; struct kvm_vcpu_fault_info { diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index c8dcf719ee99..139d7e929266 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1822,6 +1822,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp, int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot *memslot) { void *ctx = NULL; + unsigned long read_buffer; struct pt_config config; int r; @@ -1836,16 +1837,31 @@ int kvm_arch_enable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot return -ENOENT; kvm->arch.page_tracking_ctx = ctx; + + read_buffer = __get_free_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (!read_buffer) { + r = -ENOMEM; + goto out_free; + } + + kvm->arch.page_tracking_pg = (gpa_t *)read_buffer; } r = page_tracking_enable(kvm->arch.page_tracking_ctx, -1); +out_free: if (r) { if (ctx) { page_tracking_release(ctx); kvm->arch.page_tracking_ctx = NULL; } + + if (read_buffer) { + free_page(read_buffer); + kvm->arch.page_tracking_pg = NULL; + } } + return r; } @@ -1866,13 +1882,41 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot } else { page_tracking_release(kvm->arch.page_tracking_ctx); kvm->arch.page_tracking_ctx = NULL; + + if (kvm->arch.page_tracking_pg) { + free_page((unsigned long)kvm->arch.page_tracking_pg); + kvm->arch.page_tracking_pg = NULL; + } } return r; } -void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) +int kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot) { + int r; + u32 i; + u32 max_pages = PAGE_SIZE/sizeof(gpa_t); + if (!kvm->arch.page_tracking_ctx) + return 0; + + if (!kvm->arch.page_tracking_pg) + return -ENOENT; + + r = page_tracking_read_dirty_pages(kvm->arch.page_tracking_ctx, -1, + kvm->arch.page_tracking_pg, max_pages); + + while (r > 0) { + for (i = 0; i < r; ++i) { + u32 gfn = kvm->arch.page_tracking_pg[i] >> PAGE_SHIFT; + + memslot = gfn_to_memslot(kvm, gfn); + mark_page_dirty_in_slot(kvm, memslot, gfn); + } + r = page_tracking_read_dirty_pages(kvm->arch.page_tracking_ctx, -1, + kvm->arch.page_tracking_pg, max_pages); + } + return r < 0 ? r : 0; } static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm, From patchwork Wed Sep 18 15:28:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806990 Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EF5F1C984E for ; Wed, 18 Sep 2024 15:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.95.49.90 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; cv=none; b=RgiJi/sSXj1NYpAahdFS7d0pIoIQF2mw9fZ3hHLEqe+soWwkFITkAccwNoT4exXGJu3+8bm8UlLhj+mos2bIOHhA026ezznrM25vr5VMDLZt1snZlF8e13CQIEO4E/qw9vDAb/8R97qnqz+EEZKBhVoFmv3o0+isi9Nj0kXnv2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; c=relaxed/simple; bh=TXO4quz4Mn+UlPlrUbatQb4WzsclfIuhR8Ks5+h7PRc=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uBAZF4xOC+xdgvmNqS9p092IuksJ0J90RUV7xuzfCkrCXQRXliD0+wdLDK1P343d3YshtlO03Cl13BzoKBGjoQ7BEIgg3Iy+3DLuTIcfyS72U0ZXOczPPL+JygFjaZsoiP8hrcgz4DMZacrQ2PFnQSLTImC+L3PAT2ewGYeCK7Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=eQ6up9F0; arc=none smtp.client-ip=52.95.49.90 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="eQ6up9F0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673304; x=1758209304; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=o+1ltl0tHyU5VT0TXuY0ad53zHIQ+Akmv2TZGDLX2Mo=; b=eQ6up9F0R/MqtbXEYkCsmNcT3sSel+JL5np9FsVzlL39tidQpJ79Xa56 WifhFeeib63E3RxqLqKVy3hl9mfGLGJ/mKIubWal9kyv5jk4rWofKw6+s 9+l+nDHJ+BW63sw2BIH7Q/+vpNUxDrFW1z0Kv40cTqv0dKBaoyqdmLVFY 0=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="434305177" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:21 +0000 Received: from EX19MTAEUA002.ant.amazon.com [10.0.43.254:28715] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.41.159:2525] with esmtp (Farcaster) id 3fb400e6-91db-4d62-bc61-38f4ea3463bd; Wed, 18 Sep 2024 15:28:19 +0000 (UTC) X-Farcaster-Flow-ID: 3fb400e6-91db-4d62-bc61-38f4ea3463bd Received: from EX19D018EUA002.ant.amazon.com (10.252.50.146) by EX19MTAEUA002.ant.amazon.com (10.252.50.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:19 +0000 Received: from EX19MTAUWB001.ant.amazon.com (10.250.64.248) by EX19D018EUA002.ant.amazon.com (10.252.50.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:19 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.254) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:18 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id AB47D4041D; Wed, 18 Sep 2024 15:28:17 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 6/8] KVM: arm64: flush dirty logging data Date: Wed, 18 Sep 2024 15:28:05 +0000 Message-ID: <20240918152807.25135-7-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Make sure we do not miss last dirty pages and flush the data after disabling dirty logging. Flush only when dirty logging is actually disabled i.e. when page_tracking_disable returns 0. Signed-off-by: Lilit Janpoladyan --- arch/arm64/kvm/arm.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 139d7e929266..5ed049accb3e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1877,17 +1877,21 @@ int kvm_arch_disable_dirty_logging(struct kvm *kvm, const struct kvm_memory_slot r = page_tracking_disable(kvm->arch.page_tracking_ctx, -1); - if (r == -EBUSY) { - r = 0; - } else { - page_tracking_release(kvm->arch.page_tracking_ctx); - kvm->arch.page_tracking_ctx = NULL; + if (r == -EBUSY) + return 0; - if (kvm->arch.page_tracking_pg) { - free_page((unsigned long)kvm->arch.page_tracking_pg); - kvm->arch.page_tracking_pg = NULL; - } + /* Flush only when dirty tracking is disabled */ + if (!r) + r = page_tracking_flush(kvm->arch.page_tracking_ctx); + + /* But release resources anyway */ + page_tracking_release(kvm->arch.page_tracking_ctx); + kvm->arch.page_tracking_ctx = NULL; + if (kvm->arch.page_tracking_pg) { + free_page((unsigned long)kvm->arch.page_tracking_pg); + kvm->arch.page_tracking_pg = NULL; } + return r; } From patchwork Wed Sep 18 15:28:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806989 Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com [99.78.197.219]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 852291BAEF4 for ; Wed, 18 Sep 2024 15:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.219 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; cv=none; b=U5YEK8xGchT1AAqZyGfLn2WqildbD5dNN43qTUelsy1yCsc6OWaQSG48KhW80p+PKZfyStYInxk2wSlr9iv5T06sXKo+tENhsySJMopN9Gt/8LhueA10B3A9RfIpz2eVLAcObqfF9Z77waVuoVECUe6bu1ACAdZuIl6gpJcNN4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; c=relaxed/simple; bh=HvbYUAUuqxfvZVUyVuSjHY5g7ZmMcbW5eX1OPbSI7ak=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NFpeWzdq20M7mkTgHwFwe41A3hliAparvidHB7SBJ2Qrz2qg9mVuCoyKjZ5eLfiP6JmJXsSrBuIijfwKOigCcqOlhupexJVczMbnB/yiM1TuzCrCnPaLOE3bC0tTAk3sxYnE6i7sEWDly1Qc1/tWdxJVqd0cvQLlhXwbEdHoKqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=V6N/T07X; arc=none smtp.client-ip=99.78.197.219 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="V6N/T07X" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673303; x=1758209303; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=Mrvf5QEc5i3VRcPPRer2E9I3RCr3GLlTIVg8suSOwps=; b=V6N/T07XY/pYp1J6oHHuVvTmKUTrk3BhzbjOBuTdBwxIvPzrPEtJ6I4l 80U0W0y9v/7i/1akI90CMHNyRJapqymRAmhc8r3xIicNIhOZcnbi6BNjk psEhfPW/EcKRjKylI/8JjoAX0Hr/Eqa4OX8B5mNohgC8diWYjTjFOc5kH I=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="127584803" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-80008.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:22 +0000 Received: from EX19MTAEUA002.ant.amazon.com [10.0.43.254:54141] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.8.19:2525] with esmtp (Farcaster) id 4c766550-61e7-4d25-8d36-b3acac605531; Wed, 18 Sep 2024 15:28:20 +0000 (UTC) X-Farcaster-Flow-ID: 4c766550-61e7-4d25-8d36-b3acac605531 Received: from EX19D018EUA003.ant.amazon.com (10.252.50.163) by EX19MTAEUA002.ant.amazon.com (10.252.50.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:20 +0000 Received: from EX19MTAUWB002.ant.amazon.com (10.250.64.231) by EX19D018EUA003.ant.amazon.com (10.252.50.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:20 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.228) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:19 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id 869FC4043E; Wed, 18 Sep 2024 15:28:18 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 7/8] KVM: arm64: enable hardware dirty state management for stage-2 Date: Wed, 18 Sep 2024 15:28:06 +0000 Message-ID: <20240918152807.25135-8-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Enable hardware management of dirty state for stage-2 (VTCR_EL2.HD) translations. Set VTCR_EL2.HD unconditionally. This won't allow hardware dirty state management yet as page descriptors are considered as candidates for hardware dirty state updates when DBM (51) bit is set and by default page descriptors are created with DBM = 0. Signed-off-by: Lilit Janpoladyan --- arch/arm64/kvm/hyp/pgtable.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 9e2bbee77491..d507931ab10c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -658,7 +658,7 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) #ifdef CONFIG_ARM64_HW_AFDBM /* - * Enable the Hardware Access Flag management, unconditionally + * Enable the Hardware Access Flag and Dirty State management, unconditionally * on all CPUs. In systems that have asymmetric support for the feature * this allows KVM to leverage hardware support on the subset of cores * that implement the feature. @@ -669,8 +669,10 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) * happen to be running on a design that has unadvertised support for * HAFDBS. Here be dragons. */ - if (!cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38)) + if (!cpus_have_final_cap(ARM64_WORKAROUND_AMPERE_AC03_CPU_38)) { vtcr |= VTCR_EL2_HA; + vtcr |= VTCR_EL2_HD; + } #endif /* CONFIG_ARM64_HW_AFDBM */ if (kvm_lpa2_is_enabled()) From patchwork Wed Sep 18 15:28:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lilit Janpoladyan X-Patchwork-Id: 13806991 Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D80B21C9855 for ; Wed, 18 Sep 2024 15:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.119.213.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; cv=none; b=ZpXDO+u9xmsS1bvdceOS2OOPM7xMb8Z2Kq/U/KLtSQVMxAZBMCe8BY647JS3QJE56Ncdq3uS04Ydt4/+1Cp28PbWiXT15fN0xtlVcaGT8q8gZoAqdEjqnppSYp1UxN6Rw+5kdl878NI3b/fSzpxUKSXbfm8wlymqPsUqtL9uleM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726673305; c=relaxed/simple; bh=/fiwKLJNU+6tNWT6VQd7KBpnBT+w41Rr0g3Co8I4NZQ=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=iFv4GtWkbgxeAFAEPlnh0uqunBP42r1Lc/SjkeHJdkp9HBM9pas2huT5b/4z9N3tB45ZjsTCdSAIecQWtXVOJrzVkr+fRXRYoVNrLpSqlmoS3YO+nJNCli7HSNbNhc1eO+CEs0WTCLCMq1Hz47i1lrhZE9RCrQneNMMphZ0HsiQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=l77kZWU+; arc=none smtp.client-ip=52.119.213.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="l77kZWU+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1726673304; x=1758209304; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=cBuhl0qXKMJntrBkdWcKjcLFuCE565FUUFhI9H/DaWM=; b=l77kZWU+2A9Zp6h1wtzMQco35vJ/sQEM/qF5gJgKY/9kXZvBy0HSqXvv htih3b6x0h4UZYn3NjSV2Fijbg8oRLAEwUBee6TFU99Dmlgv5Ox7cHzB0 1apiZqrDKb55cOTuSZ6CNqiSDWsR5kHY99gqrZpqL+sBeRoGplhtLwEnn w=; X-IronPort-AV: E=Sophos;i="6.10,239,1719878400"; d="scan'208";a="232771200" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Sep 2024 15:28:23 +0000 Received: from EX19MTAEUB002.ant.amazon.com [10.0.10.100:54299] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.16.194:2525] with esmtp (Farcaster) id 4bd0e212-e6c1-41ad-a1ff-6bfdfe5b07a3; Wed, 18 Sep 2024 15:28:22 +0000 (UTC) X-Farcaster-Flow-ID: 4bd0e212-e6c1-41ad-a1ff-6bfdfe5b07a3 Received: from EX19D018EUA001.ant.amazon.com (10.252.50.145) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:20 +0000 Received: from EX19MTAUEB001.ant.amazon.com (10.252.135.35) by EX19D018EUA001.ant.amazon.com (10.252.50.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34; Wed, 18 Sep 2024 15:28:20 +0000 Received: from email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (10.43.8.2) by mail-relay.amazon.com (10.252.135.35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.34 via Frontend Transport; Wed, 18 Sep 2024 15:28:20 +0000 Received: from dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com (dev-dsk-lilitj-1a-5039c68b.eu-west-1.amazon.com [172.19.104.233]) by email-imr-corp-prod-iad-all-1a-93a35fb4.us-east-1.amazon.com (Postfix) with ESMTPS id 6258C4063D; Wed, 18 Sep 2024 15:28:19 +0000 (UTC) From: Lilit Janpoladyan To: , , , , , , , Subject: [PATCH 8/8] KVM: arm64: make hardware manage dirty state after write faults Date: Wed, 18 Sep 2024 15:28:07 +0000 Message-ID: <20240918152807.25135-9-lilitj@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240918152807.25135-1-lilitj@amazon.com> References: <20240918152807.25135-1-lilitj@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In case of hardware dirty logging, fault in pages with their dirty state managed by hardware. This will allow further writes to the faulted in pages to be logged by the page tracking device. The first write will still be logged on write fault. To avoid faults on first writes we need to set DBM bit when eagerly splitting huge pages (to be added). Add KVM_PTE_LEAF_ATTR_HI_S2_DBM for the hardware DBM flag and KVM_PGTABLE_PROT_HWDBM as a software page protection flag. Hardware dirty state management changes the way KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W is interpreted. Pages whose dirty state is managed by the hardware are always writable and KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W bit denotes their dirty state. Signed-off-by: Lilit Janpoladyan --- arch/arm64/include/asm/kvm_pgtable.h | 1 + arch/arm64/kvm/hyp/pgtable.c | 23 ++++++++++++++++++++--- arch/arm64/kvm/mmu.c | 8 ++++++++ 3 files changed, 29 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 19278dfe7978..d3b81d7e923b 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -210,6 +210,7 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_DEVICE = BIT(3), KVM_PGTABLE_PROT_NORMAL_NC = BIT(4), + KVM_PGTABLE_PROT_HWDBM = BIT(5), KVM_PGTABLE_PROT_SW0 = BIT(55), KVM_PGTABLE_PROT_SW1 = BIT(56), diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index d507931ab10c..c4d654e7189c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -46,6 +46,8 @@ #define KVM_PTE_LEAF_ATTR_HI_S1_GP BIT(50) +#define KVM_PTE_LEAF_ATTR_HI_S2_DBM BIT(51) + #define KVM_PTE_LEAF_ATTR_S2_PERMS (KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R | \ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) @@ -746,7 +748,13 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p if (prot & KVM_PGTABLE_PROT_R) attr |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R; - if (prot & KVM_PGTABLE_PROT_W) + /* + * If hardware dirty state management is enabled then S2AP_W is interpreted + * as dirty state, don't set S2AP_W in this case + */ + if (prot & KVM_PGTABLE_PROT_HWDBM) + attr |= KVM_PTE_LEAF_ATTR_HI_S2_DBM; + else if (prot & KVM_PGTABLE_PROT_W) attr |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W; if (!kvm_lpa2_is_enabled()) @@ -768,7 +776,10 @@ enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte) if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R) prot |= KVM_PGTABLE_PROT_R; - if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W) + + if (pte & KVM_PTE_LEAF_ATTR_HI_S2_DBM) + prot |= KVM_PGTABLE_PROT_HWDBM | KVM_PGTABLE_PROT_W; + else if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W) prot |= KVM_PGTABLE_PROT_W; if (!(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN)) prot |= KVM_PGTABLE_PROT_X; @@ -1367,7 +1378,13 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr, if (prot & KVM_PGTABLE_PROT_R) set |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R; - if (prot & KVM_PGTABLE_PROT_W) + /* + * If hardware dirty state management is enabled then S2AP_W is interpreted + * as dirty state, don't set S2AP_W in this case + */ + if (prot & KVM_PGTABLE_PROT_HWDBM) + set |= KVM_PTE_LEAF_ATTR_HI_S2_DBM; + else if (prot & KVM_PGTABLE_PROT_W) set |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W; if (prot & KVM_PGTABLE_PROT_X) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd..a5bcc7f11083 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1418,6 +1418,11 @@ static bool kvm_vma_mte_allowed(struct vm_area_struct *vma) return vma->vm_flags & VM_MTE_ALLOWED; } +static bool is_hw_logging_enabled(struct kvm *kvm) +{ + return kvm->arch.page_tracking_ctx != NULL; +} + static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, struct kvm_s2_trans *nested, struct kvm_memory_slot *memslot, unsigned long hva, @@ -1658,6 +1663,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (writable) prot |= KVM_PGTABLE_PROT_W; + if (is_hw_logging_enabled(kvm)) + prot |= KVM_PGTABLE_PROT_HWDBM; + if (exec_fault) prot |= KVM_PGTABLE_PROT_X;