From patchwork Thu Sep 15 10:10:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivam Kumar X-Patchwork-Id: 12977216 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05C15ECAAA1 for ; Thu, 15 Sep 2022 10:11:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229933AbiIOKLu (ORCPT ); Thu, 15 Sep 2022 06:11:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229690AbiIOKLr (ORCPT ); Thu, 15 Sep 2022 06:11:47 -0400 Received: from mx0b-002c1b01.pphosted.com (mx0b-002c1b01.pphosted.com [148.163.155.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 425C3402D1 for ; Thu, 15 Sep 2022 03:11:46 -0700 (PDT) Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28F2riI8022919; Thu, 15 Sep 2022 03:11:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=proofpoint20171006; bh=LsODNu59GdJ9q5LrpIHhoFfeD2kUXXL0jbHHXH21wh4=; b=qbDP/Kb8Y8L3Nb0JlgSUBXLlPGQB3b2A/LxfNdl+P4Zhsnz0GCB6BWOiQUKTHWyjNykO Yq3UROYQRDqKnboHBpCRLwaH2lL8dd51A+gpHmXxtB94s7MX9h5sZPOZgvWe5I6aN164 QpAwJGsZeQXKYBqrckZ+DMb37akdEysXI/KFMEF+zC2G59xY/1FAp0qqu3VzM77ervfq rVC+LL59PpPz6P4oBfyE5BN7ne4huaykkRw8yBEQNt7/0+DUR9W3KcjsZivbNtjsYtKC 4VOo12IpvE9T99myJf6kicbwp9B1NJUV/u0D/P6EQ33/WluVrr2rOCiySYBvpgQihqu4 Vw== Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2171.outbound.protection.outlook.com [104.47.57.171]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 3jjy074aja-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 03:11:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TFyx3UxtfbEMcfRRrNsMPpLE8pjcMnEzRHfUc6NIxQfFl6vuVmca+CHZQ6ZFpBlS4mRVmrBGFUbYeUZzbOg0jIZgkOQO4RAO2EaYoTW0kuH5fv8h+SCyd6hZljlfjbm7cmyCbjSifAagMsHpk7FAdtttR9CiZGsECZ/me8maGloN7F6YHY+aMlKXbgAawcIdkVkMaxfKQaGRU1CCP9mR+99/gz96k53wJs5Ge4JGtt+GqRNx5nxUH1gkC4gCay/gBg6Vw1qiIGLNtAdwuI9EtaH2b74/D9noljM1UIngTMKvhV5emGG2kWsiIeJ2NGA3fCjl1QEaz1xQ43nav1wNgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LsODNu59GdJ9q5LrpIHhoFfeD2kUXXL0jbHHXH21wh4=; b=YEBz5zgprOGV+LZWvdtrbYL/z4E4myCMtH+BuZc3R7fHN8Dt4EANq9Ck6h4Uau89KfVt1krb5Fv1t8mcjmU97lk7WEEOmnj5z3PGVJ56Pxu1OPfAtQVW6ru2PLKuPtsz664R8R+5y1NqG0M+lysuDuxk+LI5xlfMg0XpAxSpBN5OV/pU37PIdp8vuDmEeN2LvsTXZ9FfvNcybrRl8saJ92DPU0cDEocMeulzvkl7opaYphBrZh5lBg+1K9b5fVX6XZ+tetv0Z/+MzFax/h7wje6mRjEwuAdZsHjfeHQceox4ww6937aW5JyMbC3haZ0cUjQ/XzFx++6QbsKiiqpr4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Received: from CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) by SA0PR02MB7370.namprd02.prod.outlook.com (2603:10b6:806:e0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 15 Sep 2022 10:11:27 +0000 Received: from CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf]) by CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf%8]) with mapi id 15.20.5632.015; Thu, 15 Sep 2022 10:11:27 +0000 From: Shivam Kumar To: pbonzini@redhat.com, seanjc@google.com, maz@kernel.org, james.morse@arm.com, borntraeger@linux.ibm.com, david@redhat.com Cc: kvm@vger.kernel.org, Shivam Kumar , Shaju Abraham , Manish Mishra , Anurag Madnawat Subject: [PATCH v6 1/5] KVM: Implement dirty quota-based throttling of vcpus Date: Thu, 15 Sep 2022 10:10:46 +0000 Message-Id: <20220915101049.187325-2-shivam.kumar1@nutanix.com> X-Mailer: git-send-email 2.22.3 In-Reply-To: <20220915101049.187325-1-shivam.kumar1@nutanix.com> References: <20220915101049.187325-1-shivam.kumar1@nutanix.com> X-ClientProxiedBy: SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) To CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR02MB7555:EE_|SA0PR02MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: 3ccf8aa4-609d-4673-2b8d-08da9702a72e x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WY0wXsOLt5KGgTvKCZvVAR41A7SiVW2OMGZVymXXSL1m/9k3hm4CdemdrVICLFR50FNuYCHw0pfav38biiE9dSUupEmlnjcXfXNfY6FMn4W8u3pg5kltDzbSS17W74DKZ40g04GFkVReW9cGwLFW6hjwgonVz5VCpNF2sH6uHou//Z8fxxTG/XtUjWhUjmBL39VRNFPtC26oftBzq5sA3iTOIAIatHj6SCJihV1z6jEX88t9Wu2YmaY3Avb8Cq5/YjJhSc/SlvIgIwkZJP2hkuTH41NSk9aDTxuAOLIDJBabiAH/QKRsVn+uMpRhmzHNwlLK0G3RRehqiym5lgjScmnKTIe/G/QOydwZn6h61ztppakQtwy9rzfWPmP74ev+S23pQUz8fyFnUXigFXXx0+gEGd1VqqGobb6heKvAhNRSWXpWYK10vA6M4uT2IcfKxGIrcqX1YAPNMO7fLlqrA7NrBNMgu3TaqvOmU/L/16wIif6vyXmqbsYoxEVIBJwQ2ToD0lr7wJYbBHEqo2OjXHC4P0n1oll+6ho0/I/FQMABGwLO44nQ9eFV/Jq3VKynA3LI7T4hKT+fHTMyb5ztBmpuvl4AX1vyJ9ReZlfHLvuhc7kaFaSqEOUU4XZNTAPKS2jkUhtSTcg0ELSen2moz1n6BqdDlXxf2E4EOZ/QQy5N4Dc3siTVsL8zb8N1oAekm1kmtmxLMlQulopcKlAGiQPoguzrdXw14AkWc7jHUy0VUd3YWNUZw6zMHi58KYxW1tWK3Z8oEv9R29/R371HW9IcSQbruP4QJgWvClU6x7ua4MO51KeWTcBHlJFe8+L9 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR02MB7555.namprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(366004)(346002)(396003)(136003)(376002)(451199015)(6666004)(52116002)(107886003)(2906002)(38100700002)(41300700001)(66946007)(83380400001)(2616005)(966005)(186003)(6486002)(8936002)(6512007)(316002)(66556008)(478600001)(54906003)(6506007)(4326008)(5660300002)(86362001)(66476007)(8676002)(38350700002)(26005)(15650500001)(36756003)(1076003)(14143004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: iUwourZk3TNztcVX00hgM95wMVbfc3TWbfhPBQmrB5WBhH03pCxuzPbmCjdqmiV+eRaerzkVioaTF56+mj+S/Chq19UE5bm61UkE8E21GpIyPNz79qNIznwvo7yjwoyuMvn1CSNqhhuTkEGFtlIi0yg1eRBx5AOrmk+N4unavE++ZDv1c8v/T6t1CuDeCG3l7WBtrb60KjoWFoo9phnNiekapxSepXE1Krl+9o8Wsze9tfRIzfBGIwwg3pBlKyPKR8IJG/7KomiE2waGDYNaIA+rCSS2zdnc4NUNiBV6mYC9Yive0Cn0SUIkfk2NIOvH93nZ3gBEvjKJZBv0DOaMsNf1fl4UqQvRADlx8EXyb4jRGWjfXoby+l/5NpSTH9mqTKqDFfMFJ8+CrE+vT7M6a25n4IwpaWy70QnDRcFQ4yHlPTrUDyNwQFec1FWS3eOk//uOWd9UPciyHmexbq9RuvKD39IvSbnT2nAuPuE32Dq5OLEExP0Zp7SROlINRJYXwIiogGB7TImHXndrQJbmdAvA2tX5ygziH1F/wUfWSx7BLC/W5/MFPg68GEwr5dU+rEpSDKGlhF6O72Urs4TyY4PlSVHuH7E/vcGPA1qRDMXW6FAe0dwo4ppRnClZX6NpCaz9R7J0FzKliL6pJA06a7uZjDNuInaGNb3Gxl+fhSu+4FLYb+Rku3NBUWpIyCkMUvohlUJhO0Ld1kDYoOOV3jrmHUhbRi9gTF7niLlOUS8Y/rmySNB9Mp0meZOIcvTHy1OUXoQE2ANEPtlJbdnmiEWyqq6XaQ7w72I/EwKi9i2bI6d3KOLHR7PjshcZwCjTEQZS4xemJr0O1kepUOxehcrcrlFZwQVzUtmEB71HUv3bHOGyaL46/DBTlK7+QDKcSYKj/cku59ipvjHE8bbRrzof9rDB9D7T/vTxWxbum93Ap60bddU78x4HsM6PUe39K721nG4L0ElmIyApoIzs3Eq5csDZAFaOTaLYc291Y7mArxIPcPtri5PDzcyZT8UUwzoVk54m0qxXKqGyin5CfLSM3ZEO30E2yT9WgjlVzPhCbsHv7GfF0CcYn1kP0kmkg9lAxrNrlSDwrO5g6lGdZ1JW2MmYXqM5w4xlxIbYlTOLbPI2XhF3fbhvs1t88cdCtDP/F/4o/TR/bvSrPp1ajnOAdjuCa60/XPUZDxf5UIkTLuStLgbU2CNxlhubdYnnPUTIxWrrf8P0qzMh3BPUZECD8rmi+N1AfAEkjaRxrpFRrj9ziuaI/fp1i6HCFDqxIV/EEFJXsmQpAu5/+KkmtM3f/EbLZ79vKC7uIxwG1SjkkvzII7nycbdi8os2iINwaKrlW3S+Euo8EQVUz0aan1vOYA82Qjy7ZEYNooQyCFMM+oXyF5xvpgs8YY4MU4Pd6lU7GUrvBlTr8dFdss29YP7GRhKnBJzILeR4YCnyvRkZziJcm7cnrmkHTQWTXdlrB3abxjBGolfYakVfLOmHM1v5/Dm70gjPvEcgx82a9Uizydznp0HwL411Uc8R0hYt1EToihw6AzqdicD9K51QnzSjsAeNOxooIJo54Et0ilZKUe0Md4CpL7h0U1xB2eZY8OjGYPf51X3P4FEv8coTdg== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3ccf8aa4-609d-4673-2b8d-08da9702a72e X-MS-Exchange-CrossTenant-AuthSource: CO6PR02MB7555.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2022 10:11:27.8446 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZEWpfHXnWjrt18bTSKKTZws//1NcTCG7OJO0vekn2/hrFWXWMaG4vu7YrqC4MB83fDEdBaBoVWS9LFQ05ikFd1vyS1fVv1TYWnJ//KFkesk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7370 X-Proofpoint-GUID: uSEve-juSIRLGfhtpS5pOeVU86bWeY-Y X-Proofpoint-ORIG-GUID: uSEve-juSIRLGfhtpS5pOeVU86bWeY-Y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_06,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Reason: safe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Define variables to track and throttle memory dirtying for every vcpu. dirty_count: Number of pages the vcpu has dirtied since its creation, while dirty logging is enabled. dirty_quota: Number of pages the vcpu is allowed to dirty. To dirty more, it needs to request more quota by exiting to userspace. Implement the flow for throttling based on dirty quota. i) Increment dirty_count for the vcpu whenever it dirties a page. ii) Exit to userspace whenever the dirty quota is exhausted (i.e. dirty count equals/exceeds dirty quota) to request more dirty quota. Suggested-by: Shaju Abraham Suggested-by: Manish Mishra Co-developed-by: Anurag Madnawat Signed-off-by: Anurag Madnawat Signed-off-by: Shivam Kumar --- Documentation/virt/kvm/api.rst | 35 ++++++++++++++++++++++++++++++++++ include/linux/kvm_host.h | 20 ++++++++++++++++++- include/linux/kvm_types.h | 1 + include/uapi/linux/kvm.h | 12 ++++++++++++ virt/kvm/kvm_main.c | 26 ++++++++++++++++++++++--- 5 files changed, 90 insertions(+), 4 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index abd7c32126ce..97030a6a35b4 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6614,6 +6614,26 @@ array field represents return values. The userspace should update the return values of SBI call before resuming the VCPU. For more details on RISC-V SBI spec refer, https://github.com/riscv/riscv-sbi-doc. +:: + + /* KVM_EXIT_DIRTY_QUOTA_EXHAUSTED */ + struct { + __u64 count; + __u64 quota; + } dirty_quota_exit; + +If exit reason is KVM_EXIT_DIRTY_QUOTA_EXHAUSTED, it indicates that the VCPU has +exhausted its dirty quota. The 'dirty_quota_exit' member of kvm_run structure +makes the following information available to the userspace: + count: the current count of pages dirtied by the VCPU, can be + skewed based on the size of the pages accessed by each vCPU. + quota: the observed dirty quota just before the exit to userspace. + +The userspace can design a strategy to allocate the overall scope of dirtying +for the VM among the vcpus. Based on the strategy and the current state of dirty +quota throttling, the userspace can make a decision to either update (increase) +the quota or to put the VCPU to sleep for some time. + :: /* KVM_EXIT_NOTIFY */ @@ -6668,6 +6688,21 @@ values in kvm_run even if the corresponding bit in kvm_dirty_regs is not set. :: + /* + * Number of pages the vCPU is allowed to have dirtied over its entire + * lifetime. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if the quota + * is reached/exceeded. + */ + __u64 dirty_quota; + +Please note that enforcing the quota is best effort, as the guest may dirty +multiple pages before KVM can recheck the quota. However, unless KVM is using +a hardware-based dirty ring buffer, e.g. Intel's Page Modification Logging, +KVM will detect quota exhaustion within a handful of dirtied pages. If a +hardware ring buffer is used, the overrun is bounded by the size of the buffer +(512 entries for PML). + +:: }; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index f4519d3689e1..9acb28635d94 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -151,12 +151,13 @@ static inline bool is_error_page(struct page *page) #define KVM_REQUEST_NO_ACTION BIT(10) /* * Architecture-independent vcpu->requests bit members - * Bits 4-7 are reserved for more arch-independent bits. + * Bits 5-7 are reserved for more arch-independent bits. */ #define KVM_REQ_TLB_FLUSH (0 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_VM_DEAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_UNBLOCK 2 #define KVM_REQ_UNHALT 3 +#define KVM_REQ_DIRTY_QUOTA_EXIT 4 #define KVM_REQUEST_ARCH_BASE 8 /* @@ -380,6 +381,8 @@ struct kvm_vcpu { */ struct kvm_memory_slot *last_used_slot; u64 last_used_slot_gen; + + u64 dirty_quota; }; /* @@ -542,6 +545,21 @@ static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) return cmpxchg(&vcpu->mode, IN_GUEST_MODE, EXITING_GUEST_MODE); } +static inline int kvm_vcpu_check_dirty_quota(struct kvm_vcpu *vcpu) +{ + struct kvm_run *run = vcpu->run; + u64 dirty_quota = READ_ONCE(run->dirty_quota); + u64 pages_dirtied = vcpu->stat.generic.pages_dirtied; + + if (!dirty_quota || (pages_dirtied < dirty_quota)) + return 1; + + run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED; + run->dirty_quota_exit.count = pages_dirtied; + run->dirty_quota_exit.quota = dirty_quota; + return 0; +} + /* * Some of the bitops functions do not support too long bitmaps. * This number must be determined not to exceed such limits. diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h index 3ca3db020e0e..263a588f3cd3 100644 --- a/include/linux/kvm_types.h +++ b/include/linux/kvm_types.h @@ -118,6 +118,7 @@ struct kvm_vcpu_stat_generic { u64 halt_poll_fail_hist[HALT_POLL_HIST_COUNT]; u64 halt_wait_hist[HALT_POLL_HIST_COUNT]; u64 blocking; + u64 pages_dirtied; }; #define KVM_STATS_NAME_SIZE 48 diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index eed0315a77a6..4c4a65b0f0a5 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -272,6 +272,7 @@ struct kvm_xen_exit { #define KVM_EXIT_RISCV_SBI 35 #define KVM_EXIT_RISCV_CSR 36 #define KVM_EXIT_NOTIFY 37 +#define KVM_EXIT_DIRTY_QUOTA_EXHAUSTED 38 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -510,6 +511,11 @@ struct kvm_run { #define KVM_NOTIFY_CONTEXT_INVALID (1 << 0) __u32 flags; } notify; + /* KVM_EXIT_DIRTY_QUOTA_EXHAUSTED */ + struct { + __u64 count; + __u64 quota; + } dirty_quota_exit; /* Fix the size of the union. */ char padding[256]; }; @@ -531,6 +537,12 @@ struct kvm_run { struct kvm_sync_regs regs; char padding[SYNC_REGS_SIZE_BYTES]; } s; + /* + * Number of pages the vCPU is allowed to have dirtied over its entire + * lifetime. KVM_RUN exits with KVM_EXIT_DIRTY_QUOTA_EXHAUSTED if the + * quota is reached/exceeded. + */ + __u64 dirty_quota; }; /* for KVM_REGISTER_COALESCED_MMIO / KVM_UNREGISTER_COALESCED_MMIO */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 584a5bab3af3..f315af50037d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3298,18 +3298,36 @@ int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len) } EXPORT_SYMBOL_GPL(kvm_clear_guest); +static void kvm_vcpu_is_dirty_quota_exhausted(struct kvm_vcpu *vcpu) +{ + u64 dirty_quota = READ_ONCE(vcpu->run->dirty_quota); + + if (!dirty_quota || (vcpu->stat.generic.pages_dirtied < dirty_quota)) + return; + + /* + * Snapshot the quota to report it to userspace. The dirty count will be + * captured when the request is processed. + */ + vcpu->dirty_quota = dirty_quota; + kvm_make_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu); +} + void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn) { struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); -#ifdef CONFIG_HAVE_KVM_DIRTY_RING if (WARN_ON_ONCE(!vcpu) || WARN_ON_ONCE(vcpu->kvm != kvm)) return; -#endif - if (memslot && kvm_slot_dirty_track_enabled(memslot)) { + if (!memslot) + return; + + WARN_ON_ONCE(!vcpu->stat.generic.pages_dirtied++); + + if (kvm_slot_dirty_track_enabled(memslot)) { unsigned long rel_gfn = gfn - memslot->base_gfn; u32 slot = (memslot->as_id << 16) | memslot->id; @@ -3318,6 +3336,8 @@ void mark_page_dirty_in_slot(struct kvm *kvm, slot, rel_gfn); else set_bit_le(rel_gfn, memslot->dirty_bitmap); + + kvm_vcpu_is_dirty_quota_exhausted(vcpu); } } EXPORT_SYMBOL_GPL(mark_page_dirty_in_slot); From patchwork Thu Sep 15 10:10:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivam Kumar X-Patchwork-Id: 12977217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D35B2C6FA86 for ; Thu, 15 Sep 2022 10:12:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229991AbiIOKMA (ORCPT ); Thu, 15 Sep 2022 06:12:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229972AbiIOKL5 (ORCPT ); Thu, 15 Sep 2022 06:11:57 -0400 Received: from mx0b-002c1b01.pphosted.com (mx0b-002c1b01.pphosted.com [148.163.155.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8806402D1 for ; Thu, 15 Sep 2022 03:11:56 -0700 (PDT) Received: from pps.filterd (m0127842.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28F2o3gl000455; Thu, 15 Sep 2022 03:11:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=proofpoint20171006; bh=FmlryYL2InGcck2u5Qwd/mhswalDsNcAqMgOctimGn4=; b=WKlZsEklgkMqz5RuvT68B/HDCEvwDz+oB2Iww9AcCOfq/tdgT4xWu4cQoa54HDSFJsyC YPJjkqJaOibMcysCTvJ9cdjGBR9Iqv/THpfpQoqOHC2VxmyMlKaXO+91tpjy1Q8GD20Z UX7mFM7smqHoU8OmzF/NSI8ViPBUwbSH/cpXhgOovCV8HAnQM9RzPHBEwBMh0bvwJ9Zr 9x16MTe1SnQFrALTqV1ObaUNiXAfXkG1831e9F4DCOjnNxWkaljL48++1fddfJSiHMZL VY4K1S+AN+LYmk0kQ4SkUJhz3ewGeYqiJFDSxQvOL+eoTddmHCAyQKdvBLeAnpi3eKpT 6Q== Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2170.outbound.protection.outlook.com [104.47.57.170]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 3jjy05cap8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 03:11:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BtUZPCsmECBu8HM3U3Lw1mwKzKoIWUVyNPJhruP0XJau5QOUZoLUGxuOJyErxVoZGgR5sAGFLkZx/8cIlNdUf4nAQ9D4W+y+6wlzliJR4yH+p6Hs14SXMn69CJAKq4tKigJLN4Lfd0IjZwOkvWTq8B/Q5xw4r3bdvepFnVMhzZbiyc4r3pPz4sSI2tD3+2uD+EvaR7iM0N5OPpjTpqDZKBxvN7of7+tqzqaMsxTp0XqO3fcf8NJPVL75Urn/Xpo87nBWPrgubk3m9kEpFL6TFZqnKDTxMi/Zk0f4ieN+ET4U/vGLV9BHhnqh7TF4+X+mlAyOuWmUpV4Y3BY069vkpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FmlryYL2InGcck2u5Qwd/mhswalDsNcAqMgOctimGn4=; b=g9i82DYhJDSpqgwmadCaYPbjAaqVvqMgVnS2a44cA1ger4KbNane8hgrKbIAfIGzmr0IlmXNHvvrUBIxzi3y/56HdbxT2eu1XxzLEWI9TUokVOti05i08JXMJYG97bCg6lsjGpg2YQJ87bsQRdpiTmslRgaE7dSum/lHi6cgCvDOWkoSP77JEc3Ubzz/nH+A6juvLjumGYHwt34+kTRRGGjrVnKuSUUH59lJSFWBp7BpLuFe3K9e5c+8lBhM4L2dzUIvJn2whHmNUm++OsnQlviOws0JGwLt2TuKACR+pwQVYmRUcNYHiL3OCLPodet/qul/0IVypi64J0oObcovTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Received: from CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) by SA0PR02MB7370.namprd02.prod.outlook.com (2603:10b6:806:e0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 15 Sep 2022 10:11:46 +0000 Received: from CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf]) by CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf%8]) with mapi id 15.20.5632.015; Thu, 15 Sep 2022 10:11:46 +0000 From: Shivam Kumar To: pbonzini@redhat.com, seanjc@google.com, maz@kernel.org, james.morse@arm.com, borntraeger@linux.ibm.com, david@redhat.com Cc: kvm@vger.kernel.org, Shivam Kumar , Shaju Abraham , Manish Mishra , Anurag Madnawat Subject: [PATCH v6 2/5] KVM: x86: Dirty quota-based throttling of vcpus Date: Thu, 15 Sep 2022 10:10:48 +0000 Message-Id: <20220915101049.187325-3-shivam.kumar1@nutanix.com> X-Mailer: git-send-email 2.22.3 In-Reply-To: <20220915101049.187325-1-shivam.kumar1@nutanix.com> References: <20220915101049.187325-1-shivam.kumar1@nutanix.com> X-ClientProxiedBy: SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) To CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR02MB7555:EE_|SA0PR02MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: cb3ce053-0ea3-4c9a-2460-08da9702b20a x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ctvrl8IdvRT3/pjEke0hhNLu3sUm8rbFyBbchLYQW5Tj4l7hPliuj1rBvjw/gHuVM9NE51vD/LZ8m9EFipOe275epUmK6GUrR/L1rqlybWjowVgyFZo5BQO//pPNOsFtdCp6YmrzGK1k8zs/MxBAfonxvqiqGBMQ0o2fgFcQ6nN79hLP2/bGEtKjHCxlPkwvFfo5YEK3vrgW5bHSbjEdNRqLbrEZJarH2unAGFTQeUVGnZue8TXhkFiszVVvBiRWXlMnozVtl72aO3y3xFND4iYBdfWd59xTVV32ejzcNFWVgLGbevcrfebPtqHSMsh3CgbIZeFtXjfLHfXlveKV9yJdn6YRCOIAltom2LvOMD/eO0ZNK+VTbkLXwAflnZY/T44ltDl6q+K8uTcn1UZKWx3VN5ahc+ljSZxHFNn3q3J1L58BEVjjIYSnR31j47AVmyjBsGPpKSOaeqQ0PepzmbURbpDBJUcSI4pkK4ZsvZfbvF+yV/DtMJEaE8+3tdXOZou+hjI1T23pWpBTl8vQCVtg92zRjbQVley8sg1V/W6BUrM7F7DqTrIN6Xv+qi32eDAkQ8pbD96vPyYsO0wwKtDFu8j5dlR7MvL+kN91GYgzlUWC7Ou77wvXEkkzr2SSFXhFI890gaj28dsV/xDgIHLRlNjc7gKiL5tONnGYDvC6tbikBp3DLvexlr48diqJNUr98GoopRe4sTMOR55c0cQkcpVHb5iJHZr95Vi1SkL6Hz2wVMp364ObxKkbIqeTMA9YlGiXzbbbbgMhLzjm+6BwX/1q0VSg4cB3IFbfSuw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR02MB7555.namprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(366004)(346002)(396003)(136003)(376002)(451199015)(52116002)(107886003)(2906002)(38100700002)(41300700001)(66946007)(83380400001)(2616005)(186003)(6486002)(8936002)(6512007)(316002)(66556008)(478600001)(54906003)(6506007)(4326008)(5660300002)(86362001)(66476007)(8676002)(38350700002)(26005)(15650500001)(36756003)(1076003)(14143004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PJaixZBQIp5VfXDXCWGisGMT4HFFsb6yghbaPJmu25fJp+YQx1AIqy1uAy7izB1GSJmgz9b/Wj50OK1W999LvBjF7yPVHMcQD4llGt8MpMS9HNLgbu2ZnY6L5A1KGEO1T2k3Pn0YEvdqGDc/f3eaetwKFTLHwDDCK67lVU4jzkgadReX+W3W1ZIL8KyLfs2Cl8a6pcuQUzUy5Q1oHc50mNsjTjT4l8yJ3ARUC0Z/LIZO/ws6Zdj1hQWjrgTyj2gz6oyIXXyeqPeppu5gDYs8do0daWOCQ2zQto+U/ztkt5BzXrb8vmsfzzKlsofvbmjS2zyToJEMWK1XCi3kau+oyP4mGK5JLKPwzawLF+lGMMVZ/dOicH/Qz4GcQ2+wryyf5jUPU3dWI5YyqEzd3F6i4sIOB42FcP49fQWZRW2iojdDjUv0smtghq/yQdICgT0hudKw253uumpsI9uI/ls6YVU/Z5oaGtPxzxullLkmOsDAMa93vgB0BwtNnnTUvtWM+xTepDfX24K+vure9urtHhDM44/Ajr76uzrXY/PoiZOmIo8ApHgS9dMFDKWQdII3Z8Q5Mh5xlqcrJK2A5bIooyZm5E70OrnS4OhfYTEk4vje1NPJ4m0NKX33MIpdfyL43HkjFHWcjmPiT7bploTZ86DcosKMNcHF6jvFgUdHnxwLSA5Huxk9fw6af3GDdg0dN3jg7wX59QBaP36cCQ2h7BYH2XMlra+k0h9ULSPk6+vLb5CqTKzy9+Y/0Zdph0AxmE7KswFisUQt3gBXCNv5quaCP3iHgDs6d6N57zEmT/4qVMAAPmuvI3X5hjjVFvlsdcNBfZT6sbKPgrQKdnjxtnR8kiTeJoj+P/GQ6HdX/RT0BgClw+f0KggNV85iCW7DnS07UXHgNe2OcHAARs4rhMjLpmoxSPzvSM8VqUEGrYra1lPml6oefmaxaBeMEUdnow4Z0TQqXKi4c9tpOiNCMjqHdof9XaRVgFEkqK6IqCELYEZ084S6YoNO5REXN4NuyQtooiTxPFEHsAamnH4VpwIG+Vm/oPSmWa7WBwEAP/VLLv8WdnDC0qoz1HAXqU9rii8DgiOMUCyQOfz9Kokzc+CcAa4wZ5SBXXHe/CLobJfH30v+joMlyG86WaMsZd81dCKx5gi65cMR8DY/SMrl+w7k6LfAPxA3xnbZFfq6dgGvtaLgFQkFz5jDIAK3iyXstDlAUf20sLAdfFsOCtLiXtboOL6554TZ2VUvN96/keoN9nNb6SWVZ95GIx3HbS1qo5XujGEWkKZkzT0V1rHtedYD1vTQ9ca2n5y22/gCFIklHjbw3FnUPGr0Iphi8CVq3AjKyET6tCUpomOReb14hDKTVpnvYBTy7hWhnIBklWSKhIrqP0V1I2LDCOSRruUEWW4tFPVR6+ojJu7MvK1XOosSHrWkqz88fMy+NLfMBg7sCJX0QNmvWpQdu2CFj9McatNmPQAlgkGCryFGWFgkDvHdMrLs2TcgZAt1ROt2R2DqRJwuMJC/k1E00s4078VD8MP5L6XOud8fKrS7Nwgfi5UJ57x3E2WRBDttD/esFpNE6KJH1h7UlxKlgUYByjqk77Xk/EpkMnMFEsvm4Ehr6A== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: cb3ce053-0ea3-4c9a-2460-08da9702b20a X-MS-Exchange-CrossTenant-AuthSource: CO6PR02MB7555.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2022 10:11:46.0021 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: DxDwACn3sJLnXE6qfzMFldtmLs0XBEllYkYEsTJN/bccL5dnuuCc4WnA9M52w314+Ke3aexmev2E8Kd3aaEi26KMthOdiZ5RwBstmT+fRNI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7370 X-Proofpoint-GUID: NGOiF_jMF0psvEzL9GUXwks-rHAfgWL9 X-Proofpoint-ORIG-GUID: NGOiF_jMF0psvEzL9GUXwks-rHAfgWL9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_06,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Reason: safe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Exit to userspace whenever the dirty quota is exhausted (i.e. dirty count equals/exceeds dirty quota) to request more dirty quota. Suggested-by: Shaju Abraham Suggested-by: Manish Mishra Co-developed-by: Anurag Madnawat Signed-off-by: Anurag Madnawat Signed-off-by: Shivam Kumar --- arch/x86/kvm/mmu/spte.c | 4 ++-- arch/x86/kvm/vmx/vmx.c | 3 +++ arch/x86/kvm/x86.c | 9 +++++++++ 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 2e08b2a45361..c0ed35abbf2d 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -228,9 +228,9 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, "spte = 0x%llx, level = %d, rsvd bits = 0x%llx", spte, level, get_rsvd_bits(&vcpu->arch.mmu->shadow_zero_check, spte, level)); - if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) { + if (spte & PT_WRITABLE_MASK) { /* Enforced by kvm_mmu_hugepage_adjust. */ - WARN_ON(level > PG_LEVEL_4K); + WARN_ON(level > PG_LEVEL_4K && kvm_slot_dirty_track_enabled(slot)); mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index c9b49a09e6b5..140a5511ed45 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5749,6 +5749,9 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) */ if (__xfer_to_guest_mode_work_pending()) return 1; + + if (!kvm_vcpu_check_dirty_quota(vcpu)) + return 0; } return 1; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 43a6a7efc6ec..8447e70b26f5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10379,6 +10379,15 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) r = 0; goto out; } + if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) { + struct kvm_run *run = vcpu->run; + + run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED; + run->dirty_quota_exit.count = vcpu->stat.generic.pages_dirtied; + run->dirty_quota_exit.quota = vcpu->dirty_quota; + r = 0; + goto out; + } /* * KVM_REQ_HV_STIMER has to be processed after From patchwork Thu Sep 15 10:10:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivam Kumar X-Patchwork-Id: 12977218 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67ABFECAAD3 for ; Thu, 15 Sep 2022 10:12:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230083AbiIOKMS (ORCPT ); Thu, 15 Sep 2022 06:12:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230033AbiIOKMQ (ORCPT ); Thu, 15 Sep 2022 06:12:16 -0400 Received: from mx0a-002c1b01.pphosted.com (mx0a-002c1b01.pphosted.com [148.163.151.68]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 450AB83049 for ; Thu, 15 Sep 2022 03:12:15 -0700 (PDT) Received: from pps.filterd (m0127837.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28F4AFSS008387; Thu, 15 Sep 2022 03:12:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=proofpoint20171006; bh=39Ppz8D5yUE2kLAAbJqLn+DP2snZCikuOwcRuOeO2W8=; b=i4oxhAeDXg7M2dGJkVghyweYtFUke7E7HXgon40FRHrmgazBvora6laB3V7K32+XVd/j hkq813YIH01+5qNqb+vmEVVnLL40CD5Y7eq940AO7Hkjhvm47ZbzUsxJ7+R5b29xLF88 6MWkvmNLeCgGJDVj8zCEXwLVoomI8QGSqD/I+KN7TFw2FXTL01mhrALWMOZgjvX25+cZ 7cdtamVnXEpAoPITyMOTd9JFzpUzd9Ze9BEK47A9sm1UCOgvKhcXadGDmwFBzBr4aRV0 BFwuHmQ/nE+bCCMnbeBsDzK/ftNmSLaRg2YzB8Tnn/vIB/Sjw0nDGWgospB+0cgf+5Wa vg== Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2108.outbound.protection.outlook.com [104.47.55.108]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 3jjy0pv8bp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 03:12:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IouGVwfwPPbKkRs9KBSSjlQc8HL8qgUZBXLoiFrTFdh+bbq/Zd9ymI7p3Qq4Lf8WCZEM458nSyY+UV3Ia7G8hX/TufIgr2YKEr+L5e81dEzKC7XfE0OCjuTRBzKfI2UxK5jdJyPp+W4ZL/ID85apKHDoDXSW8xs0jsmlXtck9dh1oJbbx66vRb+1bmDfEsijuAl80xET6XsTwKIm7Ay5Yqeo1YUaqkbrb78EGavv18DdOXckqiJm5OrRnMFiNCHJNTgC+SKhuiar251sSlO2nveVIK6HhApXGTJjQ+B7Fc/xwmVQn8jsNm1ZcjpRmCXOrRgPSflKxzPo03Q9onjyTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=39Ppz8D5yUE2kLAAbJqLn+DP2snZCikuOwcRuOeO2W8=; b=AbrfXZrc7pgfRbExCZ3CdOse+MLub2x5XCYI4Jwu3T8cBq/4smf3R7mPFPYbPTQNKqqPijroijHItLMvQhf0ER0E8nB7N6ggNBWLdKNUn0+NXWr6zdTK0bFB0k195v/V4b/s/IoaRTRMGgfowi0MAp3GLFBLaVF5RNJncEG9VOQI+gTH3IVq69VxiNSQ9nWdX+e7c48/JhLhcMxBhwSpLRfnCvXg5lqj2bXlV8GGPiriCPMJqQ2PvUPAoSfQwTc2EgwwQzZlsXVzXMvb+BXYgi2n7kGJxsu59XP/dNccVx5kfQ6PNNf+tD6dq3a+p/3IdeVRab0WbviNbddpC90u/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Received: from CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) by SA0PR02MB7370.namprd02.prod.outlook.com (2603:10b6:806:e0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 15 Sep 2022 10:12:03 +0000 Received: from CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf]) by CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf%8]) with mapi id 15.20.5632.015; Thu, 15 Sep 2022 10:12:03 +0000 From: Shivam Kumar To: pbonzini@redhat.com, seanjc@google.com, maz@kernel.org, james.morse@arm.com, borntraeger@linux.ibm.com, david@redhat.com Cc: kvm@vger.kernel.org, Shivam Kumar , Shaju Abraham , Manish Mishra , Anurag Madnawat Subject: [PATCH v6 3/5] KVM: arm64: Dirty quota-based throttling of vcpus Date: Thu, 15 Sep 2022 10:10:50 +0000 Message-Id: <20220915101049.187325-4-shivam.kumar1@nutanix.com> X-Mailer: git-send-email 2.22.3 In-Reply-To: <20220915101049.187325-1-shivam.kumar1@nutanix.com> References: <20220915101049.187325-1-shivam.kumar1@nutanix.com> X-ClientProxiedBy: SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) To CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR02MB7555:EE_|SA0PR02MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: e8d2f4df-74a1-4eb8-c4ee-08da9702bc06 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wY2njLG2+b8V3rkYhos9yCRdTUJkZGQs0BMbnof/HIBTJjmuvKzD9mpzoKRYopexT62RI9lhPlxyeLUCgazVjCNujpTuQ/1i3i89Qm+PZMzSNUtE4HmWBsPde+VPuM2w6Gnzb67UNEdwkUXxK62UcHZ4unVkDNiW6DdLWWf4IziUAiLv4CDqrdqM2QezGJ4ba4IkpqGy0XdnyisHCzPMJR1cy/f17lczdYh9wa6n+UP/OC2/YP2iL0QKDAL5znuUnaw29jFLuD0fvAO4QBkaTHrM0ivLB4kI6WMY39WS1BQ9LKRA5BmD3lbXOhAVi6/VgJshGZK3u6dL9Xjfirvxiu7WMf2RhSp56NiNAuuC38+zudwOuw8wzzUbpVKhGf+0kXnvoe83ixHHj56wwZgK6FXLwyjGkX0RVkUcOL806YqyBVb5zzikYiSmZCXVqpZXAMNiy4iRy3NUFZ1nDE6CzFt21dOIkShA2Znb83w0+snQBKqijJP0IKL/1WPbRwlpa3vi5TmGjxFEMCEQF3u4IlHNKxtCbG7/7c0Srta9qAITUtCjQsfEV0lnEwILJDyAjCvC93JNiIDRoL8RXy4g5n2R2ux0ZrH7+3uCMM4HhXNqqpf7iFFg8YLuMhDZ+7YGkm5lBsusUEVkrPpwzBxZUZgC3t+3M1Kxiq8QFfeURtu6J6sIZokuRVZ2U6VBykNZ3Sfi9jcm9tSbOhUYkfnN88vfye7PllGMm6JO6UC1+mEGi6Dxi0vQg8Sbp8EsBzMjihvIW/j64fYoR8mS9Sy90ZDyEcjHX52GcwjrVG421x0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR02MB7555.namprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(366004)(346002)(396003)(136003)(376002)(451199015)(6666004)(52116002)(107886003)(2906002)(38100700002)(41300700001)(66946007)(83380400001)(2616005)(186003)(6486002)(8936002)(6512007)(316002)(66556008)(478600001)(54906003)(6506007)(4326008)(5660300002)(86362001)(66476007)(8676002)(38350700002)(26005)(15650500001)(36756003)(1076003)(14143004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AZUDKkU0MTdqod1wTvihQIDo94wRLuczuziMHV4zTZClFb1m0VVqTS+A3bgEyfYZx+HYYKwfv/JBvKSd4qk9uuBDvf70cxXCz4G52SLTFxHgIg1qpuHgh2BSv3hMxwqOYXgPCBwpRemmZb0mjYbBrlYXSFMClyOShEIWdEUdGrDNI/9TIBtesj+l8DuJgFsetsnQL7blIZN7l5O21CcQ/nAe92dMOkAuCEOn9MVFerhOlUA5lHR6qNtfpc6WM48QTZQH3JWY2KwX8kc4yBO3IdB8NxZPr4nPbctmi10WHfohsINech/GOSEa1As6wx87gLQdasQyYTC0csyollevWA+KJKazH4Q07regOgrxzUJO1+RPqCaARNxP8pMsJlzl8UV6uJGcwiN1fvm6rbpmcclt3yzsF0fEh+Mx9LcXb6cC79GqIz9xCtSVbxLVHPD5Mjc9qX13YDdlB88hunzSBOMc5lQ4VwrvLj4Dla4KQaJMc+I/oewUz3pV2p0ZCXkT1aSH95IppNifl0f10/A3YN5kZQCeLmBFu2jOiVgNnI6I6cHUB/PdZiASLC4bAuMIVPCEeZTtVKWF4oJ0WrihiyCijpd/QsOz/DYMqEb2InANKVL3baNVZ+JhqwYFMJmqjpp8SnPq/8e8fg0BdJsAhmWxmxGR3vt8voGi19S085SngjwWuUkoed1tOaP/4rjFIERpdab8nVjd12qKbD18PTg0wq02+77TTljs+fF1q2CtvBybk8GPmS0jTNRclZkkvAoiwmULCRnHuLkKUK8VzP7J8iX/badSRJdkgCSjDoXbCFV3OmNW1UBWL5EDGlIj/Ml5qKHUVE6108YgED8u8ClS+Vc/ATlnfC0qp4QnM3l8SaBDbIUo89WTbSSPwj3W80P6kh2Gh2eIk1jpEU+6ONJKr3yvYSjmKjr/L13MVtKvQZdlknoENQUlFAAEkSO9R/XtiJSIz/hY7hfbdm7gLHbLx/nHwGlAVbQv3Ou5GSX95bN7PjWj5xaWHtYKsYqfabp5logAlJ4X5YanO/k7bnJ5xcvVFH8p2MRRHIwBrP9bGfNc4Lja140eaDE46UO+F6jR4bM3/KwBYuUroHaFeN7QgQKN8PewpYUd6YuU+PYwpuq6Z3MJa8HRRgyW1tK8LCMFSTLfRajreKFyy+boowtcrfTwOxFjD6TEVdbnkXH5yxM7smEZpU8Gt/4lTUtgX46VZc4u5KW9CGsEkRASm7pd2J8aW27RtsOAWW+EyafATL0n8z6E0ST3LT7mrmljJcZReri5auJetbyaGkmAIKdaLghqblefxX5PUqVdB6ULmGttbOTqVNxhKEar/Ys6DwBnW/l35PFDAmC0kOR/+UCvDVKYqNO24WmDLAczDqRt+gzIe6pF+/7hTUv3mFRkT4bNlLH6DW6FPKp152DLGnQOqJPlbS7QMzKp4CjBHX/XhZALDQqC+6W1DwpeAOOyAEkDUVPAZIFglYxR0pQiw4VKs1lHtTO0ovUwxrhPVEMIRUMBrEooBJi0mGYzBPU6Vr4vf8AA7bmUt9ZEIFLyDbb0umh+gcgX41K38DrBvrrrwSOlCi9qdgq5UDtzBf6cFrt3mGvjyqaokOWX4KGFew== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8d2f4df-74a1-4eb8-c4ee-08da9702bc06 X-MS-Exchange-CrossTenant-AuthSource: CO6PR02MB7555.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2022 10:12:02.8645 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7hQNQYMTsdRMqywb/3qiZwWwh07+SUNF9E1J3cDbLD97LvR0F++Yohw7u7qr8nIDVF2zU0UUTj+m923CdSAgON3g6jFdt4JUBPbiZkMj9rQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7370 X-Proofpoint-GUID: thf4L5_Q65LvpIpMgoJfwrWStKrYt0P6 X-Proofpoint-ORIG-GUID: thf4L5_Q65LvpIpMgoJfwrWStKrYt0P6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_06,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Reason: safe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Exit to userspace whenever the dirty quota is exhausted (i.e. dirty count equals/exceeds dirty quota) to request more dirty quota. Suggested-by: Shaju Abraham Suggested-by: Manish Mishra Co-developed-by: Anurag Madnawat Signed-off-by: Anurag Madnawat Signed-off-by: Shivam Kumar --- arch/arm64/kvm/arm.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 2ff0ef62abad..ff17d14aa5b0 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -747,6 +747,15 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) return kvm_vcpu_suspend(vcpu); + + if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) { + struct kvm_run *run = vcpu->run; + + run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED; + run->dirty_quota_exit.count = vcpu->stat.generic.pages_dirtied; + run->dirty_quota_exit.quota = vcpu->dirty_quota; + return 0; + } } return 1; From patchwork Thu Sep 15 10:10:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivam Kumar X-Patchwork-Id: 12977219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BED5FECAAA1 for ; Thu, 15 Sep 2022 10:12:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229588AbiIOKMl (ORCPT ); Thu, 15 Sep 2022 06:12:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229693AbiIOKMj (ORCPT ); Thu, 15 Sep 2022 06:12:39 -0400 Received: from mx0b-002c1b01.pphosted.com (mx0b-002c1b01.pphosted.com [148.163.155.12]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5F2B7B7BB for ; Thu, 15 Sep 2022 03:12:38 -0700 (PDT) Received: from pps.filterd (m0127844.ppops.net [127.0.0.1]) by mx0b-002c1b01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28F3Ld3d022908; Thu, 15 Sep 2022 03:12:28 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=proofpoint20171006; bh=YyfVSbyTRijCayA0e318Az45XK6QFnxj79/isEOk73k=; b=JNQnGzTiu5w31auSp5XsXnXQhb9Q5KFM9GcFuFkfbHnceiJ41tyqsm1X7V9SrKDO1lsP 2Zzyx3QdWmlc5sXHAWwSp7Mo7TAQ70C+8xvpq8wkMYRvXA1adxF49vE3DDpRp0yrT0JZ AtOfAuTINw9r1lL7AR0szIjGrA0CxCEVkEd3J6LERVDLqX8J2MdyPyMBkEEZlaQLk62V BMvIsxE80WW942LwRpXSGzKlzzMAztllbHKaqmHm2232YJI2l9JtyqNc3lfvibB52pRT RMqCKDSIxWry/DMFqo5p9BF8WNmq32NeICsWugp6u5tm4ltNzaw+zUKaR6fi7HAm1NrQ Sw== Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2107.outbound.protection.outlook.com [104.47.55.107]) by mx0b-002c1b01.pphosted.com (PPS) with ESMTPS id 3jjy074aks-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 03:12:28 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PSVww/Q0wHlHlJxBzPCTfvrGtqmh270OVybhBsjzHmyQLdZgfREHnULgNP0koNUgyUe0R2YnaFK0gDb+3gBY3qyNXARMqjrKJPOtC4HInXCX+HibwQMQw7GkwFUh2lRxw6xLdstOXAbToGe8V/kiKJQGwfG5TdXVKC6nh2PudvQqolj0pYxKC5OUVDTos40pbRiFezKkYwOthbIofnChPi/gzaObqPrfbSoPKyPYFwPrm7JJ97Dc9ZFiyxefsRsAnx5iApdGMJonoEWBy05AeqzG/Yiq8XF/P2Cz4h4BZVGlF749T6iKK2RLk6Axfbuq+CZC7+WWI2EkjGaECjI6nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YyfVSbyTRijCayA0e318Az45XK6QFnxj79/isEOk73k=; b=XfrtGFTytVaTBTBGJzWK2FFRbAB5R3qZD7bHnBasUfpwDD34+FJ9xmI67k4LarNCWpi1hPDVewVq+qObFIWBVbUallD7Fo7ErosiD6nxEUHcLSm4/3O/HC2+IFJ/YlFcEDGcEc3EcvvdqWWOLHPphl0nL3IgaRVyOrhsGXQ2GE8EPVRAmoVb05M++kLvkr90H6KTL2kMr6pHEAzHYFaMRGACgT3/rsbuXA6fYz0D9y+lhgKs/LmwvLHe84KnVHWaiiBatkfxbY4reSyFAJTifQjoOlQc3I1DHD57ahw3igMS2J8aGcXjma/UUYoLkEUW14VgKluJkB9+wChouKjizw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Received: from CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) by SA0PR02MB7370.namprd02.prod.outlook.com (2603:10b6:806:e0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 15 Sep 2022 10:12:26 +0000 Received: from CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf]) by CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf%8]) with mapi id 15.20.5632.015; Thu, 15 Sep 2022 10:12:26 +0000 From: Shivam Kumar To: pbonzini@redhat.com, seanjc@google.com, maz@kernel.org, james.morse@arm.com, borntraeger@linux.ibm.com, david@redhat.com Cc: kvm@vger.kernel.org, Shivam Kumar , Shaju Abraham , Manish Mishra , Anurag Madnawat Subject: [PATCH v6 4/5] KVM: s390x: Dirty quota-based throttling of vcpus Date: Thu, 15 Sep 2022 10:10:52 +0000 Message-Id: <20220915101049.187325-5-shivam.kumar1@nutanix.com> X-Mailer: git-send-email 2.22.3 In-Reply-To: <20220915101049.187325-1-shivam.kumar1@nutanix.com> References: <20220915101049.187325-1-shivam.kumar1@nutanix.com> X-ClientProxiedBy: SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) To CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR02MB7555:EE_|SA0PR02MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: d9a95b7c-3c5c-4432-4510-08da9702c9c8 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sof20DH9WIgNR7HNfydNEZ80XzCaE0gkIJgiVeDCtZWnpJjLJRFPDZUGV7zl9KwmGUVrWdC6IY/NanjL+C7FhqcYZ4IdrIfKb2zLL/J5FHp17xOZ3EofZzq2rXw9la+38AAem3n1DMJMXQhDVA3sUUjPnWEtZBgzdS+QPKQ79i5Vuw1FKUYsJl7mz0EvCgzMkG6795lbrrsfnNL+DZImDMsEEQyHp2KRGTjY5EbkAFV628isrr7fERLTB52oI5LpwRLZmYUUDwPJXfPEdFMieQen1DrutV9ClDfPJIvHsBEDvCBs4vFs/bzkopaCpPaZ9gX4r3rIPcue+sWw+E4aGFzENBrb4hSRvUtVFtMx0WrSzbf+xukYIY8nqc/HXWz5iAgtWgL+ocwaZpBrfUxKVRaUKmFOIEiFw42dIPaQmKZ3z+lgmcW9BCx47LfbTGDo+gUI68IusDLUbksml1P2xKB3cvoaCxWhj9W/TmrBH+UaoY40+ZTLLVzVZhq0TbVixXPdLb+uXiPJVf56paAXbE3esnNweQCE80SuOkuTBT6V491ZNYtikdstekbxVHFQQHwu5GwhrHQX02UNM1lql1TbrSDwLwNa1uEeDLP0tYsRQncWnXJN/09gy80iqAVhNod/x9h58NSmLaiqTuZCCRFl31jZuhzmzyY67UoQH4sY44ksT/aup1VAlu4sVeCna4yykjD4u70BxgSkFhDV7HI06zMXcDqp9eGTBENPOrtmddVT2JB9iq3H1f/AKa18akVC8L2xdE4E8VdKBz0zstQLBEsHgrhFXE6p2+h6nq0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR02MB7555.namprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(366004)(346002)(396003)(136003)(376002)(451199015)(6666004)(52116002)(107886003)(2906002)(38100700002)(41300700001)(66946007)(83380400001)(2616005)(186003)(6486002)(8936002)(6512007)(316002)(66556008)(478600001)(54906003)(6506007)(4326008)(5660300002)(86362001)(66476007)(8676002)(38350700002)(26005)(15650500001)(36756003)(1076003)(14143004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Ghki8xvAUGycNTR9P1Ch2fxi7Lz9BEb7/R6Va8VAqPIc9/Z2ZXC+P59zBqxFxeCDPd6UEhfG+p2YmXQAWrdZv7hz+YnvsQwv+F+NqQYng6xSBhIvQqyCmLh5miMVU7ySDhbrLxTfe+7WucBqQGondKK3TfAlNWZ5inW6vn2OZmGWCmMQc16j7L9TvBgm2dhtzOd6mp3Z+j3LL5GuEGtuaJdSkmCEbxSahj1TqAfU6evIX3BQ+4wLyKwYqO29iddoLPIeQbGnHhRE9E+sOcqpVOJeH2spskHLBp8zqGIwIAMcI8I6Lc1YMx84IR/8huDfceTYjgZyw3snGCj1INg0Cxf1Zre9/qgBX8NGOuJCaEHcKBWjIIX28ufG7zlxZv/EZeQeXjFnetNGhMVSEuqveWYp6Zx5+s47P8nRrkHfScJaPm1RiSegr6djYinqOHYDeTAIDOdYdVqVNRFz3/BH7MqcyR0WiXFnQFCQ7pBeCH0soR+qFplWgM7DoNjoEabmcaJl7lhwwwLLrZc5S/Fl8J7d6zjY4LwkGNRgT+lfRpEbIypHF/1ZkBpS5325PXizdwUQwAK5CI3wGaGcBLXQXr2lxJY3KMuQRYcW4Ij+bG12DK18ZRNOQFeBuPirXS3d6jHAY0W7ZKfd4xJXvhdc/Ol3IFF/H7ZnEHVyGZwWLoNrgzegTKO3fhRhCYzsON076vUdgnE4AtlcQXEkgB/YHhC568ZRBTkardcThcEmqnlV3LS1MAy2o9AnhhI+afn8uBgXot9adPGdnq+J7XjWpP7r30s/ivrRKmZoNObiXQWYHAx7fdYbh/7tX6XbkzhRu7l/S3X9/VxwhcVe2HjNF0ceU73fZj6WGil+/ZYwR6p/NbIGREIpx4ZvnRu44X7Q6O3Za+Cic8t6tzC2V3zaZkQKH+nvLdXXHPDgam6idkMFALxsw1iCn+SibeLRxuroZ59ZebU2B+8S+85EIF6cLrfsCBd9vxCXjlK8K53HTq+LRpJxYQ51Np9wtJvAQbInd5PHmmbmeLE3yyKeWhOr+Sl7enuijN5mf7e4fjY17PAs7QCo6Se8R+w3OzoeuUq/qOTaheEvOZ3SndehgrIzKDXnXPJpG3dVWHxuXT35R/nFLwZ9AOEExh66iIssbDCCvGdljHf/iGjZL3Tj0QE3bxqEfnj8Ohu0rA0NXbxsfj+SNFsqR8ybAlNFif/DoBTjGQ+pu+PTaXtBGfvs87F5wxIkWkmI8w8N99HopY1JhlFL6TPK1O2vR+8I+zE4JJ6QWurIvpNiPtHV2S540bhnuVgou5rJWx+mzTYPu6y5UysgRFiS78up6Po3pmnZCVX7mgoBA2eznGY4Y0EpexXpHRGtP6cst8IFRl15FJ0JerJb15Wn4Ono79Enc37pGm4K10i6eAUtQLhAP6Sg8ff4tgemW7gAnngbQLZ3vEsLUClIASSXRNROLaza+oU0Ts3KJ+x9UdlklpSi0B8oY8zZtspvZfJQw5AOxadzh2S2/Bmk2n/Y427fnJ86pHibqtF8WPSmrMhObVRnZRjHYc0nPByLTWKTBwfLpT70bBBCLVz8vQ3aA+jdhxtqSUIRLY5hYjMM2L62rXQ513n3UijTDw== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: d9a95b7c-3c5c-4432-4510-08da9702c9c8 X-MS-Exchange-CrossTenant-AuthSource: CO6PR02MB7555.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2022 10:12:26.0076 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kEwzN03BQHprzab+9HFK2fqTJPttyA3/air+3tAQJ8/swbWLl5HdIJRZf3y0ut/Gw8mW/OJ23mY3JcifNG/zeVRSKicLe5svtLUjJaV6OEM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7370 X-Proofpoint-GUID: u7OYYwh613PQFqLZd6ZacILB7bKqZTHa X-Proofpoint-ORIG-GUID: u7OYYwh613PQFqLZd6ZacILB7bKqZTHa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_06,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Reason: safe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Exit to userspace whenever the dirty quota is exhausted (i.e. dirty count equals/exceeds dirty quota) to request more dirty quota. Suggested-by: Shaju Abraham Suggested-by: Manish Mishra Co-developed-by: Anurag Madnawat Signed-off-by: Anurag Madnawat Signed-off-by: Shivam Kumar --- arch/s390/kvm/kvm-s390.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index edfd4bbd0cba..2fe2933a7064 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -4343,6 +4343,15 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) goto retry; } + if (kvm_check_request(KVM_REQ_DIRTY_QUOTA_EXIT, vcpu)) { + struct kvm_run *run = vcpu->run; + + run->exit_reason = KVM_EXIT_DIRTY_QUOTA_EXHAUSTED; + run->dirty_quota_exit.count = vcpu->stat.generic.pages_dirtied; + run->dirty_quota_exit.quota = vcpu->dirty_quota; + return 1; + } + /* nothing to do, just clear the request */ kvm_clear_request(KVM_REQ_UNHALT, vcpu); /* we left the vsie handler, nothing to do, just clear the request */ From patchwork Thu Sep 15 10:10:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shivam Kumar X-Patchwork-Id: 12977220 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFB2CECAAD3 for ; Thu, 15 Sep 2022 10:13:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229880AbiIOKNI (ORCPT ); Thu, 15 Sep 2022 06:13:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229612AbiIOKNG (ORCPT ); Thu, 15 Sep 2022 06:13:06 -0400 Received: from mx0a-002c1b01.pphosted.com (mx0a-002c1b01.pphosted.com [148.163.151.68]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BAE07B7BB for ; Thu, 15 Sep 2022 03:13:05 -0700 (PDT) Received: from pps.filterd (m0127838.ppops.net [127.0.0.1]) by mx0a-002c1b01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28F4R7cV016296; Thu, 15 Sep 2022 03:12:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nutanix.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=proofpoint20171006; bh=2ruFcqmhurpAOnfoBpr0QPVZrCJT/wNpd0j3CjSIO1k=; b=dEIsNvqbAod7CHh2KrCj5C1EiwYNcEZzeahBZIiZhFd/TWo3H04LOx4QIEMJa8J6xXZX xoLEVt1b3Mk+yRFjDnEC3sI3aHYpGlfo8wLgJMtuQzQjU6ooFl+Mju7fL99FetsT8Rnz cQE7FHmrYLIP9roBdIZQu1pgb5BJCygfGV8XmSY4ZmFN8OoaJM3PbFFVggW6M8Q2ph/u 8jWOzjUyU7PWfDdv5FNa0oIUSdnakjwpvRzkg38S+szvtizaLCuAQtR03hf/Pb7ldltY e22oMTYwQehIbpPepOGV1NRSapV9la/gbdkMvPYlFeSwPBGphoxwH2pVFAugFWwUrJd9 CA== Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2105.outbound.protection.outlook.com [104.47.55.105]) by mx0a-002c1b01.pphosted.com (PPS) with ESMTPS id 3jjy0tc8df-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 03:12:53 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W7HLfrCbx1HEvCPL3HYckA82PJ9XVJLu+EqIMkGHjyi8PZWozYr+oO7Qe5mgJDIkoeqgZZw07nEUv+2E5aVu2zP1DOkdrF8wxvmnO4+NQU5NCsM7s8lOcUNW0dUcZgLKraHBBw9RcqbLdnIrOd9qYIupOTVfIqNoEheEl6zyCtRxnZoEmeIwEtW/ECWeSNd+ola2/wBExZULolGXTvuZBJBlctiDa3HXfZTgUZjopMRkR7rJxF1KG2Bvfdf5/xd8O4Pg4L640sP1nlLJH4tPVRh1WefNkiTlYc0rHdaez1G+jy1YvZ4J0K/Jhxt7bcy5vagk5cnJo3XyDtgtcUMnkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2ruFcqmhurpAOnfoBpr0QPVZrCJT/wNpd0j3CjSIO1k=; b=Vgnt/DaJQCR2vghNBkYjlGBZnjRNEbqEuWq6J64imK+jm637SbQflxdWbXJ25j4e8/FMPKsJ7gLvmieGiqpdxB1/14++oTyT8kW6QW2nSJRpQJ9iqS5fFU/zDREGuo+IYDhM4YWTdqsGXKWelhnE3ZB7COYDPUMvaeFL8jZMuAz1x7tT9GWPbw31vAlNzu39HaBt4LBw8CMQVYMwsULw/CzjqcXjtQxhiXlVtO4/VlikLG63yFPtnq4vmq13dNDE46hK9Xzj+zoXWc1C16cj1hAsXM2pq8ocmSWrjVaU3Hes4yWvomcHeJri6/2U0rykiLiV8PW5aln31/4NSufO5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nutanix.com; dmarc=pass action=none header.from=nutanix.com; dkim=pass header.d=nutanix.com; arc=none Received: from CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) by SA0PR02MB7370.namprd02.prod.outlook.com (2603:10b6:806:e0::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19; Thu, 15 Sep 2022 10:12:52 +0000 Received: from CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf]) by CO6PR02MB7555.namprd02.prod.outlook.com ([fe80::bcdf:6d6f:e6e:b9cf%8]) with mapi id 15.20.5632.015; Thu, 15 Sep 2022 10:12:52 +0000 From: Shivam Kumar To: pbonzini@redhat.com, seanjc@google.com, maz@kernel.org, james.morse@arm.com, borntraeger@linux.ibm.com, david@redhat.com Cc: kvm@vger.kernel.org, Shivam Kumar , Shaju Abraham , Manish Mishra , Anurag Madnawat Subject: [PATCH v6 5/5] KVM: selftests: Add selftests for dirty quota throttling Date: Thu, 15 Sep 2022 10:10:54 +0000 Message-Id: <20220915101049.187325-6-shivam.kumar1@nutanix.com> X-Mailer: git-send-email 2.22.3 In-Reply-To: <20220915101049.187325-1-shivam.kumar1@nutanix.com> References: <20220915101049.187325-1-shivam.kumar1@nutanix.com> X-ClientProxiedBy: SJ0PR03CA0235.namprd03.prod.outlook.com (2603:10b6:a03:39f::30) To CO6PR02MB7555.namprd02.prod.outlook.com (2603:10b6:303:b3::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO6PR02MB7555:EE_|SA0PR02MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: c5835724-b54c-4586-b218-08da9702d970 x-proofpoint-crosstenant: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tzYj5TMrB2Wv/HLNOhdewnk6CQ2u6CNmLmEZJdyeV7rorQhruhHZ9bxMP/oxKisdFl0R/r29TNWR9GvhnwRv0j5lYRVAjGo30BZbsymo+RWoOGG1nBkPeEdsWT150uYns4pxs5Iwc8hdUxVriTrl90C83t71KEfZ3BM1MA2MvENpJGuB4icLxHxLIIauJIw6Zx0hoyTFiMNMh9gU1p0hnC7MkXszjl9JGwEllOrH1Ch8pVVFZcxcXI2YAnWrqw8qw66hIjFVcy7haroCrOBJLj/BB+UjgzpqqIqRiXu+Ac98i2mTeKLoPvhU3lN4lTsOtgimqPozuXCv9XxABnxWoGjcZZoGKRN85n4sI+tYAQZLhrsTbS3fr2wI1esWcmaYMhoysfSCUsbSAUC3iXjp/7E1L48HhSayKcak4nuUOPLUE39qQvAxlZTW3qLqohYvMEvvKzQYRBuOFjl4vkibsuxb+/69f6ZpDdjE98qT2JsVAFbeFLgVurquWHTPOPg913VKIhR3xgMmvjOprlN6cTywErOWOjdeObQ7vtl8X2R+4fSrf3EtoeOmMCUlkH/Bwut386iyDCUSdS/ZY5/HITakz6O0tOA0ImI2FVIDrZVr/mlrh5z4quPy6AYAYnNiQswYfoHAS16l2D3ldNZVItTypV+gtpYwGVWRiX9s/arwkguG4cXoGQEabaHdbGGXryiaETD6eL6n+uJG+CbNFVlNrjS89xf9I9IhLoCcPXMmvlHLBgdYTS2lO+4z3x90EeHBtyLrLv3H+rpq/UktPIxQxHiSc9s1SD2S0VIrCP8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CO6PR02MB7555.namprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(39860400002)(366004)(346002)(396003)(136003)(376002)(451199015)(52116002)(107886003)(2906002)(38100700002)(41300700001)(66946007)(83380400001)(2616005)(186003)(6486002)(8936002)(6512007)(316002)(66556008)(478600001)(54906003)(6506007)(4326008)(5660300002)(86362001)(66476007)(8676002)(38350700002)(26005)(15650500001)(36756003)(1076003)(45080400002)(14143004);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9bFDcFUwEAep5H+pF8X5BkZfUsvvftUlWMLEJorjUaGgU8bl5RlzzjXd7nKGraWMKcLwWq/yifgvdb+PLxVD73PkSpnJjS99g5UQkVGgjK4QK4Z9yX3lkS7bpSNDX9viK/lqYst/qamxPd8ZBkYmgewH54gNJwPsB1d5OnEFgdL6X81ig1mVO+o1E7e7YTT6ujhs5Yrt2feKuaPsNFRwnoyCHp5Uc9Z0FBkER4Y1mqXl801qrevyzC3jbtUmK5ioVHgM1VlqKP6eM/1kgAM1bybV/XzuiNXwaAlnlHK8qUZ4vUaLbWUbCjikofcBlbZNcCLT8WwfTZ1qP6dwPv6msJlEous14ZCXHkibD4WHmGAEC3nGckTr1SF1t1RYVPI3DthvVVxPIG1vZfJL4f1y9940H5OqkXDEfBzwZ8q5issEfciKurTYCQIIw4ucPJYPDJwupACQY83DwE1XxGlm9Y66QxDYYjWZGHuocBS+1rfdxxUeRFxwnw11l5BP3K5CUdCJVniFr9uB3iCUC+ht0vPTiMx8l4fVNAqzlwj4vjve2abed3Xm7LV8H/BqnQt7Xx0Fp3xPbMgwCH4oDjWdw3NC4BVN1kxqqtDUsHl0BaWe9zkGHCaM5DdNB/NG+WAfc7Ri42tkQoBdSlijLlO4tjMNAQN9qG0DEt6pYCAjdtpoOHk53SG3YSDgQOQQNdXn0zFr4O2QqWoz6mvL6dUePBVtY+CwfRSRwvO3ddrRpM0kKrK+PnCrSQ/39izin9s1oLL7a5Oxc/0V/Vb37vWWjhW3qQqWMCQ7059VNPAnBUzSx4/072kYg2C/8Ryk8z4s03rg3hDvdlQX+FZaYrAxxEVClCJab0El4UyRC4RIgWZyYGNTQ5c4ofkCoVrvmgTagw35+frpsUgbq5iQ15lezz5eHCmUZvVsjy+/2fG39sAkZUhaDVe1KReOeKrC7RieuLg9+f6ODgd/bVnZ6sJ/1E4RFFdUW14aisXVuAo6AgpqGP+f1ZnlZH9yJNl8+RNK/2nSHjU9/24plUvfFFjZd5GK8X9lB/H90uMoWdGXYlvR4PwSDSrjaYvOxvvWUeOzzGap1HvjzVPnEPOS3ivSO9pvv843KeYdodTmzNlaf2irY9FhtK0PFMhuv+0bX1WgazEfh+4UQudr4pcV9J56A+fWMerw/kgAUWWIPsZRBFhZ9AIkqzgfHiF79MQcXOMIXnVJiIOxkU4BY3k6h7rXBB910yQWTckwLlOhF80A767TJR0Q53qJhP2yeCF1imK2+TWmH6WMK3Cwz20D1mCn+zvuhI+8u7C8+G2c3kfKKuPVJ6S0ceFczkgMCya58yunyge4OQJY6B85shRwxZd9P3unKNgXvQhjw1sIUcVOJRAoNYAxXBzxc5ZPnKDnZWJsegNjt9hby8/UBbPRjfqgA7HcCWbdUXZWRFM90Dx8EQmQSTMSM4zMTx5j0QKcp8Sc2pND4DRWYRHz51vsmsJRnwPGQDIa6XIyG4h3SOFDrxiKlxo6/aXSiL2nWY0VMJfc1b5KCOpphldaWb0wMSAz1hSmM34KzzlveCeYtzjkiFkLCR9wJhoJYLc0eBSSzK3Qg4eszkw0MCKl9mCE/8AbQg== X-OriginatorOrg: nutanix.com X-MS-Exchange-CrossTenant-Network-Message-Id: c5835724-b54c-4586-b218-08da9702d970 X-MS-Exchange-CrossTenant-AuthSource: CO6PR02MB7555.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2022 10:12:52.1835 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: bb047546-786f-4de1-bd75-24e5b6f79043 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WIw3kSvT+LSLzUWtrhVhiu36kqprmiapamsQGJh064h3UMSKSr0AuLCUxzvP5Z0JFsdTioTFrUL9PUjbNiHnQrgvK58p51jKbNMwXHfAZco= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR02MB7370 X-Proofpoint-ORIG-GUID: 2ijBEhSl69MmAs-JOGskva1oaYOA-2-- X-Proofpoint-GUID: 2ijBEhSl69MmAs-JOGskva1oaYOA-2-- X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_06,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Reason: safe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add selftests for dirty quota throttling with an optional -d parameter to configure by what value dirty quota should be incremented after each dirty quota exit. With very small intervals, a smaller value of dirty quota can ensure that the dirty quota exit code is tested. A zero value disables dirty quota throttling and thus dirty logging, without dirty quota throttling, can be tested. Suggested-by: Shaju Abraham Suggested-by: Manish Mishra Co-developed-by: Anurag Madnawat Signed-off-by: Anurag Madnawat Signed-off-by: Shivam Kumar --- tools/testing/selftests/kvm/dirty_log_test.c | 33 +++++++++++++++-- .../selftests/kvm/include/kvm_util_base.h | 4 +++ tools/testing/selftests/kvm/lib/kvm_util.c | 36 +++++++++++++++++++ 3 files changed, 71 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 9c883c94d478..81c46eff7e1d 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -63,6 +63,8 @@ #define SIG_IPI SIGUSR1 +#define TEST_DIRTY_QUOTA_INCREMENT 8 + /* * Guest/Host shared variables. Ensure addr_gva2hva() and/or * sync_global_to/from_guest() are used when accessing from @@ -189,6 +191,7 @@ static enum log_mode_t host_log_mode_option = LOG_MODE_ALL; static enum log_mode_t host_log_mode; static pthread_t vcpu_thread; static uint32_t test_dirty_ring_count = TEST_DIRTY_RING_COUNT; +static uint64_t test_dirty_quota_increment = TEST_DIRTY_QUOTA_INCREMENT; static void vcpu_kick(void) { @@ -208,6 +211,13 @@ static void sem_wait_until(sem_t *sem) while (ret == -1 && errno == EINTR); } +static void set_dirty_quota(struct kvm_vm *vm, uint64_t dirty_quota) +{ + struct kvm_run *run = vcpu_state(vm, VCPU_ID); + + vcpu_set_dirty_quota(run, dirty_quota); +} + static bool clear_log_supported(void) { return kvm_has_cap(KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2); @@ -255,7 +265,11 @@ static void default_after_vcpu_run(struct kvm_vcpu *vcpu, int ret, int err) TEST_ASSERT(ret == 0 || (ret == -1 && err == EINTR), "vcpu run failed: errno=%d", err); - TEST_ASSERT(get_ucall(vcpu, NULL) == UCALL_SYNC, + if (test_dirty_quota_increment && + run->exit_reason == KVM_EXIT_DIRTY_QUOTA_EXHAUSTED) + vcpu_handle_dirty_quota_exit(run, test_dirty_quota_increment); + else + TEST_ASSERT(get_ucall(vcpu, NULL) == UCALL_SYNC, "Invalid guest sync status: exit_reason=%s\n", exit_reason_str(run->exit_reason)); @@ -372,6 +386,9 @@ static void dirty_ring_after_vcpu_run(struct kvm_vcpu *vcpu, int ret, int err) if (get_ucall(vcpu, NULL) == UCALL_SYNC) { /* We should allow this to continue */ ; + } else if (test_dirty_quota_increment && + run->exit_reason == KVM_EXIT_DIRTY_QUOTA_EXHAUSTED) { + vcpu_handle_dirty_quota_exit(run, test_dirty_quota_increment); } else if (run->exit_reason == KVM_EXIT_DIRTY_RING_FULL || (ret == -1 && err == EINTR)) { /* Update the flag first before pause */ @@ -762,6 +779,10 @@ static void run_test(enum vm_guest_mode mode, void *arg) sync_global_to_guest(vm, guest_test_virt_mem); sync_global_to_guest(vm, guest_num_pages); + /* Initialise dirty quota */ + if (test_dirty_quota_increment) + set_dirty_quota(vm, test_dirty_quota_increment); + /* Start the iterations */ iteration = 1; sync_global_to_guest(vm, iteration); @@ -803,6 +824,9 @@ static void run_test(enum vm_guest_mode mode, void *arg) /* Tell the vcpu thread to quit */ host_quit = true; log_mode_before_vcpu_join(); + /* Terminate dirty quota throttling */ + if (test_dirty_quota_increment) + set_dirty_quota(vm, 0); pthread_join(vcpu_thread, NULL); pr_info("Total bits checked: dirty (%"PRIu64"), clear (%"PRIu64"), " @@ -824,6 +848,8 @@ static void help(char *name) printf(" -c: specify dirty ring size, in number of entries\n"); printf(" (only useful for dirty-ring test; default: %"PRIu32")\n", TEST_DIRTY_RING_COUNT); + printf(" -q: specify incemental dirty quota (default: %"PRIu32")\n", + TEST_DIRTY_QUOTA_INCREMENT); printf(" -i: specify iteration counts (default: %"PRIu64")\n", TEST_HOST_LOOP_N); printf(" -I: specify interval in ms (default: %"PRIu64" ms)\n", @@ -852,11 +878,14 @@ int main(int argc, char *argv[]) guest_modes_append_default(); - while ((opt = getopt(argc, argv, "c:hi:I:p:m:M:")) != -1) { + while ((opt = getopt(argc, argv, "c:q:hi:I:p:m:M:")) != -1) { switch (opt) { case 'c': test_dirty_ring_count = strtol(optarg, NULL, 10); break; + case 'q': + test_dirty_quota_increment = strtol(optarg, NULL, 10); + break; case 'i': p.iterations = strtol(optarg, NULL, 10); break; diff --git a/tools/testing/selftests/kvm/include/kvm_util_base.h b/tools/testing/selftests/kvm/include/kvm_util_base.h index 24fde97f6121..68be66b9ac67 100644 --- a/tools/testing/selftests/kvm/include/kvm_util_base.h +++ b/tools/testing/selftests/kvm/include/kvm_util_base.h @@ -834,4 +834,8 @@ static inline int __vm_disable_nx_huge_pages(struct kvm_vm *vm) return __vm_enable_cap(vm, KVM_CAP_VM_DISABLE_NX_HUGE_PAGES, 0); } +void vcpu_set_dirty_quota(struct kvm_run *run, uint64_t dirty_quota); +void vcpu_handle_dirty_quota_exit(struct kvm_run *run, + uint64_t test_dirty_quota_increment); + #endif /* SELFTEST_KVM_UTIL_BASE_H */ diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 9889fe0d8919..4c7bd9807d0b 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -18,6 +18,7 @@ #include #define KVM_UTIL_MIN_PFN 2 +#define PML_BUFFER_SIZE 512 static int vcpu_mmap_sz(void); @@ -1703,6 +1704,7 @@ static struct exit_reason { {KVM_EXIT_X86_RDMSR, "RDMSR"}, {KVM_EXIT_X86_WRMSR, "WRMSR"}, {KVM_EXIT_XEN, "XEN"}, + {KVM_EXIT_DIRTY_QUOTA_EXHAUSTED, "DIRTY_QUOTA_EXHAUSTED"}, #ifdef KVM_EXIT_MEMORY_NOT_PRESENT {KVM_EXIT_MEMORY_NOT_PRESENT, "MEMORY_NOT_PRESENT"}, #endif @@ -1979,3 +1981,37 @@ void __vm_get_stat(struct kvm_vm *vm, const char *stat_name, uint64_t *data, break; } } + +void vcpu_set_dirty_quota(struct kvm_run *run, uint64_t dirty_quota) +{ + run->dirty_quota = dirty_quota; + + if (dirty_quota) + pr_info("Dirty quota throttling enabled with initial quota %"PRIu64"\n", + dirty_quota); + else + pr_info("Dirty quota throttling disabled\n"); +} + +void vcpu_handle_dirty_quota_exit(struct kvm_run *run, + uint64_t test_dirty_quota_increment) +{ + uint64_t quota = run->dirty_quota_exit.quota; + uint64_t count = run->dirty_quota_exit.count; + + /* + * Due to PML, number of pages dirtied by the vcpu can exceed its dirty + * quota by PML buffer size. + */ + TEST_ASSERT(count <= quota + PML_BUFFER_SIZE, "Invalid number of pages + dirtied: count=%"PRIu64", quota=%"PRIu64"\n", count, quota); + + TEST_ASSERT(count >= quota, "Dirty quota exit happened with quota yet to + be exhausted: count=%"PRIu64", quota=%"PRIu64"\n", count, quota); + + if (count > quota) + pr_info("Dirty quota exit with unequal quota and count: + count=%"PRIu64", quota=%"PRIu64"\n", count, quota); + + run->dirty_quota = count + test_dirty_quota_increment; +}