From patchwork Thu Aug 8 17:42:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Colton Lewis X-Patchwork-Id: 13758031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63C82C52D71 for ; Thu, 8 Aug 2024 17:44:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:Mime-Version:Date:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=4+1jkb/kSvflaKrQy8nLqEfji7x/G0gMXefyaO4Ttzs=; b=TAzBv6k2jLCtf9aW8LPNMOuEpF Yx2NyTxvEdFcxWCLxWN6UdsOwv6bZnCfhtYojA6iDT3AXgHCBsMOomD/W/nf9wHW7+/eSO+4sdFz4 Y+nC0bANDQpik2PFBqLRyjC+pXlEB9WPxCpYF+aAq6AW4Wl0P1+GgeD6SBpPxT0Lflf7UshLNbcrS nalPxjXp3UwhtB85s9SoZb5rpz3A+pq8b7stslYrJHjB/9pzqWvbJAmMUPA36el5JEw2QpjdBUjfu d2ykmUzN+PKRMXsZLtMMuMShpT/SVCUNDhYn+0U8oHJnmGpc1lyPYVyQYJw2y5zcp7+jd2mIlLHZ1 hKiqacFw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sc7B1-0000000980D-28Jl; Thu, 08 Aug 2024 17:43:51 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sc7AR-000000097un-2g9W for linux-arm-kernel@lists.infradead.org; Thu, 08 Aug 2024 17:43:17 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-68f95e37bbfso28637937b3.0 for ; Thu, 08 Aug 2024 10:43:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723138993; x=1723743793; darn=lists.infradead.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=4+1jkb/kSvflaKrQy8nLqEfji7x/G0gMXefyaO4Ttzs=; b=yyl7173WMgtOoQbm8XNgn5MfYgtyIZZhLMmM42XvDsxU8CVdt+IrboTt5B/aALeM7w /0Q1rQ9kM2Xz5ToTBI9ktwNVNKfjIKEcX2aA38amd92qM/VnY/lTi8aBy48yp+HvJmeF MlOXi22PA4NcAKZN+lImXuqgu34kbfYVj+olXmiTESRH5ox9IPqX05n+o6Al+6ETsAWE Ed7rtG+cyBApBjrOgCi1PfMFnl7N8i0pumU8cenrbCWblGoXQatpwliMCo31Su9dxYuL WG91xG2oHr8UhEw8cWxmefJr3LAI3Z8oj2CrZW0AyfCrGD5qFmC/Shx9DLf/fbguQN0p lcqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723138993; x=1723743793; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=4+1jkb/kSvflaKrQy8nLqEfji7x/G0gMXefyaO4Ttzs=; b=cw4kgaRwUHmXy6x9nJKbsC/3wCaaySftcohL61R3IagfFmpReDDr2DgLqIQM4XLuqy l4DIiagFNAUU9cs9MsRXaYMkKYcYVYfWIj/ZjJAC0LVM2548wBHkcZdsPnUgYZivQB1X SIfMbulSQ22Ym6h08s/U6YyRUBHKv3u2WBRyFYTanWlcxzXeLfvBUAqMBa8bSeztECD4 74tvk/xyeTbeNaWF2QJ1bvN06AAA1YAqv5dkPHSp0WYV4VfLikOfFmUUgj6fcYaLuY/s JjrvS4iFxKLXke+zO0t6/lQIF9z05b29dzdYJwR/hSzdfXNe1DSGnTXnKIUkscmpA7qE jolQ== X-Forwarded-Encrypted: i=1; AJvYcCUVJ37uxzLSbN3rwicm6mA/Tq/KMeWDyA+6ntmxemXaREk4C35W9nJ44cwEK2KFvfbaOtPp8Afvty5F77F3Qc2SECRn1vuIEIMVwD98QPott4nGu14= X-Gm-Message-State: AOJu0YwKMfCWB5qXZNxa9FAiL5gIGlk+5jG97abDuLnY8BLkDNATJnY9 XkB5iBH8l2RD8AAREF+hYf9ZalVlKoiVRXU97Md423hjFKfQsJlJ0YYZBvgcXiTHpn+swgCRXuI qmzx3IpPejphdskJ/HAKRNg== X-Google-Smtp-Source: AGHT+IFI0rnzef7ISFTxPTkBDSrqwcN8rzvq+R6/y07Cu867kKIpT/4oVZnVFq4JrZ5GCRkcJ1KbtYBJHpkWdkmDpA== X-Received: from coltonlewis-kvm.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:14ce]) (user=coltonlewis job=sendgmr) by 2002:a05:690c:961:b0:62f:1f63:ae4f with SMTP id 00721157ae682-69bf6f7ec1fmr200267b3.1.1723138993566; Thu, 08 Aug 2024 10:43:13 -0700 (PDT) Date: Thu, 8 Aug 2024 17:42:43 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240808174243.2836363-1-coltonlewis@google.com> Subject: [PATCH v2] KVM: arm64: Move data barrier to end of split walk From: Colton Lewis To: kvm@vger.kernel.org Cc: Marc Zyngier , Oliver Upton , Ricardo Koller , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, Colton Lewis X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240808_104315_700098_8D2540F4 X-CRM114-Status: GOOD ( 13.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This DSB guarantees page table updates have been made visible to the hardware table walker. Moving the DSB from stage2_split_walker() to after the walk is finished in kvm_pgtable_stage2_split() results in a roughly 70% reduction in Clear Dirty Log Time in dirty_log_perf_test (modified to use eager page splitting) when using huge pages. This gain holds steady through a range of vcpus used (tested 1-64) and memory used (tested 1-64GB). This is safe to do because nothing else is using the page tables while they are still being mapped and this is how other page table walkers already function. None of them have a data barrier in the walker itself because relative ordering of table PTEs to table contents comes from the release semantics of stage2_make_pte(). Signed-off-by: Colton Lewis Acked-by: Oliver Upton --- v2: * Added more information about purpose of DSB to commit message * Rebased to v6.11-rc2 v1: https://lore.kernel.org/kvmarm/20240718223519.1673835-1-coltonlewis@google.com/ arch/arm64/kvm/hyp/pgtable.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.46.0.76.ge559c4bf1a-goog diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 9e2bbee77491..9788af2ca8c0 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1547,7 +1547,6 @@ static int stage2_split_walker(const struct kvm_pgtable_visit_ctx *ctx, */ new = kvm_init_table_pte(childp, mm_ops); stage2_make_pte(ctx, new); - dsb(ishst); return 0; } @@ -1559,8 +1558,11 @@ int kvm_pgtable_stage2_split(struct kvm_pgtable *pgt, u64 addr, u64 size, .flags = KVM_PGTABLE_WALK_LEAF, .arg = mc, }; + int ret; - return kvm_pgtable_walk(pgt, addr, size, &walker); + ret = kvm_pgtable_walk(pgt, addr, size, &walker); + dsb(ishst); + return ret; } int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_s2_mmu *mmu,