From patchwork Tue Jan 26 13:41:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanan Wang X-Patchwork-Id: 12046673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDB9AC433E0 for ; Tue, 26 Jan 2021 13:50:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C48222255F for ; Tue, 26 Jan 2021 13:50:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404890AbhAZNto (ORCPT ); Tue, 26 Jan 2021 08:49:44 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:11445 "EHLO szxga06-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404850AbhAZNmy (ORCPT ); Tue, 26 Jan 2021 08:42:54 -0500 Received: from DGGEMS406-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4DQ7DT5ZtPzjCXP; Tue, 26 Jan 2021 21:41:13 +0800 (CST) Received: from DESKTOP-TMVL5KK.china.huawei.com (10.174.187.128) by DGGEMS406-HUB.china.huawei.com (10.3.19.206) with Microsoft SMTP Server id 14.3.498.0; Tue, 26 Jan 2021 21:42:04 +0800 From: Yanan Wang To: , , , , Marc Zyngier , Will Deacon , Catalin Marinas CC: Mark Rutland , James Morse , Julien Thierry , Suzuki K Poulose , , , , , Yanan Wang Subject: [RFC PATCH v1 0/5] Enable CPU TTRem feature for stage-2 Date: Tue, 26 Jan 2021 21:41:57 +0800 Message-ID: <20210126134202.381996-1-wangyanan55@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 MIME-Version: 1.0 X-Originating-IP: [10.174.187.128] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi all, This series enable CPU TTRem feature for stage-2 page table and a RFC is sent for some comments, thanks. The ARMv8.4 TTRem feature offers 3 levels of support when changing block size without changing any other parameters that are listed as requiring use of break-before-make. And I found that maybe we can use this feature to make some improvement for stage-2 page table and the following explains what TTRem exactly does for the improvement. If migration of a VM with hugepages is canceled midway, KVM will adjust the stage-2 table mappings back to block mappings. We currently use BBM to replace the table entry with a block entry. Take adjustment of 1G block mapping as an example, with BBM procedures, we have to invalidate the old table entry first, flush TLB and unmap the old table mappings, right before installing the new block entry. So there will be a bit long period when the old table entry is invalid before installation of the new block entry, if other vCPUs access any guest page within the 1G range during this period and find the table entry invalid, they will all exit from guest with a translation fault. Actually, these translation faults are not necessary, because the block mapping will be built later. Besides, KVM will still try to build 1G block mappings for these spurious translation faults, and will perform cache maintenance operations, page table walk, etc. In summary, the spurious faults are caused by invalidation in BBM procedures. Approaches of TTRem level 1,2 ensure that there will not be a moment when the old table entry is invalid before installation of the new block entry. However, level-2 method will possibly lead to a TLB conflict which is bothering, so we use nT both at level-1 and level-2 case to avoid handling TLB conflict aborts. For an implementation which meets level 1 or level 2, the CPU has two responses to choose when accessing a block table entry with nT bit set: Firstly, CPU will generate a translation fault, the effect of this response is simier to BBM. Secondly, CPU can use the block entry for translation. So with the second kind of implementation, the above described spurious translations can be prevented. Yanan Wang (5): KVM: arm64: Detect the ARMv8.4 TTRem feature KVM: arm64: Add an API to get level of TTRem supported by hardware KVM: arm64: Support usage of TTRem in guest stage-2 translation KVM: arm64: Add handling of coalescing tables into a block mapping KVM: arm64: Adapt page-table code to new handling of coalescing tables arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/cpufeature.h | 13 ++++++ arch/arm64/kernel/cpufeature.c | 10 +++++ arch/arm64/kvm/hyp/pgtable.c | 62 +++++++++++++++++++++++------ 4 files changed, 74 insertions(+), 14 deletions(-)