From patchwork Tue Jun 25 13:35:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711132 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F40316193C; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; cv=none; b=QshdJvTS26naAOxZ3y4bkKIVs58f7Kz9HYmraZxEQkvMSYfpXHKojTcXF2wQ8lTt01Quv4jG2ByazCCyfb/ZBUTiE7svJONBMHFRvkW9lO8S1TMEBApAgbnWIbqm/RMty2I6SL3jn1ePMClZijldIvQw1z2wrRHGx/1VKUhhQGk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; c=relaxed/simple; bh=Z5W8y2Ri1k0XaNhTI73pukGuPrGrEy3whjorOmdnemc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AwJc2gF6VSRYKEGc7462lpLux2sz9Kk2zsFwF7pkjNfvMCkhu/XRjCu1nmAJL0mgoPSRYBj7X60httbMVza3vS13WF+OzJWXTyhhrQh3UuPR58IbLOtnguHEZ0Usw9eHkRaNC7zrsFvFdjcJD/wSpr6uhzbcfXqpZGQKBFAwKNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MApEtANP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MApEtANP" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E79F8C32786; Tue, 25 Jun 2024 13:35:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322523; bh=Z5W8y2Ri1k0XaNhTI73pukGuPrGrEy3whjorOmdnemc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MApEtANP/FQVQfIUI2nuQag+jwmsncFPyFRDcdPyGXjAMcgMpTt8QIzkMcXEr2mDO 1/kCTvx0/RNm5bja4P1fvJTEE8C9P7S2nKp/F9c92hWdKKf/9AahPWRwrOmyhjuaqn vfWYUfvIEctLk1YGV4EAkwyleIEFNKcx4/5lM4PNfWICx73AhHoOxrGIdAMGIcwaH7 JesC22GdBAglMZt8sTuRq6zr1vg5iQ9cnYUve3ouWOs4Y+1m5XgltfUngqkxhzQlU3 M4MH2xGh+Tcizng5SO7Me1IqWYdL2hDuVuIAPPFMWeXXMEfr8WfGdtespgQdJsCl0m KfwazE7tvZTFw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-1I; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 01/12] arm64: Add missing APTable and TCR_ELx.HPD masks Date: Tue, 25 Jun 2024 14:35:00 +0100 Message-Id: <20240625133508.259829-2-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Although Linux doesn't make use of hierarchical permissions (TFFT!), KVM needs to know where the various bits related to this feature live in the TCR_ELx registers as well as in the page tables. Add the missing bits. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_arm.h | 1 + arch/arm64/include/asm/pgtable-hwdef.h | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index b2adc2c6c82a5..c93ee1036cb09 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -108,6 +108,7 @@ /* TCR_EL2 Registers bits */ #define TCR_EL2_DS (1UL << 32) #define TCR_EL2_RES1 ((1U << 31) | (1 << 23)) +#define TCR_EL2_HPD (1 << 24) #define TCR_EL2_TBI (1 << 20) #define TCR_EL2_PS_SHIFT 16 #define TCR_EL2_PS_MASK (7 << TCR_EL2_PS_SHIFT) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 9943ff0af4c96..f75c9a7e6bd68 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -146,6 +146,7 @@ #define PMD_SECT_UXN (_AT(pmdval_t, 1) << 54) #define PMD_TABLE_PXN (_AT(pmdval_t, 1) << 59) #define PMD_TABLE_UXN (_AT(pmdval_t, 1) << 60) +#define PMD_TABLE_AP (_AT(pmdval_t, 3) << 61) /* * AttrIndx[2:0] encoding (mapping attributes defined in the MAIR* registers). @@ -307,6 +308,12 @@ #define TCR_TCMA1 (UL(1) << 58) #define TCR_DS (UL(1) << 59) +#define TCR_HPD0_SHIFT 41 +#define TCR_HPD0 BIT(TCR_HPD0_SHIFT) + +#define TCR_HPD1_SHIFT 42 +#define TCR_HPD1 BIT(TCR_HPD1_SHIFT) + /* * TTBR. */ From patchwork Tue Jun 25 13:35:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711133 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7107C161B43; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; cv=none; b=R1D9KbBs6QT7PnxB60DHlVCWCd/G5ZarV+HkrD6zqFSXRt5GwTbhIHqPK5e9ZCmeAs0cFVCwxROJbyXYW6szvGc1djAtQ+8bX4nfC4ymZ0AElXz9n1/GCvXuN/wZsdQ24M4wO+GJLRCeis3gnsHRUbEPhw3e2odTZI6zJncFcI4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; c=relaxed/simple; bh=T2CXo4D4ISy0D/ALYS9ezM0wHkOH2cIDAXg9SuVgoHo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BCQb2IVtgQ+jKwlTNTQJsQW2rAfbYjrc6NwMBp2RXC72WSzarTvEJB3DlBies6Y/h5LhSWyr9UognjtNAl51/qtLOn7N/cOoNPXFF4i9Efr7JG0xMtbq6Z8N2Gibuhf7ixivOuyOdm2jfDaA9eeGPrBkWtiUMG6noz4r9fKUeiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j07EHgQ5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j07EHgQ5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0583CC4AF09; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=T2CXo4D4ISy0D/ALYS9ezM0wHkOH2cIDAXg9SuVgoHo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j07EHgQ5Xkn4rLuGZkzAZLLsaxQpEQlKHfKGIkGglBmWSEFFfsF+ZzDC1DY0oVxIM MM6x72LEICDrnl33D6uGj5fWglR3o2Aq117RpU1geG5r9AEKduIEkTRlZZBvzPYYNh ES2J+YBWmK1rRJ8bEZ57hOQGNuh1GT8m3SW0h1v0WU/9uWrLQIrPR/df7md8Xb0YQW usmb/2JdCuhQJ2hNq8EMDbwq2ftB0lLTbVRzVO16RBWNPXxfktrh3wibOzlDGudnCF +lQNwX4tcfzqc8GxHCD0xZJXJBZ4Rf+LBzI9n8nzqSFAs2Vb5rIq+uWvZk2lysewrv FrUbgwTBLruVw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-6n; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 02/12] arm64: Add PAR_EL1 field description Date: Tue, 25 Jun 2024 14:35:01 +0100 Message-Id: <20240625133508.259829-3-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false As KVM is about to grow a full emulation for the AT instructions, add the layout of the PAR_EL1 register in its non-D128 configuration. Note that the constants are a bit ugly, as the register has two layouts, based on the state of the F bit. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/sysreg.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index be41528194569..15c073359c9e9 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -325,7 +325,25 @@ #define SYS_PAR_EL1 sys_reg(3, 0, 7, 4, 0) #define SYS_PAR_EL1_F BIT(0) +/* When PAR_EL1.F == 1 */ #define SYS_PAR_EL1_FST GENMASK(6, 1) +#define SYS_PAR_EL1_PTW BIT(8) +#define SYS_PAR_EL1_S BIT(9) +#define SYS_PAR_EL1_AssuredOnly BIT(12) +#define SYS_PAR_EL1_TopLevel BIT(13) +#define SYS_PAR_EL1_Overlay BIT(14) +#define SYS_PAR_EL1_DirtyBit BIT(15) +#define SYS_PAR_EL1_F1_IMPDEF GENMASK_ULL(63, 48) +#define SYS_PAR_EL1_F1_RES0 (BIT(7) | BIT(10) | GENMASK_ULL(47, 16)) +#define SYS_PAR_EL1_RES1 BIT(11) +/* When PAR_EL1.F == 0 */ +#define SYS_PAR_EL1_SH GENMASK_ULL(8, 7) +#define SYS_PAR_EL1_NS BIT(9) +#define SYS_PAR_EL1_F0_IMPDEF BIT(10) +#define SYS_PAR_EL1_NSE BIT(11) +#define SYS_PAR_EL1_PA GENMASK_ULL(51, 12) +#define SYS_PAR_EL1_ATTR GENMASK_ULL(63, 56) +#define SYS_PAR_EL1_F0_RES0 (GENMASK_ULL(6, 1) | GENMASK_ULL(55, 52)) /*** Statistical Profiling Extension ***/ #define PMSEVFR_EL1_RES0_IMP \ From patchwork Tue Jun 25 13:35:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711134 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AC43167D80; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; cv=none; b=JnflOt9jMCo6zuOiHWKe8IsdjYAjr//8oQlEVNnZ56+Dudym6Z4B+RZMDuiX4eUJTe7auh6IxL+09+Q+ZxSX0BxmQONgmVtw1q0TxNV8/MbFIFxs9crXlT70LVqzgOW5IovKS5y0DZWdvQ43+C5fPcBM/nh0pKne5NwLWplKczM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; c=relaxed/simple; bh=d9HZ26PlskN9lywxTEIiCLlDZUf4MEeqnxDDEB1Fjxw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EK724/WWcq3hpRAGhfLmhYsNSLTffGfEHteIIJH7oH8EFfyYlLhnsxHb1ULc8fCAsNCibOm/PWnn4yLAE+VNvwuVdhP3v6XMauYKA3zsrycwQ3EDozmpDUntNDvACcbHmvc7CTxewwTasXZYR47zb6ibDA2q0ubrq4YKtQxXzQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NwhPIru+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NwhPIru+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 346D9C4AF07; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=d9HZ26PlskN9lywxTEIiCLlDZUf4MEeqnxDDEB1Fjxw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NwhPIru+70FySj9FRvmCmZ3rHVSASk0mD5JlZ7mRZq3Rzw2wTEWk8gzI0uSX5bk45 3dneBWgh4rcQguvcZgaORlI4SkRUUmZIOS0jVcpHpo5H926noc6C7oOl232KDiNSZJ QhuRZv7inOCB4+v0uPHB2oGKb7ucJY3E9z0Olg8+awEAM7kqbnXiJDRZRpE+434jSV TLDHPd3f+BCaLLp/tSfBX87mfehjETGKjaiAG7FeVoel0+vHMYc/bdTry+bMYGn0T7 GSYPVcEEFuh+UM4H6xKgsrVOKz6zKe8BSsg6DSc6NQI7aiXdWTkpzBOKipsY0GaBLk QzM0XezzeF73g== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-CU; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 03/12] KVM: arm64: nv: Turn upper_attr for S2 walk into the full descriptor Date: Tue, 25 Jun 2024 14:35:02 +0100 Message-Id: <20240625133508.259829-4-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false The upper_attr attribute has been badly named, as it most of the time carries the full "last walked descriptor". Rename it to "desc" and make ti contain the full 64bit descriptor. This will be used by the S1 PTW. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_nested.h | 4 ++-- arch/arm64/kvm/nested.c | 12 ++++++------ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h index 5b06c31035a24..b2fe759964d83 100644 --- a/arch/arm64/include/asm/kvm_nested.h +++ b/arch/arm64/include/asm/kvm_nested.h @@ -85,7 +85,7 @@ struct kvm_s2_trans { bool readable; int level; u32 esr; - u64 upper_attr; + u64 desc; }; static inline phys_addr_t kvm_s2_trans_output(struct kvm_s2_trans *trans) @@ -115,7 +115,7 @@ static inline bool kvm_s2_trans_writable(struct kvm_s2_trans *trans) static inline bool kvm_s2_trans_executable(struct kvm_s2_trans *trans) { - return !(trans->upper_attr & BIT(54)); + return !(trans->desc & BIT(54)); } extern int kvm_walk_nested_s2(struct kvm_vcpu *vcpu, phys_addr_t gipa, diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 96029a95d1062..73544e0e64dcb 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -256,7 +256,7 @@ static int walk_nested_s2_pgd(phys_addr_t ipa, /* Check for valid descriptor at this point */ if (!(desc & 1) || ((desc & 3) == 1 && level == 3)) { out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT); - out->upper_attr = desc; + out->desc = desc; return 1; } @@ -266,7 +266,7 @@ static int walk_nested_s2_pgd(phys_addr_t ipa, if (check_output_size(wi, desc)) { out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ); - out->upper_attr = desc; + out->desc = desc; return 1; } @@ -278,7 +278,7 @@ static int walk_nested_s2_pgd(phys_addr_t ipa, if (level < first_block_level) { out->esr = compute_fsc(level, ESR_ELx_FSC_FAULT); - out->upper_attr = desc; + out->desc = desc; return 1; } @@ -289,13 +289,13 @@ static int walk_nested_s2_pgd(phys_addr_t ipa, if (check_output_size(wi, desc)) { out->esr = compute_fsc(level, ESR_ELx_FSC_ADDRSZ); - out->upper_attr = desc; + out->desc = desc; return 1; } if (!(desc & BIT(10))) { out->esr = compute_fsc(level, ESR_ELx_FSC_ACCESS); - out->upper_attr = desc; + out->desc = desc; return 1; } @@ -307,7 +307,7 @@ static int walk_nested_s2_pgd(phys_addr_t ipa, out->readable = desc & (0b01 << 6); out->writable = desc & (0b10 << 6); out->level = level; - out->upper_attr = desc & GENMASK_ULL(63, 52); + out->desc = desc; return 0; } From patchwork Tue Jun 25 13:35:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711135 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F3E4169AE4; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; cv=none; b=KmJasiE4uj2gNP7TVdi8K4aW74UArQLtzThMDXAcbsRToDPIHWxTNE1haCcOPboU0c4/45+5UkvWjp9Ylp2QEFp60LyPCYP0r7oG+b5rgbenE+cfa0YknKHinAmWbgTG2TOXTcYkAvIOgYM5ZbJ0wtOsxfOVA+WVPqU4R4HjMCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322524; c=relaxed/simple; bh=URJ463LZf+NhFwc+Irio0BcLleDjSg1ELTU95sX4nPM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=p/aEV3BNiw+8Wjo2R4OdMGXgyuFr2AdOQgzx5bIe2eYY9jCtEMTewOOAZ9AUMIBl46iJVQmTlZ9pY+N1ZraRwQUBYmKS8N6iAucsvApMOxd0WiaGMrWomvGKDR6sTbEvtrFWAlFi64OBJDd/F+EL0Mmq/gGgFXkdO/ijZxr4aEQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kxAVPM/q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kxAVPM/q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7C7ABC4AF0A; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=URJ463LZf+NhFwc+Irio0BcLleDjSg1ELTU95sX4nPM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kxAVPM/qxi5uVu5SpiZE4GKIl22W7F9yzlC8UKeU3+9GUDwwI1cRwp2mtl0ptZLBW /ypK+ROFNW6HB5RtPZqEDWw4MU89+32iGh2dmjb1nUSj8TgxfAwqVmkWtb0jVeLr86 v8YBtQiSdqbJHvJIosf5cnE7gRp+yCP2Op3L0zmI5tmpE557nmgPz7EwsCBUt3pLeD hfwNCbrBRZbVnSb/EzmJypFu7wOBCJGPTxAuyGn8F5yQfrczUMRVmFwIv7FkNuAPmM Bi8VT51e/m4/KqisgOKlu9G2xO7MeJpLlPD5QQc9ZV0ntQy62u5xabElwVdMHF1clY zIv4OWQdxDV2Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-I8; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 04/12] KVM: arm64: nv: Honor absence of FEAT_PAN2 Date: Tue, 25 Jun 2024 14:35:03 +0100 Message-Id: <20240625133508.259829-5-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false If our guest has been configured without PAN2, make sure that AT S1E1{R,W}P will generate an UNDEF. Signed-off-by: Marc Zyngier Reviewed-by: Anshuman Khandual --- arch/arm64/kvm/sys_regs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 832c6733db307..06c39f191b5ec 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -4585,6 +4585,10 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) HFGITR_EL2_TLBIRVAAE1OS | HFGITR_EL2_TLBIRVAE1OS); + if (!kvm_has_feat(kvm, ID_AA64MMFR1_EL1, PAN, PAN2)) + kvm->arch.fgu[HFGITR_GROUP] |= (HFGITR_EL2_ATS1E1RP | + HFGITR_EL2_ATS1E1WP); + if (!kvm_has_feat(kvm, ID_AA64MMFR3_EL1, S1PIE, IMP)) kvm->arch.fgu[HFGxTR_GROUP] |= (HFGxTR_EL2_nPIRE0_EL1 | HFGxTR_EL2_nPIR_EL1); From patchwork Tue Jun 25 13:35:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711140 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD6FB16E879; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; cv=none; b=p6AaR1eSERnRRM0nxZGq1PiMJHvipXgf8FVWloaIVjZBGYs2iIuSGymhrmqIMDoXSbEa9fJFn/BuOkODee1GtV8x0KGx7ygmL4pbewsB6lB9C19uZVRpUi3/y2TfmjgXQAeyOVTopS1jTED8Hd/aZ28IQoyx2WTRTOPfL19/AY8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; c=relaxed/simple; bh=O2khzE1fdyGIte4KZD309c05XGX8LSqpr8bp0t6E7fg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aYnsP876qEhP4kI5YeoWR4LWMDKdf1IsTa4cZPy1oO5goFBEhci4nNRhNXmPuM3dyit4x7PE7Ti+GEktUhnDbk2DEFkqY+I1rPkIUcUMSK/ecGlZsvYs6DLj0i1cwNQn8AD8zyPWPBu+QU2gfo0VLn7VikcztsXFoaPCnLWFKhc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lLX+nqOH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lLX+nqOH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6CE0CC32781; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=O2khzE1fdyGIte4KZD309c05XGX8LSqpr8bp0t6E7fg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lLX+nqOHNTx8D/kGeB8eYVf0gddOZM7yDKUgumJ/Y+vK102AnxX16Y909cO5qVgG2 d/nC8/ievD3Ub4VpqW1u7IKU8PSSJ9D9qW9//T9s3waeq53+yZr84akLWXBGrrUKp7 tI/gg0Fy+vUD3AicZsKY+eGD3RPB5J+bF1if+yx0qPpxykMD6TK8JqJdkMRC/Zlx2y zZYE8W+yXJrd/6khoRakP+xS0+gUXaxZADp5RQVCaPM4mlxUN/jvfKsxmHfsuZQLbv 7ZRI7yLTnPgmxw1EG5ljztSTyrjl+25sIex3uLMrCfjYNqZ6AOKzSHL0lHxSNzuP2g Rm+dyyzkaWvag== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-No; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 05/12] KVM: arm64: make kvm_at() take an OP_AT_* Date: Tue, 25 Jun 2024 14:35:04 +0100 Message-Id: <20240625133508.259829-6-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false From: Joey Gouly To allow using newer instructions that current assemblers don't know about, replace the `at` instruction with the underlying SYS instruction. Signed-off-by: Joey Gouly Cc: Marc Zyngier Cc: Oliver Upton Cc: Catalin Marinas Cc: Will Deacon Reviewed-by: Marc Zyngier Signed-off-by: Marc Zyngier Reviewed-by: Anshuman Khandual --- arch/arm64/include/asm/kvm_asm.h | 3 ++- arch/arm64/kvm/hyp/include/hyp/fault.h | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 2181a11b9d925..25f49f5fc4a63 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -10,6 +10,7 @@ #include #include #include +#include #define ARM_EXIT_WITH_SERROR_BIT 31 #define ARM_EXCEPTION_CODE(x) ((x) & ~(1U << ARM_EXIT_WITH_SERROR_BIT)) @@ -259,7 +260,7 @@ extern u64 __kvm_get_mdcr_el2(void); asm volatile( \ " mrs %1, spsr_el2\n" \ " mrs %2, elr_el2\n" \ - "1: at "at_op", %3\n" \ + "1: " __msr_s(at_op, "%3") "\n" \ " isb\n" \ " b 9f\n" \ "2: msr spsr_el2, %1\n" \ diff --git a/arch/arm64/kvm/hyp/include/hyp/fault.h b/arch/arm64/kvm/hyp/include/hyp/fault.h index 9e13c1bc2ad54..487c06099d6fc 100644 --- a/arch/arm64/kvm/hyp/include/hyp/fault.h +++ b/arch/arm64/kvm/hyp/include/hyp/fault.h @@ -27,7 +27,7 @@ static inline bool __translate_far_to_hpfar(u64 far, u64 *hpfar) * saved the guest context yet, and we may return early... */ par = read_sysreg_par(); - if (!__kvm_at("s1e1r", far)) + if (!__kvm_at(OP_AT_S1E1R, far)) tmp = read_sysreg_par(); else tmp = SYS_PAR_EL1_F; /* back to the guest */ From patchwork Tue Jun 25 13:35:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711136 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD89716B39A; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; cv=none; b=DSTBlrX6uzzgKdjYMqIncyp9AovCA2t1C35llJnvlRoclTcVNaTIpPM7Tps67ibv3p2qKPgqa04ITEgHk+6GUGByeG2gSpYS9VEVCW928EhD4e1XZKhQlDJjf9ZwcP+BImtkqZ5L67KKOA3l/tkwNjcZj+YIc/UyzUOBAVqGe9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; c=relaxed/simple; bh=c7fOw1CSlWTYN7dAgxetZtYg+b+8xyVjBZ2uZ5Ivxkg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=e+F0OMpdt4KfGOF0GM8MWNfd7jEZg13mQjrY8mQsZnILtGw+gaHnzmHT7Qgo15RiabYQiMIMee0Wx7n5N+w9BnSZekxU6/kdY9ogwrPVJrBS3FgbfQylTZijKVlwNQIpbDYzoGyiXDauH+7B9oRQ13aO8ZOfgQGcmaoeFRZYrus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AIfNGpWm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AIfNGpWm" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1B45C4AF0C; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=c7fOw1CSlWTYN7dAgxetZtYg+b+8xyVjBZ2uZ5Ivxkg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AIfNGpWmTbvYfgnzYCrxBdCDPLkuUp1zmD4u4JGHiwCLzKq0BWosXKyAU3aNG3mzM WNQzJfGXK+YZPhMP3+4mJOQA0rZOyzWGv2QLWHmBPOG6AmExuRe7O09Zkrjd8PfKWf 8GtyVLMhJ50MREdO8HBOgF1wCJL4EGK3QCyJZj6ImloCuD2KavSp7+vNmFLKwTTaZH yK2Y2ljTPR82NjQ7XPwSWFSf5/YVd0LS1GMMGqh8D8cMvvocLHka+1HHM1UdeYQqPU nzUnn4LOYzaxCzqJei9PtcvDHry2JLxAjYEXPbvFyjSCloawFBZ+Bcrd8F7OWBxBeU Vr4wc+9SinwaA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KQ-007A6l-Ta; Tue, 25 Jun 2024 14:35:22 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 06/12] KVM: arm64: nv: Add basic emulation of AT S1E{0,1}{R,W}[P] Date: Tue, 25 Jun 2024 14:35:05 +0100 Message-Id: <20240625133508.259829-7-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Emulating AT instructions is one the tasks devolved to the host hypervisor when NV is on. Here, we take the basic approach of emulating AT S1E{0,1}{R,W}[P] using the AT instructions themselves. While this mostly work, it doesn't *always* work: - S1 page tables can be swapped out - shadow S2 can be incomplete and not contain mappings for the S1 page tables We are not trying to handle these case here, and defer it to a later patch. Suitable comments indicate where we are in dire need of better handling. Co-developed-by: Jintack Lim Signed-off-by: Jintack Lim Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/Makefile | 2 +- arch/arm64/kvm/at.c | 197 +++++++++++++++++++++++++++++++ 3 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/at.c diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 25f49f5fc4a63..9b6c9f4f4d885 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -236,6 +236,7 @@ extern void __kvm_tlb_flush_vmid(struct kvm_s2_mmu *mmu); extern int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding); extern void __kvm_timer_set_cntvoff(u64 cntvoff); +extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index a6497228c5a8c..8a3ae76b4da22 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -14,7 +14,7 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ inject_fault.o va_layout.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o stacktrace.o \ vgic-sys-reg-v3.o fpsimd.o pkvm.o \ - arch_timer.o trng.o vmid.o emulate-nested.o nested.o \ + arch_timer.o trng.o vmid.o emulate-nested.o nested.o at.o \ vgic/vgic.o vgic/vgic-init.o \ vgic/vgic-irqfd.o vgic/vgic-v2.o \ vgic/vgic-v3.o vgic/vgic-v4.o \ diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c new file mode 100644 index 0000000000000..eb0aa49e61f68 --- /dev/null +++ b/arch/arm64/kvm/at.c @@ -0,0 +1,197 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017 - Linaro Ltd + * Author: Jintack Lim + */ + +#include +#include + +struct mmu_config { + u64 ttbr0; + u64 ttbr1; + u64 tcr; + u64 mair; + u64 sctlr; + u64 vttbr; + u64 vtcr; + u64 hcr; +}; + +static void __mmu_config_save(struct mmu_config *config) +{ + config->ttbr0 = read_sysreg_el1(SYS_TTBR0); + config->ttbr1 = read_sysreg_el1(SYS_TTBR1); + config->tcr = read_sysreg_el1(SYS_TCR); + config->mair = read_sysreg_el1(SYS_MAIR); + config->sctlr = read_sysreg_el1(SYS_SCTLR); + config->vttbr = read_sysreg(vttbr_el2); + config->vtcr = read_sysreg(vtcr_el2); + config->hcr = read_sysreg(hcr_el2); +} + +static void __mmu_config_restore(struct mmu_config *config) +{ + write_sysreg_el1(config->ttbr0, SYS_TTBR0); + write_sysreg_el1(config->ttbr1, SYS_TTBR1); + write_sysreg_el1(config->tcr, SYS_TCR); + write_sysreg_el1(config->mair, SYS_MAIR); + write_sysreg_el1(config->sctlr, SYS_SCTLR); + write_sysreg(config->vttbr, vttbr_el2); + write_sysreg(config->vtcr, vtcr_el2); + /* + * ARM errata 1165522 and 1530923 require the actual execution of the + * above before we can switch to the EL1/EL0 translation regime used by + * the guest. + */ + asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); + + write_sysreg(config->hcr, hcr_el2); + + isb(); +} + +static bool check_at_pan(struct kvm_vcpu *vcpu, u64 vaddr, u64 *res) +{ + u64 par_e0; + bool fail; + + /* + * For PAN-involved AT operations, perform the same translation, + * using EL0 this time. Twice. Much fun. + */ + fail = __kvm_at(OP_AT_S1E0R, vaddr); + if (fail) + return true; + + par_e0 = read_sysreg_par(); + if (!(par_e0 & SYS_PAR_EL1_F)) + goto out; + + fail = __kvm_at(OP_AT_S1E0W, vaddr); + if (fail) + return true; + + par_e0 = read_sysreg_par(); +out: + *res = par_e0; + return false; +} + +void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) +{ + struct mmu_config config; + struct kvm_s2_mmu *mmu; + unsigned long flags; + bool fail; + u64 par; + + write_lock(&vcpu->kvm->mmu_lock); + + /* + * We've trapped, so everything is live on the CPU. As we will + * be switching contexts behind everybody's back, disable + * interrupts... + */ + local_irq_save(flags); + __mmu_config_save(&config); + + /* + * If HCR_EL2.{E2H,TGE} == {1,1}, the MMU context is already + * the right one (as we trapped from vEL2). We have done too + * much work by saving the full MMU context, but who cares? + */ + if (vcpu_el2_e2h_is_set(vcpu) && vcpu_el2_tge_is_set(vcpu)) + goto skip_mmu_switch; + + /* + * FIXME: Obtaining the S2 MMU for a L2 is horribly racy, and + * we may not find it (recycled by another vcpu, for example). + * See the other FIXME comment below about the need for a SW + * PTW in this case. + */ + mmu = lookup_s2_mmu(vcpu); + if (WARN_ON(!mmu)) + goto out; + + write_sysreg_el1(vcpu_read_sys_reg(vcpu, TTBR0_EL1), SYS_TTBR0); + write_sysreg_el1(vcpu_read_sys_reg(vcpu, TTBR1_EL1), SYS_TTBR1); + write_sysreg_el1(vcpu_read_sys_reg(vcpu, TCR_EL1), SYS_TCR); + write_sysreg_el1(vcpu_read_sys_reg(vcpu, MAIR_EL1), SYS_MAIR); + write_sysreg_el1(vcpu_read_sys_reg(vcpu, SCTLR_EL1), SYS_SCTLR); + __load_stage2(mmu, mmu->arch); + +skip_mmu_switch: + /* Clear TGE, enable S2 translation, we're rolling */ + write_sysreg((config.hcr & ~HCR_TGE) | HCR_VM, hcr_el2); + isb(); + + switch (op) { + case OP_AT_S1E1R: + case OP_AT_S1E1RP: + fail = __kvm_at(OP_AT_S1E1R, vaddr); + break; + case OP_AT_S1E1W: + case OP_AT_S1E1WP: + fail = __kvm_at(OP_AT_S1E1W, vaddr); + break; + case OP_AT_S1E0R: + fail = __kvm_at(OP_AT_S1E0R, vaddr); + break; + case OP_AT_S1E0W: + fail = __kvm_at(OP_AT_S1E0W, vaddr); + break; + default: + WARN_ON_ONCE(1); + fail = true; + break; + } + + if (!fail) + par = read_sysreg(par_el1); + else + par = SYS_PAR_EL1_F; + + vcpu_write_sys_reg(vcpu, par, PAR_EL1); + + /* + * Failed? let's leave the building now. + * + * FIXME: how about a failed translation because the shadow S2 + * wasn't populated? We may need to perform a SW PTW, + * populating our shadow S2 and retry the instruction. + */ + if (par & SYS_PAR_EL1_F) + goto nopan; + + /* No PAN? No problem. */ + if (!(*vcpu_cpsr(vcpu) & PSR_PAN_BIT)) + goto nopan; + + switch (op) { + case OP_AT_S1E1RP: + case OP_AT_S1E1WP: + fail = check_at_pan(vcpu, vaddr, &par); + break; + default: + goto nopan; + } + + /* + * If the EL0 translation has succeeded, we need to pretend + * the AT operation has failed, as the PAN setting forbids + * such a translation. + * + * FIXME: we hardcode a Level-3 permission fault. We really + * should return the real fault level. + */ + if (fail || !(par & SYS_PAR_EL1_F)) + vcpu_write_sys_reg(vcpu, (0xf << 1) | SYS_PAR_EL1_F, PAR_EL1); + +nopan: + __mmu_config_restore(&config); +out: + local_irq_restore(flags); + + write_unlock(&vcpu->kvm->mmu_lock); +} From patchwork Tue Jun 25 13:35:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711137 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBE6C16B3A0; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; cv=none; b=YDvojlvIwv7vqQ6LgazOA0YU2eLO9ygqW+L6xkNfZ1SgmMPkIGqWcYM4lMBf1+OCFC8TIRagg6Whs4ovpStAAk6vwb8ftHq4DB/X56lN63XmEuMX71LNHrelHsKifn6tWCR1/b67Im1y+6maCYVr8EbFjmrr5bRqHUGfGYCN/Bs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; c=relaxed/simple; bh=GYYU7oaazXbNaqz2ULj2k5bsMFiur2S8hJufSai2nx4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kHup9P4WziDKpyfeoChe9MK26nPlnGQeR2JRfRC3O9tw/oxbp4IVGDJhHhDDvBPi1iLfqs5Soahn91w6xgHVpxlcXrbm6JGx5D5RSJx95MIyHZ3leYC/YEb+GhPB8gspMub8puQaK6zHuUUWVevyp8AlDSKr5QxQwmhmzB6G9KQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Xpzyr36q; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Xpzyr36q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD8A6C4AF0D; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322524; bh=GYYU7oaazXbNaqz2ULj2k5bsMFiur2S8hJufSai2nx4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xpzyr36qQjQUJMMnP7fnMA9Ks5+IJtwuMhwkX9wE+iek3+SvXAVtZyFJLNZeA5EUf sqpMNtyUWDGsLGo+fEb+dLWJXU6eRXR2OBs3PkVUb6WHKb9LURkRJQ5OtyS7fF/xUH p+8W/FcrWU/zy/iZ0L0s/q2Hek63XEMqoTW011y80UyBOpxTO8KrkbNLlJW04hsAQd g+dKyWXKgjVAPZ9cRO8/6b/VSKlOCuEfInyqyNQcuadX1lPwaxCpO+63VSN9HF9oAH 4MT8fvBwm5+hSFAHVK5zhhdw22N4saA3LXtFsIEcjajAhKMPOOklOhWe0CLodqKLDA zE1VFYnj4Yg7Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KR-007A6l-2e; Tue, 25 Jun 2024 14:35:23 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 07/12] KVM: arm64: nv: Add basic emulation of AT S1E2{R,W} Date: Tue, 25 Jun 2024 14:35:06 +0100 Message-Id: <20240625133508.259829-8-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Similar to our AT S1E{0,1} emulation, we implement the AT S1E2 handling. This emulation of course suffers from the same problems, but is somehow simpler due to the lack of PAN2 and the fact that we are guaranteed to execute it from the correct context. Co-developed-by: Jintack Lim Signed-off-by: Jintack Lim Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/at.c | 57 ++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 9b6c9f4f4d885..6ec0622969766 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -237,6 +237,7 @@ extern int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding); extern void __kvm_timer_set_cntvoff(u64 cntvoff); extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); +extern void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c index eb0aa49e61f68..147df5a9cc4e0 100644 --- a/arch/arm64/kvm/at.c +++ b/arch/arm64/kvm/at.c @@ -195,3 +195,60 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) write_unlock(&vcpu->kvm->mmu_lock); } + +void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) +{ + struct kvm_s2_mmu *mmu; + unsigned long flags; + u64 val, hcr, par; + bool fail; + + write_lock(&vcpu->kvm->mmu_lock); + + mmu = &vcpu->kvm->arch.mmu; + + /* + * We've trapped, so everything is live on the CPU. As we will + * be switching context behind everybody's back, disable + * interrupts... + */ + local_irq_save(flags); + + val = hcr = read_sysreg(hcr_el2); + val &= ~HCR_TGE; + val |= HCR_VM; + + if (!vcpu_el2_e2h_is_set(vcpu)) + val |= HCR_NV | HCR_NV1; + + write_sysreg(val, hcr_el2); + isb(); + + switch (op) { + case OP_AT_S1E2R: + fail = __kvm_at(OP_AT_S1E1R, vaddr); + break; + case OP_AT_S1E2W: + fail = __kvm_at(OP_AT_S1E1W, vaddr); + break; + default: + WARN_ON_ONCE(1); + fail = true; + } + + isb(); + + if (!fail) + par = read_sysreg_par(); + else + par = SYS_PAR_EL1_F; + + write_sysreg(hcr, hcr_el2); + isb(); + + local_irq_restore(flags); + + write_unlock(&vcpu->kvm->mmu_lock); + + vcpu_write_sys_reg(vcpu, par, PAR_EL1); +} From patchwork Tue Jun 25 13:35:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711138 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34CF416C438; Tue, 25 Jun 2024 13:35:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; cv=none; b=sPPoD6qF8PoJiqazxJt3ikM/rbNHesOIIt/I6beSkTWlT/IQLupWtgmepwag1pLx5GIq0AobowumTyVNsm6zeLuwuEDLmwAj56223HJazb69apAmzjs+ah9SOI9u/K76eiTG907n3GE2F52JIwgzob1xKVfu9PVUJKKW/wz4lB4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; c=relaxed/simple; bh=rn9J8T1s1vtwoAIycftZ9EaEpIfuyGrD/4j0MdkPZd0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=amoOISVfD55HVktGgHpJ2YpsDrNK3KbtlAmABgjEQykozulsXtS43GQ39ANzelxJcINPCYlyVlTF5L6wZCP+SSm2g/bpy0zZFj+zFsZEIC3KIrWOUYCjVPyfIGBD8s0ib6uoKm88MxZPGWjV4acxmguM6HVaplTqduRarSc7GJw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YSwBi9Jx; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YSwBi9Jx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EEC6FC4AF0F; Tue, 25 Jun 2024 13:35:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322525; bh=rn9J8T1s1vtwoAIycftZ9EaEpIfuyGrD/4j0MdkPZd0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YSwBi9JxQL5EuDs0g9PQ4kfbl1I3XJMq4T+pMZ2Pvv9HIqcL99WjcevqYzOr5BXKd XlA4JTL45PQqqA4MwcKmsQaoh8W5RWhTx1nkuWEkpBJ+M+Xh4HCEmd9redzICNm+sL kTzwCenxaBuQ9K3HEuXnm7v0sxJJGXBDSGyCJIPINjvN6EDND/17hEyY1pg31Fs24K 0mr2sOEe/cqdDD/j4UFpEe4JfVyiABustmxh4SDM12Re0u7qsuo4cXJJjVZoeXiYsO KHgonORNwGE6FAAMYoxYniFwvb88yVALu/jrK2aPSV0CtKIdi9dNQv0UYpN5UhTRHC r39/UK9r6ypGw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KR-007A6l-83; Tue, 25 Jun 2024 14:35:23 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 08/12] KVM: arm64: nv: Add emulation of AT S12E{0,1}{R,W} Date: Tue, 25 Jun 2024 14:35:07 +0100 Message-Id: <20240625133508.259829-9-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On the face of it, AT S12E{0,1}{R,W} is pretty simple. It is the combination of AT S1E{0,1}{R,W}, followed by an extra S2 walk. However, there is a great deal of complexity coming from combining the S1 and S2 attributes to report something consistent in PAR_EL1. This is an absolute mine field, and I have a splitting headache. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/at.c | 242 +++++++++++++++++++++++++++++++ 2 files changed, 243 insertions(+) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 6ec0622969766..b36a3b6cc0116 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -238,6 +238,7 @@ extern int __kvm_tlbi_s1e2(struct kvm_s2_mmu *mmu, u64 va, u64 sys_encoding); extern void __kvm_timer_set_cntvoff(u64 cntvoff); extern void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); extern void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); +extern void __kvm_at_s12(struct kvm_vcpu *vcpu, u32 op, u64 vaddr); extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c index 147df5a9cc4e0..71e3390b43b4c 100644 --- a/arch/arm64/kvm/at.c +++ b/arch/arm64/kvm/at.c @@ -51,6 +51,189 @@ static void __mmu_config_restore(struct mmu_config *config) isb(); } +#define MEMATTR(ic, oc) (MEMATTR_##oc << 4 | MEMATTR_##ic) +#define MEMATTR_NC 0b0100 +#define MEMATTR_Wt 0b1000 +#define MEMATTR_Wb 0b1100 +#define MEMATTR_WbRaWa 0b1111 + +#define MEMATTR_IS_DEVICE(m) (((m) & GENMASK(7, 4)) == 0) + +static u8 s2_memattr_to_attr(u8 memattr) +{ + memattr &= 0b1111; + + switch (memattr) { + case 0b0000: + case 0b0001: + case 0b0010: + case 0b0011: + return memattr << 2; + case 0b0100: + return MEMATTR(Wb, Wb); + case 0b0101: + return MEMATTR(NC, NC); + case 0b0110: + return MEMATTR(Wt, NC); + case 0b0111: + return MEMATTR(Wb, NC); + case 0b1000: + /* Reserved, assume NC */ + return MEMATTR(NC, NC); + case 0b1001: + return MEMATTR(NC, Wt); + case 0b1010: + return MEMATTR(Wt, Wt); + case 0b1011: + return MEMATTR(Wb, Wt); + case 0b1100: + /* Reserved, assume NC */ + return MEMATTR(NC, NC); + case 0b1101: + return MEMATTR(NC, Wb); + case 0b1110: + return MEMATTR(Wt, Wb); + case 0b1111: + return MEMATTR(Wb, Wb); + default: + unreachable(); + } +} + +static u8 combine_s1_s2_attr(u8 s1, u8 s2) +{ + bool transient; + u8 final = 0; + + /* Upgrade transient s1 to non-transient to simplify things */ + switch (s1) { + case 0b0001 ... 0b0011: /* Normal, Write-Through Transient */ + transient = true; + s1 = MEMATTR_Wt | (s1 & GENMASK(1,0)); + break; + case 0b0101 ... 0b0111: /* Normal, Write-Back Transient */ + transient = true; + s1 = MEMATTR_Wb | (s1 & GENMASK(1,0)); + break; + default: + transient = false; + } + + /* S2CombineS1AttrHints() */ + if ((s1 & GENMASK(3, 2)) == MEMATTR_NC || + (s2 & GENMASK(3, 2)) == MEMATTR_NC) + final = MEMATTR_NC; + else if ((s1 & GENMASK(3, 2)) == MEMATTR_Wt || + (s2 & GENMASK(3, 2)) == MEMATTR_Wt) + final = MEMATTR_Wt; + else + final = MEMATTR_Wb; + + if (final != MEMATTR_NC) { + /* Inherit RaWa hints form S1 */ + if (transient) { + switch (s1 & GENMASK(3, 2)) { + case MEMATTR_Wt: + final = 0; + break; + case MEMATTR_Wb: + final = MEMATTR_NC; + break; + } + } + + final |= s1 & GENMASK(1, 0); + } + + return final; +} + +static u8 compute_sh(u8 attr, u64 desc) +{ + /* Any form of device, as well as NC has SH[1:0]=0b10 */ + if (MEMATTR_IS_DEVICE(attr) || attr == MEMATTR(NC, NC)) + return 0b10; + + return FIELD_GET(PTE_SHARED, desc) == 0b11 ? 0b11 : 0b10; +} + +static u64 compute_par_s12(struct kvm_vcpu *vcpu, u64 s1_par, + struct kvm_s2_trans *tr) +{ + u8 s1_parattr, s2_memattr, final_attr; + u64 par; + + /* If S2 has failed to translate, report the damage */ + if (tr->esr) { + par = SYS_PAR_EL1_RES1; + par |= SYS_PAR_EL1_F; + par |= SYS_PAR_EL1_S; + par |= FIELD_PREP(SYS_PAR_EL1_FST, tr->esr); + return par; + } + + s1_parattr = FIELD_GET(SYS_PAR_EL1_ATTR, s1_par); + s2_memattr = FIELD_GET(GENMASK(5, 2), tr->desc); + + if (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_FWB) { + if (!kvm_has_feat(vcpu->kvm, ID_AA64PFR2_EL1, MTEPERM, IMP)) + s2_memattr &= ~BIT(3); + + /* Combination of R_VRJSW and R_RHWZM */ + switch (s2_memattr) { + case 0b0101: + if (MEMATTR_IS_DEVICE(s1_parattr)) + final_attr = s1_parattr; + else + final_attr = MEMATTR(NC, NC); + break; + case 0b0110: + case 0b1110: + final_attr = MEMATTR(WbRaWa, WbRaWa); + break; + case 0b0111: + case 0b1111: + /* Preserve S1 attribute */ + final_attr = s1_parattr; + break; + case 0b0100: + case 0b1100: + case 0b1101: + /* Reserved, do something non-silly */ + final_attr = s1_parattr; + break; + default: + /* MemAttr[2]=0, Device from S2 */ + final_attr = s2_memattr & GENMASK(1,0) << 2; + } + } else { + /* Combination of R_HMNDG, R_TNHFM and R_GQFSF */ + u8 s2_parattr = s2_memattr_to_attr(s2_memattr); + + if (MEMATTR_IS_DEVICE(s1_parattr) || + MEMATTR_IS_DEVICE(s2_parattr)) { + final_attr = min(s1_parattr, s2_parattr); + } else { + /* At this stage, this is memory vs memory */ + final_attr = combine_s1_s2_attr(s1_parattr & 0xf, + s2_parattr & 0xf); + final_attr |= combine_s1_s2_attr(s1_parattr >> 4, + s2_parattr >> 4) << 4; + } + } + + if ((__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_CD) && + !MEMATTR_IS_DEVICE(final_attr)) + final_attr = MEMATTR(NC, NC); + + par = FIELD_PREP(SYS_PAR_EL1_ATTR, final_attr); + par |= tr->output & GENMASK(47, 12); + par |= FIELD_PREP(SYS_PAR_EL1_SH, + compute_sh(final_attr, tr->desc)); + + return par; +} + static bool check_at_pan(struct kvm_vcpu *vcpu, u64 vaddr, u64 *res) { u64 par_e0; @@ -252,3 +435,62 @@ void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) vcpu_write_sys_reg(vcpu, par, PAR_EL1); } + +void __kvm_at_s12(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) +{ + struct kvm_s2_trans out = {}; + u64 ipa, par; + bool write; + int ret; + + /* Do the stage-1 translation */ + switch (op) { + case OP_AT_S12E1R: + op = OP_AT_S1E1R; + write = false; + break; + case OP_AT_S12E1W: + op = OP_AT_S1E1W; + write = true; + break; + case OP_AT_S12E0R: + op = OP_AT_S1E0R; + write = false; + break; + case OP_AT_S12E0W: + op = OP_AT_S1E0W; + write = true; + break; + default: + WARN_ON_ONCE(1); + return; + } + + __kvm_at_s1e01(vcpu, op, vaddr); + par = vcpu_read_sys_reg(vcpu, PAR_EL1); + if (par & SYS_PAR_EL1_F) + return; + + /* + * If we only have a single stage of translation (E2H=0 or + * TGE=1), exit early. Same thing if {VM,DC}=={0,0}. + */ + if (!vcpu_el2_e2h_is_set(vcpu) || vcpu_el2_tge_is_set(vcpu) || + !(vcpu_read_sys_reg(vcpu, HCR_EL2) & (HCR_VM | HCR_DC))) + return; + + /* Do the stage-2 translation */ + ipa = (par & GENMASK_ULL(47, 12)) | (vaddr & GENMASK_ULL(11, 0)); + out.esr = 0; + ret = kvm_walk_nested_s2(vcpu, ipa, &out); + if (ret < 0) + return; + + /* Check the access permission */ + if (!out.esr && + ((!write && !out.readable) || (write && !out.writable))) + out.esr = ESR_ELx_FSC_PERM | (out.level & 0x3); + + par = compute_par_s12(vcpu, par, &out); + vcpu_write_sys_reg(vcpu, par, PAR_EL1); +} From patchwork Tue Jun 25 13:35:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13711139 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F17916C684; Tue, 25 Jun 2024 13:35:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; cv=none; b=M2vRN0C4jL0H3PNTiSQgUdFz10KjxhPuEfR27j4M3FaKkoqFS5ee7zO7w6v1QAGYpw0FLsyPsdOwgkAVmdtXoK6kDV8BmQ91SA8HGgM9lp5DyWgctU/nFICOSU6f45fF/H14ozAbeO7jvNGBqfbdWt5nb2EZiJlvGqn46qMmi0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719322525; c=relaxed/simple; bh=de5Lj1pevoVMn753FTewXOjvi0z3RxqGLbk+fPFuegw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ro51co2Gqw/V3WJnFwzY7RE0oARCX5M1Oo+ZmCItYLDik29XW18jHCZOgKGmdYt94YWlLcmddSsGfOemAp08fRVU1vMeGf2RQrw0HFnhmAMriovcXOWJKEnezJEqGnnYaO4h1dGTgL+JxG85cjfzL8zi5Nfx1xWDADmFa7kVXZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pHy0NuGp; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pHy0NuGp" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 20F82C4AF12; Tue, 25 Jun 2024 13:35:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1719322525; bh=de5Lj1pevoVMn753FTewXOjvi0z3RxqGLbk+fPFuegw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pHy0NuGpJIYywB8F5GPKsPikRNpJ+kQLbRP9XVsB8251LDsBZdwqKr7xQ2B7azBil FSgoyJWEDaku94x+lOfa3wZH/Z0s40MseiN6Zu2fp8KlbxmAdNPueaFgLxT6RdV1ni mRTwiUyum5lKrx8WY2zLtzvmOv6dxprMKMjGUddg1p/44yJDGT63+5a7/XYmJyjFPO IzLqRGtTT4JeQNdz9l0D9nJ/BC4Qb4o0Xwqxf6pnFErJacPq4o2aFTQVhhuoQIT320 JLeelokZLCADVFBLWDGBA0oJwBea/h3IOFTEaXP79mxMBVb4O0YG8w4bKzqJgWXc14 DMfb1zfqHwC5A== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sM6KR-007A6l-DZ; Tue, 25 Jun 2024 14:35:23 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 09/12] KVM: arm64: nv: Make ps_to_output_size() generally available Date: Tue, 25 Jun 2024 14:35:08 +0100 Message-Id: <20240625133508.259829-10-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Make this helper visible to at.c, we are going to need it. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_nested.h | 14 ++++++++++++++ arch/arm64/kvm/nested.c | 14 -------------- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h index b2fe759964d83..c7adbddbab33a 100644 --- a/arch/arm64/include/asm/kvm_nested.h +++ b/arch/arm64/include/asm/kvm_nested.h @@ -205,4 +205,18 @@ static inline u64 kvm_encode_nested_level(struct kvm_s2_trans *trans) return FIELD_PREP(KVM_NV_GUEST_MAP_SZ, trans->level); } +static inline unsigned int ps_to_output_size(unsigned int ps) +{ + switch (ps) { + case 0: return 32; + case 1: return 36; + case 2: return 40; + case 3: return 42; + case 4: return 44; + case 5: + default: + return 48; + } +} + #endif /* __ARM64_KVM_NESTED_H */ diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 73544e0e64dcb..a77b3181cd65d 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -103,20 +103,6 @@ struct s2_walk_info { bool be; }; -static unsigned int ps_to_output_size(unsigned int ps) -{ - switch (ps) { - case 0: return 32; - case 1: return 36; - case 2: return 40; - case 3: return 42; - case 4: return 44; - case 5: - default: - return 48; - } -} - static u32 compute_fsc(int level, u32 fsc) { return fsc | (level & 0x3); From patchwork Mon Jul 8 16:57:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13726875 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30F23147C82; Mon, 8 Jul 2024 16:58:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; cv=none; b=plRkWEKgvyp54m8tnhT8Luoc9jzBItzaKfynIrhHD0P6ZjioaDdFg55pZHKaxrLtsyxz9AzYWLkuntAAdo0PnaiHsWYrILAgzSPII+mK8xz27fxLofK/81vCWU3+cNzC+q6eurq0XO0co1r4ECItPM0wwzlhTtGNuV7snuntqCI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; c=relaxed/simple; bh=/TykIOsi0c427lagmyC8xWzajxZQOo1+fEc6oZtgjFY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iBopqzYvFyZ9xX89YLXydW5w1juUNLjNBZltI4sEtSfoIPKggRGRgnqOwmyXOrgVyKA5hsgLkesadcsRLMx0c4d7XBQM9nMNm4S4NM600S9qzXfmg/8W9OoBnhD1C/Zwzm8rOCrByYD+jVwsuWv1qONT1bL+wZb9ey4qiDYXMnQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DY0GYHMe; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DY0GYHMe" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C307CC116B1; Mon, 8 Jul 2024 16:58:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720457890; bh=/TykIOsi0c427lagmyC8xWzajxZQOo1+fEc6oZtgjFY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DY0GYHMeofNJjrLdQ0tkV1vuO/ogNEtKPlfdcfiREd9+vPWOom6DJ4TUD6p3Xfw+r U3zbmdMOaCIoqMhwRTHWVzgJhtE5MDrdH8QPAopqQcq0SbHQiWPkZfVwDT/8omFTBQ 4FUjeMoRqS3THkNvMls+yx8F2TgQXJ4nvT39h4pYGvxtJlknrFA4Ve8Zd9lK0Yji51 KrwQb7SoZuX57LXN6p100Zcsk0C6qqZxMEc0nu1mH+ZJ2G8rYoraGxQ1tclpgm8yJj znufjDou2qoAu8qXOG7H1rub7490Alx2F9NmsPvBqV52n/sfVC2QYGXODTf2Lc5847 RRJ9MbUt85FUQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sQrgl-00AfMX-KV; Mon, 08 Jul 2024 17:58:07 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 10/12] KVM: arm64: nv: Add SW walker for AT S1 emulation Date: Mon, 8 Jul 2024 17:57:58 +0100 Message-Id: <20240708165800.1220065-1-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240625133508.259829-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false In order to plug the brokenness of our current AT implementation, we need a SW walker that is going to... err.. walk the S1 tables and tell us what it finds. Of course, it builds on top of our S2 walker, and share similar concepts. The beauty of it is that since it uses kvm_read_guest(), it is able to bring back pages that have been otherwise evicted. This is then plugged in the two AT S1 emulation functions as a "slow path" fallback. I'm not sure it is that slow, but hey. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/at.c | 538 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 520 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c index 71e3390b43b4c..8452273cbff6d 100644 --- a/arch/arm64/kvm/at.c +++ b/arch/arm64/kvm/at.c @@ -4,9 +4,305 @@ * Author: Jintack Lim */ +#include + +#include #include #include +struct s1_walk_info { + u64 baddr; + unsigned int max_oa_bits; + unsigned int pgshift; + unsigned int txsz; + int sl; + bool hpd; + bool be; + bool nvhe; + bool s2; +}; + +struct s1_walk_result { + union { + struct { + u64 desc; + u64 pa; + s8 level; + u8 APTable; + bool UXNTable; + bool PXNTable; + }; + struct { + u8 fst; + bool ptw; + bool s2; + }; + }; + bool failed; +}; + +static void fail_s1_walk(struct s1_walk_result *wr, u8 fst, bool ptw, bool s2) +{ + wr->fst = fst; + wr->ptw = ptw; + wr->s2 = s2; + wr->failed = true; +} + +#define S1_MMU_DISABLED (-127) + +static int setup_s1_walk(struct kvm_vcpu *vcpu, struct s1_walk_info *wi, + struct s1_walk_result *wr, const u64 va, const int el) +{ + u64 sctlr, tcr, tg, ps, ia_bits, ttbr; + unsigned int stride, x; + bool va55, tbi; + + wi->nvhe = el == 2 && !vcpu_el2_e2h_is_set(vcpu); + + va55 = va & BIT(55); + + if (wi->nvhe && va55) + goto addrsz; + + wi->s2 = el < 2 && (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_VM); + + switch (el) { + case 1: + sctlr = vcpu_read_sys_reg(vcpu, SCTLR_EL1); + tcr = vcpu_read_sys_reg(vcpu, TCR_EL1); + ttbr = (va55 ? + vcpu_read_sys_reg(vcpu, TTBR1_EL1) : + vcpu_read_sys_reg(vcpu, TTBR0_EL1)); + break; + case 2: + sctlr = vcpu_read_sys_reg(vcpu, SCTLR_EL2); + tcr = vcpu_read_sys_reg(vcpu, TCR_EL2); + ttbr = (va55 ? + vcpu_read_sys_reg(vcpu, TTBR1_EL2) : + vcpu_read_sys_reg(vcpu, TTBR0_EL2)); + break; + default: + BUG(); + } + + /* Let's put the MMU disabled case aside immediately */ + if (!(sctlr & SCTLR_ELx_M) || + (__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_DC)) { + if (va >= BIT(kvm_get_pa_bits(vcpu->kvm))) + goto addrsz; + + wr->level = S1_MMU_DISABLED; + wr->desc = va; + return 0; + } + + wi->be = sctlr & SCTLR_ELx_EE; + + wi->hpd = kvm_has_feat(vcpu->kvm, ID_AA64MMFR1_EL1, HPDS, IMP); + wi->hpd &= (wi->nvhe ? + FIELD_GET(TCR_EL2_HPD, tcr) : + (va55 ? + FIELD_GET(TCR_HPD1, tcr) : + FIELD_GET(TCR_HPD0, tcr))); + + tbi = (wi->nvhe ? + FIELD_GET(TCR_EL2_TBI, tcr) : + (va55 ? + FIELD_GET(TCR_TBI1, tcr) : + FIELD_GET(TCR_TBI0, tcr))); + + if (!tbi && sign_extend64(va, 55) != (s64)va) + goto addrsz; + + /* Someone was silly enough to encode TG0/TG1 differently */ + if (va55) { + wi->txsz = FIELD_GET(TCR_T1SZ_MASK, tcr); + tg = FIELD_GET(TCR_TG1_MASK, tcr); + + switch (tg << TCR_TG1_SHIFT) { + case TCR_TG1_4K: + wi->pgshift = 12; break; + case TCR_TG1_16K: + wi->pgshift = 14; break; + case TCR_TG1_64K: + default: /* IMPDEF: treat any other value as 64k */ + wi->pgshift = 16; break; + } + } else { + wi->txsz = FIELD_GET(TCR_T0SZ_MASK, tcr); + tg = FIELD_GET(TCR_TG0_MASK, tcr); + + switch (tg << TCR_TG0_SHIFT) { + case TCR_TG0_4K: + wi->pgshift = 12; break; + case TCR_TG0_16K: + wi->pgshift = 14; break; + case TCR_TG0_64K: + default: /* IMPDEF: treat any other value as 64k */ + wi->pgshift = 16; break; + } + } + + ia_bits = 64 - wi->txsz; + + /* AArch64.S1StartLevel() */ + stride = wi->pgshift - 3; + wi->sl = 3 - (((ia_bits - 1) - wi->pgshift) / stride); + + /* Check for SL mandating LPA2 (which we don't support yet) */ + switch (BIT(wi->pgshift)) { + case SZ_4K: + if (wi->sl == -1 && + !kvm_has_feat(vcpu->kvm, ID_AA64MMFR0_EL1, TGRAN4, 52_BIT)) + goto addrsz; + break; + case SZ_16K: + if (wi->sl == 0 && + !kvm_has_feat(vcpu->kvm, ID_AA64MMFR0_EL1, TGRAN16, 52_BIT)) + goto addrsz; + break; + } + + ps = (wi->nvhe ? + FIELD_GET(TCR_EL2_PS_MASK, tcr) : FIELD_GET(TCR_IPS_MASK, tcr)); + + wi->max_oa_bits = min(get_kvm_ipa_limit(), ps_to_output_size(ps)); + + /* Compute minimal alignment */ + x = 3 + ia_bits - ((3 - wi->sl) * stride + wi->pgshift); + + wi->baddr = ttbr & TTBRx_EL1_BADDR; + wi->baddr &= GENMASK_ULL(wi->max_oa_bits - 1, x); + + return 0; + +addrsz: /* Address Size Fault level 0 */ + fail_s1_walk(wr, ESR_ELx_FSC_ADDRSZ, false, false); + + return -EFAULT; +} + +static int get_ia_size(struct s1_walk_info *wi) +{ + return 64 - wi->txsz; +} + +static int walk_s1(struct kvm_vcpu *vcpu, struct s1_walk_info *wi, + struct s1_walk_result *wr, u64 va) +{ + u64 va_top, va_bottom, baddr, desc; + int level, stride, ret; + + level = wi->sl; + stride = wi->pgshift - 3; + baddr = wi->baddr; + + va_top = get_ia_size(wi) - 1; + + while (1) { + u64 index, ipa; + + va_bottom = (3 - level) * stride + wi->pgshift; + index = (va & GENMASK_ULL(va_top, va_bottom)) >> (va_bottom - 3); + + ipa = baddr | index; + + if (wi->s2) { + struct kvm_s2_trans s2_trans = {}; + + ret = kvm_walk_nested_s2(vcpu, ipa, &s2_trans); + if (ret) { + fail_s1_walk(wr, + (s2_trans.esr & ~ESR_ELx_FSC_LEVEL) | level, + true, true); + return ret; + } + + if (!kvm_s2_trans_readable(&s2_trans)) { + fail_s1_walk(wr, ESR_ELx_FSC_PERM | level, + true, true); + + return -EPERM; + } + + ipa = kvm_s2_trans_output(&s2_trans); + } + + ret = kvm_read_guest(vcpu->kvm, ipa, &desc, sizeof(desc)); + if (ret) { + fail_s1_walk(wr, ESR_ELx_FSC_SEA_TTW(level), + true, false); + return ret; + } + + if (wi->be) + desc = be64_to_cpu((__force __be64)desc); + else + desc = le64_to_cpu((__force __le64)desc); + + if (!(desc & 1) || ((desc & 3) == 1 && level == 3)) { + fail_s1_walk(wr, ESR_ELx_FSC_FAULT | level, + true, false); + return -ENOENT; + } + + /* We found a leaf, handle that */ + if ((desc & 3) == 1 || level == 3) + break; + + if (!wi->hpd) { + wr->APTable |= FIELD_GET(PMD_TABLE_AP, desc); + wr->UXNTable |= FIELD_GET(PMD_TABLE_UXN, desc); + wr->PXNTable |= FIELD_GET(PMD_TABLE_PXN, desc); + } + + baddr = GENMASK_ULL(47, wi->pgshift); + + /* Check for out-of-range OA */ + if (wi->max_oa_bits < 48 && + (baddr & GENMASK_ULL(47, wi->max_oa_bits))) { + fail_s1_walk(wr, ESR_ELx_FSC_ADDRSZ | level, + true, false); + return -EINVAL; + } + + /* Prepare for next round */ + va_top = va_bottom - 1; + level++; + } + + /* Block mapping, check the validity of the level */ + if (!(desc & BIT(1))) { + bool valid_block = false; + + switch (BIT(wi->pgshift)) { + case SZ_4K: + valid_block = level == 1 || level == 2; + break; + case SZ_16K: + case SZ_64K: + valid_block = level == 2; + break; + } + + if (!valid_block) { + fail_s1_walk(wr, ESR_ELx_FSC_FAULT | level, + true, false); + return -EINVAL; + } + } + + wr->failed = false; + wr->level = level; + wr->desc = desc; + wr->pa = desc & GENMASK(47, va_bottom); + if (va_bottom > 12) + wr->pa |= va & GENMASK_ULL(va_bottom - 1, 12); + + return 0; +} + struct mmu_config { u64 ttbr0; u64 ttbr1; @@ -234,6 +530,177 @@ static u64 compute_par_s12(struct kvm_vcpu *vcpu, u64 s1_par, return par; } +static u64 compute_par_s1(struct kvm_vcpu *vcpu, struct s1_walk_result *wr) +{ + u64 par; + + if (wr->failed) { + par = SYS_PAR_EL1_RES1; + par |= SYS_PAR_EL1_F; + par |= FIELD_PREP(SYS_PAR_EL1_FST, wr->fst); + par |= wr->ptw ? SYS_PAR_EL1_PTW : 0; + par |= wr->s2 ? SYS_PAR_EL1_S : 0; + } else if (wr->level == S1_MMU_DISABLED) { + /* MMU off or HCR_EL2.DC == 1 */ + par = wr->pa & GENMASK_ULL(47, 12); + + if (!(__vcpu_sys_reg(vcpu, HCR_EL2) & HCR_DC)) { + par |= FIELD_PREP(SYS_PAR_EL1_ATTR, 0); /* nGnRnE */ + par |= FIELD_PREP(SYS_PAR_EL1_SH, 0b10); /* OS */ + } else { + par |= FIELD_PREP(SYS_PAR_EL1_ATTR, + MEMATTR(WbRaWa, WbRaWa)); + par |= FIELD_PREP(SYS_PAR_EL1_SH, 0b00); /* NS */ + } + } else { + u64 mair, sctlr; + int el; + u8 sh; + + el = (vcpu_el2_e2h_is_set(vcpu) && + vcpu_el2_tge_is_set(vcpu)) ? 2 : 1; + + mair = ((el == 2) ? + vcpu_read_sys_reg(vcpu, MAIR_EL2) : + vcpu_read_sys_reg(vcpu, MAIR_EL1)); + + mair >>= FIELD_GET(PTE_ATTRINDX_MASK, wr->desc) * 8; + mair &= 0xff; + + sctlr = ((el == 2) ? + vcpu_read_sys_reg(vcpu, SCTLR_EL2) : + vcpu_read_sys_reg(vcpu, SCTLR_EL1)); + + /* Force NC for memory if SCTLR_ELx.C is clear */ + if (!(sctlr & SCTLR_EL1_C) && !MEMATTR_IS_DEVICE(mair)) + mair = MEMATTR(NC, NC); + + par = FIELD_PREP(SYS_PAR_EL1_ATTR, mair); + par |= wr->pa & GENMASK_ULL(47, 12); + + sh = compute_sh(mair, wr->desc); + par |= FIELD_PREP(SYS_PAR_EL1_SH, sh); + } + + return par; +} + +static u64 handle_at_slow(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) +{ + bool perm_fail, ur, uw, ux, pr, pw, pan; + struct s1_walk_result wr = {}; + struct s1_walk_info wi = {}; + int ret, idx, el; + + /* + * We only get here from guest EL2, so the translation regime + * AT applies to is solely defined by {E2H,TGE}. + */ + el = (vcpu_el2_e2h_is_set(vcpu) && + vcpu_el2_tge_is_set(vcpu)) ? 2 : 1; + + ret = setup_s1_walk(vcpu, &wi, &wr, vaddr, el); + if (ret) + goto compute_par; + + if (wr.level == S1_MMU_DISABLED) + goto compute_par; + + idx = srcu_read_lock(&vcpu->kvm->srcu); + + ret = walk_s1(vcpu, &wi, &wr, vaddr); + + srcu_read_unlock(&vcpu->kvm->srcu, idx); + + if (ret) + goto compute_par; + + /* FIXME: revisit when adding indirect permission support */ + if (kvm_has_feat(vcpu->kvm, ID_AA64MMFR1_EL1, PAN, PAN3) && + !wi.nvhe) { + u64 sctlr; + + if (el == 1) + sctlr = vcpu_read_sys_reg(vcpu, SCTLR_EL1); + else + sctlr = vcpu_read_sys_reg(vcpu, SCTLR_EL2); + + ux = (sctlr & SCTLR_EL1_EPAN) && !(wr.desc & PTE_UXN); + } else { + ux = false; + } + + pw = !(wr.desc & PTE_RDONLY); + + if (wi.nvhe) { + ur = uw = false; + pr = true; + } else { + if (wr.desc & PTE_USER) { + ur = pr = true; + uw = pw; + } else { + ur = uw = false; + pr = true; + } + } + + /* Apply the Hierarchical Permission madness */ + if (wi.nvhe) { + wr.APTable &= BIT(1); + wr.PXNTable = wr.UXNTable; + } + + ur &= !(wr.APTable & BIT(0)); + uw &= !(wr.APTable != 0); + ux &= !wr.UXNTable; + + pw &= !(wr.APTable & BIT(1)); + + pan = *vcpu_cpsr(vcpu) & PSR_PAN_BIT; + + perm_fail = false; + + switch (op) { + case OP_AT_S1E1RP: + perm_fail |= pan && (ur || uw || ux); + fallthrough; + case OP_AT_S1E1R: + case OP_AT_S1E2R: + perm_fail |= !pr; + break; + case OP_AT_S1E1WP: + perm_fail |= pan && (ur || uw || ux); + fallthrough; + case OP_AT_S1E1W: + case OP_AT_S1E2W: + perm_fail |= !pw; + break; + case OP_AT_S1E0R: + perm_fail |= !ur; + break; + case OP_AT_S1E0W: + perm_fail |= !uw; + break; + default: + BUG(); + } + + if (perm_fail) { + struct s1_walk_result tmp; + + tmp.failed = true; + tmp.fst = ESR_ELx_FSC_PERM | wr.level; + tmp.s2 = false; + tmp.ptw = false; + + wr = tmp; + } + +compute_par: + return compute_par_s1(vcpu, &wr); +} + static bool check_at_pan(struct kvm_vcpu *vcpu, u64 vaddr, u64 *res) { u64 par_e0; @@ -266,9 +733,11 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) struct mmu_config config; struct kvm_s2_mmu *mmu; unsigned long flags; - bool fail; + bool fail, retry_slow; u64 par; + retry_slow = false; + write_lock(&vcpu->kvm->mmu_lock); /* @@ -288,14 +757,15 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) goto skip_mmu_switch; /* - * FIXME: Obtaining the S2 MMU for a L2 is horribly racy, and - * we may not find it (recycled by another vcpu, for example). - * See the other FIXME comment below about the need for a SW - * PTW in this case. + * Obtaining the S2 MMU for a L2 is horribly racy, and we may not + * find it (recycled by another vcpu, for example). When this + * happens, use the SW (slow) path. */ mmu = lookup_s2_mmu(vcpu); - if (WARN_ON(!mmu)) + if (!mmu) { + retry_slow = true; goto out; + } write_sysreg_el1(vcpu_read_sys_reg(vcpu, TTBR0_EL1), SYS_TTBR0); write_sysreg_el1(vcpu_read_sys_reg(vcpu, TTBR1_EL1), SYS_TTBR1); @@ -331,18 +801,17 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) } if (!fail) - par = read_sysreg(par_el1); + par = read_sysreg_par(); else par = SYS_PAR_EL1_F; + retry_slow = !fail; + vcpu_write_sys_reg(vcpu, par, PAR_EL1); /* - * Failed? let's leave the building now. - * - * FIXME: how about a failed translation because the shadow S2 - * wasn't populated? We may need to perform a SW PTW, - * populating our shadow S2 and retry the instruction. + * Failed? let's leave the building now, unless we retry on + * the slow path. */ if (par & SYS_PAR_EL1_F) goto nopan; @@ -354,29 +823,58 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) switch (op) { case OP_AT_S1E1RP: case OP_AT_S1E1WP: + retry_slow = false; fail = check_at_pan(vcpu, vaddr, &par); break; default: goto nopan; } + if (fail) { + vcpu_write_sys_reg(vcpu, SYS_PAR_EL1_F, PAR_EL1); + goto nopan; + } + /* * If the EL0 translation has succeeded, we need to pretend * the AT operation has failed, as the PAN setting forbids * such a translation. - * - * FIXME: we hardcode a Level-3 permission fault. We really - * should return the real fault level. */ - if (fail || !(par & SYS_PAR_EL1_F)) - vcpu_write_sys_reg(vcpu, (0xf << 1) | SYS_PAR_EL1_F, PAR_EL1); - + if (par & SYS_PAR_EL1_F) { + u8 fst = FIELD_GET(SYS_PAR_EL1_FST, par); + + /* + * If we get something other than a permission fault, we + * need to retry, as we're likely to have missed in the PTs. + */ + if ((fst & ESR_ELx_FSC_TYPE) != ESR_ELx_FSC_PERM) + retry_slow = true; + } else { + /* + * The EL0 access succeded, but we don't have the full + * syndrom information to synthetize the failure. Go slow. + */ + retry_slow = true; + } nopan: __mmu_config_restore(&config); out: local_irq_restore(flags); write_unlock(&vcpu->kvm->mmu_lock); + + /* + * If retry_slow is true, then we either are missing shadow S2 + * entries, have paged out guest S1, or something is inconsistent. + * + * Either way, we need to walk the PTs by hand so that we can either + * fault things back, in or record accurate fault information along + * the way. + */ + if (retry_slow) { + par = handle_at_slow(vcpu, op, vaddr); + vcpu_write_sys_reg(vcpu, par, PAR_EL1); + } } void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) @@ -433,6 +931,10 @@ void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) write_unlock(&vcpu->kvm->mmu_lock); + /* We failed the translation, let's replay it in slow motion */ + if (!fail && (par & SYS_PAR_EL1_F)) + par = handle_at_slow(vcpu, op, vaddr); + vcpu_write_sys_reg(vcpu, par, PAR_EL1); } From patchwork Mon Jul 8 16:57:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13726876 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C2031482F5; Mon, 8 Jul 2024 16:58:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; cv=none; b=YHyLqwQoPu3R6+05bKq5CIqkpzK1qwQmouj03kFlCYr/TNZ0irVIGHqE6U/sUsbzwDHyriktGlJcCEgrkK0QRbNA0M7iv7T2jR/gqtRj1NG/Pe12lpwmdsVRxazwihaCNFy9EFZRgXHywrHP1t4pm9aBZQAr9GenwXe7/mdxG+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; c=relaxed/simple; bh=xgtxveXTJwvzkpFohpoJENXHtiwU/tbP8ceu6f2Dbas=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=a/URqhglStVIm1m6ot4sropCzDg6fiSZdU6pGyfSPv6VpCz8oXKiYfxpFCXkvOVnb5hbKCUMqK8bwagrhwHDvnQ8yft1oI7bSRG6DPKPHGDNvgH7QWIgKyqXmnMGii2Ybx66HNvOO4A+umqvpNd03Jav06oSc2nwpx5zd/NOTVI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sgcjIrpE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sgcjIrpE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D777FC32786; Mon, 8 Jul 2024 16:58:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720457889; bh=xgtxveXTJwvzkpFohpoJENXHtiwU/tbP8ceu6f2Dbas=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sgcjIrpE8/hXKbX4dsTSZ/xShU/txDYQhtGmh3imUp3kesCuHfPgYCbfgLuILi6Eg tMIhitmyhzEsC2QvAXcbafCzduKTabWxyK8/OeT/3G20nYAIXdPznn0tvUihqfOwA1 Ans+5XMx38FGlmXbvpwL0P512LK+jE16JBzvARlwRcGIWuYaQOVpOkHTCmSTtFZxoe Q5P1fKl0Vq97OOZcse/JUT89cMDoDUamDObnybm9A0sC0WMm5GC7L8QdftWlQpnOEk ZoI75m4FV+SsubHZVEPk6kydIU2T75v8HIKZheU7ZicQ/2/s2SUhgElTKN2344bvR7 4jx0OTwXLRx9Q== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sQrgl-00AfMX-W1; Mon, 08 Jul 2024 17:58:08 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 11/12] KVM: arm64: nv: Plumb handling of AT S1* traps from EL2 Date: Mon, 8 Jul 2024 17:57:59 +0100 Message-Id: <20240708165800.1220065-2-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240708165800.1220065-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> <20240708165800.1220065-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Hooray, we're done. Plug the AT traps into the system instruction table, and let it rip. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/sys_regs.c | 45 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 06c39f191b5ec..d8dadcb9b5e3f 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -2797,6 +2797,36 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG(SP_EL2, NULL, reset_unknown, 0), }; +static bool handle_at_s1e01(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + u32 op = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); + + __kvm_at_s1e01(vcpu, op, p->regval); + + return true; +} + +static bool handle_at_s1e2(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + u32 op = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); + + __kvm_at_s1e2(vcpu, op, p->regval); + + return true; +} + +static bool handle_at_s12(struct kvm_vcpu *vcpu, struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + u32 op = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); + + __kvm_at_s12(vcpu, op, p->regval); + + return true; +} + static bool kvm_supported_tlbi_s12_op(struct kvm_vcpu *vpcu, u32 instr) { struct kvm *kvm = vpcu->kvm; @@ -3059,6 +3089,14 @@ static struct sys_reg_desc sys_insn_descs[] = { { SYS_DESC(SYS_DC_ISW), access_dcsw }, { SYS_DESC(SYS_DC_IGSW), access_dcgsw }, { SYS_DESC(SYS_DC_IGDSW), access_dcgsw }, + + SYS_INSN(AT_S1E1R, handle_at_s1e01), + SYS_INSN(AT_S1E1W, handle_at_s1e01), + SYS_INSN(AT_S1E0R, handle_at_s1e01), + SYS_INSN(AT_S1E0W, handle_at_s1e01), + SYS_INSN(AT_S1E1RP, handle_at_s1e01), + SYS_INSN(AT_S1E1WP, handle_at_s1e01), + { SYS_DESC(SYS_DC_CSW), access_dcsw }, { SYS_DESC(SYS_DC_CGSW), access_dcgsw }, { SYS_DESC(SYS_DC_CGDSW), access_dcgsw }, @@ -3138,6 +3176,13 @@ static struct sys_reg_desc sys_insn_descs[] = { SYS_INSN(TLBI_VALE1NXS, handle_tlbi_el1), SYS_INSN(TLBI_VAALE1NXS, handle_tlbi_el1), + SYS_INSN(AT_S1E2R, handle_at_s1e2), + SYS_INSN(AT_S1E2W, handle_at_s1e2), + SYS_INSN(AT_S12E1R, handle_at_s12), + SYS_INSN(AT_S12E1W, handle_at_s12), + SYS_INSN(AT_S12E0R, handle_at_s12), + SYS_INSN(AT_S12E0W, handle_at_s12), + SYS_INSN(TLBI_IPAS2E1IS, handle_ipas2e1is), SYS_INSN(TLBI_RIPAS2E1IS, handle_ripas2e1is), SYS_INSN(TLBI_IPAS2LE1IS, handle_ipas2e1is), From patchwork Mon Jul 8 16:58:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 13726877 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 754E31487C5; Mon, 8 Jul 2024 16:58:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; cv=none; b=pOR5fyxcF6KsZSwFx+CpirEAV6u6RIOiV4s5hPMXSKmMx4bf9qDeyrrw5bLb5HtKpWgWNwKo3GGNLJVj9fD3IJSyDND85BLdUTgcqPw4t94ty1lFsONHZaot6dxhTG0R3/jKLVFr5aapGgrKhOKDsVHD9N59Pjs8uFeoga3rLm0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720457890; c=relaxed/simple; bh=UBQp9Bke6cyie441wh33sIArUOtqMteq33hP74aLxOo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qKN6M3HKiKWhPvMuJxLuFxICwCG2xSg/3tspoRHA6kdFntBVZzR/AM3tHASj+RD8DJpJfCqnJy9eVVQ38L0iaa79h667PN4J77q7uI91nGiuMrCFbRgGRSfKRU9oFqoPCFdKAHoLcJ7tUfiUc9Y6RkE9EhzCdT2wI0TDwgTZ7eU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=J6uZBWca; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="J6uZBWca" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1EE4DC4AF0C; Mon, 8 Jul 2024 16:58:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720457890; bh=UBQp9Bke6cyie441wh33sIArUOtqMteq33hP74aLxOo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J6uZBWcaDhlnB5bd+Dyl/g87653RP4Ez2mRDqMKmw07BvsWbenz/6nU1/ldzF6rxM UYKC2bu2okmrjurKBuKE1gWdJqe0nVE+CO3qi8Dw34mei3VJej20OARdN5lc9f5xf5 VMmIXJpKUSMWHs6ad9YlDemDvJbotg189OAi19SzecSIiWcfVVqt8PbZaVZk9oom0/ NcORtX2R3mfFF4f/Mh1F6/zGGFv0KgKWG2AbFzAYr+Drw3lGJOITNBxzunASpOoQ1g egNpm2EqBbIsz8KNz3WLjThaWbuxLI/5RIVdRnI8cfB53mS3aj5H7VOIkKq7Gl/MlY c3MmDsA8/YIaw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sQrgm-00AfMX-7R; Mon, 08 Jul 2024 17:58:08 +0100 From: Marc Zyngier To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu , Joey Gouly Subject: [PATCH 12/12] KVM: arm64: nv: Add support for FEAT_ATS1A Date: Mon, 8 Jul 2024 17:58:00 +0100 Message-Id: <20240708165800.1220065-3-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240708165800.1220065-1-maz@kernel.org> References: <20240625133508.259829-1-maz@kernel.org> <20240708165800.1220065-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com, joey.gouly@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Handling FEAT_ATS1A (which provides the AT S1E{1,2}A instructions) is pretty easy, as it is just the usual AT without the permission check. This basically amounts to plumbing the instructions in the various dispatch tables, and handling FEAT_ATS1A being disabled in the ID registers. Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kvm/at.c | 9 +++++++++ arch/arm64/kvm/emulate-nested.c | 2 ++ arch/arm64/kvm/sys_regs.c | 11 +++++++++++ 4 files changed, 23 insertions(+) diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 15c073359c9e9..73fa79b5a51d1 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -670,6 +670,7 @@ #define OP_AT_S12E1W sys_insn(AT_Op0, 4, AT_CRn, 8, 5) #define OP_AT_S12E0R sys_insn(AT_Op0, 4, AT_CRn, 8, 6) #define OP_AT_S12E0W sys_insn(AT_Op0, 4, AT_CRn, 8, 7) +#define OP_AT_S1E2A sys_insn(AT_Op0, 4, AT_CRn, 9, 2) /* TLBI instructions */ #define TLBI_Op0 1 diff --git a/arch/arm64/kvm/at.c b/arch/arm64/kvm/at.c index 8452273cbff6d..1e1255d244712 100644 --- a/arch/arm64/kvm/at.c +++ b/arch/arm64/kvm/at.c @@ -682,6 +682,9 @@ static u64 handle_at_slow(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) case OP_AT_S1E0W: perm_fail |= !uw; break; + case OP_AT_S1E1A: + case OP_AT_S1E2A: + break; default: BUG(); } @@ -794,6 +797,9 @@ void __kvm_at_s1e01(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) case OP_AT_S1E0W: fail = __kvm_at(OP_AT_S1E0W, vaddr); break; + case OP_AT_S1E1A: + fail = __kvm_at(OP_AT_S1E1A, vaddr); + break; default: WARN_ON_ONCE(1); fail = true; @@ -912,6 +918,9 @@ void __kvm_at_s1e2(struct kvm_vcpu *vcpu, u32 op, u64 vaddr) case OP_AT_S1E2W: fail = __kvm_at(OP_AT_S1E1W, vaddr); break; + case OP_AT_S1E2A: + fail = __kvm_at(OP_AT_S1E1A, vaddr); + break; default: WARN_ON_ONCE(1); fail = true; diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c index 96b837fe51562..b5ac298f76705 100644 --- a/arch/arm64/kvm/emulate-nested.c +++ b/arch/arm64/kvm/emulate-nested.c @@ -774,6 +774,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = { SR_TRAP(OP_AT_S12E1W, CGT_HCR_NV), SR_TRAP(OP_AT_S12E0R, CGT_HCR_NV), SR_TRAP(OP_AT_S12E0W, CGT_HCR_NV), + SR_TRAP(OP_AT_S1E2A, CGT_HCR_NV), SR_TRAP(OP_TLBI_IPAS2E1, CGT_HCR_NV), SR_TRAP(OP_TLBI_RIPAS2E1, CGT_HCR_NV), SR_TRAP(OP_TLBI_IPAS2LE1, CGT_HCR_NV), @@ -855,6 +856,7 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = { SR_TRAP(OP_AT_S1E0W, CGT_HCR_AT), SR_TRAP(OP_AT_S1E1RP, CGT_HCR_AT), SR_TRAP(OP_AT_S1E1WP, CGT_HCR_AT), + SR_TRAP(OP_AT_S1E1A, CGT_HCR_AT), SR_TRAP(SYS_ERXPFGF_EL1, CGT_HCR_nFIEN), SR_TRAP(SYS_ERXPFGCTL_EL1, CGT_HCR_nFIEN), SR_TRAP(SYS_ERXPFGCDN_EL1, CGT_HCR_nFIEN), diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index d8dadcb9b5e3f..834893e461451 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -2812,6 +2812,13 @@ static bool handle_at_s1e2(struct kvm_vcpu *vcpu, struct sys_reg_params *p, { u32 op = sys_insn(p->Op0, p->Op1, p->CRn, p->CRm, p->Op2); + /* There is no FGT associated with AT S1E2A :-( */ + if (op == OP_AT_S1E2A && + !kvm_has_feat(vcpu->kvm, ID_AA64ISAR2_EL1, ATS1A, IMP)) { + kvm_inject_undefined(vcpu); + return false; + } + __kvm_at_s1e2(vcpu, op, p->regval); return true; @@ -3182,6 +3189,7 @@ static struct sys_reg_desc sys_insn_descs[] = { SYS_INSN(AT_S12E1W, handle_at_s12), SYS_INSN(AT_S12E0R, handle_at_s12), SYS_INSN(AT_S12E0W, handle_at_s12), + SYS_INSN(AT_S1E2A, handle_at_s1e2), SYS_INSN(TLBI_IPAS2E1IS, handle_ipas2e1is), SYS_INSN(TLBI_RIPAS2E1IS, handle_ripas2e1is), @@ -4630,6 +4638,9 @@ void kvm_calculate_traps(struct kvm_vcpu *vcpu) HFGITR_EL2_TLBIRVAAE1OS | HFGITR_EL2_TLBIRVAE1OS); + if (!kvm_has_feat(kvm, ID_AA64ISAR2_EL1, ATS1A, IMP)) + kvm->arch.fgu[HFGITR_GROUP] |= HFGITR_EL2_ATS1E1A; + if (!kvm_has_feat(kvm, ID_AA64MMFR1_EL1, PAN, PAN2)) kvm->arch.fgu[HFGITR_GROUP] |= (HFGITR_EL2_ATS1E1RP | HFGITR_EL2_ATS1E1WP);