From patchwork Tue Oct 1 11:29:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13817879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60A5FCE7D0D for ; Tue, 1 Oct 2024 11:33:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=f7KYUHlikFonHy6zpqftYFKm1C9dSv1KviRYYbtRmMU=; b=OuKU8C3gP+L640NDulsuqoG+ob fAKddCBm26EVEFuDLJpxWuI1INtsuMt3l4xlbgntcrkozQ/zIAQNoirRm92pTD8nnxPwNxAYn+p7C 0EGZXDg94eDQG9QD7vS+xUjIKDheccsy/oDwRuZODzAQzJpjw4FCUbFNy+mvyBe4yMsON8feq1OfY kJNcv+VjroBDl745dgJ9iVq14CH4VvfjVoOdolZL2eL16HEsMkxPZhDnH86gII7gaT5vRlHyzf5xV UZAIGoIjyeUPtvBgXy7aaqvIrTO4e0ant0WjytZciSHlSbXUvxCf7OOumVuzR9L+ig9pkaQqmAgmq FuKy7IHg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1svb8V-00000002Wz3-2i0a; Tue, 01 Oct 2024 11:33:47 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1svb65-00000002WFY-3MUO for linux-arm-kernel@lists.infradead.org; Tue, 01 Oct 2024 11:31:19 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 95F51A4359C; Tue, 1 Oct 2024 11:31:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30BC4C4CECD; Tue, 1 Oct 2024 11:31:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782276; bh=/8kAC4uyafUpiRRGXlRqbKMN4dLMMXoXHxOXxaPQDDQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=T5Umuq7ljlHWXhSLe/eIw8E95ukpdJgREL82FZYdJJxHAVrGCZSWDIckIz7LVxQKp 3tIWa9HnmYvQl2nd2+VeaCoz0P3x9R+DWElTguisIqt3Wosl/cFoZyIEb9Sr6WiPUx rsNggS0pfUwCI5P0uYcNEUYH6FBQftvAJZn92YAx01DQS9eyiFqReamTnES6Vw0q/L rqcrPFCFINIW7a3eSb3ySpan+9FOrd1kn0HMD0Mjkvrp4+w2D26bQbLHMwZDeqXeM+ qyPtRFnc0ZjgMjtQxYSobYY4RkH9iV1FuiGbjsWgNGmVrqDdpxeDPnX85em1jr7JSg BZxCJowBjoO/A== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:06 +0100 Subject: [PATCH v7 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper MIME-Version: 1.0 Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-1-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3064; i=broonie@kernel.org; h=from:subject:message-id; bh=/8kAC4uyafUpiRRGXlRqbKMN4dLMMXoXHxOXxaPQDDQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+918K2dkt+RyTe17B6hQuE/kqMhTsruB9gMWraGq ay4fjSqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfAAKCRAk1otyXVSH0ADvB/ 47xx2/156+0n1vm9GXaOjaUWX7fxRYKsC/VK02LjPI2bwp0DxUXHhJAidXhkijDjq6DuXua6jkqdDD kEQN3rBkcn/KcMLk/jkYToMkAcWfeoRFqWzIXZrGQR+3j5E4NHoGxv5yFTKNPNyNkkd3sNW2ReTBne ni4Hi3asHXhG20ls6oh42qwpnDXWAvCG3InygpC60/mxi7LuGdvDeIW6y5gxMzqdjRmP/vy0A1MoYr wdM8ices0RXqLg/lepRarbl0U+kacax9uEc9njx4VrUGsjIXne2rIMqG6bTgYK43yXlruLL0J3lRPj Mscn4z+5Oi8cIZwzIk2cHF7yKWo2S9 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241001_043118_003618_BEDCE168 X-CRM114-Status: GOOD ( 13.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index f2a84efc3618..3c30da35c285 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -170,6 +170,10 @@ static inline unsigned int __bit_to_vq(unsigned int bit) return SVE_VQ_MAX - bit; } +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 77006df20a75..8a080dbd8988 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum vec_type type, bit = find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <= info->max_virtualisable_vl) { + if (__bit_to_vl(b) <= info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); /* * For the default VL, pick the maximum supported value <= 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl = SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl = __bit_to_vl(b + 1); if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl = info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); min_bit = find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl = sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl = __bit_to_vl(min_bit); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); WARN_ON(info->min_vl > info->max_vl); From patchwork Tue Oct 1 11:29:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13817886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6825BCE7D10 for ; Tue, 1 Oct 2024 11:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KTlH0kDHxQ/etob9Ii4h9i0gT/eiSHWYDzfVxAx1uVk=; b=wl05/yJpiddBWS0xM4WPjTLApx U9fkA7CmYPzSKKFoA7TqgMCJexDGpt0yh+AeaJO54cCpUjEp/cs/ZYmOCCKTSrZQr+E9Tm9xkIxtp iPiEtNc3uDB+mlmPpoRGqJnXh4VmnJ/C3HdibMdi0k6Sj0jhRhsjq4rXVrcy+EXiVEPkCGPT6HUzk nKUEDZNEQo/IgNFdjosVZ8Dy97kjFoBd6HX+Y/ANxPjqIUPSq3T9QJ5FVX3JHsGC2FRMszNtgKs4u sWwiekOTsAWpBrRAuXn9FNwPqsD3MuK2tfDkkVCPLdMOI6ca+5yZkx7ueQjmYsx6hH3ROTmYvdXcm 8MdZ08Jw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1svbC9-00000002Xzg-3tR5; Tue, 01 Oct 2024 11:37:34 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1svb68-00000002WGV-1NcZ for linux-arm-kernel@lists.infradead.org; Tue, 01 Oct 2024 11:31:22 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 181D8A435A8; Tue, 1 Oct 2024 11:31:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08E53C4CEC6; Tue, 1 Oct 2024 11:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782279; bh=T8FjmI+5TxAQIynPAIQoQkfPmrHt9l4M/rmCIDXWNk8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Niupd6pTw/ZNvqwxoEImGBsdNK8Psgwq8K6LKc4ziELCbvvS0OVtx/T0EN6fBiPnd pK8ioArLpJv2NZWn/en39KA5U3aZxJ5xDZgQyWgwrTvrRakq1H1La3O0Pnjt5MVn/j JLQKYgTtF2yfhUs/+j2BS0YHQE8vmptykquie25DsL2apAXGouXTx6nveONf6Vlg9w G8On63/UDgyvG4R85fTUfi3kkUTR5MGXXwNf3lJI/TAEV2HZpNcqFr8hQOhLAkedU6 yWVwbIo8sfF0/bdaEWixDvv3Vzy4qnFTVFtVs5Up8V9QjfxAGyyCCb7/fJOPj/jWCP hvED56DYkN/Ag== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:07 +0100 Subject: [PATCH v7 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU MIME-Version: 1.0 Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-2-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=4205; i=broonie@kernel.org; h=from:subject:message-id; bh=T8FjmI+5TxAQIynPAIQoQkfPmrHt9l4M/rmCIDXWNk8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+919ztNN3f1p7RnUQbLl+kj+u0eNguOZjwmLwrCH RRxsJ7uJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfQAKCRAk1otyXVSH0HE+B/ 9TyWHjKOKfGCjZIC6IfjLAEsXXFeXkrEwn6BlS9R81DdltxbYEk4weYzWvbTIe7A2JO52fqD51KLVn 4Fl7BLZny2+KsuDXIjp8LRdmXg+vmeKJ6DPY8pLQYCDUy4JtexwtLVDT/ptuX+FYC5gZaV84wgCM5Y E0kZPQMf3JM3vRzMo7SbQiQdZF9Rkio1ZRlvaNriPgg9SJdXitsGSr9ciTQ3iIInNXbHuYVl+wH5yM /DnCYlZRIs13UaBO2wxIqrFCcIIzL15k1c8cgZXWaeh6YlxTmcD+mBBbRYTRAaJgzpu7IZvRlCR08Y aD7eXunfqinvaatVVlbQ2Jb7R/a8tx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241001_043120_582104_E84EB8BB X-CRM114-Status: GOOD ( 17.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 3c30da35c285..754b0eb09fe8 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -186,6 +186,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -276,6 +279,11 @@ static inline int vec_max_virtualisable_vl(enum vec_type type) return vl_info[type].max_virtualisable_vl; } +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -286,6 +294,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8a080dbd8988..0bf0837d4adb 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = { .min_vl = SVE_VL_MIN, .max_vl = SVE_VL_MIN, .max_virtualisable_vl = SVE_VL_MIN, + .max_cpu_vl = SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b = find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b = find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1069,10 +1080,23 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; vec_probe_vqs(info, tmp_map); + /* + * Currently the maximum VL is only used for pKVM which + * doesn't allow late CPUs but we don't expect asymmetry and + * if we encounter any then future users will need handling so + * warn if we see anything. + */ + max_vl = __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: increases maximum VL to %lu\n", + info->name, smp_processor_id(), max_vl); + info->max_cpu_vl = max_vl; + } + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { pr_warn("%s: cpu%d: Required vector length(s) missing\n", From patchwork Tue Oct 1 11:29:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13817887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1549CE7D10 for ; Tue, 1 Oct 2024 11:38:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=EMn+G/Hqtr1xLFXhEJrUZRizkVtxOr6MSEo1E8c+y6k=; b=Hod8RuG/ByIUOoP2BJVK2lgnpS Tm92yZdBldAs3WggV709TGFLCuwsHOImB0UzBRBSd66PXhjiZq7g2pTOsaKX7TGw90MZtFIY9eoXl 5NyEok1sHwx7v8BI0DI62s4o7c5xs8SjW2B4MnbmPqPn8QHSsZ5Ptzdn1WdZyh63mGhV47y4o56wm KLAhAoXsuj74yvfZ68YrRUAy1l4EC0KH730S0xNBHGm7JEGLuuCYY2kSG/99Qg/QGY4Tr66ofqAZh SdXLVqYxj/NmWURmLwiipzk5GJXHEsZ26/PdRcei9SDwlf6oAQBy3j99PmZn6e3Q/iRvwToaqp4sa TLsQRgWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1svbDM-00000002YCf-22MR; Tue, 01 Oct 2024 11:38:48 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1svb6A-00000002WHt-245F for linux-arm-kernel@lists.infradead.org; Tue, 01 Oct 2024 11:31:24 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 70631A435A6; Tue, 1 Oct 2024 11:31:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96B0CC4CEC6; Tue, 1 Oct 2024 11:31:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782281; bh=KBunKuidQbGsl1sTPDqT1ibl0GGL5dnEQFUfFDn0MP8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=f2utS53aPNMtAdR2kLn87aubYqKj6soMe/MmzTiwRNusjBY9Kvtd6t8lsrYC66qcT 2caAWmgTZjnL+MHHlShC8N1E5MZh1IxjUtqpkiWJaPvzht7rXS+SsapTNPfR7AnDBL kd0KFzAU0ScB+S3JML/yRqKEEu9pw3uN7bwg+5xpWUHyNm1helyuJvmXq9TPBplb8A nPPMnzlWOtlnFGdwgGboF1PdVpxfvmyVjzv6qNv1tFMQljPYocbhF/KUky0C4mpNup HMgVOpIx79OsWdmel+ctIodRvkcMSPFpOxF+QM8QYCqtz6ulqYb8ZUfQ85kJvuaUhG HMwZEG1sWDMQQ== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:08 +0100 Subject: [PATCH v7 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore MIME-Version: 1.0 Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-3-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3206; i=broonie@kernel.org; h=from:subject:message-id; bh=KBunKuidQbGsl1sTPDqT1ibl0GGL5dnEQFUfFDn0MP8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+91952Fz2ONJP3ZDAU2Oql7Admcy1BKuIxh9+12i 7GtyhbuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfQAKCRAk1otyXVSH0PTkB/ 407zY0DEiqK+eF3/i3xkdy/woU0hcDfHxTybQd/T6pvgXBawysTNgYjQ2CogGCSBbw2pDvtNcedfI3 0IVNF17OOUZiVB7M0DGQjX4C85sN4i+JFZ/Hm0uVzONX/hAWX3wPafc6YTKZc2kzsmQGz4IE7PnNNt sVBUmV/EeSNHYzoqbJfdHp2+sb36lNGBL3fzSUOPs8Ubt16DA8/+rYRzbtQcM+RjSw/cJ0kY8k0FTA rax6ye6OXZGFwiJGCDaYpnZlUDwxpl9QKgYDr11tRok3EbtntYmW6c4HzPhv1i/k7JWARv+cB3Dod7 6JZh3CHiXQ9r4iQB/izB/VgNa4fdJm X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241001_043122_709771_7D139D37 X-CRM114-Status: GOOD ( 17.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption in the case where the PE supports a VL larger than the maximum usable VL for the host. We can just read the current VL from the hardware with an instruction so use that instead of a saved value, we need to correct the value used to lay out the stored data and this makes it clear that the layout is consistent with the hardware configuration. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index c838309e4ec4..6b074f4d48b2 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 46d52e8a3df3..58f60ea95a35 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -339,7 +339,7 @@ static inline void __hyp_sve_save_host(void) sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 87692b566d90..48adcd005079 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); From patchwork Tue Oct 1 11:29:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13817892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48EE4CE7D10 for ; Tue, 1 Oct 2024 11:41:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BGx83iyGWzEFgd75nZGwIyaKs4qouYnS6mMcY6Bh9rw=; b=a7f3sxMp2CoSjBQFxjwQFGMrTB EMjktmZZ7PgYzokkGu3/wjIumL24/uRN9GFdSTaD4Ag/jf///F6SsO9+3woQM43q3IUwheCjNVeef PwQeOLmuAlo4Jbl27gb3UCUZOeXhItRkSiDtxcHFe1eigsRGIC3SLiUcSltMFAkmMCaVVyAYx/tW9 kNWQ6U0WErGk8vqGlcQUOVUSHhu7h478ibkZwEt1asWxdWi5KoJ9s+Q9C+LfGVq4qyTqjAWZO/X3u CHsBM5R2/DSA2LSud4sFArPJx9/lyNDsxAPTi6rJrb6Y61XLd9+oqwfQCGNFJNiQpQPZx8CoK1W5N QnHhJEaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1svbFo-00000002Yae-23Vo; Tue, 01 Oct 2024 11:41:20 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1svb6C-00000002WIt-3y2H for linux-arm-kernel@lists.infradead.org; Tue, 01 Oct 2024 11:31:27 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id CFDB6A435A2; Tue, 1 Oct 2024 11:31:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 065F8C4CECD; Tue, 1 Oct 2024 11:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782284; bh=3H3rN27/T4dSvW1Uedlxa4dx04M+fhPbfJn2xQeiUto=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ev0zcnejLlg4untSFDKPs0uu9yRjLpeF29NXm6Fyn2yh3/+5F6eKQ+i/efPs3+p7m Ms97pZVloGyzj0h7PgGqWHkpDCdU2MDWFiWnNyBAlGYlbBbgkT8of9E3yp0dHCzQjc FG49kMw6VnsY+KD8+Bu2FZAUZQFnS/gitVh92IdizoZi2bkwMrboc7ogwwjzfbrFq/ itKBJbxfIL3lh2wVsGkt/RfR3TrVW5P6/hfsI6PJHU46H27qv9ZFa4NUTB/rCpWR1l qJewLv3219nczc442IMtmEliB4fnOxgEqrFnASQNOgAQaA24w9cHZVlGiF76UuT7fU RsaaVHjOy2elA== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:09 +0100 Subject: [PATCH v7 4/4] KVM: arm64: Avoid underallocating storage for host SVE state MIME-Version: 1.0 Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-4-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=3H3rN27/T4dSvW1Uedlxa4dx04M+fhPbfJn2xQeiUto=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+91+bqwK4f/pJu2CGuxMaixni9UrdS9SNp0K0H+g BdW8tDmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfgAKCRAk1otyXVSH0O8MB/ 9PffATY1K7tJ1hlmtmShlhAHdohSC3qAnH0uEWLq5dujSFDSuS8JMjvRLtBWwAApYUjohwazjqUMGr S6MQmMdtfoV58SR9WNsybgfttZz3/uUpVu8hWjJLRHIKm7drRnIjuhK2oerBmf4UaS6EmeiZ816hvb qV4L4r53Jp/fm1zDfziPzD8Vu5vCb5UwJ7HOmd7vAhyehJ4jkNnLG/kF63IHnl/hDaw2po5EjH/fCp oNaK0u1sYxpx592tHvJj/Orl/hKz2D7Wtuu6M199308Qhe+CKvn4rLBNdxudXs0uEG5R1ot344mBT2 y5b51UB7U5839ImBX7zF9B/CbsLJlo X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241001_043125_234143_52A3E986 X-CRM114-Status: GOOD ( 16.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 329619c6fa96..612a5adc2dbf 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 6b074f4d48b2..19f3ae9f05a9 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 48adcd005079..234372bef85c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -99,8 +99,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_cpu_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 187a5f4d56c0..770d66491b76 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 0b0ae5ae7bc2..6c87d01514ff 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl = sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) = kvm_host_sve_max_cpu_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need