From patchwork Thu Apr 11 00:51:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hector Martin X-Patchwork-Id: 13625210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93C43CD1297 for ; Thu, 11 Apr 2024 00:52:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OUxnTwh2zsU+/Svs6ZZjeldBb5WiD5zn/bnLGG+cTcQ=; b=Djx0RNoUqh8CRc UZMvH6MKgcZaUA1i3hD0FRBofrVGHuOYOigu5pekDzbL5rsuEiqeHlAiePG00IUV2eIzbeUc0xoG+ R6w7+I5l+60OoKJCy68GdyEmvKZgDbaLoaNBCTjcOAhNhsZg5AaYGTEpCkcSCJUHnYg/OVyz4pTsl QewIQni7BnyY/QdwUY0QqNO4ppw9Prwjm8Zf0efwTsLBimrI9GlhUPo1jTW9+CnvlNjuGllq+n8XB 75Ll2Bs7wKTJX+Pm8jjXFCarhdr8jFeVyv2vvKzNQGo61resE1ub5m+umdHoi4OJiTXad+4L7jMwY wZykBkvZ7iddSLYk6R4w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruifY-00000009io4-2CTD; Thu, 11 Apr 2024 00:52:00 +0000 Received: from marcansoft.com ([212.63.210.85] helo=mail.marcansoft.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruifP-00000009ii8-1vdg for linux-arm-kernel@lists.infradead.org; Thu, 11 Apr 2024 00:51:53 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sendonly@marcansoft.com) by mail.marcansoft.com (Postfix) with ESMTPSA id A9CB8425C7; Thu, 11 Apr 2024 00:51:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=marcan.st; s=default; t=1712796710; bh=A6cfBeGeoux68NpHgjY1yK+Y/P3gva5wYhlCSyteIQ8=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=hQsQ2KMFH+62Ms4wBxA9kOprBD45PuiudDTCI4Jc4FGX9LpD74wyBTzgEPPVjWUnB 953bY8Dqz9SO3OO4aOQL27CegKMagwfpLuV+N2j2IubpZ7mJZRXeV17q646tvLmMdj AQDO6gvir7cpikZZ9v9b5p9KjuO929lju6gRqXmEl6ayhx28tondrxz9ioiCDG4zG/ k8Tkna5i6wcEZy8+4LyPyH56PrRn/PB64V2/cZ1Ga6JK07yFX/7ipVjJ20wK9jbGED /rxmNI9VBoHwwwL0I2ru1B1ACbf8DbqhzbbOC/lshwf4EIjeUqmSxkMii2YRtjO8Uj BZfDAJl3lm9Rg== From: Hector Martin Date: Thu, 11 Apr 2024 09:51:23 +0900 Subject: [PATCH 4/4] arm64: Implement Apple IMPDEF TSO memory model control MIME-Version: 1.0 Message-Id: <20240411-tso-v1-4-754f11abfbff@marcan.st> References: <20240411-tso-v1-0-754f11abfbff@marcan.st> In-Reply-To: <20240411-tso-v1-0-754f11abfbff@marcan.st> To: Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland Cc: Zayd Qumsieh , Justin Lu , Ryan Houdek , Mark Brown , Ard Biesheuvel , Mateusz Guzik , Anshuman Khandual , Oliver Upton , Miguel Luis , Joey Gouly , Christoph Paasch , Kees Cook , Sami Tolvanen , Baoquan He , Joel Granados , Dawei Li , Andrew Morton , Florent Revest , David Hildenbrand , Stefan Roesch , Andy Chiu , Josh Triplett , Oleg Nesterov , Helge Deller , Zev Weiss , Ondrej Mosnacek , Miguel Ojeda , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Asahi Linux , Hector Martin X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5582; i=marcan@marcan.st; h=from:subject:message-id; bh=A6cfBeGeoux68NpHgjY1yK+Y/P3gva5wYhlCSyteIQ8=; b=owGbwMvMwCUm+yP4NEe/cRLjabUkhjRxE94WxUt9IucjBTdtv6v15/UKdQNda43uC39P229ef uim5u6wjlIWBjEuBlkxRZbGE72nuj2nn1NXTZkOM4eVCWQIAxenAEzkhCrDf8/nnCam8mvmbv3j rVJcmMGmb8BzQcSso17uv+gqxuKo9Qz/7O4K+/y0uul+TvFG7uNAH1NVHblvejunNfWVqW+SPqj PBAA= X-Developer-Key: i=marcan@marcan.st; a=openpgp; fpr=FC18F00317968B7BE86201CBE22A629A4C515DD5 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240410_175151_724455_25363E9C X-CRM114-Status: GOOD ( 19.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apple CPUs may implement the TSO memory model as an optional configurable mode. This allows x86 emulators to simplify their load/store handling, greatly increasing performance. Expose this via the prctl PR_SET_MEM_MODEL_TSO mechanism. We use the Apple IMPDEF AIDR_EL1 register to check for the availability of TSO mode, and enable this codepath on all CPUs with an Apple implementer. This relies on the ACTLR_EL1 thread state scaffolding introduced earlier. Signed-off-by: Hector Martin --- arch/arm64/Kconfig | 2 ++ arch/arm64/include/asm/apple_cpufeature.h | 15 +++++++++++++++ arch/arm64/include/asm/cpufeature.h | 3 ++- arch/arm64/kernel/cpufeature_impdef.c | 23 +++++++++++++++++++++++ arch/arm64/kernel/process.c | 22 ++++++++++++++++++++++ arch/arm64/tools/cpucaps | 1 + 6 files changed, 65 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 9b3593b34cce..2f3eedd955c9 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2167,6 +2167,8 @@ endif # ARM64_PSEUDO_NMI config ARM64_MEMORY_MODEL_CONTROL bool "Runtime memory model control" + default ARCH_APPLE + select ARM64_ACTLR_STATE help Some ARM64 CPUs support runtime switching of the CPU memory model, which can be useful to emulate other CPU architectures diff --git a/arch/arm64/include/asm/apple_cpufeature.h b/arch/arm64/include/asm/apple_cpufeature.h new file mode 100644 index 000000000000..4370d91ffa3e --- /dev/null +++ b/arch/arm64/include/asm/apple_cpufeature.h @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 + +#ifndef __ASM_APPLE_CPUFEATURES_H +#define __ASM_APPLE_CPUFEATURES_H + +#include +#include + +#define AIDR_APPLE_TSO_SHIFT 9 +#define AIDR_APPLE_TSO BIT(9) + +#define ACTLR_APPLE_TSO_SHIFT 1 +#define ACTLR_APPLE_TSO BIT(1) + +#endif diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 46ab37f8f4d8..a191000d88c2 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -911,7 +911,8 @@ static inline unsigned int get_vmid_bits(u64 mmfr1) static __always_inline bool system_has_actlr_state(void) { - return false; + return IS_ENABLED(CONFIG_ARM64_ACTLR_STATE) && + alternative_has_cap_unlikely(ARM64_HAS_TSO_APPLE); } s64 arm64_ftr_safe_value(const struct arm64_ftr_bits *ftrp, s64 new, s64 cur); diff --git a/arch/arm64/kernel/cpufeature_impdef.c b/arch/arm64/kernel/cpufeature_impdef.c index bb04a8e3d79d..9325d1eb12f4 100644 --- a/arch/arm64/kernel/cpufeature_impdef.c +++ b/arch/arm64/kernel/cpufeature_impdef.c @@ -4,8 +4,21 @@ */ #include +#include #ifdef CONFIG_ARM64_MEMORY_MODEL_CONTROL +static bool has_apple_feature(const struct arm64_cpu_capabilities *entry, int scope) +{ + u64 val; + WARN_ON(scope != SCOPE_SYSTEM); + + if (read_cpuid_implementor() != ARM_CPU_IMP_APPLE) + return false; + + val = read_sysreg(aidr_el1); + return cpufeature_matches(val, entry); +} + static bool has_tso_fixed(const struct arm64_cpu_capabilities *entry, int scope) { /* List of CPUs that always use the TSO memory model */ @@ -22,6 +35,16 @@ static bool has_tso_fixed(const struct arm64_cpu_capabilities *entry, int scope) static const struct arm64_cpu_capabilities arm64_impdef_features[] = { #ifdef CONFIG_ARM64_MEMORY_MODEL_CONTROL + { + .desc = "TSO memory model (Apple)", + .capability = ARM64_HAS_TSO_APPLE, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_apple_feature, + .field_pos = AIDR_APPLE_TSO_SHIFT, + .field_width = 1, + .sign = FTR_UNSIGNED, + .min_field_value = 1, + }, { .desc = "TSO memory model (Fixed)", .capability = ARM64_HAS_TSO_FIXED, diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 117f80e16aac..34a19ecfb630 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -44,6 +44,7 @@ #include #include +#include #include #include #include @@ -522,6 +523,10 @@ void update_sctlr_el1(u64 sctlr) #ifdef CONFIG_ARM64_MEMORY_MODEL_CONTROL int arch_prctl_mem_model_get(struct task_struct *t) { + if (alternative_has_cap_unlikely(ARM64_HAS_TSO_APPLE) && + t->thread.actlr & ACTLR_APPLE_TSO) + return PR_SET_MEM_MODEL_TSO; + return PR_SET_MEM_MODEL_DEFAULT; } @@ -531,6 +536,23 @@ int arch_prctl_mem_model_set(struct task_struct *t, unsigned long val) val == PR_SET_MEM_MODEL_TSO) return 0; + if (alternative_has_cap_unlikely(ARM64_HAS_TSO_APPLE)) { + WARN_ON(!system_has_actlr_state()); + + switch (val) { + case PR_SET_MEM_MODEL_TSO: + t->thread.actlr |= ACTLR_APPLE_TSO; + break; + case PR_SET_MEM_MODEL_DEFAULT: + t->thread.actlr &= ~ACTLR_APPLE_TSO; + break; + default: + return -EINVAL; + } + write_sysreg(t->thread.actlr, actlr_el1); + return 0; + } + if (val == PR_SET_MEM_MODEL_DEFAULT) return 0; diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index daa6b9495402..62f9ca9ce44b 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -52,6 +52,7 @@ HAS_STAGE2_FWB HAS_TCR2 HAS_TIDCP1 HAS_TLB_RANGE +HAS_TSO_APPLE HAS_TSO_FIXED HAS_VA52 HAS_VIRT_HOST_EXTN