From patchwork Thu Sep 12 05:55:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Charlie Jenkins X-Patchwork-Id: 13801514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E4B6EE6457 for ; Thu, 12 Sep 2024 07:01:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yVPfKrfmMayrZvq3NGnpJCojPOyWLN1iFzFuN877+lI=; b=BRYxCnCTHROfzR rdnRsH4JAsne3hUDpo6v7vVTEz/lxCVtybkVr1mbFGXBQCi4PMz03JPcaG8a7Zui151WgPSgavOBy ZacyyinKKaM/O/pWbi1aaL3dVsmwEuNhy3GNb6i1Blpo3fwzpVhA5xweNSQtoaAj7AoaAhZCIMC7A 8hqFbxGQN7HgSG/ugSbuce5azZRpbzi5q2egn3BX7N/DgHPSyfwDprr90OMK/vt7Dq9LyahHy2MER oA3gLBDFXFAsTdMvXPp90QByRlRXjF6oBbAgIZWkkk+k1TvnzoLFVkKzGSeuus5GmhIovS9UcQ+X5 K8PHSFaaU7KJjYZKFmnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sodos-0000000C5kq-0Wjc; Thu, 12 Sep 2024 07:00:46 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1socoY-0000000BwI3-2Gl5 for linux-riscv@bombadil.infradead.org; Thu, 12 Sep 2024 05:56:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=lZGiXXErbLpwtJqQgRszC/20gA32B7VjfeYUlTIyz2Y=; b=NC82jZnAxm4oZbabt2q5CjePE4 A2IHuBQ6Pi9Gji9zxkfce5PZ3iHkdBAMJzfcIR4afK+ndOMWeuBUoRZooGheA+QKmuNoc59Mt2Rzz Kj2h7Lr0AlZx3y6gCRiBSWLiRltRXtlP7BkpMTYXiWGtF4lV624Jy2sA7gW4IfBJfwQAxfnze4OAy ch9q1VqP4dsBG/UZxYvp3wEmQxlrKJCV39o/o7cY+Tj60qXFMJsvBi86CcIP5hqK9puH7lZCuEIEi UsOqHbWe7Ahiq9BXcKPGUTYw+BEAqWURTWE1txoLaHl+DSc9UD3+ZbglE7QZZRtqEfIoYWY4GvDGT MT9TQdXw==; Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1socoT-00000000MDi-277W for linux-riscv@lists.infradead.org; Thu, 12 Sep 2024 05:56:21 +0000 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-20688fbaeafso7444945ad.0 for ; Wed, 11 Sep 2024 22:56:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1726120575; x=1726725375; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lZGiXXErbLpwtJqQgRszC/20gA32B7VjfeYUlTIyz2Y=; b=oIVh+BGDWj8kbTl94GpOjrznhiH175ClTjk+dxKEky0dDZVJ630r327s7jXTlTe9Ht PrxPJ62H6Rv1ZlKgbeuAhe+jZqFnO4VZ5Qy51PXtXLlCqzVuopEaFByyZOVtrXrbfmN/ ONnG1qWKMiJyz/P77RculEvxReRceWPWRpdIZ3YtW5eavwZxFeNb8oujYkcghsyKLn0S HH+vbX5bb5SZJd4QGU8ktvVjUC32aVW4ElK167cAw7ihADVV4zuLsdbvsAd+J5iQOO9N 88FkNGYaTylAQZCLpyCW8uZxisFmdtzOHd+0dI2Lx6sZ5s2k5AEOb02DugwW4Y6syXsA enMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726120575; x=1726725375; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lZGiXXErbLpwtJqQgRszC/20gA32B7VjfeYUlTIyz2Y=; b=RAsXkFu8q8FYk2zc102ssG+AoTqfg2F77GxQXEIjz9rPdouYlDpB3BAQ5v5fDLGUXK 1rI5/KLSs3sl2hHw9HircKoHOJZd6uz9MAr5zsHDoX1HZqaC8N3DZ0v5BB8jHCQDYBye 19KydBbdGmhWg+dEtw6PohiICvzng/23ZbDOtCtSUPIn8+o68TjdpcEJ56BhVj9Zqsf1 EDMbi8FMWLzPlaR5a+sCA0UUxsaEp0dAmo6lV5meoqFqASJnP4ddRAdiGZeKjU0ST+0I ODOfKc4EY3mibDamU/gfk9NS8G5XzISj1uDnkE5oxqymq4h7V4bxcu/uG178QO5SXSGA X9pA== X-Gm-Message-State: AOJu0YwiFTywvToY1s6RLOqf3fNFwHaWJxbyowGPKCyQxSSihwtwIQs7 hHQ4zt8h/5rF06sJHddprVdtnKmx2riV47SwHYLgLUxLmG/cMUvk3Fr5xJMov4I= X-Google-Smtp-Source: AGHT+IEP6KKwUZUW8OkMBb/fzUUdV4ELJioR4iSAdt1JCFUkRDRHnlZjxlI1+ScQqzaBoVsDlqO0MA== X-Received: by 2002:a17:902:e852:b0:205:8407:6321 with SMTP id d9443c01a7336-2076e32e467mr26718025ad.9.1726120574663; Wed, 11 Sep 2024 22:56:14 -0700 (PDT) Received: from charlie.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2076afe99b6sm7870465ad.209.2024.09.11.22.56.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Sep 2024 22:56:13 -0700 (PDT) From: Charlie Jenkins Date: Wed, 11 Sep 2024 22:55:22 -0700 Subject: [PATCH v10 14/14] riscv: Add ghostwrite vulnerability MIME-Version: 1.0 Message-Id: <20240911-xtheadvector-v10-14-8d3930091246@rivosinc.com> References: <20240911-xtheadvector-v10-0-8d3930091246@rivosinc.com> In-Reply-To: <20240911-xtheadvector-v10-0-8d3930091246@rivosinc.com> To: Conor Dooley , Rob Herring , Krzysztof Kozlowski , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jisheng Zhang , Chen-Yu Tsai , Jernej Skrabec , Samuel Holland , Samuel Holland , Jonathan Corbet , Shuah Khan , Guo Ren , Evan Green , Andy Chiu , Jessica Clarke , Andrew Jones Cc: linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Charlie Jenkins X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9377; i=charlie@rivosinc.com; h=from:subject:message-id; bh=2MZb8AJQZs0wG05Wps2DKInawqFPfqgCL1L5PflC7B4=; b=owGbwMvMwCHWx5hUnlvL8Y3xtFoSQ9qjptiFMw5nP9trcGhyZOLbtRZvpOqP7b6X2+nx55Y4v 1rzq847HaUsDGIcDLJiiiw81xqYW+/olx0VLZsAM4eVCWQIAxenAEzkRikjw8kZu9hvVjGoXk3b vvRgabHQ1BfzTnhorbhbdZajvjUkTp7hf/4B+f1yv02/VcdquBxsSyl53Xbll6Ms+zxRa+/jDSu DWAE= X-Developer-Key: i=charlie@rivosinc.com; a=openpgp; fpr=7D834FF11B1D8387E61C776FFB10D1F27D6B1354 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240912_065617_706671_75F9B59C X-CRM114-Status: GOOD ( 26.88 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Follow the patterns of the other architectures that use GENERIC_CPU_VULNERABILITIES for riscv to introduce the ghostwrite vulnerability and mitigation. The mitigation is to disable all vector which is accomplished by clearing the bit from the cpufeature field. Ghostwrite only affects thead c9xx CPUs that impelment xtheadvector, so the vulerability will only be mitigated on these CPUs. Signed-off-by: Charlie Jenkins --- arch/riscv/Kconfig.errata | 11 ++++++++ arch/riscv/errata/thead/errata.c | 28 ++++++++++++++++++ arch/riscv/include/asm/bugs.h | 22 +++++++++++++++ arch/riscv/include/asm/errata_list.h | 3 +- arch/riscv/kernel/Makefile | 2 ++ arch/riscv/kernel/bugs.c | 55 ++++++++++++++++++++++++++++++++++++ arch/riscv/kernel/cpufeature.c | 9 +++++- drivers/base/cpu.c | 3 ++ include/linux/cpu.h | 1 + 9 files changed, 132 insertions(+), 2 deletions(-) diff --git a/arch/riscv/Kconfig.errata b/arch/riscv/Kconfig.errata index 2acc7d876e1f..e318119d570d 100644 --- a/arch/riscv/Kconfig.errata +++ b/arch/riscv/Kconfig.errata @@ -119,4 +119,15 @@ config ERRATA_THEAD_PMU If you don't know what to do here, say "Y". +config ERRATA_THEAD_GHOSTWRITE + bool "Apply T-Head Ghostwrite errata" + depends on ERRATA_THEAD && RISCV_ISA_XTHEADVECTOR + default y + help + The T-Head C9xx cores have a vulnerability in the xtheadvector + instruction set. When this errata is enabled, the CPUs will be probed + to determine if they are vulnerable and disable xtheadvector. + + If you don't know what to do here, say "Y". + endmenu # "CPU errata selection" diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index f5120e07c318..5cc008ab41a8 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -142,6 +143,31 @@ static bool errata_probe_pmu(unsigned int stage, return true; } +static bool errata_probe_ghostwrite(unsigned int stage, + unsigned long arch_id, unsigned long impid) +{ + if (!IS_ENABLED(CONFIG_ERRATA_THEAD_GHOSTWRITE)) + return false; + + /* + * target-c9xx cores report arch_id and impid as 0 + * + * While ghostwrite may not affect all c9xx cores that implement + * xtheadvector, there is no futher granularity than c9xx. Assume + * vulnerable for this entire class of processors when xtheadvector is + * enabled. + */ + if (arch_id != 0 || impid != 0) + return false; + + if (stage != RISCV_ALTERNATIVES_EARLY_BOOT) + return false; + + ghostwrite_set_vulnerable(); + + return true; +} + static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid) { @@ -155,6 +181,8 @@ static u32 thead_errata_probe(unsigned int stage, if (errata_probe_pmu(stage, archid, impid)) cpu_req_errata |= BIT(ERRATA_THEAD_PMU); + errata_probe_ghostwrite(stage, archid, impid); + return cpu_req_errata; } diff --git a/arch/riscv/include/asm/bugs.h b/arch/riscv/include/asm/bugs.h new file mode 100644 index 000000000000..e294b15bf78e --- /dev/null +++ b/arch/riscv/include/asm/bugs.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Interface for managing mitigations for riscv vulnerabilities. + * + * Copyright (C) 2024 Rivos Inc. + */ + +#ifndef __ASM_BUGS_H +#define __ASM_BUGS_H + +/* Watch out, ordering is important here. */ +enum mitigation_state { + UNAFFECTED, + MITIGATED, + VULNERABLE, +}; + +void ghostwrite_set_vulnerable(void); +void ghostwrite_enable_mitigation(void); +enum mitigation_state ghostwrite_get_state(void); + +#endif /* __ASM_BUGS_H */ diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 7c8a71a526a3..6e426ed7919a 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -25,7 +25,8 @@ #ifdef CONFIG_ERRATA_THEAD #define ERRATA_THEAD_MAE 0 #define ERRATA_THEAD_PMU 1 -#define ERRATA_THEAD_NUMBER 2 +#define ERRATA_THEAD_GHOSTWRITE 2 +#define ERRATA_THEAD_NUMBER 3 #endif #ifdef __ASSEMBLY__ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 06d407f1b30b..d7a54e34178e 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -113,3 +113,5 @@ obj-$(CONFIG_COMPAT) += compat_vdso/ obj-$(CONFIG_64BIT) += pi/ obj-$(CONFIG_ACPI) += acpi.o obj-$(CONFIG_ACPI_NUMA) += acpi_numa.o + +obj-$(CONFIG_GENERIC_CPU_VULNERABILITIES) += bugs.o diff --git a/arch/riscv/kernel/bugs.c b/arch/riscv/kernel/bugs.c new file mode 100644 index 000000000000..0c19691b4cd5 --- /dev/null +++ b/arch/riscv/kernel/bugs.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Rivos Inc. + */ + +#include +#include +#include + +#include +#include + +static enum mitigation_state ghostwrite_state; + +void ghostwrite_set_vulnerable(void) +{ + ghostwrite_state = VULNERABLE; +} + +/* + * Vendor extension alternatives will use the value set at the time of boot + * alternative patching, thus this must be called before boot alternatives are + * patched (and after extension probing) to be effective. + */ +void ghostwrite_enable_mitigation(void) +{ + if (IS_ENABLED(CONFIG_RISCV_ISA_XTHEADVECTOR) && + ghostwrite_state == VULNERABLE && !cpu_mitigations_off()) { + disable_xtheadvector(); + ghostwrite_state = MITIGATED; + } +} + +enum mitigation_state ghostwrite_get_state(void) +{ + return ghostwrite_state; +} + +ssize_t cpu_show_ghostwrite(struct device *dev, struct device_attribute *attr, char *buf) +{ + if (IS_ENABLED(CONFIG_RISCV_ISA_XTHEADVECTOR)) { + switch (ghostwrite_state) { + case UNAFFECTED: + return sprintf(buf, "Not affected\n"); + case MITIGATED: + return sprintf(buf, "Mitigation: xtheadvector disabled\n"); + case VULNERABLE: + fallthrough; + default: + return sprintf(buf, "Vulnerable\n"); + } + } else { + return sprintf(buf, "Not affected\n"); + } +} diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 56b5054b8f86..1f4329bb8a9d 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -867,7 +868,13 @@ static int __init riscv_fill_hwcap_from_ext_list(unsigned long *isa2hwcap) riscv_fill_vendor_ext_list(cpu); } - if (has_xtheadvector_no_alternatives() && has_thead_homogeneous_vlenb() < 0) { + /* + * Execute ghostwrite mitigation immediately after detecting extensions + * to disable xtheadvector if necessary. + */ + if (ghostwrite_get_state() == VULNERABLE) { + ghostwrite_enable_mitigation(); + } else if (has_xtheadvector_no_alternatives() && has_thead_homogeneous_vlenb() < 0) { pr_warn("Unsupported heterogeneous vlenb detected, vector extension disabled.\n"); disable_xtheadvector(); } diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c index fdaa24bb641a..a7e511849875 100644 --- a/drivers/base/cpu.c +++ b/drivers/base/cpu.c @@ -599,6 +599,7 @@ CPU_SHOW_VULN_FALLBACK(retbleed); CPU_SHOW_VULN_FALLBACK(spec_rstack_overflow); CPU_SHOW_VULN_FALLBACK(gds); CPU_SHOW_VULN_FALLBACK(reg_file_data_sampling); +CPU_SHOW_VULN_FALLBACK(ghostwrite); static DEVICE_ATTR(meltdown, 0444, cpu_show_meltdown, NULL); static DEVICE_ATTR(spectre_v1, 0444, cpu_show_spectre_v1, NULL); @@ -614,6 +615,7 @@ static DEVICE_ATTR(retbleed, 0444, cpu_show_retbleed, NULL); static DEVICE_ATTR(spec_rstack_overflow, 0444, cpu_show_spec_rstack_overflow, NULL); static DEVICE_ATTR(gather_data_sampling, 0444, cpu_show_gds, NULL); static DEVICE_ATTR(reg_file_data_sampling, 0444, cpu_show_reg_file_data_sampling, NULL); +static DEVICE_ATTR(ghostwrite, 0444, cpu_show_ghostwrite, NULL); static struct attribute *cpu_root_vulnerabilities_attrs[] = { &dev_attr_meltdown.attr, @@ -630,6 +632,7 @@ static struct attribute *cpu_root_vulnerabilities_attrs[] = { &dev_attr_spec_rstack_overflow.attr, &dev_attr_gather_data_sampling.attr, &dev_attr_reg_file_data_sampling.attr, + &dev_attr_ghostwrite.attr, NULL }; diff --git a/include/linux/cpu.h b/include/linux/cpu.h index bdcec1732445..6a0a8f1c7c90 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -77,6 +77,7 @@ extern ssize_t cpu_show_gds(struct device *dev, struct device_attribute *attr, char *buf); extern ssize_t cpu_show_reg_file_data_sampling(struct device *dev, struct device_attribute *attr, char *buf); +extern ssize_t cpu_show_ghostwrite(struct device *dev, struct device_attribute *attr, char *buf); extern __printf(4, 5) struct device *cpu_device_create(struct device *parent, void *drvdata,