From patchwork Tue Jun 25 00:49:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F7C7C2BD09 for ; Tue, 25 Jun 2024 00:50:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rIBJwFngq5T9ru77WLY6HeKwsFmP2TjWdS04E4lAFEc=; b=FRnZY95mj9xkjR EiyjoZCwdRPZThCMmrOwoH/sXf003MDP1stXfR9WsQdO0vAF1918ZZapTWKaOnfBhGIkaKTRoCZon HEKGAvnK01fIJC7ZW7VjlB3Wq7jxZm/zLaNhO5xJ0/ZV//FmNMexGS3uaLu40R8MP2UaF8ma/9ghM JhNiNeRXPRKnIpLqYKHC04srWYVB6Xhk5ykeBj94PAM8RAFksgSocSvrhQshMY7AOdml+mw6orvu0 p9SBlTBmVl1QZqrglJd4B0rUQ7Gtwb+6ZEViqbcTtFFS6D+vjrs96R0FLOhEK19HBAD+6rVnI3BhR QU2zpd2JHfSwYQ83UBOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOL-000000018E0-3u1v; Tue, 25 Jun 2024 00:50:37 +0000 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOJ-000000018Cp-2Hfh for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:50:36 +0000 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-71910dfb8c0so1658349a12.3 for ; Mon, 24 Jun 2024 17:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276634; x=1719881434; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qsFL4pVPRli+inXg9hiGEw7k4kBD3MX99ZHq/8uxkhA=; b=wFcHRQY9gcFB9CTqUsQF4sbCh+2WxLwSWXgSMbMFaF+NE/tb51fhRn+sRSxbOcXjS5 k4jPwqr6xxOSkBikMfGhbnu/YIYqdXxW+/Ny3u5vnv1LMrnjpEZUkTEWIw2HTVRom9nx 8rNffPHayf+dK9ycK9Wbo5arzRcak/VAz+UytZB84AG08BWDxFlc64XIgTvlv3Mxqd0d EPNvYHxVNPAo9L/UfBQg3RG4JTJaqi+WlVU1p0EXvJKfdpaNw7ZYrJNDCuzQ9Eymmwuz yeEEK/42VwsEvGdQAPyLAwqTj+3eDRFvYYeWBArvJjYy9Wmw+qCFM33Sq/WqGGZb4yqK M+DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276634; x=1719881434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qsFL4pVPRli+inXg9hiGEw7k4kBD3MX99ZHq/8uxkhA=; b=wJnG3q4Fq31eURGnat78uep5LwwnL14jLmlRoz+99LYNAC4vk9xKX1b5b5TGWWjhqt FUmQ2ZsftOhJO89jD06klU4pVHV0s5c63afz8NkMS5lpWW20pr0Zi3z+dOO3wWQc+dtk jiOdb7tCZ0rrLtT78jxNOsF4W+T6Q4NtCjlC8ioDfEs5Og3/0siYxbJeiCAlsVYg2R0P W78LCUz0yj0/ao/PEw0hFHbWV5sRIg1zaGi4irF0s5znBKrKV2JmT+P3AspTJn7NwtXH 8p5xxkyJPUw1+bmjs7z2frx5pHPMigYkTX670DtCeOPomkCzSmKLriMoIgKLdCA6PTIc WeWw== X-Gm-Message-State: AOJu0YxP3jGJN8A7UjzAleF8TehQI25Fy3RsL4CgAsICu0B3NXpfdtO2 hFQ2HEsOc0aZwMD56ubBVSu96Y8++8rQB25qNtvGFLPlRpyRbDcCm7cEH5j4401MW/pyLk8RPvB J X-Google-Smtp-Source: AGHT+IEpgOILuNHe0y3CrK2jbU7V6dFxD2gSind2RbvoHSr7ZIOp9rwdXozB4OgBe0a+9B3PWo9AGg== X-Received: by 2002:a17:902:f68f:b0:1fa:3f05:209d with SMTP id d9443c01a7336-1fa3f052432mr46261095ad.45.1719276634455; Mon, 24 Jun 2024 17:50:34 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.50.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:50:34 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v3 1/8] RISC-V: Add Zicclsm to cpufeature and hwprobe Date: Mon, 24 Jun 2024 20:49:54 -0400 Message-ID: <20240625005001.37901-2-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175035_611784_B3DB01DC X-CRM114-Status: GOOD ( 18.04 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org > Zicclsm Misaligned loads and stores to main memory regions with both > the cacheability and coherence PMAs must be supported. > Note: > This introduces a new extension name for this feature. > This requires misaligned support for all regular load and store > instructions (including scalar and vector) but not AMOs or other > specialized forms of memory access. Even though mandated, misaligned > loads and stores might execute extremely slowly. Standard software > distributions should assume their existence only for correctness, > not for performance. Detecing zicclsm allows the kernel to report if the hardware supports misaligned accesses even if support wasn't probed. This is useful for usermode to know if vector misaligned accesses are supported. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley Reviewed-by: Andy Chiu --- V1 -> V2: - Add documentation for Zicclsm - Move Zicclsm to correct location V2 -> V3: - No changes --- Documentation/arch/riscv/hwprobe.rst | 3 +++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/sys_hwprobe.c | 1 + 5 files changed, 7 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index df5045103e73..7085a694b801 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -207,6 +207,9 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_EXT_ZVE64D`: The Vector sub-extension Zve64d is supported, as defined by version 1.0 of the RISC-V Vector extension manual. + * :c:macro:`RISCV_HWPROBE_EXT_ZICCLSM`: The Zicclsm extension is supported as + defined in the RISC-V RVA Profiles Specification. + * :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar values to :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_PERF`, but the key was mistakenly classified as a bitmask rather than a value. diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index f64d4e98e67c..0b3bd8885a2b 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -86,6 +86,7 @@ #define RISCV_ISA_EXT_ZVE64X 77 #define RISCV_ISA_EXT_ZVE64F 78 #define RISCV_ISA_EXT_ZVE64D 79 +#define RISCV_ISA_EXT_ZICCLSM 80 #define RISCV_ISA_EXT_XLINUXENVCFG 127 diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 2fb8a8185e7a..023b7771d1b7 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -65,6 +65,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_EXT_ZVE64X (1ULL << 39) #define RISCV_HWPROBE_EXT_ZVE64F (1ULL << 40) #define RISCV_HWPROBE_EXT_ZVE64D (1ULL << 41) +#define RISCV_HWPROBE_EXT_ZICCLSM (1ULL << 42) #define RISCV_HWPROBE_KEY_CPUPERF_0 5 #define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) #define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 1d6e4fda00f8..83c5ae16ad5e 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -283,6 +283,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h), __RISCV_ISA_EXT_SUPERSET(zicbom, RISCV_ISA_EXT_ZICBOM, riscv_xlinuxenvcfg_exts), __RISCV_ISA_EXT_SUPERSET(zicboz, RISCV_ISA_EXT_ZICBOZ, riscv_xlinuxenvcfg_exts), + __RISCV_ISA_EXT_DATA(zicclsm, RISCV_ISA_EXT_ZICCLSM), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), __RISCV_ISA_EXT_DATA(zicond, RISCV_ISA_EXT_ZICOND), __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR), diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index e4ec9166339f..e910e2971984 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -96,6 +96,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair, EXT_KEY(ZBB); EXT_KEY(ZBS); EXT_KEY(ZICBOZ); + EXT_KEY(ZICCLSM); EXT_KEY(ZBC); EXT_KEY(ZBKB); From patchwork Tue Jun 25 00:49:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CD6CBC2BD09 for ; Tue, 25 Jun 2024 00:50:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=W9ObkAdVMXkLZ6dJa37gNEt3fAr6JbkItJq3D2Vr8RQ=; b=QzEMOTHQHMhA6i gtUhoDcs35+d9mZuP5PC9gtZqO/MieKAw+bq8Rq/VtOv3lDetVFIt8nW3eck8iSztavEd+4ymu4MW MwFBLOK4i0ridVxzwVkxmfjyYClquI3IEExE3Wwus/Ji7sn2rQuu0wAO65Hyntg+mK1BLoGgBh18Z O/yFwjMvsDTwpAn8hF8kXp69QEM1Om8eLKBXzWaRjSqYs4m1RYN/fQqymPgBUpmrW8M6MYT5zdvXp 9cuHD7/BtO+bh1h8XtTGFbLvBm5UmUn9qGgdgYQa0PQIcR0V7ZPer6fHFOP9B9XvzQGzkPgMtfJyy zEI8atpeRCuEwFkOfOCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOb-000000018JP-2STu; Tue, 25 Jun 2024 00:50:53 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOY-000000018I7-10I8 for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:50:51 +0000 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1f9de13d6baso33195855ad.2 for ; Mon, 24 Jun 2024 17:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276649; x=1719881449; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EJQbdAAUHpsJIDvU82Jg8JV8IiFb5YWn87NOrnsuEk4=; b=VZS/I/Xbsx4D3GEVR/62TO/urRM0wktc7DuSm9Ll6Yside4Blo44e7KtUG1k6agfM7 Ld8C8anBFxDzGcwxKxaYsytu8In5ZGjuE09AMU+udyVhH/aMlctFiSfhgswLluN4T/fJ PtMXSJOHULSmTD9RCMY/JCZT+6xOyEj9aDgQMXfT6/l3VuoTBIJsDp5dM+VU2QZkBAcR QolcfFRJ8/O1liFXGvbMJ4gqnGIY7md3/V4orHtfhVoIn6SdKhfDRYccn3t26vAN2V/r OKdl/aE/wkwU/AJJSawZD+uN/386vy3Og7NSusBbfScrpP1dDvrtK0Fej+NIU8Y0f/1S ZSIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276649; x=1719881449; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EJQbdAAUHpsJIDvU82Jg8JV8IiFb5YWn87NOrnsuEk4=; b=g8xo7Mop2BchmdavuxGBnAGrw3s/tpqNOl1gqy3gYi40CrGCttMlnj287oCxy2oCZ6 OpFsQhXzfErMnztcmFVd1HymVG5uytFQxUA5tlDl/RXGLOrQFIyznVW0LExWZhQGtP2B KR424DCVWFJLeAJfZNVOpY0BeAuuKojHPyKhGb496wrJqQJQhYQa9QOaI76EkSPw9p4Y a3IzFMceNd37gCP4DbmOvRulWltW+N5LPhzEdtH8SYhG+GMp1Fqiwr8RLvvIfDOaNME4 YxLssOvuq2HsgmiPM9PGoru8emiChrx6i+W/t3Yy7hnPs5B8IofKcsfjej++t4mcBfKW v53Q== X-Gm-Message-State: AOJu0YyGUYv2OYnLF1HtSprz/BxoZGiIOinkXYBeGRT/J2iPHTr5Ow/7 eifsd/3rThAnXEcUpfpLogdAeuZSz4i+ok3E8UJsrMCN3Bj0hCSeQ0E/W5D8XnnC3rxffMPhoDP I X-Google-Smtp-Source: AGHT+IH7iNOTk2dACUhxktOMTo7VpX1zJFsXLc4rOo9b8V0SM2fkV7dKQh6SXUdWY9e9smQZdIjpvg== X-Received: by 2002:a17:902:e811:b0:1f9:bb24:b21a with SMTP id d9443c01a7336-1fa2401c0c3mr69837055ad.41.1719276649116; Mon, 24 Jun 2024 17:50:49 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.50.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:50:48 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, Conor Dooley Subject: [PATCH v3 2/8] dt-bindings: riscv: Add Zicclsm ISA extension description. Date: Mon, 24 Jun 2024 20:49:55 -0400 Message-ID: <20240625005001.37901-3-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175050_304251_AD4B2441 X-CRM114-Status: GOOD ( 10.05 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add description for Zicclsm ISA extension. Signed-off-by: Jesse Taube Acked-by: Conor Dooley --- V1 -> V2: - New patch V2 -> V3: - No changes --- Documentation/devicetree/bindings/riscv/extensions.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Documentation/devicetree/bindings/riscv/extensions.yaml index cfed80ad5540..9f6aae1f5b65 100644 --- a/Documentation/devicetree/bindings/riscv/extensions.yaml +++ b/Documentation/devicetree/bindings/riscv/extensions.yaml @@ -317,6 +317,13 @@ properties: The standard Zicboz extension for cache-block zeroing as ratified in commit 3dd606f ("Create cmobase-v1.0.pdf") of riscv-CMOs. + - const: zicclsm + description: + The standard Zicclsm extension for misaligned support for all regular + load and store instructions (including scalar and vector) but not AMOs + or other specialized forms of memory access. Defined in the + RISC-V RVA Profiles Specification. + - const: zicntr description: The standard Zicntr extension for base counters and timers, as From patchwork Tue Jun 25 00:49:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F721C3064D for ; Tue, 25 Jun 2024 00:51:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=K4oxY1ZcFFrbFnzKog8LxYlqdjOupKiF8x43vulcc7k=; b=zLIln5o22tgLNM cZRLDKdSPm0njXeiRykChZ6LgCu7oAoxuN31JJnPjYWaPOUH8i5zGn5EDL2efJuw7Sbkdk0JJ5zAV WmuEutMmeezsqx3dV5SbVhEugZdE7YK/2tVqtyA4gwK5wrq/YUt3mHciFs/xuIVq4eMFRnq8FNbfe qjguwZjFYkGmq50Ry3XHJrLbjmviqbToit/kCcksRfgiRFa/AMXjGNSvD3gEo1nZ5Sc/iEjdepANz KkLkmYf4NBGX5qd6LqX8kStWkqOuF3fSWYrPcPQw4xv3kMfM9GcZXOYDONGE77/NvrIH1tD6fot8+ DN60SpGTQCEjREeKrEKA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOp-000000018Pn-084t; Tue, 25 Jun 2024 00:51:07 +0000 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuOm-000000018O5-3C1c for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:51:05 +0000 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1fa1a542466so16909355ad.0 for ; Mon, 24 Jun 2024 17:51:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276664; x=1719881464; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S8PvHLug8r0Usox0R6aX+Qr/KVZBRToAr5dZQC5ZPHA=; b=CYrajDYQHkWbpRZRl81l2a7YOFQQ4Cbj69ydMK0ltQlOCHqjFiv4tuNuyUj+AgHZEX e3k73blUbpkwjbKp7ZwtMpCk0L3a3NiVmm+oGu6L266qelsi0+cdCeFyRR00J5/vVvte 13SqEluKCwetM4Qx/ls2oa5g2JlzYbAX4HEL6xrlErqEpGIgpy9/qL5V8qlqvQMVFw8z Zkn6dmVFme1RhQXIMFjdW3WSL2HePbNKUOiXZRQXnV9cEDbqHwyK+R7EaO50NOxQuZkT UOWDayAkEJDqDfSUhlU3NKzf1MrE+GGx7HV2qBxgB8cfsCkqICpWSJS3spMAAF0WVoaV leSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276664; x=1719881464; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S8PvHLug8r0Usox0R6aX+Qr/KVZBRToAr5dZQC5ZPHA=; b=SetwxQikFecGJq5cyomWbD/FEFUA7jOiWvGzAkKq3as6X3QKtwFZKmRNfEkPuv45uY HHmEpEE88dmdobvRk4wsJakZJYZ3ZkXE7SkrdxnXonw9wSVkLk76fD42S79naHNbANiG 7UOXNszA0yxVr9G5B4/UapyZVTGcRK0ctKpPQmKAe6euQ5TJSxLh82Zbits8sLwi0gvf WSmu9286Bzb4koFQAFB7LI/7rbAZnu5Zlds0OGfklhUA5ejhd1hQiADbAVfSTtscFjY9 jh5QRFueqsDqI5nIbneROeh3ayaKXsagH5MCEYB00VaILGcOE3ee3ZtjBEyNUDyD+2hP l9GQ== X-Gm-Message-State: AOJu0Yyuit9BnkJzkWknJagD3k5RyYdjvIPoZdgSJbkuJ1vUEGHb1WtC +Fz73tbfW/ToL53OIuNwn8hKMpPSwQra226TXDiIy5XbfSr0sWgKFObPfq70/OYEm2W06nT7PWd 6 X-Google-Smtp-Source: AGHT+IEAP5AqPSLGnRbl2LW9e1HkZo4YnWE4Xye4a5nUkIMhcY2MADnRmTPJFFoHvz1mEZkNhxwJ0g== X-Received: by 2002:a17:903:c1:b0:1f7:1b42:42f3 with SMTP id d9443c01a7336-1fa0f8cdcd6mr55557855ad.18.1719276663833; Mon, 24 Jun 2024 17:51:03 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.50.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:51:03 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, stable@vger.kernel.org Subject: [PATCH v3 3/8] RISC-V: Check scalar unaligned access on all CPUs Date: Mon, 24 Jun 2024 20:49:56 -0400 Message-ID: <20240625005001.37901-4-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175104_865871_E4442C1C X-CRM114-Status: GOOD ( 14.27 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Originally, the check_unaligned_access_emulated_all_cpus function only checked the boot hart. This fixes the function to check all harts. Fixes: 71c54b3d169d ("riscv: report misaligned accesses emulation to hwprobe") Signed-off-by: Jesse Taube Cc: stable@vger.kernel.org --- V1 -> V2: - New patch V2 -> V3: - Split patch --- arch/riscv/kernel/traps_misaligned.c | 23 ++++++----------------- 1 file changed, 6 insertions(+), 17 deletions(-) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index b62d5a2f4541..8fadbe00dd62 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -526,31 +526,17 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } -static bool check_unaligned_access_emulated(int cpu) +static void check_unaligned_access_emulated(struct work_struct *unused) { + int cpu = smp_processor_id(); long *mas_ptr = per_cpu_ptr(&misaligned_access_speed, cpu); unsigned long tmp_var, tmp_val; - bool misaligned_emu_detected; *mas_ptr = RISCV_HWPROBE_MISALIGNED_UNKNOWN; __asm__ __volatile__ ( " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); - - misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_EMULATED); - /* - * If unaligned_ctl is already set, this means that we detected that all - * CPUS uses emulated misaligned access at boot time. If that changed - * when hotplugging the new cpu, this is something we don't handle. - */ - if (unlikely(unaligned_ctl && !misaligned_emu_detected)) { - pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)\n"); - while (true) - cpu_relax(); - } - - return misaligned_emu_detected; } bool check_unaligned_access_emulated_all_cpus(void) @@ -562,8 +548,11 @@ bool check_unaligned_access_emulated_all_cpus(void) * accesses emulated since tasks requesting such control can run on any * CPU. */ + schedule_on_each_cpu(check_unaligned_access_emulated); + for_each_online_cpu(cpu) - if (!check_unaligned_access_emulated(cpu)) + if (per_cpu(misaligned_access_speed, cpu) + != RISCV_HWPROBE_MISALIGNED_EMULATED) return false; unaligned_ctl = true; From patchwork Tue Jun 25 00:49:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7AB6CC3064D for ; Tue, 25 Jun 2024 00:51:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wiRDvZj0SijcyhRH83dwvSn/sFKzDIYkuXO53ZhyuPM=; b=x4iu8w3V5nhhSB HFW4Y/TfRDMVlT1HvxwtoV3VnHwPQc8uRSYbXEpCE94s25FeqYOxiwGL5JrYOk+B4aetyIYP8m9jk uGIdg4aXmKJ1toQ21TOKfqGPELpDjCbkFlBYWc6KcMw4kA2kk6I+lRKH9mWoZ+e43t+TVYOADCxRT 6XQdQVIm8+gKhJ+JFryDVmKoF7ngL7luCCbDREQE2C7e6lCQk+DDqAPeimKBDLSf8MQa+0EvMKESp DYeR6xp9s3AnhGDvGNkmFeboOITED5G7feNawMW/+bpaGGpPq5GfzVMJpYSeOQh/4oFSsQfYCu4Q9 GAygVVXmcZxYDy+gEmLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuP4-000000018Xn-2giY; Tue, 25 Jun 2024 00:51:22 +0000 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuP1-000000018VO-0qhx for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:51:20 +0000 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1f9ffd24262so22663365ad.0 for ; Mon, 24 Jun 2024 17:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276678; x=1719881478; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vbZ5afsJ9snzdZoHI92IYxp4PE+4CuBIhAjqET55SFU=; b=j9zVXALDhBPYYNRNgi3glqxo2j7Yr51hep+4DSAbUuv0fIRGcqpOM6shNIKz0icdc4 36upL4Gyu/7LXZ47iy+wNZnO9J11KQrAl5YcIvv88WarO4qwbj4GoM7og6EHM2DGtxvc e7Tb1Gh0ZxghqBUnr7OiZAHmQgnUIycEdanSpMBBCp5HV7YvLr5PQ9GXKM5kTMJWmKx1 4bQwZa+8KgB9yqZCOtblGrAzUgTHOOCq0M8bPbzzx44uKBytjI0TwhuPJKPUetKAhuyS 4GyITZg8IHm89L51J6Q0BtzUWsgF6lqsmMmFMhhTImShlRCJQ/gyyqshqJjOmMi9ZJzL fNNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276678; x=1719881478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vbZ5afsJ9snzdZoHI92IYxp4PE+4CuBIhAjqET55SFU=; b=W5Bdrzx8d+TDn/FhIhTf9Vg8JZjGHxQeSVeI80Bqv8vKsF97Y9dasnucBMjA2GOmwc GR2Qu9MuhQ2mPKCDJhzu4K5j9uKqsjBV6WZSUqanXA1tkugMyuZcBnbh0Da/ZFqLHQvU ua8D8XHmLVwzGahR6yAM9D4Oio5UIOM0FKr4MZkxUpTHbo5r1Q1QaQHHZz+G9a3cdIW/ yg0EsUlT62rcP7KGEPpG5nC28cKzBS03g2aAYV1voQeUradLeT3LW/qP2mQHQ1AzVg80 QFiVQmlW/ugQ/aaquKWXgtkMAsLnEqC2eepsIoHWKLs9hYLSUYwTxqWI18dfXJ+Nou+K 5KQg== X-Gm-Message-State: AOJu0YxtoN5ZnfKqMCytcW/W0R/E2ssJvVAr1f7SR+lfzdUn5kw8X6kV 7JQT20HLqYrNbF5q8XaoCb4u99YmCEw8va6Zz63WRmRo5mINNWsaH5jta1tigDUzPqfdNgXCp3u k X-Google-Smtp-Source: AGHT+IFrptaOTah+yZVkTRLhwy8L5y0nIsOmh0E3MizJhpkRcr8WQrdivG0Q840vHNP+HRAolE2fNA== X-Received: by 2002:a17:902:cecc:b0:1f8:67e4:397f with SMTP id d9443c01a7336-1fa23f227b5mr66212145ad.7.1719276678136; Mon, 24 Jun 2024 17:51:18 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:51:17 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v3 4/8] RISC-V: Check Zicclsm to set unaligned access speed Date: Mon, 24 Jun 2024 20:49:57 -0400 Message-ID: <20240625005001.37901-5-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175119_289029_AA6DC8DD X-CRM114-Status: GOOD ( 12.88 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Check for Zicclsm before checking for unaligned access speed. This will greatly reduce the boot up time as finding the access speed is no longer necessary. Signed-off-by: Jesse Taube --- V2 -> V3: - New patch split from previous patch --- arch/riscv/kernel/unaligned_access_speed.c | 26 ++++++++++++++-------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index a9a6bcb02acf..329fd289b5c8 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -259,23 +259,31 @@ static int check_unaligned_access_speed_all_cpus(void) kfree(bufs); return 0; } +#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ +static int check_unaligned_access_speed_all_cpus(void) +{ + return 0; +} +#endif static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); + bool all_cpus_emulated; + int cpu; + + if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { + for_each_online_cpu(cpu) { + per_cpu(misaligned_access_speed, cpu) = RISCV_HWPROBE_MISALIGNED_FAST; + } + return 0; + } + + all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); return 0; } -#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ -static int check_unaligned_access_all_cpus(void) -{ - check_unaligned_access_emulated_all_cpus(); - - return 0; -} -#endif arch_initcall(check_unaligned_access_all_cpus); From patchwork Tue Jun 25 00:49:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E324C3064D for ; Tue, 25 Jun 2024 00:51:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PIcOt2dExj+Qn3lHTeolWSKrFN4+3YK/QBrFoGpzfiQ=; b=BVgvRDIkWHsXLA vwWf/CaXG0DxEkNnFiuf7Bbz+/yiehGB9JQYwsjUuc3mejl65+Sme1lQa6WufX2nkSNBR3UTTJgzk SViBSa/CtjUcMka8QJxd7qHuA5R2jstb1vEhEddXPAopFwHhqYF/dWTQJXEvlcMTZ3MZRjzetSOjE U7mA1nEPaHQUxtQmz1bhLhZYL2toVC/o4sLGX78tvc/+qpUXkdYvryUvLNVOdkd6WUHFYsP07mbxf 531Ndwq1HNw5qYx+Fc21nVxm13z3AvU0Z1cxo9l/mDSjtNi1hyTe+3uShq6LrDMMuiKKalBEydRlg b2UR6jbPBcxyAh67S87Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPI-000000018ek-0i36; Tue, 25 Jun 2024 00:51:36 +0000 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPF-000000018de-1zCw for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:51:34 +0000 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1f9c2847618so41395785ad.1 for ; Mon, 24 Jun 2024 17:51:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276692; x=1719881492; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1oa3I+3DutwwBi8qvOET2dkVnq58k4yUHgUCWbbrOmQ=; b=ILXzNBxS6J7POP0QgnV2Q2bfDBl1NUHMfMh2v/fG1MM/AQ3+/X6iLW/I1fDSNXqJaD EGSGZTlPw4RtQu6NuqOYBaygLlXh6Vk0q6h9TD+y9iAod7IDO1fHVPgWLa7GSn6FNe1T v7mKj9oq8OxprzDOAyGwZkBAWh8okl4BvYiEJAGDD2qbF+4/h2dwq9aDsETy+9Ke4GUC DyPaSqJuDihh/wyLeh7K2I+0U9y2ud+TX4TKkKQnb91aRR3By4Zn5ZybQ+i6mUViOpqW h0QsCQex0Hrl5ZVXVrqDrWnDQKO0SjkQ/68VwF5MSzXiBOlXebFT0Vh8b/yI1lIAu4fr vpFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276692; x=1719881492; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1oa3I+3DutwwBi8qvOET2dkVnq58k4yUHgUCWbbrOmQ=; b=uKchfzQLpdQ1m7ovoTKL7AZl+R5i+tWTenuSs4iVle3yf+UCSwoNpc5wI/STMdKYOO S0IpJ/ESN6OlMANOid3PiL0iSNxaoOHBE3kBvXhzH773jl5DA7pDRK+P/eNe2cBO+LHJ TlbsmBC07kD2KaXOh0XO1Vue5PUekqYDnA7k6gJZ++GiV1JeGUjZOO0AxQI3a9S93LsX fU5zSU5t2XcWHVnkwSaUDjydKeyet9ZrgA69JdgaRZ+BI/JImMe2/M7SZ4LwBGFn0D9n q/+0hrNOPI2jkY/3LsBhZ6AwLcdfVcaxC7biaWy7sWZxYWs7YLyMWvix0JeENfdKBuY0 rrmg== X-Gm-Message-State: AOJu0YzrJzXTt+ft9uPqZVyTs22pnch4o/MyIi8i7few7XwDoz0vxvl7 7XkBOAekJ0xX+GRLg/Al+TqoGK9Q5+YM18C9LQFvBKgSyjBB5LljgzKYLL7NBrYyB+1Jc0y56tG L X-Google-Smtp-Source: AGHT+IErTLldPHfIA/rX16NWeo1oMIVqSiKmNwkGaeTo+WEnyEbdKZRROZeAVNmp1AXXIm8ZcE7X/w== X-Received: by 2002:a17:903:1c3:b0:1f7:2d45:2f1 with SMTP id d9443c01a7336-1fa158de5bcmr75992885ad.15.1719276692445; Mon, 24 Jun 2024 17:51:32 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.51.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:51:32 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v3 5/8] RISC-V: Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED Date: Mon, 24 Jun 2024 20:49:58 -0400 Message-ID: <20240625005001.37901-6-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175133_538654_CE6315F7 X-CRM114-Status: GOOD ( 14.71 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Replace RISCV_MISALIGNED with RISCV_SCALAR_MISALIGNED to allow for the addition of RISCV_VECTOR_MISALIGNED in a later patch. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley --- V2 -> V3: - New patch --- arch/riscv/Kconfig | 6 +++--- arch/riscv/include/asm/cpufeature.h | 2 +- arch/riscv/include/asm/entry-common.h | 2 +- arch/riscv/kernel/Makefile | 4 ++-- arch/riscv/kernel/fpu.S | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..34d24242e37a 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -717,7 +717,7 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. -config RISCV_MISALIGNED +config RISCV_SCALAR_MISALIGNED bool select SYSCTL_ARCH_UNALIGN_ALLOW help @@ -734,7 +734,7 @@ choice config RISCV_PROBE_UNALIGNED_ACCESS bool "Probe for hardware unaligned access support" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help During boot, the kernel will run a series of tests to determine the speed of unaligned accesses. This probing will dynamically determine @@ -745,7 +745,7 @@ config RISCV_PROBE_UNALIGNED_ACCESS config RISCV_EMULATED_UNALIGNED_ACCESS bool "Emulate unaligned access where system support is missing" - select RISCV_MISALIGNED + select RISCV_SCALAR_MISALIGNED help If unaligned memory accesses trap into the kernel as they are not supported by the system, the kernel will emulate the unaligned diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..0ed7d99c14dd 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -33,8 +33,8 @@ extern struct riscv_isainfo hart_isa[NR_CPUS]; void riscv_user_isa_enable(void); -#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_SCALAR_MISALIGNED) void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm/entry-common.h index 2293e535f865..0a4e3544c877 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,7 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); #else diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..8d4e7d40e42f 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ -obj-$(CONFIG_RISCV_MISALIGNED) += traps_misaligned.o -obj-$(CONFIG_RISCV_MISALIGNED) += unaligned_access_speed.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) += traps_misaligned.o +obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) += unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o obj-$(CONFIG_FPU) += fpu.o diff --git a/arch/riscv/kernel/fpu.S b/arch/riscv/kernel/fpu.S index 327cf527dd7e..f74f6b60e347 100644 --- a/arch/riscv/kernel/fpu.S +++ b/arch/riscv/kernel/fpu.S @@ -170,7 +170,7 @@ SYM_FUNC_END(__fstate_restore) __access_func(f31) -#ifdef CONFIG_RISCV_MISALIGNED +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED /* * Disable compressed instructions set to keep a constant offset between FP @@ -224,4 +224,4 @@ SYM_FUNC_START(get_f64_reg) fp_access_epilogue SYM_FUNC_END(get_f64_reg) -#endif /* CONFIG_RISCV_MISALIGNED */ +#endif /* CONFIG_RISCV_SCALAR_MISALIGNED */ From patchwork Tue Jun 25 00:49:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03BA3C3064D for ; Tue, 25 Jun 2024 00:51:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=X0MD7zzVthkNSDfpRJsbl3qB9DZRXfGK+nXDWSoucrA=; b=WQoanZWvU39MNg lZGd9P8jtXC68NrUUIcYXVZ0yvrAFYAI991GjtKJ2Iha1UGnYYQpXjL/SqujF29nOEkpNZtnoQ/lS 8QU+UcfvuM554zy/wWYYI5DdM7bOzNKEQ5c5cGV9DdjN8aYu24gZkKleJHJX9qp5tdOA5oF0sLexb FA7UcBAeI5iJCUt1l84RA1TVOXcZ9unWPQcyokvJmOzZdVevpNmygUJFNIo6nrm0LjC6FDw0yIQCs 8NUE5VeBxsC14FgPhOzrkwPuLchqHKAySThu47LAf07hkiLnxiA7BwJhpSuolOR5UokRvPboDL8GN NNcxzm8TeHdIJByVANgg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPX-000000018mm-0MyE; Tue, 25 Jun 2024 00:51:51 +0000 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPT-000000018kp-3jMn for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:51:49 +0000 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-7066a4a611dso1677502b3a.3 for ; Mon, 24 Jun 2024 17:51:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276707; x=1719881507; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QKl6zIDaJ/Av5Rwcnw+etMZXm6iL8TVMqp6HLgQqRb4=; b=GDDF6NoWko/SFysotReU3EWsBvpUCOmgAAZ6IYys0g5drbPnu4xZsRpmtuFADB4eSp ryWxL4i2wvvaD9sz+/TKlAW7x8YfOPav0l8VRq/qPvwambc6fpK3xJ1w/L454ZjUI/ly eNEIh4cMLeFYXtuD1P/fU3PdvUntbM3ce2Pb5gUFOOSytle6xrdozUsTepTdXA1hZ9gT dJlDc7kSfOzxsPjd7jBJiRq6eblumnm7ddb4D2lc1vqVE3ik1ilQvbeKZOdadO2IQk3Z 1z788qitLRnSzjKgy6UQKlKEkH0ib9ZLLgu0b+h5gSBCmMK5Vp8CpBQN2GfF589TUwaI SECg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276707; x=1719881507; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QKl6zIDaJ/Av5Rwcnw+etMZXm6iL8TVMqp6HLgQqRb4=; b=qwskceh0bKwHeyHgXpfrnFz9SGuB+wuOiqxdCwps0iBYSZxISKTQ6MnrQoT6s7sSxb OyQpAXqeKPO9rsbmUKuyeT2gOxczCD4cME1ILF/0OGKjFubGfpNj6T6zqkbqYtaLPIin HSix0Nmv1EYcgkrBK/6oSyp6487Hhun+S4acvJDt+vJL/4eKan6M1YUBRbYaSOvEkVqg OqSoCyIvuAxcut+B7gFky18NhZxrd21p+IcGHHRHSjzmz78ydNbAjJC4Y2/kqFe2QnL+ oXOp2oSKdXHsmFUcVp5/ywaBC0ZrB03ovlOrzgyYNzIcYWP74OCoaEs+Y3oIPEEeqf5J 64YQ== X-Gm-Message-State: AOJu0YwjfHeA6PWcldhwlD8Is78hAB2nyIMA8n3T7KDrvRqlQRZErhML OHiwg4JEcNvK2su4VRpTeK3D4HzfG1/C2aziyEGpDi3JGH/JhPHtrHWWIaBiYftE/5o31rnrl+g k X-Google-Smtp-Source: AGHT+IG7makXmY+D39VjvdE+J+r7ZeCxtrYsv+Y9WTpDUJ1QDs72R7ahLQSLrIfF4IVjTeLNQIw48w== X-Received: by 2002:a05:6a20:be09:b0:1bd:a39:df8c with SMTP id adf61e73a8af0-1bd0a39e16dmr3173556637.49.1719276706766; Mon, 24 Jun 2024 17:51:46 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.51.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:51:46 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v3 6/8] RISC-V: Detect unaligned vector accesses supported Date: Mon, 24 Jun 2024 20:49:59 -0400 Message-ID: <20240625005001.37901-7-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175147_975876_DECE24FD X-CRM114-Status: GOOD ( 24.25 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Run a unaligned vector access to test if the system supports vector unaligned access. Add the result to a new key in hwprobe. This is useful for usermode to know if vector misaligned accesses are supported and if they are faster or slower than equivalent byte accesses. Signed-off-by: Jesse Taube V1 -> V2: - Add Kconfig options - Add insn_is_vector - Add handle_vector_misaligned_load - Fix build - Seperate vector from scalar misaligned access - This patch was almost completely rewritten V2 -> V3: - Fixed CONFIG_ in Kconfig - Fixed check_vector_unaligned_access_emulated leaving vector_misaligned_access as unknown. - Remove local_irq_enable - Remove RISCV_DETECT_VECTOR_UNALIGNED_ACCESS - Remove RISCV_VEC_UNALIGNED_ACCESS_UNSUPPORTED --- arch/riscv/Kconfig | 35 ++++++ arch/riscv/include/asm/cpufeature.h | 5 + arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/sys_hwprobe.c | 35 ++++++ arch/riscv/kernel/traps_misaligned.c | 120 ++++++++++++++++++++- arch/riscv/kernel/unaligned_access_speed.c | 9 +- arch/riscv/kernel/vector.c | 2 +- 11 files changed, 209 insertions(+), 20 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 34d24242e37a..ffbe0fdd7fb3 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -717,12 +717,26 @@ config THREAD_SIZE_ORDER Specify the Pages of thread stack size (from 4KB to 64KB), which also affects irq stack size, which is equal to thread stack size. +config RISCV_MISALIGNED + bool + help + Embed support for detecting and emulating misaligned + scalar or vector loads and stores. + config RISCV_SCALAR_MISALIGNED bool + select RISCV_MISALIGNED select SYSCTL_ARCH_UNALIGN_ALLOW help Embed support for emulating misaligned loads and stores. +config RISCV_VECTOR_MISALIGNED + bool + select RISCV_MISALIGNED + depends on RISCV_ISA_V + help + Enable detecting support for vector misaligned loads and stores. + choice prompt "Unaligned Accesses Support" default RISCV_PROBE_UNALIGNED_ACCESS @@ -774,6 +788,27 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS endchoice +choice + prompt "Vector unaligned Accesses Support" + depends on RISCV_ISA_V + default RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + help + This determines the level of support for vector unaligned accesses. This + information is used by the kernel to perform optimizations. It is also + exposed to user space via the hwprobe syscall. The hardware will be + probed at boot by default. + +config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + bool "Probe speed of vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will run a series of tests to determine the + speed of vector unaligned accesses if they are supported. This probing + will dynamically determine the speed of vector unaligned accesses on + the underlying system if they are supported. + +endchoice + endmenu # "Platform type" menu "Kernel features" diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 0ed7d99c14dd..f25f56f9bfaa 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -45,6 +45,11 @@ static inline bool unaligned_ctl_available(void) } #endif +bool check_vector_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_VECTOR_MISALIGNED) +DECLARE_PER_CPU(long, vector_misaligned_access); +#endif + #if defined(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) DECLARE_STATIC_KEY_FALSE(fast_unaligned_access_speed_key); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm/entry-common.h index 0a4e3544c877..7b32d2b08bb6 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,18 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); -#ifdef CONFIG_RISCV_SCALAR_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); -#else -static inline int handle_misaligned_load(struct pt_regs *regs) -{ - return -1; -} -static inline int handle_misaligned_store(struct pt_regs *regs) -{ - return -1; -} -#endif #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h index 150a9877b0af..ef01c182af2b 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ #include -#define RISCV_HWPROBE_MAX_KEY 7 +#define RISCV_HWPROBE_MAX_KEY 8 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index be7d309cca8a..99b0f91db9ee 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -21,6 +21,7 @@ extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); +bool insn_is_vector(u32 insn_buf); bool riscv_v_first_use_handler(struct pt_regs *regs); void kernel_vector_begin(void); void kernel_vector_end(void); diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 023b7771d1b7..2fee870e41bb 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -75,6 +75,11 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 #define RISCV_HWPROBE_KEY_MISALIGNED_PERF 7 +#define RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF 8 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN 0 +#define RISCV_HWPROBE_VEC_MISALIGNED_SLOW 2 +#define RISCV_HWPROBE_VEC_MISALIGNED_FAST 3 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED 4 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ /* Flags */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 8d4e7d40e42f..5b243d46f4b1 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) += traps_misaligned.o -obj-$(CONFIG_RISCV_SCALAR_MISALIGNED) += unaligned_access_speed.o +obj-$(CONFIG_RISCV_MISALIGNED) += traps_misaligned.o +obj-$(CONFIG_RISCV_MISALIGNED) += unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o obj-$(CONFIG_FPU) += fpu.o diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index e910e2971984..5b78ea5a84d1 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -194,6 +194,37 @@ static u64 hwprobe_misaligned(const struct cpumask *cpus) } #endif +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + int cpu; + u64 perf = -1ULL; + + /* Return if supported or not even if speed wasn't probed */ + for_each_cpu(cpu, cpus) { + int this_perf = per_cpu(vector_misaligned_access, cpu); + + if (perf == -1ULL) + perf = this_perf; + + if (perf != this_perf) { + perf = RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + break; + } + } + + if (perf == -1ULL) + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + return perf; +} +#else +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; +} +#endif + static void hwprobe_one_pair(struct riscv_hwprobe *pair, const struct cpumask *cpus) { @@ -222,6 +253,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, pair->value = hwprobe_misaligned(cpus); break; + case RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF: + pair->value = hwprobe_vec_misaligned(cpus); + break; + case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE: pair->value = 0; if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ)) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index 8fadbe00dd62..55c43962115f 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include #include #include +#include #define INSN_MATCH_LB 0x3 #define INSN_MASK_LB 0x707f @@ -322,12 +323,37 @@ union reg_data { u64 data_u64; }; -static bool unaligned_ctl __read_mostly; - /* sysctl hooks */ int unaligned_enabled __read_mostly = 1; /* Enabled by default */ -int handle_misaligned_load(struct pt_regs *regs) +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc = regs->epc; + unsigned long insn; + + if (get_insn(regs, epc, &insn)) + return -1; + + /* Only return 0 when in check_vector_unaligned_access_emulated */ + if (*this_cpu_ptr(&vector_misaligned_access) == RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN) { + *this_cpu_ptr(&vector_misaligned_access) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + regs->epc = epc + INSN_LEN(insn); + return 0; + } + + /* If vector instruction we don't emulate it yet */ + regs->epc = epc; + return -1; +} +#else +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + return -1; +} +#endif + +static int handle_scalar_misaligned_load(struct pt_regs *regs) { union reg_data val; unsigned long epc = regs->epc; @@ -435,7 +461,7 @@ int handle_misaligned_load(struct pt_regs *regs) return 0; } -int handle_misaligned_store(struct pt_regs *regs) +static int handle_scalar_misaligned_store(struct pt_regs *regs) { union reg_data val; unsigned long epc = regs->epc; @@ -526,6 +552,86 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } +int handle_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc = regs->epc; + unsigned long insn; + + if (IS_ENABLED(CONFIG_RISCV_VECTOR_MISALIGNED)) { + if (get_insn(regs, epc, &insn)) + return -1; + + if (insn_is_vector(insn)) + return handle_vector_misaligned_load(regs); + } + + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_load(regs); + + return -1; +} + +int handle_misaligned_store(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_SCALAR_MISALIGNED)) + return handle_scalar_misaligned_store(regs); + + return -1; +} + +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static void check_vector_unaligned_access_emulated(struct work_struct *unused) +{ + long *mas_ptr = this_cpu_ptr(&vector_misaligned_access); + unsigned long tmp_var; + + *mas_ptr = RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + kernel_vector_begin(); + __asm__ __volatile__ ( + ".balign 4\n\t" + ".option push\n\t" + ".option arch, +zve32x\n\t" + " vsetivli zero, 1, e16, m1, ta, ma\n\t" // Vectors of 16b + " vle16.v v0, (%[ptr])\n\t" // Load bytes + ".option pop\n\t" + : : [ptr] "r" ((u8 *)&tmp_var + 1) : "v0"); + kernel_vector_end(); + + if (*mas_ptr == RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN) + *mas_ptr = RISCV_HWPROBE_VEC_MISALIGNED_SLOW; +} + +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + int cpu; + + if (!has_vector()) { + for_each_online_cpu(cpu) + per_cpu(vector_misaligned_access, cpu) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + return false; + } + + schedule_on_each_cpu(check_vector_unaligned_access_emulated); + + for_each_online_cpu(cpu) + if (per_cpu(vector_misaligned_access, cpu) + != RISCV_HWPROBE_VEC_MISALIGNED_SLOW) + return false; + + return true; +} +#else +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif + +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED + +static bool unaligned_ctl __read_mostly; + static void check_unaligned_access_emulated(struct work_struct *unused) { int cpu = smp_processor_id(); @@ -563,3 +669,9 @@ bool unaligned_ctl_available(void) { return unaligned_ctl; } +#else +bool check_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index 329fd289b5c8..8489e012cf23 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -19,7 +19,8 @@ #define MISALIGNED_BUFFER_ORDER get_order(MISALIGNED_BUFFER_SIZE) #define MISALIGNED_COPY_SIZE ((MISALIGNED_BUFFER_SIZE / 2) - 0x80) -DEFINE_PER_CPU(long, misaligned_access_speed); +DEFINE_PER_CPU(long, misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_UNKNOWN; +DEFINE_PER_CPU(long, vector_misaligned_access) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS static cpumask_t fast_misaligned_access; @@ -273,12 +274,18 @@ static int check_unaligned_access_all_cpus(void) if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { for_each_online_cpu(cpu) { +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED + per_cpu(vector_misaligned_access, cpu) = RISCV_HWPROBE_VEC_MISALIGNED_FAST; +#endif +#ifdef CONFIG_RISCV_SCALAR_MISALIGNED per_cpu(misaligned_access_speed, cpu) = RISCV_HWPROBE_MISALIGNED_FAST; +#endif } return 0; } all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); + check_vector_unaligned_access_emulated_all_cpus(); if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 682b3feee451..821818886fab 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -66,7 +66,7 @@ void __init riscv_v_setup_ctx_cache(void) #endif } -static bool insn_is_vector(u32 insn_buf) +bool insn_is_vector(u32 insn_buf) { u32 opcode = insn_buf & __INSN_OPCODE_MASK; u32 width, csr; From patchwork Tue Jun 25 00:50:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 306BEC3064D for ; Tue, 25 Jun 2024 00:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SDdbVZAb6lzGuJ7F62ftH+yAqRQk+NMN0JAxbu6/v78=; b=ICWm+CfYrmos65 jt0MgnMdI4DZC/UoIpWbxt+ctK+ncHXl+MeSO+gDRewM2Bmg2675BYrLuekLn+MyUWNFQhSEATxHu o71E9xjxfKg2rRz970NR5Z8viAECYbz4igeD14Rcobfxr9289lPfzfeo2KjbDWp2jZqQgoVh7d4+0 jkeNybpuxcdtPNLa+gLxRTZpNpX/eD53MvUihwa2iVSTryKZ0PQjHOSgFb+nnnctUBToUztobK5rA xTf0npz2Xu+72qeLemVZPd8KMjHMu3yo7oeigGXnLl5Yue87W2MAurd5AuS12OK2fBrFcgbfum/Xh 1Vb6De4QiYLmMQzlR6gg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPl-000000018vK-46g8; Tue, 25 Jun 2024 00:52:05 +0000 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPi-000000018sO-19cB for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:52:03 +0000 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1f47f07acd3so40651105ad.0 for ; Mon, 24 Jun 2024 17:52:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276721; x=1719881521; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X2tgkU9aGqZTPe2pnjnSc9vrh+00DekoLDeyb7G9kKU=; b=EJ9iB7CbkWpET+Cn05SE9Lv5aqRzsp5J9KmHC389CYn2sYtZGCCMtzM3NPlfAkySnb KeZS+w+7XF4r8Q0U1y40MebRnfZGsGutD08IpLMSVmv3T24ztmBhzQWRwRtTa3Eug8CY qejhIcCuFh8kovO4a8w1aEgT8hqOYU3fAnI+msy56GgwR8Dla18JaVER2f3DCSqwwEnS TKXKtetP/CshtdCOkPvygRzYRPNAfcIjvt1aYbca68gRvF7ZLmbxMGUY+hD31ytgLg9N KDljsqmGwI8kdA/Pr3gyyS/T722DnXyrh3Mh8n1d5vXz3eK8kb2IIVkcEAcV642GqMw8 ub/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276721; x=1719881521; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X2tgkU9aGqZTPe2pnjnSc9vrh+00DekoLDeyb7G9kKU=; b=BUaN65YDO/MiZo9old/z55+ErOQ431uVwROozvHOXYxIpIb+kHQWJMYH/qxQJikIx8 8f2EkG9gHolO6Hfgl8MfR4586rKULVV9GvarZwlK+644Z3GGQsusl2y35vKQv+Otv+tH mLVBjtODWyM4ZaRCL4r1EqiSaXnyxq/mYD2ps05Si3PDzBD397jDNWwjoscGuolZ90hb pfFDPMA6DqQg+2HKM/YvmY8/E3AtnQrgutYA+XdFDNnCASX5EARTcVjwtExcU5gaxxJl Ur6y8szEXrJ5MUhwKADG26Sk6EC1h53I9gvGDp1mJ5m8yu5+o02h49704fNcxXMrB+E8 j5kQ== X-Gm-Message-State: AOJu0Yzv82Wx5T0nVlCEijs9QLjeTZoEbYTyP0mvD/vhhXDKq4ec/2EB zRllpxFWzUfK308TObiWfpt5ZfYHpcKyQsCCR5e+0Y9LIgF767OogY8VgS7tyOtC05bhES3c79d T X-Google-Smtp-Source: AGHT+IGpdZyGHhs1Is7AfSNYZzr49EaMRTC68QJZWXouQ1fwjwreylkPVn4YjeOZWIUSu9DqSMA2EQ== X-Received: by 2002:a17:902:f685:b0:1f7:2050:9a76 with SMTP id d9443c01a7336-1fa1d3de5bcmr90910035ad.8.1719276721098; Mon, 24 Jun 2024 17:52:01 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.51.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:52:00 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v3 7/8] RISC-V: Report vector unaligned access speed hwprobe Date: Mon, 24 Jun 2024 20:50:00 -0400 Message-ID: <20240625005001.37901-8-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175202_416678_FAFF3E1C X-CRM114-Status: GOOD ( 28.92 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Detect if vector misaligned accesses are faster or slower than equivalent vector byte accesses. This is useful for usermode to know whether vector byte accesses or vector misaligned accesses have a better bandwidth for operations like memcpy. Signed-off-by: Jesse Taube --- V1 -> V2: - Add Kconfig options - Add WORD_EEW to vec-copy-unaligned.S V2 -> V3: - Remove unnecessary comment - Remove local_irq_enable --- arch/riscv/Kconfig | 18 +++ arch/riscv/kernel/Makefile | 3 +- arch/riscv/kernel/copy-unaligned.h | 5 + arch/riscv/kernel/sys_hwprobe.c | 6 + arch/riscv/kernel/unaligned_access_speed.c | 132 ++++++++++++++++++++- arch/riscv/kernel/vec-copy-unaligned.S | 58 +++++++++ 6 files changed, 219 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/kernel/vec-copy-unaligned.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index ffbe0fdd7fb3..6f9fd3748916 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -807,6 +807,24 @@ config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS will dynamically determine the speed of vector unaligned accesses on the underlying system if they are supported. +config RISCV_SLOW_VEC_UNALIGNED_ACCESS + bool "Assume the system supports slow vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports slow vector unaligned memory accesses. The + kernel and userspace programs may not be able to run at all on systems + that do not support unaligned memory accesses. + +config RISCV_EFFICIENT_VEC_UNALIGNED_ACCESS + bool "Assume the system supports fast vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports fast vector unaligned memory accesses. When + enabled, this option improves the performance of the kernel on such + systems. However, the kernel and userspace programs will run much more + slowly, or will not be able to run at all, on systems that do not + support efficient unaligned memory accesses. + endchoice endmenu # "Platform type" diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..291935a084d5 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,7 +64,8 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_MISALIGNED) += traps_misaligned.o obj-$(CONFIG_RISCV_MISALIGNED) += unaligned_access_speed.o -obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS) += vec-copy-unaligned.o obj-$(CONFIG_FPU) += fpu.o obj-$(CONFIG_FPU) += kernel_mode_fpu.o diff --git a/arch/riscv/kernel/copy-unaligned.h b/arch/riscv/kernel/copy-unaligned.h index e3d70d35b708..85d4d11450cb 100644 --- a/arch/riscv/kernel/copy-unaligned.h +++ b/arch/riscv/kernel/copy-unaligned.h @@ -10,4 +10,9 @@ void __riscv_copy_words_unaligned(void *dst, const void *src, size_t size); void __riscv_copy_bytes_unaligned(void *dst, const void *src, size_t size); +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +void __riscv_copy_vec_words_unaligned(void *dst, const void *src, size_t size); +void __riscv_copy_vec_bytes_unaligned(void *dst, const void *src, size_t size); +#endif + #endif /* __RISCV_KERNEL_COPY_UNALIGNED_H */ diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index 5b78ea5a84d1..46de3f145154 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -221,6 +221,12 @@ static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) #else static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) { + if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_VEC_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VEC_MISALIGNED_FAST; + + if (IS_ENABLED(CONFIG_RISCV_SLOW_VEC_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VEC_MISALIGNED_SLOW; + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; } #endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index 8489e012cf23..a2a8f948500b 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -8,9 +8,11 @@ #include #include #include +#include #include #include #include +#include #include "copy-unaligned.h" @@ -267,9 +269,129 @@ static int check_unaligned_access_speed_all_cpus(void) } #endif +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +static void check_vector_unaligned_access(struct work_struct *unused) +{ + int cpu = smp_processor_id(); + u64 start_cycles, end_cycles; + u64 word_cycles; + u64 byte_cycles; + int ratio; + unsigned long start_jiffies, now; + struct page *page; + void *dst; + void *src; + long speed = RISCV_HWPROBE_VEC_MISALIGNED_SLOW; + + if (per_cpu(vector_misaligned_access, cpu) != RISCV_HWPROBE_VEC_MISALIGNED_SLOW) + return; + + page = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); + if (!page) { + pr_warn("Allocation failure, not measuring vector misaligned performance\n"); + return; + } + + /* Make an unaligned destination buffer. */ + dst = (void *)((unsigned long)page_address(page) | 0x1); + /* Unalign src as well, but differently (off by 1 + 2 = 3). */ + src = dst + (MISALIGNED_BUFFER_SIZE / 2); + src += 2; + word_cycles = -1ULL; + + /* Do a warmup. */ + kernel_vector_begin(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + + start_jiffies = jiffies; + while ((now = jiffies) == start_jiffies) + cpu_relax(); + + /* + * For a fixed amount of time, repeatedly try the function, and take + * the best time in cycles as the measurement. + */ + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles = get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles = get_cycles64(); + if ((end_cycles - start_cycles) < word_cycles) + word_cycles = end_cycles - start_cycles; + } + + byte_cycles = -1ULL; + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + start_jiffies = jiffies; + while ((now = jiffies) == start_jiffies) + cpu_relax(); + + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles = get_cycles64(); + mb(); + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + mb(); + end_cycles = get_cycles64(); + if ((end_cycles - start_cycles) < byte_cycles) + byte_cycles = end_cycles - start_cycles; + } + + kernel_vector_end(); + + /* Don't divide by zero. */ + if (!word_cycles || !byte_cycles) { + pr_warn("cpu%d: rdtime lacks granularity needed to measure unaligned vector access speed\n", + cpu); + + return; + } + + if (word_cycles < byte_cycles) + speed = RISCV_HWPROBE_VEC_MISALIGNED_FAST; + + ratio = div_u64((byte_cycles * 100), word_cycles); + pr_info("cpu%d: Ratio of vector byte access time to vector unaligned word access is %d.%02d, unaligned accesses are %s\n", + cpu, + ratio / 100, + ratio % 100, + (speed == RISCV_HWPROBE_VEC_MISALIGNED_FAST) ? "fast" : "slow"); + + per_cpu(vector_misaligned_access, cpu) = speed; +} + +static int riscv_online_cpu_vec(unsigned int cpu) +{ + check_vector_unaligned_access(NULL); + return 0; +} + +/* Measure unaligned access speed on all CPUs present at boot in parallel. */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused) +{ + schedule_on_each_cpu(check_vector_unaligned_access); + + /* + * Setup hotplug callbacks for any new CPUs that come online or go + * offline. + */ + cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv:online", + riscv_online_cpu_vec, NULL); + + return 0; +} +#else /* CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused) +{ + return 0; +} +#endif + static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated; + bool all_cpus_emulated, all_cpus_vec_supported; int cpu; if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { @@ -285,7 +407,13 @@ static int check_unaligned_access_all_cpus(void) } all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); - check_vector_unaligned_access_emulated_all_cpus(); + all_cpus_vec_supported = check_vector_unaligned_access_emulated_all_cpus(); + + if (all_cpus_vec_supported && + IS_ENABLED(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS)) { + kthread_run(vec_check_unaligned_access_speed_all_cpus, + NULL, "vec_check_unaligned_access_speed_all_cpus"); + } if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); diff --git a/arch/riscv/kernel/vec-copy-unaligned.S b/arch/riscv/kernel/vec-copy-unaligned.S new file mode 100644 index 000000000000..e5bc94917e60 --- /dev/null +++ b/arch/riscv/kernel/vec-copy-unaligned.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2024 Rivos Inc. */ + +#include +#include +#include + + .text + +#define WORD_EEW 32 + +#define WORD_SEW CONCATENATE(e, WORD_EEW) +#define VEC_L CONCATENATE(vle, WORD_EEW).v +#define VEC_S CONCATENATE(vle, WORD_EEW).v + +/* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using word loads and stores. */ +/* Note: The size is truncated to a multiple of WORD_EEW */ +SYM_FUNC_START(__riscv_copy_vec_words_unaligned) + andi a4, a2, ~(WORD_EEW-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, WORD_SEW, m8, ta, ma + VEC_L v0, (a1) + VEC_S v0, (a0) + addi a0, a0, WORD_EEW + addi a1, a1, WORD_EEW + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_words_unaligned) + +/* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using only byte accesses. */ +/* Note: The size is truncated to a multiple of 8 */ +SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) + andi a4, a2, ~(8-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, e8, m8, ta, ma + vle8.v v0, (a1) + vse8.v v0, (a0) + addi a0, a0, 8 + addi a1, a1, 8 + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_bytes_unaligned) From patchwork Tue Jun 25 00:50:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13710393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1E5B6C2BD09 for ; Tue, 25 Jun 2024 00:52:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h/6uAjfKUgfnUmKz/Tn8YxNtfXWQh/AcyZEL5lYZ/Sg=; b=zuCU6oZAS4fzJl rSiKsy9HvQ0VimRA7rvS0gsIaVmClyGdOlwekQA45hPL65kB/ngBadATajiw3aIZ9CZ2x136LUD0G hRbFlJ2S9tdsRU7zVVAVhWTyDljYxmfAMo0MsqbpmlU9cvI/uhgC5ftcatrQZ/d3MqvCJm/sI1XuC 4KwYj+zw+5XQ3NmTIFp5mPKgVH69YxLot8IEcaoiv3xCi6dROHxx6wsp04T7lnYijqlFHP7O0JgDV qjm1PlaRNGWQrQC84sYzj4nUcvxNkgT3JSUEoO0rTZco2JuE3sWL40oNCIQcwER4gHkayBfgfPlSH zjl5udUjgzrtE9qwN7UQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPz-00000001921-1FRn; Tue, 25 Jun 2024 00:52:19 +0000 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sLuPw-000000018zI-1LdL for linux-riscv@lists.infradead.org; Tue, 25 Jun 2024 00:52:17 +0000 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-70699b6afddso57016b3a.1 for ; Mon, 24 Jun 2024 17:52:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1719276735; x=1719881535; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qPS8Htynt5YVDBSwFV34iezZtSuGdE5NHFiA6AMckwQ=; b=ODoqQi90n4lMHjPP3ZH8Lcedhq0jGTEWxtRKoVINq9jfyY/dfIMhbgEL76mMI185iU lDwMVdazkW/U9aSemsidHZsSmryUbrSSNiPQzkADHEizJ5pVF1ci1LXBQN7jguP6XOcY gj3+C3Ad3IgHcR4MLUoPgbHw3SPXjeFJbyCagQ6S/QsXtc8Os6JwIyd2nwnorq7nan96 LUgb/dBuHmx2hgmCuZefrqNqYMEVMHFMgedTlGVoZ2g33rmtrc//7Z+YN+Tds7Q9hBe7 5prU61bf8W/80BSn+z3II6A+ukbhgcoKkGBYKsWGic6fljWsk+w/dekGWUWp+7jkx5nl u5Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719276735; x=1719881535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qPS8Htynt5YVDBSwFV34iezZtSuGdE5NHFiA6AMckwQ=; b=FTYWt6Np4F82AipdrlW3lYHDh0EkUICZ/jK/euZc0HhRWEodE2T5TJ8o8sMzFveTEG oRZa7n4OS1YXul+ZpXGFdO02nBDI/AMljHoKVJ+9Eu1VuTxS0Ag/avW4/u3+pVeOz2qt 9NhxO3citCX1t/WJHSaBlLJJtgiNi7jIfccqIOM3A9fC74mrS5QqdKLPwX92z+lOVAFn cgrPpBpiepWzJp0u59R9+i1AzjujCDbrqFuMXsw+6T13aFa4qMmPSLEbRknKKXSqt61S SGifCGQtuDxRDEIQfQjZwohQQdUDaNNT18erb44deJQyglSgwmb8VSVv0TK7dP493bOJ LVRg== X-Gm-Message-State: AOJu0YwMFm3lFR6QRYfJb8E5gHODHVWDXeHZlsY8NqkRTeD1/9O0TxvR AIF6t4fNBEbfc7Non1034WNUV4aKY+e+A0RU9chyI3DM7j9Dje8Av2lwZaCM9dxGmxYwUgeyyPe D X-Google-Smtp-Source: AGHT+IFXUAdJamg8ygnbLMv1f2WUsJYF7iboeHyAr7Z6Y+bevFXHoxCdZstz+th6Yy52wYFD6MOksg== X-Received: by 2002:a05:6a20:da96:b0:1bc:fd7f:4545 with SMTP id adf61e73a8af0-1bcfd7f49femr6118715637.8.1719276735396; Mon, 24 Jun 2024 17:52:15 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f9ebbb2a7csm68150235ad.256.2024.06.24.17.52.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jun 2024 17:52:15 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v3 8/8] RISC-V: hwprobe: Document unaligned vector perf key Date: Mon, 24 Jun 2024 20:50:01 -0400 Message-ID: <20240625005001.37901-9-jesse@rivosinc.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240625005001.37901-1-jesse@rivosinc.com> References: <20240625005001.37901-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240624_175216_385123_84F237C2 X-CRM114-Status: GOOD ( 11.66 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Document key for reporting the speed of unaligned vector accesses. The descriptions are the same as the scalar equivalent values. Signed-off-by: Jesse Taube --- V1 -> V2: - New patch V2 -> V3: - Specify access width --- Documentation/arch/riscv/hwprobe.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index 7085a694b801..d102b4a16d55 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -236,3 +236,19 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which represents the size of the Zicboz block in bytes. + +* :c:macro:`RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF`: An enum value describing the + performance of misaligned vector accesses on the selected set of processors. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN`: The performance of misaligned + accesses is unknown. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_SLOW`: 32bit misaligned accesses are slower + than equivalent byte accesses. Misaligned accesses may be supported + directly in hardware, or trapped and emulated by software. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_FAST`: 32bit misaligned accesses are faster + than equivalent byte accesses. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED`: Misaligned accesses are + not supported at all and will generate a misaligned address fault.