From patchwork Thu Nov 10 16:49:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB96FC4332F for ; Thu, 10 Nov 2022 16:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HeOM5zad9UMw70rwG0gilzCghMMpFolVOgOvFCB8LSw=; b=LmES6jLW41m4W1 EKtNSorFVYx4AD5aibPXI7RJ74W5I5ZLJKG9VwVPbCB9xlgOu3hc0wijK2YRfVpP8hSymmS8idgWo lIpMrLcdbl39T4iP/D5VBvTTuaLkhwMlWfUrU/2jytTHiFJWmbx7OyIRuukdQ5LKSVNJY280FmIIm 0hCElia/c+5iyiD5Sa4lE/rqshUpmn/OG3Q00idsIycV4CU1QPaI3lA+cKlpjhxVkazOtPItHFuNF ULfxJwCgycu8W1pII/Dfl8RDUYZbjNrZk1wfIOvbi8mAf5p5lU/xXWl95kwJA/kcLC/kSiVTg6cWX +ZSYBMvmsq+WH1ePssoQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAl1-007KWm-MA; Thu, 10 Nov 2022 16:50:27 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAkq-007KN1-Ef for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:18 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkk-0001xU-1c; Thu, 10 Nov 2022 17:50:10 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 1/7] efi/riscv: libstub: mark when compiling libstub Date: Thu, 10 Nov 2022 17:49:18 +0100 Message-Id: <20221110164924.529386-2-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_085016_516492_CD13BCBA X-CRM114-Status: GOOD ( 12.06 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner We want to runtime-optimize some core functions (str*, mem*) but not have this leak into libstub. Instead libstub for the short while it's running should just use the generic implementation. To be able to determine whether functions are getting compiled as part of libstub or not, add a compile-flag we can check via #ifdef. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- drivers/firmware/efi/libstub/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index b1601aad7e1a..39c8e3da1937 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ -fno-builtin -fpic \ $(call cc-option,-mno-single-pic-base) cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ - -fpic + -fpic -DRISCV_EFISTUB cflags-$(CONFIG_LOONGARCH) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ -fpie From patchwork Thu Nov 10 16:49:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E7FAC4332F for ; Thu, 10 Nov 2022 16:50:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RMiVvRwu+cYHHdL4agvwyheL4noiLLT8QgzhEN5CcTI=; b=OoUHgX6EfpswTX y4BhY29Q2LUKToONcy6KZgbOEIXGWD97goEiAEozcbiQBHUzNUxE4DJcq+/AhVS4wEA2Uq4oD9Swa pjODK+FaRTE/DmujxzuvNo4NdY2OkEd1jB34dd37W95MZ0VL+jKyvVVIyW4dKZ9XOvNdAOxckMiEH UQwlw7S5Cu3hz/C1ec8zGk51XtqhLPyc8qUeNli9DK9vN9Il5+bs2U9B0du8Vb/uOsmE20b260Umd 9zC2f6Zs4zQegni3uZ7aJTv2W7VBwiyfrujO1KCVMdxB81ze7+8TfTLyktVDmqYWEE8RtlvXTav3L VBm/KdpKNr5Ot5pshcDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAku-007KQY-4Y; Thu, 10 Nov 2022 16:50:20 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAko-007KHl-6u for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:15 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkk-0001xU-EO; Thu, 10 Nov 2022 17:50:10 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 2/7] RISC-V: add auipc elements to parse_asm header Date: Thu, 10 Nov 2022 17:49:19 +0100 Message-Id: <20221110164924.529386-3-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_085014_276505_EE48A81F X-CRM114-Status: UNSURE ( 9.56 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner We will want to use the opcode parsing outside kdb as well and need at least the auipc element there. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/include/asm/parse_asm.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/riscv/include/asm/parse_asm.h b/arch/riscv/include/asm/parse_asm.h index f36368de839f..c287c1426aa7 100644 --- a/arch/riscv/include/asm/parse_asm.h +++ b/arch/riscv/include/asm/parse_asm.h @@ -100,6 +100,7 @@ #define RVC_C2_RD_OPOFF 7 /* parts of opcode for RVG*/ +#define OPCODE_AUIPC 0x17 #define OPCODE_BRANCH 0x63 #define OPCODE_JALR 0x67 #define OPCODE_JAL 0x6f @@ -129,6 +130,7 @@ #define FUNCT12_SRET 0x10200000 +#define MATCH_AUIPC (OPCODE_AUIPC) #define MATCH_JALR (FUNCT3_JALR | OPCODE_JALR) #define MATCH_JAL (OPCODE_JAL) #define MATCH_BEQ (FUNCT3_BEQ | OPCODE_BRANCH) @@ -145,6 +147,7 @@ #define MATCH_C_JR (FUNCT4_C_JR | OPCODE_C_2) #define MATCH_C_JALR (FUNCT4_C_JALR | OPCODE_C_2) +#define MASK_AUIPC 0x7f #define MASK_JALR 0x707f #define MASK_JAL 0x7f #define MASK_C_JALR 0xf07f From patchwork Thu Nov 10 16:49:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74855C4332F for ; Thu, 10 Nov 2022 16:50:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1Euu4DQ6+h6tSBM02B87cj3WuDWl6BStp1qjoTDZLhY=; b=bY+NHoNGc65Xxq rGc/t/x4TIE4xFK3DoWUCY7GoL4fUKLL8vSA5WMJRLv0s0fa5OT1zQEVznyZ6f8XARF7jd74AfyeZ 5bhUsaaULMCvXairTdW0V/NgkJbhh6a/RTByDLykewPPmOKqNJxHQ/y/HQeXDTlyDVW2oVspZQjhE 2uhUk5kJpdN748to1GfMiLhgpQC4GPyy5FXeqM0epRdUhKe1zoif4lLABVjFHtu93SHqy5U+RsgmL St1VabL/vuTAAWJhCjF2Ci9klE2ipdEvZzM/Pv3J3U7AfcC/KKAP0uxJEPVFQ34EbtOAbi+kDSdUd FvdT8d0GhCYuEwhvUxcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAkr-007KNw-79; Thu, 10 Nov 2022 16:50:17 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAko-007KIE-6k for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:15 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkk-0001xU-QO; Thu, 10 Nov 2022 17:50:10 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 3/7] RISC-V: add U-type imm parsing to parse_asm header Date: Thu, 10 Nov 2022 17:49:20 +0100 Message-Id: <20221110164924.529386-4-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_085014_276505_EC5EE3FF X-CRM114-Status: GOOD ( 12.09 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner Similar to other existing types, allow extracting the immediate for a U-type instruction. U-type immediates are special in that regard, that the value in the instruction in bits [31:12] already represents the same bits of the immediate, so no shifting is required. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/include/asm/parse_asm.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/riscv/include/asm/parse_asm.h b/arch/riscv/include/asm/parse_asm.h index c287c1426aa7..939ede0ee527 100644 --- a/arch/riscv/include/asm/parse_asm.h +++ b/arch/riscv/include/asm/parse_asm.h @@ -25,6 +25,15 @@ #define J_IMM_11_MASK GENMASK(0, 0) #define J_IMM_19_12_MASK GENMASK(7, 0) +/* + * U-type IMMs contain the upper 20bits [31:20] of an immediate with + * the rest filled in by zeros, so no shifting required. Similarly, + * bit31 contains the signed state, so no sign extension necessary. + */ +#define U_IMM_SIGN_OPOFF 31 +#define U_IMM_31_12_OPOFF 0 +#define U_IMM_31_12_MASK GENMASK(31, 12) + /* The bit field of immediate value in B-type instruction */ #define B_IMM_SIGN_OPOFF 31 #define B_IMM_10_5_OPOFF 25 @@ -183,6 +192,10 @@ static inline bool is_ ## INSN_NAME ## _insn(long insn) \ #define RV_X(X, s, mask) (((X) >> (s)) & (mask)) #define RVC_X(X, s, mask) RV_X(X, s, mask) +#define EXTRACT_UTYPE_IMM(x) \ + ({typeof(x) x_ = (x); \ + (RV_X(x_, U_IMM_31_12_OPOFF, U_IMM_31_12_MASK)); }) + #define EXTRACT_JTYPE_IMM(x) \ ({typeof(x) x_ = (x); \ (RV_X(x_, J_IMM_10_1_OPOFF, J_IMM_10_1_MASK) << J_IMM_10_1_OFF) | \ From patchwork Thu Nov 10 16:49:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10580C43219 for ; Thu, 10 Nov 2022 16:50:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2L5d7/toiIhOQwTF9NkJYLZq7qsUeqeg37BznFYRhuM=; b=z+jhr89mXtP3os pNuJ3Jru6r1Ef0hGXilvybPp5wjx8+NOMNGrIt6k/IuOs1NFfJjzlsZWCA+wa4M96W7VdvT93JSHp zV3m/0ztS4GmW4c4+mtKHZfUVRPbPeIUJS7W8CTZpypviLS/pDDI9spNrKPJUgVHwWoS435olIbX/ 4A/m4I1D/agcR2dYVR9d5TdQyHnyQJcEcN+vXh9/uFkNan+0cbhhBSJxNGJy/3umYGt0V4d0oEUax eOUxum/a2IoBoROA7AfJNDUmoKPpuz9Jq/0X7SHwHUseY0ITgdSdOTNGYFcmcZNcT63TIy3Ldcwc9 jc0tCkKIZ0SL/YZf+VZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAkw-007KRa-9T; Thu, 10 Nov 2022 16:50:22 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAko-007KIj-74 for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:16 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkl-0001xU-65; Thu, 10 Nov 2022 17:50:11 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 4/7] RISC-V: add rd reg parsing to parse_asm header Date: Thu, 10 Nov 2022 17:49:21 +0100 Message-Id: <20221110164924.529386-5-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_085014_278741_A636C0B2 X-CRM114-Status: UNSURE ( 9.84 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner Add a macro to allow parsing of the rd register from an instruction. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/include/asm/parse_asm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/riscv/include/asm/parse_asm.h b/arch/riscv/include/asm/parse_asm.h index 939ede0ee527..305b15f7d41c 100644 --- a/arch/riscv/include/asm/parse_asm.h +++ b/arch/riscv/include/asm/parse_asm.h @@ -51,6 +51,7 @@ #define RVG_RS1_OPOFF 15 #define RVG_RS2_OPOFF 20 #define RVG_RD_OPOFF 7 +#define RVG_RD_MASK GENMASK(4, 0) /* The bit field of immediate value in RVC J instruction */ #define RVC_J_IMM_SIGN_OPOFF 12 @@ -192,6 +193,10 @@ static inline bool is_ ## INSN_NAME ## _insn(long insn) \ #define RV_X(X, s, mask) (((X) >> (s)) & (mask)) #define RVC_X(X, s, mask) RV_X(X, s, mask) +#define EXTRACT_RD_REG(x) \ + ({typeof(x) x_ = (x); \ + (RV_X(x_, RVG_RD_OPOFF, RVG_RD_MASK)); }) + #define EXTRACT_UTYPE_IMM(x) \ ({typeof(x) x_ = (x); \ (RV_X(x_, U_IMM_31_12_OPOFF, U_IMM_31_12_MASK)); }) From patchwork Thu Nov 10 16:49:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8186C4332F for ; Thu, 10 Nov 2022 16:50:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sLDK1Zx6hXNHtfpuuAgYZp8FbDmuJcwZxgfe1I6GTQU=; b=qC2f84NjoTWEVa 6fVTgsPf6RQbwh2UNaHXuMlnCk3LJA22vbmopxuHSQ7+rjsiCQ/yLmX05OhzJvRtYCDUmmOlNSO8S 6kJq4nObn0OTp+vpHWDEGwhYuMVkiQbE1vGU5M0UpkbMpEEBdrBYBzGBdcimMG54kajtZojmocaeq /y4SumUjQBIslg2TJRETEjipKhublye/iEyHNrYSv4DUkz/txhi3df0AyUu4pkxgtmMJ2pa1Vhmkq 1IWO3J6Xca7VPTz4gWmen7TVA9GDBNeMmsxah6J8ld5+exSWXsw0pojW5x9kBlSeE659ys433wrEE imqVpONAxRsko6pvaNNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAky-007KTA-1d; Thu, 10 Nov 2022 16:50:24 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAko-007KIw-9p for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:17 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkl-0001xU-Hk; Thu, 10 Nov 2022 17:50:11 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 5/7] RISC-V: fix auipc-jalr addresses in patched alternatives Date: Thu, 10 Nov 2022 17:49:22 +0100 Message-Id: <20221110164924.529386-6-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_085014_374632_8BE721CE X-CRM114-Status: GOOD ( 17.08 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner Alternatives live in a different section, so addresses used by call functions will point to wrong locations after the patch got applied. Similar to arm64, adjust the location to consider that offset. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/kernel/cpufeature.c | 79 +++++++++++++++++++++++++++++++++- 1 file changed, 77 insertions(+), 2 deletions(-) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 694267d1fe81..026512ca9c4c 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -298,6 +298,74 @@ static u32 __init_or_module cpufeature_probe(unsigned int stage) return cpu_req_feature; } +#include + +DECLARE_INSN(jalr, MATCH_JALR, MASK_JALR) +DECLARE_INSN(auipc, MATCH_AUIPC, MASK_AUIPC) + +static inline bool is_auipc_jalr_pair(long insn1, long insn2) +{ + return is_auipc_insn(insn1) && is_jalr_insn(insn2); +} + +#define JALR_SIGN_MASK BIT(I_IMM_SIGN_OPOFF - I_IMM_11_0_OPOFF) +#define JALR_OFFSET_MASK I_IMM_11_0_MASK +#define AUIPC_OFFSET_MASK U_IMM_31_12_MASK +#define AUIPC_PAD (0x00001000) +#define JALR_SHIFT I_IMM_11_0_OPOFF + +#define to_jalr_imm(offset) \ + ((offset & I_IMM_11_0_MASK) << I_IMM_11_0_OPOFF) + +#define to_auipc_imm(offset) \ + ((offset & JALR_SIGN_MASK) ? \ + ((offset & AUIPC_OFFSET_MASK) + AUIPC_PAD) : \ + (offset & AUIPC_OFFSET_MASK)) + +static void riscv_alternative_fix_auipc_jalr(unsigned int *alt_ptr, + unsigned int len, int patch_offset) +{ + int num_instr = len / sizeof(u32); + unsigned int call[2]; + int i; + int imm1; + u32 rd1; + + for (i = 0; i < num_instr; i++) { + /* is there a further instruction? */ + if (i + 1 >= num_instr) + continue; + + if (!is_auipc_jalr_pair(*(alt_ptr + i), *(alt_ptr + i + 1))) + continue; + + /* call will use ra register */ + rd1 = EXTRACT_RD_REG(*(alt_ptr + i)); + if (rd1 != 1) + continue; + + /* get and adjust new target address */ + imm1 = EXTRACT_UTYPE_IMM(*(alt_ptr + i)); + imm1 += EXTRACT_ITYPE_IMM(*(alt_ptr + i + 1)); + imm1 -= patch_offset; + + /* pick the original auipc + jalr */ + call[0] = *(alt_ptr + i); + call[1] = *(alt_ptr + i + 1); + + /* drop the old IMMs */ + call[0] &= ~(U_IMM_31_12_MASK); + call[1] &= ~(I_IMM_11_0_MASK << I_IMM_11_0_OPOFF); + + /* add the adapted IMMs */ + call[0] |= to_auipc_imm(imm1); + call[1] |= to_jalr_imm(imm1); + + /* patch the call place again */ + patch_text_nosync(alt_ptr + i * sizeof(u32), call, 8); + } +} + void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, struct alt_entry *end, unsigned int stage) @@ -316,8 +384,15 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, } tmp = (1U << alt->errata_id); - if (cpu_req_feature & tmp) - patch_text_nosync(alt->old_ptr, alt->alt_ptr, alt->alt_len); + if (cpu_req_feature & tmp) { + /* do the basic patching */ + patch_text_nosync(alt->old_ptr, alt->alt_ptr, + alt->alt_len); + + riscv_alternative_fix_auipc_jalr(alt->old_ptr, + alt->alt_len, + alt->old_ptr - alt->alt_ptr); + } } } #endif From patchwork Thu Nov 10 16:49:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1F25C4332F for ; Thu, 10 Nov 2022 16:50:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sncO0mNm3Y6QVLwT7C7HIJAmqAcVwpiE9122H/ZSDDs=; b=x5UYPnQEVjSNV+ jgSDdtyttG1yj8OhHF+IqnCW3bFKn/fPV25YySaD5gFKnTMiVQoqfuhcDDlrNWzbEcUxnvjB2O3Ly BTEQ09EtNrStwU4X8DUC6E5l4xrc+CANERZwjRLRG3NPEATDZxE6UG/KMGF3B4v29WeY0IjrdKRrt 2BwnNe6mBYmMzuvvLA1hog7MqrlhVm5o7IwuAtEVlLquWkTjeWjtBOEbe165hbm3i0wQw8UwuKN+h vUBDKjKlTpG4qfByDJzu3y5WMSagYx1tGjEp5u/zx1EyIiF8wlvhQfAZTyt2VglukVYsnBtzTZ63j uPkWpGR4nasK8tREJK7w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAl4-007KYs-2n; Thu, 10 Nov 2022 16:50:30 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAky-007KSU-AR for linux-riscv@bombadil.infradead.org; Thu, 10 Nov 2022 16:50:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=dfKRhJUW55Ud3142LZA34V/1RuCQ39MiyNRuROQ/wWk=; b=IPVSq9GT49FIT4PudY7H0TLTL8 /05Z3A/zVLZN8wa03SDx5V3LUBC0W/jJPj4dWNa8GEfyUgPdcofFmuz642VsiLGvt0YeBWqTrDwFg X+4Rt79IDCqQ5SRDBcRMJ+G1Ub1WpwUV+BCE5fR76aZe0/kFCDozutwnHwEOWyNoww5sdaEKfgjlX LKk9jbpC8+5UW7nsYammsKffRCihghlfGPRVJUn04kC7NeWlPu7ti5y8GEffTkGBDmeJR98IDzUKP j1mwmgafivZC29w62IrtrH2MOS3skaQ3WAUQFYfCTrWK954AsFnagbm+f+6eoQ5qO1QopJ14ToStA oMxtm5vQ==; Received: from gloria.sntech.de ([185.11.138.130]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAku-00AKaR-7Q for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:22 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkl-0001xU-Us; Thu, 10 Nov 2022 17:50:12 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 6/7] RISC-V: add infrastructure to allow different str* implementations Date: Thu, 10 Nov 2022 17:49:23 +0100 Message-Id: <20221110164924.529386-7-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_165020_412888_2601A04A X-CRM114-Status: GOOD ( 18.57 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner Depending on supported extensions on specific RISC-V cores, optimized str* functions might make sense. This adds basic infrastructure to allow patching the function calls via alternatives later on. The main idea is to have the core str* functions be inline functions which then call the most optimized variant and this call then be replaced via alternatives. The big advantage is that we don't need additional calls. Though we need to duplicate the generic functions as the main code expects either itself or the architecture to provide the str* functions. The added *_generic functions are done in assembler (taken from disassembling the main-kernel functions for now) to allow us to control the used registers. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/include/asm/string.h | 66 +++++++++++++++++++++++++++++++++ arch/riscv/kernel/image-vars.h | 6 +-- arch/riscv/lib/Makefile | 3 ++ arch/riscv/lib/strcmp.S | 39 +++++++++++++++++++ arch/riscv/lib/strlen.S | 29 +++++++++++++++ arch/riscv/lib/strncmp.S | 41 ++++++++++++++++++++ 6 files changed, 181 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/lib/strcmp.S create mode 100644 arch/riscv/lib/strlen.S create mode 100644 arch/riscv/lib/strncmp.S diff --git a/arch/riscv/include/asm/string.h b/arch/riscv/include/asm/string.h index 909049366555..b98481d2d154 100644 --- a/arch/riscv/include/asm/string.h +++ b/arch/riscv/include/asm/string.h @@ -18,6 +18,72 @@ extern asmlinkage void *__memcpy(void *, const void *, size_t); #define __HAVE_ARCH_MEMMOVE extern asmlinkage void *memmove(void *, const void *, size_t); extern asmlinkage void *__memmove(void *, const void *, size_t); + +#define __HAVE_ARCH_STRCMP +extern asmlinkage int __strcmp_generic(const char *cs, const char *ct); + +static inline int strcmp(const char *cs, const char *ct) +{ +#ifdef RISCV_EFISTUB + return __strcmp_generic(cs, ct); +#else + register const char *a0 asm("a0") = cs; + register const char *a1 asm("a1") = ct; + register int a0_out asm("a0"); + + asm volatile("call __strcmp_generic\n\t" + : "=r"(a0_out) + : "r"(a0), "r"(a1) + : "ra", "t0", "t1", "t2"); + + return a0_out; +#endif +} + +#define __HAVE_ARCH_STRNCMP +extern asmlinkage int __strncmp_generic(const char *cs, + const char *ct, size_t count); + +static inline int strncmp(const char *cs, const char *ct, size_t count) +{ +#ifdef RISCV_EFISTUB + return __strncmp_generic(cs, ct, count); +#else + register const char *a0 asm("a0") = cs; + register const char *a1 asm("a1") = ct; + register size_t a2 asm("a2") = count; + register int a0_out asm("a0"); + + asm volatile("call __strncmp_generic\n\t" + : "=r"(a0_out) + : "r"(a0), "r"(a1), "r"(a2) + : "ra", "t0", "t1", "t2"); + + return a0_out; +#endif +} + +#define __HAVE_ARCH_STRLEN +extern asmlinkage __kernel_size_t __strlen_generic(const char *); + +static inline __kernel_size_t strlen(const char *s) +{ +#ifdef RISCV_EFISTUB + return __strlen_generic(s); +#else + register const char *a0 asm("a0") = s; + register int a0_out asm("a0"); + + asm volatile( + "call __strlen_generic\n\t" + : "=r"(a0_out) + : "r"(a0) + : "ra", "t0", "t1"); + + return a0_out; +#endif +} + /* For those files which don't want to check by kasan. */ #if defined(CONFIG_KASAN) && !defined(__SANITIZE_ADDRESS__) #define memcpy(dst, src, len) __memcpy(dst, src, len) diff --git a/arch/riscv/kernel/image-vars.h b/arch/riscv/kernel/image-vars.h index d6e5f739905e..2f270b9fde63 100644 --- a/arch/riscv/kernel/image-vars.h +++ b/arch/riscv/kernel/image-vars.h @@ -25,10 +25,10 @@ */ __efistub_memcmp = memcmp; __efistub_memchr = memchr; -__efistub_strlen = strlen; +__efistub___strlen_generic = __strlen_generic; __efistub_strnlen = strnlen; -__efistub_strcmp = strcmp; -__efistub_strncmp = strncmp; +__efistub___strcmp_generic = __strcmp_generic; +__efistub___strncmp_generic = __strncmp_generic; __efistub_strrchr = strrchr; __efistub__start = _start; diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile index 25d5c9664e57..6c74b0bedd60 100644 --- a/arch/riscv/lib/Makefile +++ b/arch/riscv/lib/Makefile @@ -3,6 +3,9 @@ lib-y += delay.o lib-y += memcpy.o lib-y += memset.o lib-y += memmove.o +lib-y += strcmp.o +lib-y += strlen.o +lib-y += strncmp.o lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o diff --git a/arch/riscv/lib/strcmp.S b/arch/riscv/lib/strcmp.S new file mode 100644 index 000000000000..f23a5c5e39d8 --- /dev/null +++ b/arch/riscv/lib/strcmp.S @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* int __strcmp_generic(const char *cs, const char *ct) */ +ENTRY(__strcmp_generic) + /* + * Returns + * a0 - comparison result, like strncmp + * + * Parameters + * a0 - string1 + * a1 - string2 + * a2 - number of characters to compare + * + * Clobbers + * t0, t1, t2, t3, t4, t5 + */ + mv t2, a1 +1: + lbu t1, 0(a0) + lbu t0, 0(a1) + addi a0, a0, 1 + addi a1, a1, 1 + beq t1, t0, 3f + li a0, 1 + bgeu t1, t0, 2f + li a0, -1 +2: + mv a1, t2 + ret +3: + bnez t1, 1b + li a0, 0 + j 2b +END(__strcmp_generic) +EXPORT_SYMBOL(__strcmp_generic) diff --git a/arch/riscv/lib/strlen.S b/arch/riscv/lib/strlen.S new file mode 100644 index 000000000000..e0e7440ac724 --- /dev/null +++ b/arch/riscv/lib/strlen.S @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* int __strlen_generic(const char *s) */ +ENTRY(__strlen_generic) + /* + * Returns + * a0 - string length + * + * Parameters + * a0 - String to measure + * + * Clobbers: + * t0, t1 + */ + mv t1, a0 +1: + lbu t0, 0(t1) + bnez t0, 2f + sub a0, t1, a0 + ret +2: + addi t1, t1, 1 + j 1b +END(__strlen_generic) +EXPORT_SYMBOL(__strlen_generic) diff --git a/arch/riscv/lib/strncmp.S b/arch/riscv/lib/strncmp.S new file mode 100644 index 000000000000..8d271cd0df72 --- /dev/null +++ b/arch/riscv/lib/strncmp.S @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include +#include +#include + +/* int __strncmp_generic(const char *cs, const char *ct, size_t count) */ +ENTRY(__strncmp_generic) + /* + * Returns + * a0 - comparison result, like strncmp + * + * Parameters + * a0 - string1 + * a1 - string2 + * a2 - number of characters to compare + * + * Clobbers + * t0, t1, t2 + */ + li t0, 0 +1: + beq a2, t0, 4f + add t1, a0, t0 + add t2, a1, t0 + lbu t1, 0(t1) + lbu t2, 0(t2) + beq t1, t2, 3f + li a0, 1 + bgeu t1, t2, 2f + li a0, -1 +2: + ret +3: + addi t0, t0, 1 + bnez t1, 1b +4: + li a0, 0 + j 2b +END(__strncmp_generic) +EXPORT_SYMBOL(__strncmp_generic) From patchwork Thu Nov 10 16:49:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 13039019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D68ABC4321E for ; Thu, 10 Nov 2022 16:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eJyLDFh0+BhQTqKHurdFX1wYZ3F/+oZ56f5L3WaBRgg=; b=Tiut8rg+r6SbXc baDie7BDKPsC5DB+aY9W6KZuOHd2mUIRqS1ZprplDyA9kFLfbzpKqEnAs/l2VmoBhp0aI4X6X7oRx dEEB+F+WO+BUvhGSfzHwRJfJTHb5NhreSi3alMPcSz8yAKtRHZj2TBC6+FYiDKr2h449W4Ahq5crP oPime6ax/M5XsQgClIoeeDTsSpKKigmZSVLqbBt8v/+0zL3TeSBIdlDzMtBuh5KGx0w9v2G8evfc0 qx7rsF/Qxd7EAsEN4HKCEUHNiOKtLX1N5HsZEAZmxUckQY3/eovpIAEKZKyHycvha/UBrFqUGaq0B 56Q5gEsO1XBhcyxB6zHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAl6-007KcR-QJ; Thu, 10 Nov 2022 16:50:32 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAky-007KSX-AQ for linux-riscv@bombadil.infradead.org; Thu, 10 Nov 2022 16:50:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=6c8Ld0lmCf3IOjJFeEjJfbwKZHEG5yrxpbxpI3+9Upg=; b=AN8FtC6x/ypFmBATFxVtd/hQ9w STV9PF7SL1b4KUwMnDQ08h2YxL1o1m8jtO4vyLOTs6r5Ox6Swfn8aI+z3t3sfDdB2Y3iVEkaPWkqY kGpzRZhN5dn20r7WCcCLONLeK7cgB6+MVatvMYX0Gk/3tM5Y9sNHkHpZxLCE6nfvUQCE+3EHoOyba fudGpEKJksKpWkVaysRPTv3j2JRRAQGBpjpTq63WVfhZl30ctxDBJjMnAmaWd1Mgc3m0OSVMrplx+ Mout5pmu863iTkqup9inbRrLDsO6RxhNKwn47EV3jM//n3OaqQHTo+855qn/mVUFCdd0OVDYB4EBy ND0CfFQw==; Received: from gloria.sntech.de ([185.11.138.130]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1otAku-00AKaS-9N for linux-riscv@lists.infradead.org; Thu, 10 Nov 2022 16:50:22 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1otAkm-0001xU-AX; Thu, 10 Nov 2022 17:50:12 +0100 From: Heiko Stuebner To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: christoph.muellner@vrull.eu, prabhakar.csengg@gmail.com, conor@kernel.org, philipp.tomsich@vrull.eu, ajones@ventanamicro.com, heiko@sntech.de, emil.renner.berthing@canonical.com, Heiko Stuebner Subject: [PATCH 7/7] RISC-V: add zbb support to string functions Date: Thu, 10 Nov 2022 17:49:24 +0100 Message-Id: <20221110164924.529386-8-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221110164924.529386-1-heiko@sntech.de> References: <20221110164924.529386-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221110_165020_659082_F6C7EE86 X-CRM114-Status: GOOD ( 27.48 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Heiko Stuebner Add handling for ZBB extension and add support for using it as a variant for optimized string functions. Signed-off-by: Heiko Stuebner Reviewed-by: Conor Dooley --- arch/riscv/Kconfig | 23 ++++++ arch/riscv/include/asm/errata_list.h | 3 +- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/string.h | 29 ++++++-- arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 18 +++++ arch/riscv/lib/Makefile | 3 + arch/riscv/lib/strcmp_zbb.S | 91 +++++++++++++++++++++++ arch/riscv/lib/strlen_zbb.S | 98 +++++++++++++++++++++++++ arch/riscv/lib/strncmp_zbb.S | 106 +++++++++++++++++++++++++++ 10 files changed, 366 insertions(+), 7 deletions(-) create mode 100644 arch/riscv/lib/strcmp_zbb.S create mode 100644 arch/riscv/lib/strlen_zbb.S create mode 100644 arch/riscv/lib/strncmp_zbb.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index acfc4d298aab..56633931e808 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -411,6 +411,29 @@ config RISCV_ISA_SVPBMT If you don't know what to do here, say Y. +config TOOLCHAIN_HAS_ZBB + bool + default y + depends on !64BIT || $(cc-option,-mabi=lp64 -march=rv64ima_zbb) + depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zbb) + depends on LLD_VERSION >= 150000 || LD_VERSION >= 23900 + +config RISCV_ISA_ZBB + bool "Zbb extension support for " + depends on TOOLCHAIN_HAS_ZBB + depends on !XIP_KERNEL && MMU + select RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZBB + extension (basic bit manipulation) and enable its usage. + + The Zbb extension provides instructions to accelerate a number + of bit-specific operations (count bit population, sign extending, + bitrotation, etc). + + If you don't know what to do here, say Y. + config TOOLCHAIN_HAS_ZICBOM bool default y diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 4180312d2a70..95e626b7281e 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -24,7 +24,8 @@ #define CPUFEATURE_SVPBMT 0 #define CPUFEATURE_ZICBOM 1 -#define CPUFEATURE_NUMBER 2 +#define CPUFEATURE_ZBB 2 +#define CPUFEATURE_NUMBER 3 #ifdef __ASSEMBLY__ diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index b22525290073..ac5555fd9788 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -59,6 +59,7 @@ enum riscv_isa_ext_id { RISCV_ISA_EXT_ZIHINTPAUSE, RISCV_ISA_EXT_SSTC, RISCV_ISA_EXT_SVINVAL, + RISCV_ISA_EXT_ZBB, RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX, }; diff --git a/arch/riscv/include/asm/string.h b/arch/riscv/include/asm/string.h index b98481d2d154..806c402c874e 100644 --- a/arch/riscv/include/asm/string.h +++ b/arch/riscv/include/asm/string.h @@ -6,6 +6,8 @@ #ifndef _ASM_RISCV_STRING_H #define _ASM_RISCV_STRING_H +#include +#include #include #include @@ -21,6 +23,7 @@ extern asmlinkage void *__memmove(void *, const void *, size_t); #define __HAVE_ARCH_STRCMP extern asmlinkage int __strcmp_generic(const char *cs, const char *ct); +extern asmlinkage int __strcmp_zbb(const char *cs, const char *ct); static inline int strcmp(const char *cs, const char *ct) { @@ -31,10 +34,14 @@ static inline int strcmp(const char *cs, const char *ct) register const char *a1 asm("a1") = ct; register int a0_out asm("a0"); - asm volatile("call __strcmp_generic\n\t" + asm volatile( + ALTERNATIVE( + "call __strcmp_generic\n\t", + "call __strcmp_zbb\n\t", + 0, CPUFEATURE_ZBB, CONFIG_RISCV_ISA_ZBB) : "=r"(a0_out) : "r"(a0), "r"(a1) - : "ra", "t0", "t1", "t2"); + : "ra", "t0", "t1", "t2", "t3", "t4", "t5"); return a0_out; #endif @@ -43,6 +50,8 @@ static inline int strcmp(const char *cs, const char *ct) #define __HAVE_ARCH_STRNCMP extern asmlinkage int __strncmp_generic(const char *cs, const char *ct, size_t count); +extern asmlinkage int __strncmp_zbb(const char *cs, + const char *ct, size_t count); static inline int strncmp(const char *cs, const char *ct, size_t count) { @@ -54,10 +63,14 @@ static inline int strncmp(const char *cs, const char *ct, size_t count) register size_t a2 asm("a2") = count; register int a0_out asm("a0"); - asm volatile("call __strncmp_generic\n\t" + asm volatile( + ALTERNATIVE( + "call __strncmp_generic\n\t", + "call __strncmp_zbb\n\t", + 0, CPUFEATURE_ZBB, CONFIG_RISCV_ISA_ZBB) : "=r"(a0_out) : "r"(a0), "r"(a1), "r"(a2) - : "ra", "t0", "t1", "t2"); + : "ra", "t0", "t1", "t2", "t3", "t4", "t5", "t6"); return a0_out; #endif @@ -65,6 +78,7 @@ static inline int strncmp(const char *cs, const char *ct, size_t count) #define __HAVE_ARCH_STRLEN extern asmlinkage __kernel_size_t __strlen_generic(const char *); +extern asmlinkage __kernel_size_t __strlen_zbb(const char *); static inline __kernel_size_t strlen(const char *s) { @@ -75,10 +89,13 @@ static inline __kernel_size_t strlen(const char *s) register int a0_out asm("a0"); asm volatile( - "call __strlen_generic\n\t" + ALTERNATIVE( + "call __strlen_generic\n\t", + "call __strlen_zbb\n\t", + 0, CPUFEATURE_ZBB, CONFIG_RISCV_ISA_ZBB) : "=r"(a0_out) : "r"(a0) - : "ra", "t0", "t1"); + : "ra", "t0", "t1", "t2", "t3"); return a0_out; #endif diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index bf9dd6764bad..66ff36a57e20 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -166,6 +166,7 @@ static struct riscv_isa_ext_data isa_ext_arr[] = { __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL), __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), + __RISCV_ISA_EXT_DATA(zbb, RISCV_ISA_EXT_ZBB), __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA(zihintpause, RISCV_ISA_EXT_ZIHINTPAUSE), __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 026512ca9c4c..f19b9d4e2dca 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -201,6 +201,7 @@ void __init riscv_fill_hwcap(void) } else { SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); + SET_ISA_EXT_MAP("zbb", RISCV_ISA_EXT_ZBB); SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); SET_ISA_EXT_MAP("zihintpause", RISCV_ISA_EXT_ZIHINTPAUSE); SET_ISA_EXT_MAP("sstc", RISCV_ISA_EXT_SSTC); @@ -278,6 +279,20 @@ static bool __init_or_module cpufeature_probe_zicbom(unsigned int stage) return true; } +static bool __init_or_module cpufeature_probe_zbb(unsigned int stage) +{ + if (!IS_ENABLED(CONFIG_RISCV_ISA_ZBB)) + return false; + + if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) + return false; + + if (!riscv_isa_extension_available(NULL, ZBB)) + return false; + + return true; +} + /* * Probe presence of individual extensions. * @@ -295,6 +310,9 @@ static u32 __init_or_module cpufeature_probe(unsigned int stage) if (cpufeature_probe_zicbom(stage)) cpu_req_feature |= BIT(CPUFEATURE_ZICBOM); + if (cpufeature_probe_zbb(stage)) + cpu_req_feature |= BIT(CPUFEATURE_ZBB); + return cpu_req_feature; } diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile index 6c74b0bedd60..b632483f851c 100644 --- a/arch/riscv/lib/Makefile +++ b/arch/riscv/lib/Makefile @@ -4,8 +4,11 @@ lib-y += memcpy.o lib-y += memset.o lib-y += memmove.o lib-y += strcmp.o +lib-$(CONFIG_RISCV_ISA_ZBB) += strcmp_zbb.o lib-y += strlen.o +lib-$(CONFIG_RISCV_ISA_ZBB) += strlen_zbb.o lib-y += strncmp.o +lib-$(CONFIG_RISCV_ISA_ZBB) += strncmp_zbb.o lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o diff --git a/arch/riscv/lib/strcmp_zbb.S b/arch/riscv/lib/strcmp_zbb.S new file mode 100644 index 000000000000..aff9b941d3ee --- /dev/null +++ b/arch/riscv/lib/strcmp_zbb.S @@ -0,0 +1,91 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 VRULL GmbH + * Author: Christoph Muellner + */ + +#include +#include +#include + +#define src1 a0 +#define result a0 +#define src2 t5 +#define data1 t0 +#define data2 t1 +#define align t2 +#define data1_orcb t3 +#define m1 t4 + +.option push +.option arch,+zbb + +/* int __strcmp_zbb(const char *cs, const char *ct) */ +ENTRY(__strcmp_zbb) + /* + * Returns + * a0 - comparison result, like strncmp + * + * Parameters + * a0 - string1 + * a1 - string2 + * a2 - number of characters to compare + * + * Clobbers + * t0, t1, t2, t3, t4, t5 + */ + mv src2, a1 + + or align, src1, src2 + li m1, -1 + and align, align, SZREG-1 + bnez align, 3f + /* Main loop for aligned string. */ + .p2align 3 +1: + REG_L data1, 0(src1) + REG_L data2, 0(src2) + orc.b data1_orcb, data1 + bne data1_orcb, m1, 2f + addi src1, src1, SZREG + addi src2, src2, SZREG + beq data1, data2, 1b + + /* Words don't match, and no null byte in the first + * word. Get bytes in big-endian order and compare. */ +#ifndef CONFIG_CPU_BIG_ENDIAN + rev8 data1, data1 + rev8 data2, data2 +#endif + /* Synthesize (data1 >= data2) ? 1 : -1 in a branchless sequence. */ + sltu result, data1, data2 + neg result, result + ori result, result, 1 + ret + +2: + /* Found a null byte. + * If words don't match, fall back to simple loop. */ + bne data1, data2, 3f + + /* Otherwise, strings are equal. */ + li result, 0 + ret + + /* Simple loop for misaligned strings. */ + .p2align 3 +3: + lbu data1, 0(src1) + lbu data2, 0(src2) + addi src1, src1, 1 + addi src2, src2, 1 + bne data1, data2, 4f + bnez data1, 3b + +4: + sub result, data1, data2 + ret +END(__strcmp_zbb) +EXPORT_SYMBOL(__strcmp_zbb) + +.option pop diff --git a/arch/riscv/lib/strlen_zbb.S b/arch/riscv/lib/strlen_zbb.S new file mode 100644 index 000000000000..bc8d3607a32f --- /dev/null +++ b/arch/riscv/lib/strlen_zbb.S @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 VRULL GmbH + * Author: Christoph Muellner + */ + +#include +#include +#include + +#define src a0 +#define result a0 +#define addr t0 +#define data t1 +#define offset t2 +#define offset_bits t2 +#define valid_bytes t3 +#define m1 t3 + +#ifdef CONFIG_CPU_BIG_ENDIAN +# define CZ clz +# define SHIFT sll +#else +# define CZ ctz +# define SHIFT srl +#endif + +.option push +.option arch,+zbb + +/* int __strlen_zbb(const char *s) */ +ENTRY(__strlen_zbb) + /* + * Returns + * a0 - string length + * + * Parameters + * a0 - String to measure + * + * Clobbers + * t0, t1, t2, t3 + */ + + /* Number of irrelevant bytes in the first word. */ + andi offset, src, SZREG-1 + /* Align pointer. */ + andi addr, src, -SZREG + + li valid_bytes, SZREG + sub valid_bytes, valid_bytes, offset + slli offset_bits, offset, RISCV_LGPTR + + /* Get the first word. */ + REG_L data, 0(addr) + /* Shift away the partial data we loaded to remove the irrelevant bytes + * preceding the string with the effect of adding NUL bytes at the + * end of the string. */ + SHIFT data, data, offset_bits + /* Convert non-NUL into 0xff and NUL into 0x00. */ + orc.b data, data + /* Convert non-NUL into 0x00 and NUL into 0xff. */ + not data, data + /* Search for the first set bit (corresponding to a NUL byte in the + * original chunk). */ + CZ data, data + /* The first chunk is special: commpare against the number + * of valid bytes in this chunk. */ + srli result, data, 3 + bgtu valid_bytes, result, 3f + + /* Prepare for the word comparison loop. */ + addi offset, addr, SZREG + li m1, -1 + + /* Our critical loop is 4 instructions and processes data in + * 4 byte or 8 byte chunks. */ + .p2align 3 +1: + REG_L data, SZREG(addr) + addi addr, addr, SZREG + orc.b data, data + beq data, m1, 1b +2: + not data, data + CZ data, data + /* Get number of processed words. */ + sub offset, addr, offset + /* Add number of characters in the first word. */ + add result, result, offset + srli data, data, 3 + /* Add number of characters in the last word. */ + add result, result, data +3: + ret +END(__strlen_zbb) +EXPORT_SYMBOL(__strlen_zbb) + +.option pop diff --git a/arch/riscv/lib/strncmp_zbb.S b/arch/riscv/lib/strncmp_zbb.S new file mode 100644 index 000000000000..852c8425d238 --- /dev/null +++ b/arch/riscv/lib/strncmp_zbb.S @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 VRULL GmbH + * Author: Christoph Muellner + */ + +#include +#include +#include + +#define src1 a0 +#define result a0 +#define src2 t6 +#define len a2 +#define data1 t0 +#define data2 t1 +#define align t2 +#define data1_orcb t3 +#define limit t4 +#define m1 t5 + +.option push +.option arch,+zbb + +/* int __strncmp_zbb(const char *cs, const char *ct, size_t count) */ +ENTRY(__strncmp_zbb) + /* + * Returns + * a0 - comparison result, like strncmp + * + * Parameters + * a0 - string1 + * a1 - string2 + * a2 - number of characters to compare + * + * Clobbers + * t0, t1, t2, t3, t4, t5, t6 + */ + mv src2, a1 + + or align, src1, src2 + li m1, -1 + and align, align, SZREG-1 + add limit, src1, len + bnez align, 4f + + /* Adjust limit for fast-path. */ + addi limit, limit, -SZREG + /* Main loop for aligned string. */ + .p2align 3 +1: + bgt src1, limit, 3f + REG_L data1, 0(src1) + REG_L data2, 0(src2) + orc.b data1_orcb, data1 + bne data1_orcb, m1, 2f + addi src1, src1, SZREG + addi src2, src2, SZREG + beq data1, data2, 1b + + /* Words don't match, and no null byte in the first + * word. Get bytes in big-endian order and compare. */ +#ifndef CONFIG_CPU_BIG_ENDIAN + rev8 data1, data1 + rev8 data2, data2 +#endif + /* Synthesize (data1 >= data2) ? 1 : -1 in a branchless sequence. */ + sltu result, data1, data2 + neg result, result + ori result, result, 1 + ret + +2: + /* Found a null byte. + * If words don't match, fall back to simple loop. */ + bne data1, data2, 3f + + /* Otherwise, strings are equal. */ + li result, 0 + ret + + /* Simple loop for misaligned strings. */ +3: + /* Restore limit for slow-path. */ + addi limit, limit, SZREG + .p2align 3 +4: + bge src1, limit, 6f + lbu data1, 0(src1) + lbu data2, 0(src2) + addi src1, src1, 1 + addi src2, src2, 1 + bne data1, data2, 5f + bnez data1, 4b + +5: + sub result, data1, data2 + ret + +6: + li result, 0 + ret +END(__strncmp_zbb) +EXPORT_SYMBOL(__strncmp_zbb) + +.option pop