From patchwork Fri Apr 14 15:58:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13211773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34C66C77B6E for ; Fri, 14 Apr 2023 15:59:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4YvH7LDuw9eKzo0P1YdgUaffCrIwFjoW3VdP6q6mwjU=; b=HniJjOW3e2jnum jZnmHNe3J0YkYDQ6YyNxPliYcKHrAC3vUKdHqdlyUcUOoshpjNQzAkglayRfbh0cWnx7fo4FjBm8H M/TI94WBgrt0dZ04C5ubvju44AslvvmNDa2mMUMaCMfESq3OJbaZtDDmK+EuQ0olcEKlQf3Hfl0Gc OGlxV8w93B4QqbFMv+3603AK5xieq09Oz5Nv3wbM13NU9gcHG+NRMbHQ5B+JtSDvDQz5EeuNXXCKd CC4h+Gn0sl2A+tfoXOnlJw/inwkIpaLovs/1PDKs9KGIbphmW3kElEruwTh4Mrv4HR/5pGdKOXHGx Ypc9kK/phQOvNClMcCTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pnLpy-00A1n7-30; Fri, 14 Apr 2023 15:59:46 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pnLpw-00A1l6-1J for linux-riscv@lists.infradead.org; Fri, 14 Apr 2023 15:59:46 +0000 Received: by mail-pl1-x633.google.com with SMTP id kh6so17138038plb.0 for ; Fri, 14 Apr 2023 08:59:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1681487982; x=1684079982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ahp5kQnTghgkZmiSTxR6+eMYSF2yeYW/B9Sl3mLm86I=; b=jWHmU/ESlEHOVioBV8QithJ4pWOgoHH5LTG4cTPCqQwFXhWGAV+CAw9vIPolQzdFKg 1O8hU9YR2eD4QNJ4eslFexI6xAN7K+8XELN9YWB/Q5ikzgY4TKcWePvNli9yKYw2Qf// UGjw4S3bQB0GRC/9puQ/wr8kgEXdBfo5dqG0yEU3DuPl4SDI5vKnvhdG5aWoD97UQuZe qI1AGxRQmg0qa/pJDjOuU/WuRmfA+ua04g027fEB2zwwZiuTlvIzlPC/I4xVtvft3Akc JRcNGAbJ9my8qAf2FFNaXFNEhXyL2mS31Upn6s6ICL1VPqmeM9RzDm2hiP5epyqX6YTu KmMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681487982; x=1684079982; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ahp5kQnTghgkZmiSTxR6+eMYSF2yeYW/B9Sl3mLm86I=; b=FZbUYEwjr3jdNCLQHUflf/H9dSZFp4+sEc/V06wINDDT7de+yg5eAeEQCBq5v4+MBW 2GlNDC18RcVJfaKsaKv48XRaPAMvqzd2hJKqgvSQOWBR7PnCe1pu4SDlnDJRpD94f+6s bg0F04CYSDvKzXrv7HO2435KG64OsaBIldH+c3u1Wis9k91JmlMAI3ijOoKPtfbJaMvO rGwvrRH1q4rMYozQUr4dxcoPuLQClkfHL9F1JBAFyjZQFL9FCfmtz45MoDoHuFtvuaNn pu0yEKn7H5nFTa8YmuepwEmOFwoYgn2nnuKAa30eoaOKP0XwJwfNCmzNcw3vNxXB6s4m qXnQ== X-Gm-Message-State: AAQBX9fccSQijlK6AXX5kFYBjwfNtUx8gza/LpqsPaCIZ7CmJvZR2/MV JZBHPmfDZnN/7ft0m0ffDK1v2YZxXYtY0Qq7rl6RrEKugWQ1XbK4gIBXRxJttGDuqA6i1oPFHev Sr6WMBzc4LeCyIDv+BZC/FUaqnUXjz0FkUiCgHoF8lopjaRxOFZvNqCAYNwtLvjiO7pyrHHtZX5 yPJxBR1vLu5bYb X-Google-Smtp-Source: AKy350ZKao8CL5faXPhsQHH/k9HmEq8p+sEw8aF/D7/B8/02SnVH9xnwups8GJsXNgURoBC0Xrs70Q== X-Received: by 2002:a17:90a:2e13:b0:23f:3f9c:7878 with SMTP id q19-20020a17090a2e1300b0023f3f9c7878mr5945626pjd.2.1681487981638; Fri, 14 Apr 2023 08:59:41 -0700 (PDT) Received: from hsinchu25.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id br8-20020a17090b0f0800b00240d4521958sm3083584pjb.18.2023.04.14.08.59.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Apr 2023 08:59:41 -0700 (PDT) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen , Andy Chiu , Paul Walmsley , Albert Ou , Guo Ren , Heiko Stuebner , Conor Dooley Subject: [PATCH -next v18 08/20] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Fri, 14 Apr 2023 15:58:31 +0000 Message-Id: <20230414155843.12963-9-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230414155843.12963-1-andy.chiu@sifive.com> References: <20230414155843.12963-1-andy.chiu@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230414_085944_443072_33D46C86 X-CRM114-Status: GOOD ( 12.14 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Greentime Hu Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_ext_state. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu Signed-off-by: Vineet Gupta Signed-off-by: Andy Chiu Acked-by: Conor Dooley Reviewed-by: Guo Ren Reviewed-by: Björn Töpel Reviewed-by: Heiko Stuebner Tested-by: Heiko Stuebner --- arch/riscv/include/asm/vector.h | 95 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 +++++ 2 files changed, 112 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 68c9fe831a41..7a56bb0769aa 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -11,8 +11,10 @@ #ifdef CONFIG_RISCV_ISA_V +#include #include #include +#include extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); @@ -22,6 +24,26 @@ static __always_inline bool has_vector(void) return riscv_has_extension_likely(RISCV_ISA_EXT_v); } +static inline void __riscv_v_vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_CLEAN; +} + +static inline void riscv_v_vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void riscv_v_vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_INITIAL; +} + +static inline bool riscv_v_vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void riscv_v_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -32,13 +54,86 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) +{ + asm volatile ( + "csrr %0, " __stringify(CSR_VSTART) "\n\t" + "csrr %1, " __stringify(CSR_VTYPE) "\n\t" + "csrr %2, " __stringify(CSR_VL) "\n\t" + "csrr %3, " __stringify(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src) +{ + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvl x0, %2, %1\n\t" + ".option pop\n\t" + "csrw " __stringify(CSR_VSTART) ", %0\n\t" + "csrw " __stringify(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + __vstate_csr_save(save_to); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + riscv_v_disable(); +} + +static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_from, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + __vstate_csr_restore(restore_from); + riscv_v_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ struct pt_regs; static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; } static __always_inline bool has_vector(void) { return false; } +static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return false; } #define riscv_v_vsize (0) +#define riscv_v_vstate_off(regs) do {} while (0) +#define riscv_v_vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..586786d023c4 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_ext_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */