From patchwork Fri Mar 10 09:11:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168926 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7941C64EC4 for ; Fri, 10 Mar 2023 09:17:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230165AbjCJJRA (ORCPT ); Fri, 10 Mar 2023 04:17:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230460AbjCJJQk (ORCPT ); Fri, 10 Mar 2023 04:16:40 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 398FA7DF8F for ; Fri, 10 Mar 2023 01:12:36 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnY-00GpVx-4O; Fri, 10 Mar 2023 09:12:24 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 01/45] target/riscv: Add zvkb cpu property Date: Fri, 10 Mar 2023 09:11:31 +0000 Message-Id: <20230310091215.931644-2-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 13 +++++++++++++ target/riscv/cpu.h | 1 + 2 files changed, 14 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 1e97473af2..69611408f9 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -109,6 +109,7 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zve64d, true, PRIV_VERSION_1_12_0, ext_zve64d), ISA_EXT_DATA_ENTRY(zvfh, true, PRIV_VERSION_1_12_0, ext_zvfh), ISA_EXT_DATA_ENTRY(zvfhmin, true, PRIV_VERSION_1_12_0, ext_zvfhmin), + ISA_EXT_DATA_ENTRY(zvkb, true, PRIV_VERSION_1_12_0, ext_zvkb), ISA_EXT_DATA_ENTRY(zhinx, true, PRIV_VERSION_1_12_0, ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia), @@ -1211,6 +1212,18 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) return; } + /* + * In principle zve*x would also suffice here, were they supported + * in qemu + */ + if (cpu->cfg.ext_zvkb && + !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || + cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { + error_setg( + errp, "Vector crypto extensions require V or Zve* extensions"); + return; + } + #ifndef CONFIG_USER_ONLY if (cpu->cfg.pmu_num) { if (!riscv_pmu_init(cpu, cpu->cfg.pmu_num) && cpu->cfg.ext_sscofpmf) { diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 638e47c75a..7d6699f718 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -470,6 +470,7 @@ struct RISCVCPUConfig { bool ext_zve32f; bool ext_zve64f; bool ext_zve64d; + bool ext_zvkb; bool ext_zmmul; bool ext_zvfh; bool ext_zvfhmin; From patchwork Fri Mar 10 09:11:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AB9EC6FD19 for ; Fri, 10 Mar 2023 09:17:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231300AbjCJJRG (ORCPT ); Fri, 10 Mar 2023 04:17:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230142AbjCJJQn (ORCPT ); Fri, 10 Mar 2023 04:16:43 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A774109758 for ; Fri, 10 Mar 2023 01:12:39 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnY-00GpVx-CB; Fri, 10 Mar 2023 09:12:24 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 02/45] target/riscv: Refactor some of the generic vector functionality Date: Fri, 10 Mar 2023 09:11:32 +0000 Message-Id: <20230310091215.931644-3-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk Summary of refactoring: * take some functions/macros out of `vector_helper` and put them in a new module called `vector_internals` * factor the non SEW-specific stuff out of `GEN_OPIVV_TRANS` into function `opivv_trans` (similar to `opivi_trans`) All this refactoring ensures more functions/macros can be used by both vector and vector-crypto helpers (latter implemented in proceeding commit). Signed-off-by: Kiran Ostrolenk --- target/riscv/insn_trans/trans_rvv.c.inc | 62 ++++++------ target/riscv/meson.build | 1 + target/riscv/vector_helper.c | 120 +----------------------- target/riscv/vector_internals.c | 57 +++++++++++ target/riscv/vector_internals.h | 118 +++++++++++++++++++++++ 5 files changed, 209 insertions(+), 149 deletions(-) create mode 100644 target/riscv/vector_internals.c create mode 100644 target/riscv/vector_internals.h diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc index f2e3d38515..4106bd6994 100644 --- a/target/riscv/insn_trans/trans_rvv.c.inc +++ b/target/riscv/insn_trans/trans_rvv.c.inc @@ -1643,38 +1643,40 @@ GEN_OPIWX_WIDEN_TRANS(vwadd_wx) GEN_OPIWX_WIDEN_TRANS(vwsubu_wx) GEN_OPIWX_WIDEN_TRANS(vwsub_wx) +static bool opivv_trans(uint32_t vd, uint32_t vs1, uint32_t vs2, uint32_t vm, + gen_helper_gvec_4_ptr *fn, DisasContext *s) +{ + uint32_t data = 0; + TCGLabel *over = gen_new_label(); + tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); + + data = FIELD_DP32(data, VDATA, VM, vm); + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); + data = FIELD_DP32(data, VDATA, VTA, s->vta); + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); + data = FIELD_DP32(data, VDATA, VMA, s->vma); + tcg_gen_gvec_4_ptr(vreg_ofs(s, vd), vreg_ofs(s, 0), vreg_ofs(s, vs1), + vreg_ofs(s, vs2), cpu_env, s->cfg_ptr->vlen / 8, + s->cfg_ptr->vlen / 8, data, fn); + mark_vs_dirty(s); + gen_set_label(over); + return true; +} + /* Vector Integer Add-with-Carry / Subtract-with-Borrow Instructions */ /* OPIVV without GVEC IR */ -#define GEN_OPIVV_TRANS(NAME, CHECK) \ -static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ -{ \ - if (CHECK(s, a)) { \ - uint32_t data = 0; \ - static gen_helper_gvec_4_ptr * const fns[4] = { \ - gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ - gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ - }; \ - TCGLabel *over = gen_new_label(); \ - tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); \ - tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ - \ - data = FIELD_DP32(data, VDATA, VM, a->vm); \ - data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ - data = FIELD_DP32(data, VDATA, VTA, s->vta); \ - data = \ - FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s);\ - data = FIELD_DP32(data, VDATA, VMA, s->vma); \ - tcg_gen_gvec_4_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0), \ - vreg_ofs(s, a->rs1), \ - vreg_ofs(s, a->rs2), cpu_env, \ - s->cfg_ptr->vlen / 8, \ - s->cfg_ptr->vlen / 8, data, \ - fns[s->sew]); \ - mark_vs_dirty(s); \ - gen_set_label(over); \ - return true; \ - } \ - return false; \ +#define GEN_OPIVV_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return opivv_trans(a->rd, a->rs1, a->rs2, a->vm, fns[s->sew], s);\ + } \ + return false; \ } /* diff --git a/target/riscv/meson.build b/target/riscv/meson.build index 5dee37a242..a94fc3f598 100644 --- a/target/riscv/meson.build +++ b/target/riscv/meson.build @@ -16,6 +16,7 @@ riscv_ss.add(files( 'gdbstub.c', 'op_helper.c', 'vector_helper.c', + 'vector_internals.c', 'bitmanip_helper.c', 'translate.c', 'm128_helper.c', diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c index 2423affe37..823aa8eb08 100644 --- a/target/riscv/vector_helper.c +++ b/target/riscv/vector_helper.c @@ -26,6 +26,7 @@ #include "fpu/softfloat.h" #include "tcg/tcg-gvec-desc.h" #include "internals.h" +#include "vector_internals.h" #include target_ulong HELPER(vsetvl)(CPURISCVState *env, target_ulong s1, @@ -95,48 +96,6 @@ target_ulong HELPER(vsetvl)(CPURISCVState *env, target_ulong s1, #define H8(x) (x) #endif -static inline uint32_t vext_nf(uint32_t desc) -{ - return FIELD_EX32(simd_data(desc), VDATA, NF); -} - -static inline uint32_t vext_vm(uint32_t desc) -{ - return FIELD_EX32(simd_data(desc), VDATA, VM); -} - -/* - * Encode LMUL to lmul as following: - * LMUL vlmul lmul - * 1 000 0 - * 2 001 1 - * 4 010 2 - * 8 011 3 - * - 100 - - * 1/8 101 -3 - * 1/4 110 -2 - * 1/2 111 -1 - */ -static inline int32_t vext_lmul(uint32_t desc) -{ - return sextract32(FIELD_EX32(simd_data(desc), VDATA, LMUL), 0, 3); -} - -static inline uint32_t vext_vta(uint32_t desc) -{ - return FIELD_EX32(simd_data(desc), VDATA, VTA); -} - -static inline uint32_t vext_vma(uint32_t desc) -{ - return FIELD_EX32(simd_data(desc), VDATA, VMA); -} - -static inline uint32_t vext_vta_all_1s(uint32_t desc) -{ - return FIELD_EX32(simd_data(desc), VDATA, VTA_ALL_1S); -} - /* * Get the maximum number of elements can be operated. * @@ -155,21 +114,6 @@ static inline uint32_t vext_max_elems(uint32_t desc, uint32_t log2_esz) return scale < 0 ? vlenb >> -scale : vlenb << scale; } -/* - * Get number of total elements, including prestart, body and tail elements. - * Note that when LMUL < 1, the tail includes the elements past VLMAX that - * are held in the same vector register. - */ -static inline uint32_t vext_get_total_elems(CPURISCVState *env, uint32_t desc, - uint32_t esz) -{ - uint32_t vlenb = simd_maxsz(desc); - uint32_t sew = 1 << FIELD_EX64(env->vtype, VTYPE, VSEW); - int8_t emul = ctzl(esz) - ctzl(sew) + vext_lmul(desc) < 0 ? 0 : - ctzl(esz) - ctzl(sew) + vext_lmul(desc); - return (vlenb << emul) / esz; -} - static inline target_ulong adjust_addr(CPURISCVState *env, target_ulong addr) { return (addr & env->cur_pmmask) | env->cur_pmbase; @@ -202,20 +146,6 @@ static void probe_pages(CPURISCVState *env, target_ulong addr, } } -/* set agnostic elements to 1s */ -static void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt, - uint32_t tot) -{ - if (is_agnostic == 0) { - /* policy undisturbed */ - return; - } - if (tot - cnt == 0) { - return; - } - memset(base + cnt, -1, tot - cnt); -} - static inline void vext_set_elem_mask(void *v0, int index, uint8_t value) { @@ -225,18 +155,6 @@ static inline void vext_set_elem_mask(void *v0, int index, ((uint64_t *)v0)[idx] = deposit64(old, pos, 1, value); } -/* - * Earlier designs (pre-0.9) had a varying number of bits - * per mask value (MLEN). In the 0.9 design, MLEN=1. - * (Section 4.5) - */ -static inline int vext_elem_mask(void *v0, int index) -{ - int idx = index / 64; - int pos = index % 64; - return (((uint64_t *)v0)[idx] >> pos) & 1; -} - /* elements operations for load and store */ typedef void vext_ldst_elem_fn(CPURISCVState *env, target_ulong addr, uint32_t idx, void *vd, uintptr_t retaddr); @@ -774,8 +692,6 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b) #define NOP_UUU_H uint16_t, uint16_t, uint32_t, uint16_t, uint32_t #define NOP_UUU_W uint32_t, uint32_t, uint64_t, uint32_t, uint64_t -/* operation of two vector elements */ -typedef void opivv2_fn(void *vd, void *vs1, void *vs2, int i); #define OPIVV2(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP) \ static void do_##NAME(void *vd, void *vs1, void *vs2, int i) \ @@ -796,40 +712,6 @@ RVVCALL(OPIVV2, vsub_vv_h, OP_SSS_H, H2, H2, H2, DO_SUB) RVVCALL(OPIVV2, vsub_vv_w, OP_SSS_W, H4, H4, H4, DO_SUB) RVVCALL(OPIVV2, vsub_vv_d, OP_SSS_D, H8, H8, H8, DO_SUB) -static void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2, - CPURISCVState *env, uint32_t desc, - opivv2_fn *fn, uint32_t esz) -{ - uint32_t vm = vext_vm(desc); - uint32_t vl = env->vl; - uint32_t total_elems = vext_get_total_elems(env, desc, esz); - uint32_t vta = vext_vta(desc); - uint32_t vma = vext_vma(desc); - uint32_t i; - - for (i = env->vstart; i < vl; i++) { - if (!vm && !vext_elem_mask(v0, i)) { - /* set masked-off elements to 1s */ - vext_set_elems_1s(vd, vma, i * esz, (i + 1) * esz); - continue; - } - fn(vd, vs1, vs2, i); - } - env->vstart = 0; - /* set tail elements to 1s */ - vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz); -} - -/* generate the helpers for OPIVV */ -#define GEN_VEXT_VV(NAME, ESZ) \ -void HELPER(NAME)(void *vd, void *v0, void *vs1, \ - void *vs2, CPURISCVState *env, \ - uint32_t desc) \ -{ \ - do_vext_vv(vd, v0, vs1, vs2, env, desc, \ - do_##NAME, ESZ); \ -} - GEN_VEXT_VV(vadd_vv_b, 1) GEN_VEXT_VV(vadd_vv_h, 2) GEN_VEXT_VV(vadd_vv_w, 4) diff --git a/target/riscv/vector_internals.c b/target/riscv/vector_internals.c new file mode 100644 index 0000000000..95efaa79cb --- /dev/null +++ b/target/riscv/vector_internals.c @@ -0,0 +1,57 @@ +/* + * RISC-V Vector Extension Internals + * + * Copyright (c) 2020 T-Head Semiconductor Co., Ltd. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#include "vector_internals.h" + +/* set agnostic elements to 1s */ +void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt, + uint32_t tot) +{ + if (is_agnostic == 0) { + /* policy undisturbed */ + return; + } + if (tot - cnt == 0) { + return ; + } + memset(base + cnt, -1, tot - cnt); +} + +void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2, + CPURISCVState *env, uint32_t desc, + opivv2_fn *fn, uint32_t esz) +{ + uint32_t vm = vext_vm(desc); + uint32_t vl = env->vl; + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + uint32_t vta = vext_vta(desc); + uint32_t vma = vext_vma(desc); + uint32_t i; + + for (i = env->vstart; i < vl; i++) { + if (!vm && !vext_elem_mask(v0, i)) { + /* set masked-off elements to 1s */ + vext_set_elems_1s(vd, vma, i * esz, (i + 1) * esz); + continue; + } + fn(vd, vs1, vs2, i); + } + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz); +} diff --git a/target/riscv/vector_internals.h b/target/riscv/vector_internals.h new file mode 100644 index 0000000000..1d26ff9514 --- /dev/null +++ b/target/riscv/vector_internals.h @@ -0,0 +1,118 @@ +/* + * RISC-V Vector Extension Internals + * + * Copyright (c) 2020 T-Head Semiconductor Co., Ltd. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#ifndef TARGET_RISCV_VECTOR_INTERNALS_H +#define TARGET_RISCV_VECTOR_INTERNALS_H + +#include "qemu/osdep.h" +#include "qemu/bitops.h" +#include "cpu.h" +#include "tcg/tcg-gvec-desc.h" +#include "internals.h" + +static inline uint32_t vext_nf(uint32_t desc) +{ + return FIELD_EX32(simd_data(desc), VDATA, NF); +} + +/* + * Encode LMUL to lmul as following: + * LMUL vlmul lmul + * 1 000 0 + * 2 001 1 + * 4 010 2 + * 8 011 3 + * - 100 - + * 1/8 101 -3 + * 1/4 110 -2 + * 1/2 111 -1 + */ +static inline int32_t vext_lmul(uint32_t desc) +{ + return sextract32(FIELD_EX32(simd_data(desc), VDATA, LMUL), 0, 3); +} + +static inline uint32_t vext_vm(uint32_t desc) +{ + return FIELD_EX32(simd_data(desc), VDATA, VM); +} + +static inline uint32_t vext_vma(uint32_t desc) +{ + return FIELD_EX32(simd_data(desc), VDATA, VMA); +} + +static inline uint32_t vext_vta(uint32_t desc) +{ + return FIELD_EX32(simd_data(desc), VDATA, VTA); +} + +static inline uint32_t vext_vta_all_1s(uint32_t desc) +{ + return FIELD_EX32(simd_data(desc), VDATA, VTA_ALL_1S); +} + +/* + * Earlier designs (pre-0.9) had a varying number of bits + * per mask value (MLEN). In the 0.9 design, MLEN=1. + * (Section 4.5) + */ +static inline int vext_elem_mask(void *v0, int index) +{ + int idx = index / 64; + int pos = index % 64; + return (((uint64_t *)v0)[idx] >> pos) & 1; +} + +/* + * Get number of total elements, including prestart, body and tail elements. + * Note that when LMUL < 1, the tail includes the elements past VLMAX that + * are held in the same vector register. + */ +static inline uint32_t vext_get_total_elems(CPURISCVState *env, uint32_t desc, + uint32_t esz) +{ + uint32_t vlenb = simd_maxsz(desc); + uint32_t sew = 1 << FIELD_EX64(env->vtype, VTYPE, VSEW); + int8_t emul = ctzl(esz) - ctzl(sew) + vext_lmul(desc) < 0 ? 0 : + ctzl(esz) - ctzl(sew) + vext_lmul(desc); + return (vlenb << emul) / esz; +} + +/* set agnostic elements to 1s */ +void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt, + uint32_t tot); + +/* operation of two vector elements */ +typedef void opivv2_fn(void *vd, void *vs1, void *vs2, int i); + +void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2, + CPURISCVState *env, uint32_t desc, + opivv2_fn *fn, uint32_t esz); + +/* generate the helpers for OPIVV */ +#define GEN_VEXT_VV(NAME, ESZ) \ +void HELPER(NAME)(void *vd, void *v0, void *vs1, \ + void *vs2, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + do_vext_vv(vd, v0, vs1, vs2, env, desc, \ + do_##NAME, ESZ); \ +} + +#endif /* TARGET_RISCV_VECTOR_INTERNALS_H */ From patchwork Fri Mar 10 09:11:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DFEFC74A44 for ; Fri, 10 Mar 2023 09:17:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231263AbjCJJRB (ORCPT ); Fri, 10 Mar 2023 04:17:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229897AbjCJJQl (ORCPT ); Fri, 10 Mar 2023 04:16:41 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 522431091C3 for ; Fri, 10 Mar 2023 01:12:38 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnY-00GpVx-LM; Fri, 10 Mar 2023 09:12:24 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter , Max Chou Subject: [PATCH 03/45] target/riscv: Add vclmul.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:33 +0000 Message-Id: <20230310091215.931644-4-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Nazar Kazakov Co-authored-by: Max Chou Signed-off-by: Max Chou Signed-off-by: Nazar Kazakov Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 3 ++ target/riscv/insn32.decode | 3 ++ target/riscv/insn_trans/trans_rvzvkb.c.inc | 40 +++++++++++++++++++++ target/riscv/meson.build | 3 +- target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 42 ++++++++++++++++++++++ 6 files changed, 91 insertions(+), 1 deletion(-) create mode 100644 target/riscv/insn_trans/trans_rvzvkb.c.inc create mode 100644 target/riscv/vcrypto_helper.c diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 37b54e0991..33060c3e2f 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1142,3 +1142,6 @@ DEF_HELPER_FLAGS_1(aes64im, TCG_CALL_NO_RWG_SE, tl, tl) DEF_HELPER_FLAGS_3(sm4ed, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl) DEF_HELPER_FLAGS_3(sm4ks, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl) + +/* Vector crypto functions */ +DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 73d5d1b045..03a0057d71 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -908,3 +908,6 @@ sm4ks .. 11010 ..... ..... 000 ..... 0110011 @k_aes # *** RV32 Zicond Standard Extension *** czero_eqz 0000111 ..... ..... 101 ..... 0110011 @r czero_nez 0000111 ..... ..... 111 ..... 0110011 @r + +# *** RV64 Zvkb vector crypto extension *** +vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc new file mode 100644 index 0000000000..7cd920e76d --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -0,0 +1,40 @@ +/* + * RISC-V translation routines for the Zvkb Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#define GEN_VV_MASKED_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + return opivv_trans(a->rd, a->rs1, a->rs2, a->vm, \ + gen_helper_##NAME, s); \ + } \ + return false; \ +} + +static bool zvkb_vv_check(DisasContext *s, arg_rmrr *a) +{ + return opivv_check(s, a) && s->cfg_ptr->ext_zvkb == true; +} + +static bool vclmul_vv_check(DisasContext *s, arg_rmrr *a) +{ + return zvkb_vv_check(s, a) && s->sew == MO_64; +} + +GEN_VV_MASKED_TRANS(vclmul_vv, vclmul_vv_check) diff --git a/target/riscv/meson.build b/target/riscv/meson.build index a94fc3f598..52a61dd66e 100644 --- a/target/riscv/meson.build +++ b/target/riscv/meson.build @@ -20,7 +20,8 @@ riscv_ss.add(files( 'bitmanip_helper.c', 'translate.c', 'm128_helper.c', - 'crypto_helper.c' + 'crypto_helper.c', + 'vcrypto_helper.c' )) riscv_ss.add(when: 'CONFIG_KVM', if_true: files('kvm.c'), if_false: files('kvm-stub.c')) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index 0ee8ee147d..939f5aeb1c 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1083,6 +1083,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvzicbo.c.inc" #include "insn_trans/trans_rvzfh.c.inc" #include "insn_trans/trans_rvk.c.inc" +#include "insn_trans/trans_rvzvkb.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c new file mode 100644 index 0000000000..b4983886bd --- /dev/null +++ b/target/riscv/vcrypto_helper.c @@ -0,0 +1,42 @@ +/* + * RISC-V Vector Crypto Extension Helpers for QEMU. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#include "qemu/osdep.h" +#include "qemu/host-utils.h" +#include "qemu/bitops.h" +#include "cpu.h" +#include "exec/memop.h" +#include "exec/exec-all.h" +#include "exec/helper-proto.h" +#include "internals.h" +#include "vector_internals.h" + +static uint64_t clmul64(uint64_t y, uint64_t x) +{ + uint64_t result = 0; + for (int j = 63; j >= 0; j--) { + if ((y >> j) & 1) { + result ^= (x << j); + } + } + return result; +} + +RVVCALL(OPIVV2, vclmul_vv, OP_UUU_D, H8, H8, H8, clmul64) +GEN_VEXT_VV(vclmul_vv, 8) From patchwork Fri Mar 10 09:11:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168924 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDA2AC74A44 for ; Fri, 10 Mar 2023 09:16:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229772AbjCJJQ7 (ORCPT ); Fri, 10 Mar 2023 04:16:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbjCJJQk (ORCPT ); Fri, 10 Mar 2023 04:16:40 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B794108695 for ; Fri, 10 Mar 2023 01:12:37 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnY-00GpVx-TK; Fri, 10 Mar 2023 09:12:24 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 04/45] target/riscv: Refactor some of the generic vector functionality Date: Fri, 10 Mar 2023 09:11:34 +0000 Message-Id: <20230310091215.931644-5-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk This refactoring ensures these functions/macros can be used by both vector and vector-crypto helpers (latter implemented in proceeding commit). Signed-off-by: Kiran Ostrolenk Reviewed-by: Wilfred Mallawa --- target/riscv/vector_helper.c | 36 --------------------------------- target/riscv/vector_internals.c | 24 ++++++++++++++++++++++ target/riscv/vector_internals.h | 16 +++++++++++++++ 3 files changed, 40 insertions(+), 36 deletions(-) diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c index 823aa8eb08..09b790653e 100644 --- a/target/riscv/vector_helper.c +++ b/target/riscv/vector_helper.c @@ -721,8 +721,6 @@ GEN_VEXT_VV(vsub_vv_h, 2) GEN_VEXT_VV(vsub_vv_w, 4) GEN_VEXT_VV(vsub_vv_d, 8) -typedef void opivx2_fn(void *vd, target_long s1, void *vs2, int i); - /* * (T1)s1 gives the real operator type. * (TX1)(T1)s1 expands the operator type of widen or narrow operations. @@ -747,40 +745,6 @@ RVVCALL(OPIVX2, vrsub_vx_h, OP_SSS_H, H2, H2, DO_RSUB) RVVCALL(OPIVX2, vrsub_vx_w, OP_SSS_W, H4, H4, DO_RSUB) RVVCALL(OPIVX2, vrsub_vx_d, OP_SSS_D, H8, H8, DO_RSUB) -static void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2, - CPURISCVState *env, uint32_t desc, - opivx2_fn fn, uint32_t esz) -{ - uint32_t vm = vext_vm(desc); - uint32_t vl = env->vl; - uint32_t total_elems = vext_get_total_elems(env, desc, esz); - uint32_t vta = vext_vta(desc); - uint32_t vma = vext_vma(desc); - uint32_t i; - - for (i = env->vstart; i < vl; i++) { - if (!vm && !vext_elem_mask(v0, i)) { - /* set masked-off elements to 1s */ - vext_set_elems_1s(vd, vma, i * esz, (i + 1) * esz); - continue; - } - fn(vd, s1, vs2, i); - } - env->vstart = 0; - /* set tail elements to 1s */ - vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz); -} - -/* generate the helpers for OPIVX */ -#define GEN_VEXT_VX(NAME, ESZ) \ -void HELPER(NAME)(void *vd, void *v0, target_ulong s1, \ - void *vs2, CPURISCVState *env, \ - uint32_t desc) \ -{ \ - do_vext_vx(vd, v0, s1, vs2, env, desc, \ - do_##NAME, ESZ); \ -} - GEN_VEXT_VX(vadd_vx_b, 1) GEN_VEXT_VX(vadd_vx_h, 2) GEN_VEXT_VX(vadd_vx_w, 4) diff --git a/target/riscv/vector_internals.c b/target/riscv/vector_internals.c index 95efaa79cb..9cf5c17cde 100644 --- a/target/riscv/vector_internals.c +++ b/target/riscv/vector_internals.c @@ -55,3 +55,27 @@ void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2, /* set tail elements to 1s */ vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz); } + +void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2, + CPURISCVState *env, uint32_t desc, + opivx2_fn fn, uint32_t esz) +{ + uint32_t vm = vext_vm(desc); + uint32_t vl = env->vl; + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + uint32_t vta = vext_vta(desc); + uint32_t vma = vext_vma(desc); + uint32_t i; + + for (i = env->vstart; i < vl; i++) { + if (!vm && !vext_elem_mask(v0, i)) { + /* set masked-off elements to 1s */ + vext_set_elems_1s(vd, vma, i * esz, (i + 1) * esz); + continue; + } + fn(vd, s1, vs2, i); + } + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, vl * esz, total_elems * esz); +} diff --git a/target/riscv/vector_internals.h b/target/riscv/vector_internals.h index 1d26ff9514..90500e5df6 100644 --- a/target/riscv/vector_internals.h +++ b/target/riscv/vector_internals.h @@ -115,4 +115,20 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, \ do_##NAME, ESZ); \ } +typedef void opivx2_fn(void *vd, target_long s1, void *vs2, int i); + +void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2, + CPURISCVState *env, uint32_t desc, + opivx2_fn fn, uint32_t esz); + +/* generate the helpers for OPIVX */ +#define GEN_VEXT_VX(NAME, ESZ) \ +void HELPER(NAME)(void *vd, void *v0, target_ulong s1, \ + void *vs2, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + do_vext_vx(vd, v0, s1, vs2, env, desc, \ + do_##NAME, ESZ); \ +} + #endif /* TARGET_RISCV_VECTOR_INTERNALS_H */ From patchwork Fri Mar 10 09:11:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80504C6FD19 for ; Fri, 10 Mar 2023 09:17:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231377AbjCJJRJ (ORCPT ); Fri, 10 Mar 2023 04:17:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230287AbjCJJQn (ORCPT ); Fri, 10 Mar 2023 04:16:43 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9453B109762 for ; Fri, 10 Mar 2023 01:12:41 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnZ-00GpVx-59; Fri, 10 Mar 2023 09:12:25 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 05/45] target/riscv: Add vclmul.vx decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:35 +0000 Message-Id: <20230310091215.931644-6-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkb.c.inc | 48 ++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 2 + 4 files changed, 52 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 33060c3e2f..352921ead6 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1145,3 +1145,4 @@ DEF_HELPER_FLAGS_3(sm4ks, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl) /* Vector crypto functions */ DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 03a0057d71..6b8466424d 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -911,3 +911,4 @@ czero_nez 0000111 ..... ..... 111 ..... 0110011 @r # *** RV64 Zvkb vector crypto extension *** vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm +vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 7cd920e76d..76efade1b6 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -38,3 +38,51 @@ static bool vclmul_vv_check(DisasContext *s, arg_rmrr *a) } GEN_VV_MASKED_TRANS(vclmul_vv, vclmul_vv_check) + +#define GEN_VX_MASKED_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + TCGv_ptr rd_v, v0_v, rs2_v; \ + TCGv rs1; \ + TCGv_i32 desc; \ + uint32_t data = 0; \ + \ + TCGLabel *over = gen_new_label(); \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + \ + rd_v = tcg_temp_new_ptr(); \ + v0_v = tcg_temp_new_ptr(); \ + rs1 = get_gpr(s, a->rs1, EXT_ZERO); \ + rs2_v = tcg_temp_new_ptr(); \ + desc = tcg_constant_i32(simd_desc(s->cfg_ptr->vlen / 8, \ + s->cfg_ptr->vlen / 8, data)); \ + tcg_gen_addi_ptr(rd_v, cpu_env, vreg_ofs(s, a->rd)); \ + tcg_gen_addi_ptr(v0_v, cpu_env, vreg_ofs(s, 0)); \ + tcg_gen_addi_ptr(rs2_v, cpu_env, vreg_ofs(s, a->rs2)); \ + gen_helper_##NAME(rd_v, v0_v, rs1, rs2_v, cpu_env, desc); \ + \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ +} + +static bool zvkb_vx_check(DisasContext *s, arg_rmrr *a) +{ + return opivx_check(s, a) && s->cfg_ptr->ext_zvkb == true; +} + +static bool vclmul_vx_check(DisasContext *s, arg_rmrr *a) +{ + return zvkb_vx_check(s, a) && s->sew == MO_64; +} + +GEN_VX_MASKED_TRANS(vclmul_vx, vclmul_vx_check) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index b4983886bd..749a9cb30b 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -40,3 +40,5 @@ static uint64_t clmul64(uint64_t y, uint64_t x) RVVCALL(OPIVV2, vclmul_vv, OP_UUU_D, H8, H8, H8, clmul64) GEN_VEXT_VV(vclmul_vv, 8) +RVVCALL(OPIVX2, vclmul_vx, OP_UUU_D, H8, H8, clmul64) +GEN_VEXT_VX(vclmul_vx, 8) From patchwork Fri Mar 10 09:11:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1120CC6FD1C for ; Fri, 10 Mar 2023 09:17:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229603AbjCJJRy (ORCPT ); Fri, 10 Mar 2023 04:17:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230520AbjCJJQz (ORCPT ); Fri, 10 Mar 2023 04:16:55 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EAA2149BE for ; Fri, 10 Mar 2023 01:13:01 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnZ-00GpVx-D4; Fri, 10 Mar 2023 09:12:25 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 06/45] target/riscv: Add vclmulh.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:36 +0000 Message-Id: <20230310091215.931644-7-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkb.c.inc | 1 + target/riscv/vcrypto_helper.c | 13 +++++++++++++ 4 files changed, 16 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 352921ead6..1c69c34a78 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1146,3 +1146,4 @@ DEF_HELPER_FLAGS_3(sm4ks, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl) /* Vector crypto functions */ DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 6b8466424d..3ad8e2055b 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -912,3 +912,4 @@ czero_nez 0000111 ..... ..... 111 ..... 0110011 @r # *** RV64 Zvkb vector crypto extension *** vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm +vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 76efade1b6..63a8778acc 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -38,6 +38,7 @@ static bool vclmul_vv_check(DisasContext *s, arg_rmrr *a) } GEN_VV_MASKED_TRANS(vclmul_vv, vclmul_vv_check) +GEN_VV_MASKED_TRANS(vclmulh_vv, vclmul_vv_check) #define GEN_VX_MASKED_TRANS(NAME, CHECK) \ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 749a9cb30b..1891c29767 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -38,7 +38,20 @@ static uint64_t clmul64(uint64_t y, uint64_t x) return result; } +static uint64_t clmulh64(uint64_t y, uint64_t x) +{ + uint64_t result = 0; + for (int j = 63; j >= 1; j--) { + if ((y >> j) & 1) { + result ^= (x >> (64 - j)); + } + } + return result; +} + RVVCALL(OPIVV2, vclmul_vv, OP_UUU_D, H8, H8, H8, clmul64) GEN_VEXT_VV(vclmul_vv, 8) RVVCALL(OPIVX2, vclmul_vx, OP_UUU_D, H8, H8, clmul64) GEN_VEXT_VX(vclmul_vx, 8) +RVVCALL(OPIVV2, vclmulh_vv, OP_UUU_D, H8, H8, H8, clmulh64) +GEN_VEXT_VV(vclmulh_vv, 8) From patchwork Fri Mar 10 09:11:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168925 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC556C74A4B for ; Fri, 10 Mar 2023 09:17:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231240AbjCJJQ7 (ORCPT ); Fri, 10 Mar 2023 04:16:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231216AbjCJJQk (ORCPT ); Fri, 10 Mar 2023 04:16:40 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F6CD1086BE for ; Fri, 10 Mar 2023 01:12:38 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnZ-00GpVx-L4; Fri, 10 Mar 2023 09:12:25 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 07/45] target/riscv: Add vclmulh.vx decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:37 +0000 Message-Id: <20230310091215.931644-8-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkb.c.inc | 1 + target/riscv/vcrypto_helper.c | 2 ++ 4 files changed, 5 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 1c69c34a78..37f2e162f6 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1147,3 +1147,4 @@ DEF_HELPER_FLAGS_3(sm4ks, TCG_CALL_NO_RWG_SE, tl, tl, tl, tl) DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vclmulh_vx, void, ptr, ptr, tl, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 3ad8e2055b..488e01ca59 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -913,3 +913,4 @@ czero_nez 0000111 ..... ..... 111 ..... 0110011 @r vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm +vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 63a8778acc..810e469e13 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -87,3 +87,4 @@ static bool vclmul_vx_check(DisasContext *s, arg_rmrr *a) } GEN_VX_MASKED_TRANS(vclmul_vx, vclmul_vx_check) +GEN_VX_MASKED_TRANS(vclmulh_vx, vclmul_vx_check) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 1891c29767..8b7c63d499 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -55,3 +55,5 @@ RVVCALL(OPIVX2, vclmul_vx, OP_UUU_D, H8, H8, clmul64) GEN_VEXT_VX(vclmul_vx, 8) RVVCALL(OPIVV2, vclmulh_vv, OP_UUU_D, H8, H8, H8, clmulh64) GEN_VEXT_VV(vclmulh_vv, 8) +RVVCALL(OPIVX2, vclmulh_vx, OP_UUU_D, H8, H8, clmulh64) +GEN_VEXT_VX(vclmulh_vx, 8) From patchwork Fri Mar 10 09:11:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38824C6FD1C for ; Fri, 10 Mar 2023 09:17:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231357AbjCJJRH (ORCPT ); Fri, 10 Mar 2023 04:17:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230248AbjCJJQn (ORCPT ); Fri, 10 Mar 2023 04:16:43 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA01A109759 for ; Fri, 10 Mar 2023 01:12:40 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnZ-00GpVx-Sl; Fri, 10 Mar 2023 09:12:25 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 08/45] target/riscv: Refactor some of the generic vector functionality Date: Fri, 10 Mar 2023 09:11:38 +0000 Message-Id: <20230310091215.931644-9-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk This refactoring ensures these functions/macros can be used by both vector and vector-crypto helpers (latter implemented in proceeding commit). Also moves the checks out of `do_opiv{v,x,i}_gvec{,_shift}` functions and into the corresponding macros. This enables the functions to be reused in proceeding commit without check duplication. Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Kiran Ostrolenk --- target/riscv/insn_trans/trans_rvv.c.inc | 28 +++++++-------- target/riscv/vector_helper.c | 45 ----------------------- target/riscv/vector_internals.h | 48 +++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 61 deletions(-) diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc index 4106bd6994..bb5e2c5407 100644 --- a/target/riscv/insn_trans/trans_rvv.c.inc +++ b/target/riscv/insn_trans/trans_rvv.c.inc @@ -1187,9 +1187,6 @@ do_opivv_gvec(DisasContext *s, arg_rmrr *a, GVecGen3Fn *gvec_fn, gen_helper_gvec_4_ptr *fn) { TCGLabel *over = gen_new_label(); - if (!opivv_check(s, a)) { - return false; - } tcg_gen_brcondi_tl(TCG_COND_EQ, cpu_vl, 0, over); tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); @@ -1223,6 +1220,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivv_check(s, a)) { \ + return false; \ + } \ return do_opivv_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ } @@ -1282,10 +1282,6 @@ static inline bool do_opivx_gvec(DisasContext *s, arg_rmrr *a, GVecGen2sFn *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i64 src1 = tcg_temp_new_i64(); @@ -1307,6 +1303,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ } @@ -1439,10 +1438,6 @@ static inline bool do_opivi_gvec(DisasContext *s, arg_rmrr *a, GVecGen2iFn *gvec_fn, gen_helper_opivx *fn, imm_mode_t imm_mode) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { gvec_fn(s->sew, vreg_ofs(s, a->rd), vreg_ofs(s, a->rs2), extract_imm(s, a->rs1, imm_mode), MAXSZ(s), MAXSZ(s)); @@ -1460,6 +1455,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##OPIVX##_b, gen_helper_##OPIVX##_h, \ gen_helper_##OPIVX##_w, gen_helper_##OPIVX##_d, \ }; \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivi_gvec(s, a, tcg_gen_gvec_##SUF, \ fns[s->sew], IMM_MODE); \ } @@ -1785,10 +1783,6 @@ static inline bool do_opivx_gvec_shift(DisasContext *s, arg_rmrr *a, GVecGen2sFn32 *gvec_fn, gen_helper_opivx *fn) { - if (!opivx_check(s, a)) { - return false; - } - if (a->vm && s->vl_eq_vlmax && !(s->vta && s->lmul < 0)) { TCGv_i32 src1 = tcg_temp_new_i32(); @@ -1810,7 +1804,9 @@ static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ }; \ - \ + if (!opivx_check(s, a)) { \ + return false; \ + } \ return do_opivx_gvec_shift(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ } diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c index 09b790653e..27fefef10e 100644 --- a/target/riscv/vector_helper.c +++ b/target/riscv/vector_helper.c @@ -76,26 +76,6 @@ target_ulong HELPER(vsetvl)(CPURISCVState *env, target_ulong s1, return vl; } -/* - * Note that vector data is stored in host-endian 64-bit chunks, - * so addressing units smaller than that needs a host-endian fixup. - */ -#if HOST_BIG_ENDIAN -#define H1(x) ((x) ^ 7) -#define H1_2(x) ((x) ^ 6) -#define H1_4(x) ((x) ^ 4) -#define H2(x) ((x) ^ 3) -#define H4(x) ((x) ^ 1) -#define H8(x) ((x)) -#else -#define H1(x) (x) -#define H1_2(x) (x) -#define H1_4(x) (x) -#define H2(x) (x) -#define H4(x) (x) -#define H8(x) (x) -#endif - /* * Get the maximum number of elements can be operated. * @@ -657,18 +637,11 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b) *** Vector Integer Arithmetic Instructions */ -/* expand macro args before macro */ -#define RVVCALL(macro, ...) macro(__VA_ARGS__) - /* (TD, T1, T2, TX1, TX2) */ #define OP_SSS_B int8_t, int8_t, int8_t, int8_t, int8_t #define OP_SSS_H int16_t, int16_t, int16_t, int16_t, int16_t #define OP_SSS_W int32_t, int32_t, int32_t, int32_t, int32_t #define OP_SSS_D int64_t, int64_t, int64_t, int64_t, int64_t -#define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t -#define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t -#define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t -#define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t #define OP_SUS_B int8_t, uint8_t, int8_t, uint8_t, int8_t #define OP_SUS_H int16_t, uint16_t, int16_t, uint16_t, int16_t #define OP_SUS_W int32_t, uint32_t, int32_t, uint32_t, int32_t @@ -692,14 +665,6 @@ GEN_VEXT_ST_WHOLE(vs8r_v, int8_t, ste_b) #define NOP_UUU_H uint16_t, uint16_t, uint32_t, uint16_t, uint32_t #define NOP_UUU_W uint32_t, uint32_t, uint64_t, uint32_t, uint64_t - -#define OPIVV2(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP) \ -static void do_##NAME(void *vd, void *vs1, void *vs2, int i) \ -{ \ - TX1 s1 = *((T1 *)vs1 + HS1(i)); \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2, s1); \ -} #define DO_SUB(N, M) (N - M) #define DO_RSUB(N, M) (M - N) @@ -721,16 +686,6 @@ GEN_VEXT_VV(vsub_vv_h, 2) GEN_VEXT_VV(vsub_vv_w, 4) GEN_VEXT_VV(vsub_vv_d, 8) -/* - * (T1)s1 gives the real operator type. - * (TX1)(T1)s1 expands the operator type of widen or narrow operations. - */ -#define OPIVX2(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP) \ -static void do_##NAME(void *vd, target_long s1, void *vs2, int i) \ -{ \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1); \ -} RVVCALL(OPIVX2, vadd_vx_b, OP_SSS_B, H1, H1, DO_ADD) RVVCALL(OPIVX2, vadd_vx_h, OP_SSS_H, H2, H2, DO_ADD) diff --git a/target/riscv/vector_internals.h b/target/riscv/vector_internals.h index 90500e5df6..749d138beb 100644 --- a/target/riscv/vector_internals.h +++ b/target/riscv/vector_internals.h @@ -30,6 +30,26 @@ static inline uint32_t vext_nf(uint32_t desc) return FIELD_EX32(simd_data(desc), VDATA, NF); } +/* + * Note that vector data is stored in host-endian 64-bit chunks, + * so addressing units smaller than that needs a host-endian fixup. + */ +#if HOST_BIG_ENDIAN +#define H1(x) ((x) ^ 7) +#define H1_2(x) ((x) ^ 6) +#define H1_4(x) ((x) ^ 4) +#define H2(x) ((x) ^ 3) +#define H4(x) ((x) ^ 1) +#define H8(x) ((x)) +#else +#define H1(x) (x) +#define H1_2(x) (x) +#define H1_4(x) (x) +#define H2(x) (x) +#define H4(x) (x) +#define H8(x) (x) +#endif + /* * Encode LMUL to lmul as following: * LMUL vlmul lmul @@ -98,9 +118,26 @@ static inline uint32_t vext_get_total_elems(CPURISCVState *env, uint32_t desc, void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt, uint32_t tot); +/* expand macro args before macro */ +#define RVVCALL(macro, ...) macro(__VA_ARGS__) + +/* (TD, T1, T2, TX1, TX2) */ +#define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t +#define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t +#define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t +#define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t + /* operation of two vector elements */ typedef void opivv2_fn(void *vd, void *vs1, void *vs2, int i); +#define OPIVV2(NAME, TD, T1, T2, TX1, TX2, HD, HS1, HS2, OP) \ +static void do_##NAME(void *vd, void *vs1, void *vs2, int i) \ +{ \ + TX1 s1 = *((T1 *)vs1 + HS1(i)); \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2, s1); \ +} + void do_vext_vv(void *vd, void *v0, void *vs1, void *vs2, CPURISCVState *env, uint32_t desc, opivv2_fn *fn, uint32_t esz); @@ -117,6 +154,17 @@ void HELPER(NAME)(void *vd, void *v0, void *vs1, \ typedef void opivx2_fn(void *vd, target_long s1, void *vs2, int i); +/* + * (T1)s1 gives the real operator type. + * (TX1)(T1)s1 expands the operator type of widen or narrow operations. + */ +#define OPIVX2(NAME, TD, T1, T2, TX1, TX2, HD, HS2, OP) \ +static void do_##NAME(void *vd, target_long s1, void *vs2, int i) \ +{ \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2, (TX1)(T1)s1); \ +} + void do_vext_vx(void *vd, void *v0, target_long s1, void *vs2, CPURISCVState *env, uint32_t desc, opivx2_fn fn, uint32_t esz); From patchwork Fri Mar 10 09:11:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B776BC6FD1E for ; Fri, 10 Mar 2023 09:17:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231269AbjCJJRE (ORCPT ); Fri, 10 Mar 2023 04:17:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230157AbjCJJQm (ORCPT ); Fri, 10 Mar 2023 04:16:42 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97840109743 for ; Fri, 10 Mar 2023 01:12:38 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYna-00GpVx-4m; Fri, 10 Mar 2023 09:12:26 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 09/45] qemu/bitops.h: Limit rotate amounts Date: Fri, 10 Mar 2023 09:11:39 +0000 Message-Id: <20230310091215.931644-10-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Dickon Hood Rotates have been fixed up to only allow for reasonable rotate amounts (ie, no rotates >7 on an 8b value etc.) This fixes a problem with riscv vector rotate instructions. Signed-off-by: Dickon Hood --- include/qemu/bitops.h | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/include/qemu/bitops.h b/include/qemu/bitops.h index 03213ce952..c443995b3b 100644 --- a/include/qemu/bitops.h +++ b/include/qemu/bitops.h @@ -218,7 +218,8 @@ static inline unsigned long find_first_zero_bit(const unsigned long *addr, */ static inline uint8_t rol8(uint8_t word, unsigned int shift) { - return (word << shift) | (word >> ((8 - shift) & 7)); + shift &= 7; + return (word << shift) | (word >> (8 - shift)); } /** @@ -228,7 +229,8 @@ static inline uint8_t rol8(uint8_t word, unsigned int shift) */ static inline uint8_t ror8(uint8_t word, unsigned int shift) { - return (word >> shift) | (word << ((8 - shift) & 7)); + shift &= 7; + return (word >> shift) | (word << (8 - shift)); } /** @@ -238,7 +240,8 @@ static inline uint8_t ror8(uint8_t word, unsigned int shift) */ static inline uint16_t rol16(uint16_t word, unsigned int shift) { - return (word << shift) | (word >> ((16 - shift) & 15)); + shift &= 15; + return (word << shift) | (word >> (16 - shift)); } /** @@ -248,7 +251,8 @@ static inline uint16_t rol16(uint16_t word, unsigned int shift) */ static inline uint16_t ror16(uint16_t word, unsigned int shift) { - return (word >> shift) | (word << ((16 - shift) & 15)); + shift &= 15; + return (word >> shift) | (word << (16 - shift)); } /** @@ -258,7 +262,8 @@ static inline uint16_t ror16(uint16_t word, unsigned int shift) */ static inline uint32_t rol32(uint32_t word, unsigned int shift) { - return (word << shift) | (word >> ((32 - shift) & 31)); + shift &= 31; + return (word << shift) | (word >> (32 - shift)); } /** @@ -268,7 +273,8 @@ static inline uint32_t rol32(uint32_t word, unsigned int shift) */ static inline uint32_t ror32(uint32_t word, unsigned int shift) { - return (word >> shift) | (word << ((32 - shift) & 31)); + shift &= 31; + return (word >> shift) | (word << (32 - shift)); } /** @@ -278,7 +284,8 @@ static inline uint32_t ror32(uint32_t word, unsigned int shift) */ static inline uint64_t rol64(uint64_t word, unsigned int shift) { - return (word << shift) | (word >> ((64 - shift) & 63)); + shift &= 63; + return (word << shift) | (word >> (64 - shift)); } /** @@ -288,7 +295,8 @@ static inline uint64_t rol64(uint64_t word, unsigned int shift) */ static inline uint64_t ror64(uint64_t word, unsigned int shift) { - return (word >> shift) | (word << ((64 - shift) & 63)); + shift &= 63; + return (word >> shift) | (word << (64 - shift)); } /** From patchwork Fri Mar 10 09:11:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0073C6FD1C for ; Fri, 10 Mar 2023 09:17:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231401AbjCJJRL (ORCPT ); Fri, 10 Mar 2023 04:17:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230319AbjCJJQo (ORCPT ); Fri, 10 Mar 2023 04:16:44 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC71110976F for ; Fri, 10 Mar 2023 01:12:41 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYna-00GpVx-CT; Fri, 10 Mar 2023 09:12:26 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 10/45] target/riscv: Add vrol.[vv,vx] and vror.[vv,vx,vi] decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:40 +0000 Message-Id: <20230310091215.931644-11-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Dickon Hood Add an implementation of the vrol.* and vror.* instructions, with mappings between the RISC-V instructions and their internal TCG accelerated implmentations. There are some missing ror helpers, so I've bodged it by converting them to rols. Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Dickon Hood --- target/riscv/helper.h | 20 ++++++++ target/riscv/insn32.decode | 7 +++ target/riscv/insn_trans/trans_rvv.c.inc | 3 ++ target/riscv/insn_trans/trans_rvzvkb.c.inc | 57 ++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 36 ++++++++++++++ 5 files changed, 123 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 37f2e162f6..841cb43f04 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1148,3 +1148,23 @@ DEF_HELPER_6(vclmul_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmul_vx, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vclmulh_vv, void, ptr, ptr, ptr, ptr, env, i32) DEF_HELPER_6(vclmulh_vx, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_6(vror_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vror_vv_d, void, ptr, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vror_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vror_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_6(vrol_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vrol_vv_d, void, ptr, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vrol_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vrol_vx_d, void, ptr, ptr, tl, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 488e01ca59..c557c063df 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -37,6 +37,7 @@ %imm_u 12:s20 !function=ex_shift_12 %imm_bs 30:2 !function=ex_shift_3 %imm_rnum 20:4 +%imm_z6 26:1 15:5 # Argument sets: &empty @@ -82,6 +83,7 @@ @r_vm ...... vm:1 ..... ..... ... ..... ....... &rmrr %rs2 %rs1 %rd @r_vm_1 ...... . ..... ..... ... ..... ....... &rmrr vm=1 %rs2 %rs1 %rd @r_vm_0 ...... . ..... ..... ... ..... ....... &rmrr vm=0 %rs2 %rs1 %rd +@r2_zimm6 ..... . vm:1 ..... ..... ... ..... ....... &rmrr %rs2 rs1=%imm_z6 %rd @r2_zimm11 . zimm:11 ..... ... ..... ....... %rs1 %rd @r2_zimm10 .. zimm:10 ..... ... ..... ....... %rs1 %rd @r2_s ....... ..... ..... ... ..... ....... %rs2 %rs1 @@ -914,3 +916,8 @@ vclmul_vv 001100 . ..... ..... 010 ..... 1010111 @r_vm vclmul_vx 001100 . ..... ..... 110 ..... 1010111 @r_vm vclmulh_vv 001101 . ..... ..... 010 ..... 1010111 @r_vm vclmulh_vx 001101 . ..... ..... 110 ..... 1010111 @r_vm +vrol_vv 010101 . ..... ..... 000 ..... 1010111 @r_vm +vrol_vx 010101 . ..... ..... 100 ..... 1010111 @r_vm +vror_vv 010100 . ..... ..... 000 ..... 1010111 @r_vm +vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm +vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 diff --git a/target/riscv/insn_trans/trans_rvv.c.inc b/target/riscv/insn_trans/trans_rvv.c.inc index bb5e2c5407..fa89a2f466 100644 --- a/target/riscv/insn_trans/trans_rvv.c.inc +++ b/target/riscv/insn_trans/trans_rvv.c.inc @@ -1374,6 +1374,7 @@ GEN_OPIVX_GVEC_TRANS(vrsub_vx, rsubs) typedef enum { IMM_ZX, /* Zero-extended */ IMM_SX, /* Sign-extended */ + IMM_ZIMM6, /* Truncate to 6 bits */ IMM_TRUNC_SEW, /* Truncate to log(SEW) bits */ IMM_TRUNC_2SEW, /* Truncate to log(2*SEW) bits */ } imm_mode_t; @@ -1389,6 +1390,8 @@ static int64_t extract_imm(DisasContext *s, uint32_t imm, imm_mode_t imm_mode) return extract64(imm, 0, s->sew + 3); case IMM_TRUNC_2SEW: return extract64(imm, 0, s->sew + 4); + case IMM_ZIMM6: + return extract64(imm, 0, 6); default: g_assert_not_reached(); } diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 810e469e13..f71383e482 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -88,3 +88,60 @@ static bool vclmul_vx_check(DisasContext *s, arg_rmrr *a) GEN_VX_MASKED_TRANS(vclmul_vx, vclmul_vx_check) GEN_VX_MASKED_TRANS(vclmulh_vx, vclmul_vx_check) + +#define GEN_OPIVI_GVEC_TRANS_CHECK(NAME, IMM_MODE, OPIVX, SUF, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_opivx * const fns[4] = { \ + gen_helper_##OPIVX##_b, gen_helper_##OPIVX##_h, \ + gen_helper_##OPIVX##_w, gen_helper_##OPIVX##_d, \ + }; \ + return do_opivi_gvec(s, a, tcg_gen_gvec_##SUF, \ + fns[s->sew], IMM_MODE); \ + } \ + return false; \ +} + +#define GEN_OPIVV_GVEC_TRANS_CHECK(NAME, SUF, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return do_opivv_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ + } \ + return false; \ +} + +#define GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(NAME, SUF, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_opivx * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return do_opivx_gvec_shift(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ + } \ + return false; \ +} + +static void tcg_gen_gvec_rotrs(unsigned vece, uint32_t dofs, uint32_t aofs, + TCGv_i32 shift, uint32_t oprsz, uint32_t maxsz) +{ + TCGv_i32 tmp = tcg_temp_new_i32(); + tcg_gen_sub_i32(tmp, tcg_constant_i32(1 << (vece + 3)), shift); + tcg_gen_gvec_rotls(vece, dofs, aofs, tmp, oprsz, maxsz); +} + +/* vrol.v[vx] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vrol_vv, rotlv, zvkb_vv_check) +GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vrol_vx, rotls, zvkb_vx_check) + +/* vror.v[vxi] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vror_vv, rotrv, zvkb_vv_check) +GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vror_vx, rotrs, zvkb_vx_check) +GEN_OPIVI_GVEC_TRANS_CHECK(vror_vi, IMM_ZIMM6, vror_vx, rotri, zvkb_vx_check) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 8b7c63d499..30ed9b1270 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -57,3 +57,39 @@ RVVCALL(OPIVV2, vclmulh_vv, OP_UUU_D, H8, H8, H8, clmulh64) GEN_VEXT_VV(vclmulh_vv, 8) RVVCALL(OPIVX2, vclmulh_vx, OP_UUU_D, H8, H8, clmulh64) GEN_VEXT_VX(vclmulh_vx, 8) + +RVVCALL(OPIVV2, vror_vv_b, OP_UUU_B, H1, H1, H1, ror8) +RVVCALL(OPIVV2, vror_vv_h, OP_UUU_H, H2, H2, H2, ror16) +RVVCALL(OPIVV2, vror_vv_w, OP_UUU_W, H4, H4, H4, ror32) +RVVCALL(OPIVV2, vror_vv_d, OP_UUU_D, H8, H8, H8, ror64) +GEN_VEXT_VV(vror_vv_b, 1) +GEN_VEXT_VV(vror_vv_h, 2) +GEN_VEXT_VV(vror_vv_w, 4) +GEN_VEXT_VV(vror_vv_d, 8) + +RVVCALL(OPIVX2, vror_vx_b, OP_UUU_B, H1, H1, ror8) +RVVCALL(OPIVX2, vror_vx_h, OP_UUU_H, H2, H2, ror16) +RVVCALL(OPIVX2, vror_vx_w, OP_UUU_W, H4, H4, ror32) +RVVCALL(OPIVX2, vror_vx_d, OP_UUU_D, H8, H8, ror64) +GEN_VEXT_VX(vror_vx_b, 1) +GEN_VEXT_VX(vror_vx_h, 2) +GEN_VEXT_VX(vror_vx_w, 4) +GEN_VEXT_VX(vror_vx_d, 8) + +RVVCALL(OPIVV2, vrol_vv_b, OP_UUU_B, H1, H1, H1, rol8) +RVVCALL(OPIVV2, vrol_vv_h, OP_UUU_H, H2, H2, H2, rol16) +RVVCALL(OPIVV2, vrol_vv_w, OP_UUU_W, H4, H4, H4, rol32) +RVVCALL(OPIVV2, vrol_vv_d, OP_UUU_D, H8, H8, H8, rol64) +GEN_VEXT_VV(vrol_vv_b, 1) +GEN_VEXT_VV(vrol_vv_h, 2) +GEN_VEXT_VV(vrol_vv_w, 4) +GEN_VEXT_VV(vrol_vv_d, 8) + +RVVCALL(OPIVX2, vrol_vx_b, OP_UUU_B, H1, H1, rol8) +RVVCALL(OPIVX2, vrol_vx_h, OP_UUU_H, H2, H2, rol16) +RVVCALL(OPIVX2, vrol_vx_w, OP_UUU_W, H4, H4, rol32) +RVVCALL(OPIVX2, vrol_vx_d, OP_UUU_D, H8, H8, rol64) +GEN_VEXT_VX(vrol_vx_b, 1) +GEN_VEXT_VX(vrol_vx_h, 2) +GEN_VEXT_VX(vrol_vx_w, 4) +GEN_VEXT_VX(vrol_vx_d, 8) From patchwork Fri Mar 10 09:11:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C163C6FD19 for ; Fri, 10 Mar 2023 09:17:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229814AbjCJJRC (ORCPT ); Fri, 10 Mar 2023 04:17:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229895AbjCJJQm (ORCPT ); Fri, 10 Mar 2023 04:16:42 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86D051091E7 for ; Fri, 10 Mar 2023 01:12:38 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYna-00GpVx-Js; Fri, 10 Mar 2023 09:12:26 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 11/45] target/riscv: Refactor some of the generic vector functionality Date: Fri, 10 Mar 2023 09:11:41 +0000 Message-Id: <20230310091215.931644-12-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk This refactoring ensures these functions/macros can be used by both vector and vector-crypto helpers (latter implemented in proceeding commit). Signed-off-by: Kiran Ostrolenk --- target/riscv/vector_helper.c | 39 -------------------------------- target/riscv/vector_internals.h | 40 +++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 39 deletions(-) diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c index 27fefef10e..7c8775fd7b 100644 --- a/target/riscv/vector_helper.c +++ b/target/riscv/vector_helper.c @@ -3412,11 +3412,6 @@ GEN_VEXT_VF(vfwnmsac_vf_h, 4) GEN_VEXT_VF(vfwnmsac_vf_w, 8) /* Vector Floating-Point Square-Root Instruction */ -/* (TD, T2, TX2) */ -#define OP_UU_H uint16_t, uint16_t, uint16_t -#define OP_UU_W uint32_t, uint32_t, uint32_t -#define OP_UU_D uint64_t, uint64_t, uint64_t - #define OPFVV1(NAME, TD, T2, TX2, HD, HS2, OP) \ static void do_##NAME(void *vd, void *vs2, int i, \ CPURISCVState *env) \ @@ -4109,40 +4104,6 @@ GEN_VEXT_CMP_VF(vmfge_vf_w, uint32_t, H4, vmfge32) GEN_VEXT_CMP_VF(vmfge_vf_d, uint64_t, H8, vmfge64) /* Vector Floating-Point Classify Instruction */ -#define OPIVV1(NAME, TD, T2, TX2, HD, HS2, OP) \ -static void do_##NAME(void *vd, void *vs2, int i) \ -{ \ - TX2 s2 = *((T2 *)vs2 + HS2(i)); \ - *((TD *)vd + HD(i)) = OP(s2); \ -} - -#define GEN_VEXT_V(NAME, ESZ) \ -void HELPER(NAME)(void *vd, void *v0, void *vs2, \ - CPURISCVState *env, uint32_t desc) \ -{ \ - uint32_t vm = vext_vm(desc); \ - uint32_t vl = env->vl; \ - uint32_t total_elems = \ - vext_get_total_elems(env, desc, ESZ); \ - uint32_t vta = vext_vta(desc); \ - uint32_t vma = vext_vma(desc); \ - uint32_t i; \ - \ - for (i = env->vstart; i < vl; i++) { \ - if (!vm && !vext_elem_mask(v0, i)) { \ - /* set masked-off elements to 1s */ \ - vext_set_elems_1s(vd, vma, i * ESZ, \ - (i + 1) * ESZ); \ - continue; \ - } \ - do_##NAME(vd, vs2, i); \ - } \ - env->vstart = 0; \ - /* set tail elements to 1s */ \ - vext_set_elems_1s(vd, vta, vl * ESZ, \ - total_elems * ESZ); \ -} - target_ulong fclass_h(uint64_t frs1) { float16 f = frs1; diff --git a/target/riscv/vector_internals.h b/target/riscv/vector_internals.h index 749d138beb..15cbc5ddac 100644 --- a/target/riscv/vector_internals.h +++ b/target/riscv/vector_internals.h @@ -121,12 +121,52 @@ void vext_set_elems_1s(void *base, uint32_t is_agnostic, uint32_t cnt, /* expand macro args before macro */ #define RVVCALL(macro, ...) macro(__VA_ARGS__) +/* (TD, T2, TX2) */ +#define OP_UU_B uint8_t, uint8_t, uint8_t +#define OP_UU_H uint16_t, uint16_t, uint16_t +#define OP_UU_W uint32_t, uint32_t, uint32_t +#define OP_UU_D uint64_t, uint64_t, uint64_t + /* (TD, T1, T2, TX1, TX2) */ #define OP_UUU_B uint8_t, uint8_t, uint8_t, uint8_t, uint8_t #define OP_UUU_H uint16_t, uint16_t, uint16_t, uint16_t, uint16_t #define OP_UUU_W uint32_t, uint32_t, uint32_t, uint32_t, uint32_t #define OP_UUU_D uint64_t, uint64_t, uint64_t, uint64_t, uint64_t +#define OPIVV1(NAME, TD, T2, TX2, HD, HS2, OP) \ +static void do_##NAME(void *vd, void *vs2, int i) \ +{ \ + TX2 s2 = *((T2 *)vs2 + HS2(i)); \ + *((TD *)vd + HD(i)) = OP(s2); \ +} + +#define GEN_VEXT_V(NAME, ESZ) \ +void HELPER(NAME)(void *vd, void *v0, void *vs2, \ + CPURISCVState *env, uint32_t desc) \ +{ \ + uint32_t vm = vext_vm(desc); \ + uint32_t vl = env->vl; \ + uint32_t total_elems = \ + vext_get_total_elems(env, desc, ESZ); \ + uint32_t vta = vext_vta(desc); \ + uint32_t vma = vext_vma(desc); \ + uint32_t i; \ + \ + for (i = env->vstart; i < vl; i++) { \ + if (!vm && !vext_elem_mask(v0, i)) { \ + /* set masked-off elements to 1s */ \ + vext_set_elems_1s(vd, vma, i * ESZ, \ + (i + 1) * ESZ); \ + continue; \ + } \ + do_##NAME(vd, vs2, i); \ + } \ + env->vstart = 0; \ + /* set tail elements to 1s */ \ + vext_set_elems_1s(vd, vta, vl * ESZ, \ + total_elems * ESZ); \ +} + /* operation of two vector elements */ typedef void opivv2_fn(void *vd, void *vs1, void *vs2, int i); From patchwork Fri Mar 10 09:11:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168940 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FE04C64EC4 for ; Fri, 10 Mar 2023 09:17:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231462AbjCJJR1 (ORCPT ); Fri, 10 Mar 2023 04:17:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230387AbjCJJQr (ORCPT ); Fri, 10 Mar 2023 04:16:47 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20F0310A2B9 for ; Fri, 10 Mar 2023 01:12:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYna-00GpVx-Rs; Fri, 10 Mar 2023 09:12:26 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, William Salmon Subject: [PATCH 12/45] target/riscv: Add vbrev8.v decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:42 +0000 Message-Id: <20230310091215.931644-13-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: William Salmon Signed-off-by: William Salmon --- target/riscv/helper.h | 5 +++ target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkb.c.inc | 38 ++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 21 ++++++++++++ 4 files changed, 65 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 841cb43f04..625f9872d0 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1168,3 +1168,8 @@ DEF_HELPER_6(vrol_vx_b, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vrol_vx_h, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vrol_vx_w, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vrol_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_5(vbrev8_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vbrev8_v_d, void, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index c557c063df..87473a77c0 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -921,3 +921,4 @@ vrol_vx 010101 . ..... ..... 100 ..... 1010111 @r_vm vror_vv 010100 . ..... ..... 000 ..... 1010111 @r_vm vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 +vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index f71383e482..7cd114ae71 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -145,3 +145,41 @@ GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vrol_vx, rotls, zvkb_vx_check) GEN_OPIVV_GVEC_TRANS_CHECK(vror_vv, rotrv, zvkb_vv_check) GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vror_vx, rotrs, zvkb_vx_check) GEN_OPIVI_GVEC_TRANS_CHECK(vror_vi, IMM_ZIMM6, vror_vx, rotri, zvkb_vx_check) + +#define GEN_OPIV_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmr * a) \ +{ \ + if (CHECK(s, a)) { \ + uint32_t data = 0; \ + static gen_helper_gvec_3_ptr * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + TCGLabel *over = gen_new_label(); \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), vreg_ofs(s, 0), \ + vreg_ofs(s, a->rs2), cpu_env, \ + s->cfg_ptr->vlen / 8, s->cfg_ptr->vlen / 8, \ + data, fns[s->sew]); \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ +} + +static bool vxrev8_check(DisasContext *s, arg_rmr *a) +{ + return s->cfg_ptr->ext_zvkb == true && + require_rvv(s) && + vext_check_isa_ill(s) && + vext_check_ss(s, a->rd, a->rs2, a->vm); +} + +GEN_OPIV_TRANS(vbrev8_v, vxrev8_check) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 30ed9b1270..5d2a995de6 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -93,3 +93,24 @@ GEN_VEXT_VX(vrol_vx_b, 1) GEN_VEXT_VX(vrol_vx_h, 2) GEN_VEXT_VX(vrol_vx_w, 4) GEN_VEXT_VX(vrol_vx_d, 8) + +static uint64_t brev8(uint64_t val) +{ + val = ((val & 0x5555555555555555ull) << 1) + | ((val & 0xAAAAAAAAAAAAAAAAull) >> 1); + val = ((val & 0x3333333333333333ull) << 2) + | ((val & 0xCCCCCCCCCCCCCCCCull) >> 2); + val = ((val & 0x0F0F0F0F0F0F0F0Full) << 4) + | ((val & 0xF0F0F0F0F0F0F0F0ull) >> 4); + + return val; +} + +RVVCALL(OPIVV1, vbrev8_v_b, OP_UU_B, H1, H1, brev8) +RVVCALL(OPIVV1, vbrev8_v_h, OP_UU_H, H2, H2, brev8) +RVVCALL(OPIVV1, vbrev8_v_w, OP_UU_W, H4, H4, brev8) +RVVCALL(OPIVV1, vbrev8_v_d, OP_UU_D, H8, H8, brev8) +GEN_VEXT_V(vbrev8_v_b, 1) +GEN_VEXT_V(vbrev8_v_h, 2) +GEN_VEXT_V(vbrev8_v_w, 4) +GEN_VEXT_V(vbrev8_v_d, 8) From patchwork Fri Mar 10 09:11:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31B75C6FD19 for ; Fri, 10 Mar 2023 09:17:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231432AbjCJJRv (ORCPT ); Fri, 10 Mar 2023 04:17:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230502AbjCJJQv (ORCPT ); Fri, 10 Mar 2023 04:16:51 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09E4F10A4E3 for ; Fri, 10 Mar 2023 01:12:53 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnb-00GpVx-2v; Fri, 10 Mar 2023 09:12:27 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 13/45] target/riscv: Add vrev8.v decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:43 +0000 Message-Id: <20230310091215.931644-14-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/helper.h | 4 ++++ target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkb.c.inc | 1 + target/riscv/vcrypto_helper.c | 11 +++++++++++ 4 files changed, 17 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 625f9872d0..b4baa22692 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1169,6 +1169,10 @@ DEF_HELPER_6(vrol_vx_h, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vrol_vx_w, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vrol_vx_d, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_5(vrev8_v_b, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_h, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_w, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vrev8_v_d, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_b, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_h, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_w, void, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 87473a77c0..bdefcd3fa2 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -922,3 +922,4 @@ vror_vv 010100 . ..... ..... 000 ..... 1010111 @r_vm vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm +vrev8_v 010010 . ..... 01001 010 ..... 1010111 @r2_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 7cd114ae71..77ba8bc713 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -183,3 +183,4 @@ static bool vxrev8_check(DisasContext *s, arg_rmr *a) } GEN_OPIV_TRANS(vbrev8_v, vxrev8_check) +GEN_OPIV_TRANS(vrev8_v, vxrev8_check) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 5d2a995de6..ecf21c50f8 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -20,6 +20,7 @@ #include "qemu/osdep.h" #include "qemu/host-utils.h" #include "qemu/bitops.h" +#include "qemu/bswap.h" #include "cpu.h" #include "exec/memop.h" #include "exec/exec-all.h" @@ -114,3 +115,13 @@ GEN_VEXT_V(vbrev8_v_b, 1) GEN_VEXT_V(vbrev8_v_h, 2) GEN_VEXT_V(vbrev8_v_w, 4) GEN_VEXT_V(vbrev8_v_d, 8) + +#define DO_IDENTITY(a) (a) +RVVCALL(OPIVV1, vrev8_v_b, OP_UU_B, H1, H1, DO_IDENTITY) +RVVCALL(OPIVV1, vrev8_v_h, OP_UU_H, H2, H2, bswap16) +RVVCALL(OPIVV1, vrev8_v_w, OP_UU_W, H4, H4, bswap32) +RVVCALL(OPIVV1, vrev8_v_d, OP_UU_D, H8, H8, bswap64) +GEN_VEXT_V(vrev8_v_b, 1) +GEN_VEXT_V(vrev8_v_h, 2) +GEN_VEXT_V(vrev8_v_w, 4) +GEN_VEXT_V(vrev8_v_d, 8) From patchwork Fri Mar 10 09:11:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07926C6FD1C for ; Fri, 10 Mar 2023 09:17:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230504AbjCJJRT (ORCPT ); Fri, 10 Mar 2023 04:17:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229751AbjCJJQq (ORCPT ); Fri, 10 Mar 2023 04:16:46 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE4A110A2B0 for ; Fri, 10 Mar 2023 01:12:43 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnb-00GpVx-AD; Fri, 10 Mar 2023 09:12:27 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 14/45] target/riscv: Add vandn.[vv,vx] decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:44 +0000 Message-Id: <20230310091215.931644-15-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- accel/tcg/tcg-runtime-gvec.c | 11 +++++++ accel/tcg/tcg-runtime.h | 1 + target/riscv/helper.h | 9 ++++++ target/riscv/insn32.decode | 2 ++ target/riscv/insn_trans/trans_rvzvkb.c.inc | 34 ++++++++++++++++++++++ target/riscv/vcrypto_helper.c | 19 ++++++++++++ 6 files changed, 76 insertions(+) diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c index ac7d28c251..322dcc0687 100644 --- a/accel/tcg/tcg-runtime-gvec.c +++ b/accel/tcg/tcg-runtime-gvec.c @@ -550,6 +550,17 @@ void HELPER(gvec_ands)(void *d, void *a, uint64_t b, uint32_t desc) clear_high(d, oprsz, desc); } +void HELPER(gvec_andsc)(void *d, void *a, uint64_t b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint64_t)) { + *(uint64_t *)(d + i) = *(uint64_t *)(a + i) & ~b; + } + clear_high(d, oprsz, desc); +} + void HELPER(gvec_xors)(void *d, void *a, uint64_t b, uint32_t desc) { intptr_t oprsz = simd_oprsz(desc); diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index e141a6ab24..d086200483 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -217,6 +217,7 @@ DEF_HELPER_FLAGS_4(gvec_nor, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_eqv, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ands, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) +DEF_HELPER_FLAGS_4(gvec_andsc, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(gvec_xors, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(gvec_ors, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index b4baa22692..1bc10ca5be 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1177,3 +1177,12 @@ DEF_HELPER_5(vbrev8_v_b, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_h, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_w, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vbrev8_v_d, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_6(vandn_vv_b, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_h, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_w, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vv_d, void, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_6(vandn_vx_b, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vandn_vx_h, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vandn_vx_w, void, ptr, ptr, tl, ptr, env, i32) +DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index bdefcd3fa2..bbf128e4dc 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -923,3 +923,5 @@ vror_vx 010100 . ..... ..... 100 ..... 1010111 @r_vm vror_vi 01010. . ..... ..... 011 ..... 1010111 @r2_zimm6 vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm vrev8_v 010010 . ..... 01001 010 ..... 1010111 @r2_vm +vandn_vv 000001 . ..... ..... 000 ..... 1010111 @r_vm +vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm diff --git a/target/riscv/insn_trans/trans_rvzvkb.c.inc b/target/riscv/insn_trans/trans_rvzvkb.c.inc index 77ba8bc713..a9a820ec0a 100644 --- a/target/riscv/insn_trans/trans_rvzvkb.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkb.c.inc @@ -146,6 +146,40 @@ GEN_OPIVV_GVEC_TRANS_CHECK(vror_vv, rotrv, zvkb_vv_check) GEN_OPIVX_GVEC_SHIFT_TRANS_CHECK(vror_vx, rotrs, zvkb_vx_check) GEN_OPIVI_GVEC_TRANS_CHECK(vror_vi, IMM_ZIMM6, vror_vx, rotri, zvkb_vx_check) + +static void tcg_gen_gvec_andsc(unsigned vece, uint32_t dofs, uint32_t aofs, + TCGv_i64 c, uint32_t oprsz, uint32_t maxsz) +{ + static GVecGen2s g = { + .fni8 = tcg_gen_andc_i64, + .fniv = tcg_gen_andc_vec, + .fno = gen_helper_gvec_andsc, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + + g.vece = vece; + + tcg_gen_dup_i64(vece, c, c); + tcg_gen_gvec_2s(dofs, aofs, oprsz, maxsz, c, &g); +} + +#define GEN_OPIVX_GVEC_TRANS_CHECK(NAME, SUF, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr *a) \ +{ \ + if (CHECK(s, a)) { \ + static gen_helper_opivx * const fns[4] = { \ + gen_helper_##NAME##_b, gen_helper_##NAME##_h, \ + gen_helper_##NAME##_w, gen_helper_##NAME##_d, \ + }; \ + return do_opivx_gvec(s, a, tcg_gen_gvec_##SUF, fns[s->sew]); \ + } \ + return false; \ +} + +/* vandn.v[vx] */ +GEN_OPIVV_GVEC_TRANS_CHECK(vandn_vv, andc, zvkb_vv_check) +GEN_OPIVX_GVEC_TRANS_CHECK(vandn_vx, andsc, zvkb_vx_check) + #define GEN_OPIV_TRANS(NAME, CHECK) \ static bool trans_##NAME(DisasContext *s, arg_rmr * a) \ { \ diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index ecf21c50f8..7d2a355418 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -125,3 +125,22 @@ GEN_VEXT_V(vrev8_v_b, 1) GEN_VEXT_V(vrev8_v_h, 2) GEN_VEXT_V(vrev8_v_w, 4) GEN_VEXT_V(vrev8_v_d, 8) + +#define DO_ANDN(a, b) ((a) & ~(b)) +RVVCALL(OPIVV2, vandn_vv_b, OP_UUU_B, H1, H1, H1, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_h, OP_UUU_H, H2, H2, H2, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_w, OP_UUU_W, H4, H4, H4, DO_ANDN) +RVVCALL(OPIVV2, vandn_vv_d, OP_UUU_D, H8, H8, H8, DO_ANDN) +GEN_VEXT_VV(vandn_vv_b, 1) +GEN_VEXT_VV(vandn_vv_h, 2) +GEN_VEXT_VV(vandn_vv_w, 4) +GEN_VEXT_VV(vandn_vv_d, 8) + +RVVCALL(OPIVX2, vandn_vx_b, OP_UUU_B, H1, H1, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_h, OP_UUU_H, H2, H2, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_w, OP_UUU_W, H4, H4, DO_ANDN) +RVVCALL(OPIVX2, vandn_vx_d, OP_UUU_D, H8, H8, DO_ANDN) +GEN_VEXT_VX(vandn_vx_b, 1) +GEN_VEXT_VX(vandn_vx_h, 2) +GEN_VEXT_VX(vandn_vx_w, 4) +GEN_VEXT_VX(vandn_vx_d, 8) From patchwork Fri Mar 10 09:11:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE8D4C6FD19 for ; Fri, 10 Mar 2023 09:17:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230116AbjCJJRR (ORCPT ); Fri, 10 Mar 2023 04:17:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230371AbjCJJQp (ORCPT ); Fri, 10 Mar 2023 04:16:45 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EDF410A29E for ; Fri, 10 Mar 2023 01:12:43 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnb-00GpVx-I0; Fri, 10 Mar 2023 09:12:27 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 15/45] target/riscv: Expose zvkb cpu property Date: Fri, 10 Mar 2023 09:11:45 +0000 Message-Id: <20230310091215.931644-16-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 69611408f9..462615140c 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1469,6 +1469,9 @@ static Property riscv_cpu_extensions[] = { DEFINE_PROP_BOOL("x-zvfh", RISCVCPU, cfg.ext_zvfh, false), DEFINE_PROP_BOOL("x-zvfhmin", RISCVCPU, cfg.ext_zvfhmin, false), + /* Vector cryptography extensions */ + DEFINE_PROP_BOOL("x-zvkb", RISCVCPU, cfg.ext_zvkb, false), + DEFINE_PROP_END_OF_LIST(), }; From patchwork Fri Mar 10 09:11:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E22AFC64EC4 for ; Fri, 10 Mar 2023 09:17:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229891AbjCJJRN (ORCPT ); Fri, 10 Mar 2023 04:17:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231231AbjCJJQp (ORCPT ); Fri, 10 Mar 2023 04:16:45 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EEAA10A2A2 for ; Fri, 10 Mar 2023 01:12:43 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnb-00GpVx-PT; Fri, 10 Mar 2023 09:12:27 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 16/45] target/riscv: Add zvkned cpu property Date: Fri, 10 Mar 2023 09:11:46 +0000 Message-Id: <20230310091215.931644-17-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 3 ++- target/riscv/cpu.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 462615140c..00e1d007a4 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -110,6 +110,7 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zvfh, true, PRIV_VERSION_1_12_0, ext_zvfh), ISA_EXT_DATA_ENTRY(zvfhmin, true, PRIV_VERSION_1_12_0, ext_zvfhmin), ISA_EXT_DATA_ENTRY(zvkb, true, PRIV_VERSION_1_12_0, ext_zvkb), + ISA_EXT_DATA_ENTRY(zvkned, true, PRIV_VERSION_1_12_0, ext_zvkned), ISA_EXT_DATA_ENTRY(zhinx, true, PRIV_VERSION_1_12_0, ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia), @@ -1216,7 +1217,7 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) * In principle zve*x would also suffice here, were they supported * in qemu */ - if (cpu->cfg.ext_zvkb && + if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg( diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 7d6699f718..4f3b97e0f1 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -471,6 +471,7 @@ struct RISCVCPUConfig { bool ext_zve64f; bool ext_zve64d; bool ext_zvkb; + bool ext_zvkned; bool ext_zmmul; bool ext_zvfh; bool ext_zvfhmin; From patchwork Fri Mar 10 09:11:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A067C74A44 for ; Fri, 10 Mar 2023 09:17:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231573AbjCJJRi (ORCPT ); Fri, 10 Mar 2023 04:17:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60448 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230417AbjCJJQt (ORCPT ); Fri, 10 Mar 2023 04:16:49 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1DC210A2BC for ; Fri, 10 Mar 2023 01:12:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnc-00GpVx-1L; Fri, 10 Mar 2023 09:12:28 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 17/45] target/riscv: Add vaesef.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:47 +0000 Message-Id: <20230310091215.931644-18-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 3 + target/riscv/insn32.decode | 4 ++ target/riscv/insn_trans/trans_rvzvkned.c.inc | 72 +++++++++++++++++++ target/riscv/op_helper.c | 5 ++ target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 73 ++++++++++++++++++++ 6 files changed, 158 insertions(+) create mode 100644 target/riscv/insn_trans/trans_rvzvkned.c.inc diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 1bc10ca5be..a402bb5d40 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1,5 +1,6 @@ /* Exceptions */ DEF_HELPER_2(raise_exception, noreturn, env, i32) +DEF_HELPER_2(restore_cpu_and_raise_exception, noreturn, env, i32) /* Floating Point - rounding mode */ DEF_HELPER_FLAGS_2(set_rounding_mode, TCG_CALL_NO_WG, void, env, i32) @@ -1186,3 +1187,5 @@ DEF_HELPER_6(vandn_vx_b, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_h, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_w, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) + +DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index bbf128e4dc..f68025755a 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -75,6 +75,7 @@ @r_rm ....... ..... ..... ... ..... ....... %rs2 %rs1 %rm %rd @r2_rm ....... ..... ..... ... ..... ....... %rs1 %rm %rd @r2 ....... ..... ..... ... ..... ....... &r2 %rs1 %rd +@r2_vm_1 ...... . ..... ..... ... ..... ....... &rmr vm=1 %rs2 %rd @r2_nfvm ... ... vm:1 ..... ..... ... ..... ....... &r2nfvm %nf %rs1 %rd @r2_vm ...... vm:1 ..... ..... ... ..... ....... &rmr %rs2 %rd @r1_vm ...... vm:1 ..... ..... ... ..... ....... %rd @@ -925,3 +926,6 @@ vbrev8_v 010010 . ..... 01000 010 ..... 1010111 @r2_vm vrev8_v 010010 . ..... 01001 010 ..... 1010111 @r2_vm vandn_vv 000001 . ..... ..... 000 ..... 1010111 @r_vm vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm + +# *** RV64 Zvkned vector crypto extension *** +vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc new file mode 100644 index 0000000000..6f3d62bef1 --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -0,0 +1,72 @@ +/* + * RISC-V translation routines for the Zvkned Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ +#define GEN_V_UNMASKED_TRANS(NAME, CHECK) \ +static bool trans_##NAME(DisasContext *s, arg_##NAME * a) \ +{ \ + if (CHECK(s, a)) { \ + TCGv_ptr rd_v, rs2_v; \ + TCGv_i32 desc; \ + uint32_t data = 0; \ + TCGLabel *over = gen_new_label(); \ + TCGLabel *vl_ok = gen_new_label(); \ + TCGv_i32 tmp = tcg_temp_new_i32(); \ + \ + /* save opcode for unwinding in case we throw an exception */ \ + decode_save_opc(s); \ + \ + /* check (vl % 4 == 0) */ \ + tcg_gen_trunc_tl_i32(tmp, cpu_vl); \ + tcg_gen_andi_i32(tmp, tmp, 0b11); \ + tcg_gen_brcondi_i32(TCG_COND_EQ, tmp, 0, vl_ok); \ + gen_helper_restore_cpu_and_raise_exception(cpu_env, \ + tcg_constant_i32(RISCV_EXCP_ILLEGAL_INST)); \ + gen_set_label(vl_ok); \ + \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + rd_v = tcg_temp_new_ptr(); \ + rs2_v = tcg_temp_new_ptr(); \ + desc = tcg_constant_i32(simd_desc(s->cfg_ptr->vlen / 8, \ + s->cfg_ptr->vlen / 8, data)); \ + tcg_gen_addi_ptr(rd_v, cpu_env, vreg_ofs(s, a->rd)); \ + tcg_gen_addi_ptr(rs2_v, cpu_env, vreg_ofs(s, a->rs2)); \ + gen_helper_##NAME(rd_v, rs2_v, cpu_env, desc); \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ +} + + +static bool vaes_check_vv(DisasContext *s, arg_rmr *a) +{ + return s->cfg_ptr->ext_zvkned == true && + require_rvv(s) && + vext_check_isa_ill(s) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + require_align(a->rd, s->lmul) && + require_align(a->rs2, s->lmul) && + s->vstart % 4 == 0 && s->sew == MO_32; +} +GEN_V_UNMASKED_TRANS(vaesef_vv, vaes_check_vv) diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c index 84ee018f7d..f26c6396a5 100644 --- a/target/riscv/op_helper.c +++ b/target/riscv/op_helper.c @@ -38,6 +38,11 @@ void helper_raise_exception(CPURISCVState *env, uint32_t exception) riscv_raise_exception(env, exception, 0); } +void helper_restore_cpu_and_raise_exception(CPURISCVState *env, uint32_t exception) +{ + riscv_raise_exception(env, exception, GETPC()); +} + target_ulong helper_csrr(CPURISCVState *env, int csr) { /* diff --git a/target/riscv/translate.c b/target/riscv/translate.c index 939f5aeb1c..e3561b0bbd 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1084,6 +1084,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvzfh.c.inc" #include "insn_trans/trans_rvk.c.inc" #include "insn_trans/trans_rvzvkb.c.inc" +#include "insn_trans/trans_rvzvkned.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 7d2a355418..bb5ce5b50a 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -22,6 +22,7 @@ #include "qemu/bitops.h" #include "qemu/bswap.h" #include "cpu.h" +#include "crypto/aes.h" #include "exec/memop.h" #include "exec/exec-all.h" #include "exec/helper-proto.h" @@ -144,3 +145,75 @@ GEN_VEXT_VX(vandn_vx_b, 1) GEN_VEXT_VX(vandn_vx_h, 2) GEN_VEXT_VX(vandn_vx_w, 4) GEN_VEXT_VX(vandn_vx_d, 8) + +static inline void aes_sub_bytes(uint8_t round_state[4][4]) +{ + for (int j = 0; j < 16; j++) { + round_state[j / 4][j % 4] = AES_sbox[round_state[j / 4][j % 4]]; + } +} + +static inline void aes_shift_bytes(uint8_t round_state[4][4]) +{ + uint8_t temp; + temp = round_state[0][1]; + round_state[0][1] = round_state[1][1]; + round_state[1][1] = round_state[2][1]; + round_state[2][1] = round_state[3][1]; + round_state[3][1] = temp; + temp = round_state[0][2]; + round_state[0][2] = round_state[2][2]; + round_state[2][2] = temp; + temp = round_state[1][2]; + round_state[1][2] = round_state[3][2]; + round_state[3][2] = temp; + temp = round_state[0][3]; + round_state[0][3] = round_state[3][3]; + round_state[3][3] = round_state[2][3]; + round_state[2][3] = round_state[1][3]; + round_state[1][3] = temp; +} + +static inline void xor_round_key(uint8_t round_state[4][4], uint8_t *round_key) +{ + for (int j = 0; j < 16; j++) { + round_state[j / 4][j % 4] = round_state[j / 4][j % 4] ^ (round_key)[j]; + } +} + +#define GEN_ZVKNED_HELPER_VV(NAME, ...) \ +void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + uint64_t *vd = vd_vptr; \ + uint64_t *vs2 = vs2_vptr; \ + uint32_t vl = env->vl; \ + uint32_t total_elems = vext_get_total_elems(env, desc, 4); \ + uint32_t vta = vext_vta(desc); \ + \ + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { \ + uint64_t round_key[2] = { \ + cpu_to_le64(vs2[i * 2 + 0]), \ + cpu_to_le64(vs2[i * 2 + 1]), \ + }; \ + uint8_t round_state[4][4]; \ + cpu_to_le64s(vd + i * 2 + 0); \ + cpu_to_le64s(vd + i * 2 + 1); \ + for (int j = 0; j < 16; j++) { \ + round_state[j / 4][j % 4] = ((uint8_t *)(vd + i * 2))[j]; \ + } \ + __VA_ARGS__; \ + for (int j = 0; j < 16; j++) { \ + ((uint8_t *)(vd + i * 2))[j] = round_state[j / 4][j % 4]; \ + } \ + le64_to_cpus(vd + i * 2 + 0); \ + le64_to_cpus(vd + i * 2 + 1); \ + } \ + env->vstart = 0; \ + /* set tail elements to 1s */ \ + vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); \ +} + +GEN_ZVKNED_HELPER_VV(vaesef_vv, aes_sub_bytes(round_state); + aes_shift_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) From patchwork Fri Mar 10 09:11:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168944 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51AB3C6FD1C for ; Fri, 10 Mar 2023 09:17:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231542AbjCJJRh (ORCPT ); Fri, 10 Mar 2023 04:17:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230411AbjCJJQs (ORCPT ); Fri, 10 Mar 2023 04:16:48 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AEA210A2BB for ; Fri, 10 Mar 2023 01:12:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnc-00GpVx-9n; Fri, 10 Mar 2023 09:12:28 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 18/45] target/riscv: Add vaesef.vs decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:48 +0000 Message-Id: <20230310091215.931644-19-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 20 +++++++++++ target/riscv/vcrypto_helper.c | 36 ++++++++++++++++++++ 4 files changed, 58 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index a402bb5d40..fb30b4d13e 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1189,3 +1189,4 @@ DEF_HELPER_6(vandn_vx_w, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index f68025755a..5d1bb6ccc6 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -929,3 +929,4 @@ vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm # *** RV64 Zvkned vector crypto extension *** vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 +vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 6f3d62bef1..69bf7f9fee 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -69,4 +69,24 @@ static bool vaes_check_vv(DisasContext *s, arg_rmr *a) require_align(a->rs2, s->lmul) && s->vstart % 4 == 0 && s->sew == MO_32; } + +static bool vaes_check_overlap(DisasContext *s, int vd, int vs2) +{ + int8_t op_size = s->lmul <= 0 ? 1 : 1 << s->lmul; + return !is_overlapped(vd, op_size, vs2, 1); +} + +static bool vaes_check_vs(DisasContext *s, arg_rmr *a) +{ + return vaes_check_overlap(s, a->rd, a->rs2) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + s->cfg_ptr->ext_zvkned == true && + require_rvv(s) && + vext_check_isa_ill(s) && + require_align(a->rd, s->lmul) && + s->vstart % 4 == 0 && + s->sew == MO_32; +} + GEN_V_UNMASKED_TRANS(vaesef_vv, vaes_check_vv) +GEN_V_UNMASKED_TRANS(vaesef_vs, vaes_check_vs) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index bb5ce5b50a..b079b543c7 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -214,6 +214,42 @@ void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); \ } +#define GEN_ZVKNED_HELPER_VS(NAME, ...) \ +void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ + uint32_t desc) \ +{ \ + uint64_t *vd = vd_vptr; \ + uint64_t *vs2 = vs2_vptr; \ + uint32_t vl = env->vl; \ + uint32_t total_elems = vext_get_total_elems(env, desc, 4); \ + uint32_t vta = vext_vta(desc); \ + \ + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { \ + uint64_t round_key[2] = { \ + cpu_to_le64(vs2[0]), \ + cpu_to_le64(vs2[1]), \ + }; \ + uint8_t round_state[4][4]; \ + cpu_to_le64s(vd + i * 2 + 0); \ + cpu_to_le64s(vd + i * 2 + 1); \ + for (int j = 0; j < 16; j++) { \ + round_state[j / 4][j % 4] = ((uint8_t *)(vd + i * 2))[j]; \ + } \ + __VA_ARGS__; \ + for (int j = 0; j < 16; j++) { \ + ((uint8_t *)(vd + i * 2))[j] = round_state[j / 4][j % 4]; \ + } \ + le64_to_cpus(vd + i * 2 + 0); \ + le64_to_cpus(vd + i * 2 + 1); \ + } \ + env->vstart = 0; \ + /* set tail elements to 1s */ \ + vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); \ +} + GEN_ZVKNED_HELPER_VV(vaesef_vv, aes_sub_bytes(round_state); aes_shift_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VS(vaesef_vs, aes_sub_bytes(round_state); + aes_shift_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) From patchwork Fri Mar 10 09:11:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78EB6C6FD1E for ; Fri, 10 Mar 2023 09:17:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230330AbjCJJRV (ORCPT ); Fri, 10 Mar 2023 04:17:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230219AbjCJJQq (ORCPT ); Fri, 10 Mar 2023 04:16:46 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 092C110A2B1 for ; Fri, 10 Mar 2023 01:12:44 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnc-00GpVx-Hn; Fri, 10 Mar 2023 09:12:28 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 19/45] target/riscv: Add vaesdf.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:49 +0000 Message-Id: <20230310091215.931644-20-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 31 ++++++++++++++++++++ 4 files changed, 34 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index fb30b4d13e..c626ddcee1 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1190,3 +1190,4 @@ DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 5d1bb6ccc6..e9ccc56915 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -930,3 +930,4 @@ vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm # *** RV64 Zvkned vector crypto extension *** vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 +vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 69bf7f9fee..134fd59df8 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -90,3 +90,4 @@ static bool vaes_check_vs(DisasContext *s, arg_rmr *a) GEN_V_UNMASKED_TRANS(vaesef_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesef_vs, vaes_check_vs) +GEN_V_UNMASKED_TRANS(vaesdf_vv, vaes_check_vv) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index b079b543c7..8f448de86e 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -181,6 +181,34 @@ static inline void xor_round_key(uint8_t round_state[4][4], uint8_t *round_key) } } +static inline void aes_inv_sub_bytes(uint8_t round_state[4][4]) +{ + for (int j = 0; j < 16; j++) { + round_state[j / 4][j % 4] = AES_isbox[round_state[j / 4][j % 4]]; + } +} + +static inline void aes_inv_shift_bytes(uint8_t round_state[4][4]) +{ + uint8_t temp; + temp = round_state[3][1]; + round_state[3][1] = round_state[2][1]; + round_state[2][1] = round_state[1][1]; + round_state[1][1] = round_state[0][1]; + round_state[0][1] = temp; + temp = round_state[0][2]; + round_state[0][2] = round_state[2][2]; + round_state[2][2] = temp; + temp = round_state[1][2]; + round_state[1][2] = round_state[3][2]; + round_state[3][2] = temp; + temp = round_state[0][3]; + round_state[0][3] = round_state[1][3]; + round_state[1][3] = round_state[2][3]; + round_state[2][3] = round_state[3][3]; + round_state[3][3] = temp; +} + #define GEN_ZVKNED_HELPER_VV(NAME, ...) \ void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ uint32_t desc) \ @@ -253,3 +281,6 @@ GEN_ZVKNED_HELPER_VV(vaesef_vv, aes_sub_bytes(round_state); GEN_ZVKNED_HELPER_VS(vaesef_vs, aes_sub_bytes(round_state); aes_shift_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VV(vaesdf_vv, aes_inv_shift_bytes(round_state); + aes_inv_sub_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) From patchwork Fri Mar 10 09:11:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC238C64EC4 for ; Fri, 10 Mar 2023 09:17:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231391AbjCJJRp (ORCPT ); Fri, 10 Mar 2023 04:17:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35904 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230447AbjCJJQu (ORCPT ); Fri, 10 Mar 2023 04:16:50 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E084010A4CD for ; Fri, 10 Mar 2023 01:12:48 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnc-00GpVx-Py; Fri, 10 Mar 2023 09:12:28 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 20/45] target/riscv: Add vaesdf.vs decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:50 +0000 Message-Id: <20230310091215.931644-21-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 3 +++ 4 files changed, 6 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index c626ddcee1..059d63b0ea 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1191,3 +1191,4 @@ DEF_HELPER_6(vandn_vx_d, void, ptr, ptr, tl, ptr, env, i32) DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index e9ccc56915..df1ae7425d 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -931,3 +931,4 @@ vandn_vx 000001 . ..... ..... 100 ..... 1010111 @r_vm vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 +vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 134fd59df8..7a14c33e01 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -91,3 +91,4 @@ static bool vaes_check_vs(DisasContext *s, arg_rmr *a) GEN_V_UNMASKED_TRANS(vaesef_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesef_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdf_vv, vaes_check_vv) +GEN_V_UNMASKED_TRANS(vaesdf_vs, vaes_check_vs) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 8f448de86e..69e0843ae4 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -284,3 +284,6 @@ GEN_ZVKNED_HELPER_VS(vaesef_vs, aes_sub_bytes(round_state); GEN_ZVKNED_HELPER_VV(vaesdf_vv, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VS(vaesdf_vs, aes_inv_shift_bytes(round_state); + aes_inv_sub_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) From patchwork Fri Mar 10 09:11:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D552C6FD19 for ; Fri, 10 Mar 2023 09:17:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231483AbjCJJRd (ORCPT ); Fri, 10 Mar 2023 04:17:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229742AbjCJJQs (ORCPT ); Fri, 10 Mar 2023 04:16:48 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 301EE10A2BE for ; Fri, 10 Mar 2023 01:12:47 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnd-00GpVx-1h; Fri, 10 Mar 2023 09:12:29 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 21/45] target/riscv: Add vaesdm.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:51 +0000 Message-Id: <20230310091215.931644-22-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 36 ++++++++++++++++++++ 4 files changed, 39 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 059d63b0ea..9b04f90240 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1192,3 +1192,4 @@ DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index df1ae7425d..cdaa320f19 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -932,3 +932,4 @@ vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 +vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 7a14c33e01..4f1105ce1d 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -92,3 +92,4 @@ GEN_V_UNMASKED_TRANS(vaesef_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesef_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdf_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdf_vs, vaes_check_vs) +GEN_V_UNMASKED_TRANS(vaesdm_vv, vaes_check_vv) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 69e0843ae4..f3b9070768 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -209,6 +209,38 @@ static inline void aes_inv_shift_bytes(uint8_t round_state[4][4]) round_state[3][3] = temp; } +static inline uint8_t xtime(uint8_t x) +{ + return (x << 1) ^ (((x >> 7) & 1) * 0x1b); +} + +static inline uint8_t multiply(uint8_t x, uint8_t y) +{ + return (((y & 1) * x) ^ ((y >> 1 & 1) * xtime(x)) ^ + ((y >> 2 & 1) * xtime(xtime(x))) ^ + ((y >> 3 & 1) * xtime(xtime(xtime(x)))) ^ + ((y >> 4 & 1) * xtime(xtime(xtime(xtime(x)))))); +} + +static inline void aes_inv_mix_cols(uint8_t round_state[4][4]) +{ + uint8_t a, b, c, d; + for (int j = 0; j < 4; ++j) { + a = round_state[j][0]; + b = round_state[j][1]; + c = round_state[j][2]; + d = round_state[j][3]; + round_state[j][0] = multiply(a, 0x0e) ^ multiply(b, 0x0b) ^ + multiply(c, 0x0d) ^ multiply(d, 0x09); + round_state[j][1] = multiply(a, 0x09) ^ multiply(b, 0x0e) ^ + multiply(c, 0x0b) ^ multiply(d, 0x0d); + round_state[j][2] = multiply(a, 0x0d) ^ multiply(b, 0x09) ^ + multiply(c, 0x0e) ^ multiply(d, 0x0b); + round_state[j][3] = multiply(a, 0x0b) ^ multiply(b, 0x0d) ^ + multiply(c, 0x09) ^ multiply(d, 0x0e); + } +} + #define GEN_ZVKNED_HELPER_VV(NAME, ...) \ void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ uint32_t desc) \ @@ -287,3 +319,7 @@ GEN_ZVKNED_HELPER_VV(vaesdf_vv, aes_inv_shift_bytes(round_state); GEN_ZVKNED_HELPER_VS(vaesdf_vs, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VV(vaesdm_vv, aes_inv_shift_bytes(round_state); + aes_inv_sub_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key); + aes_inv_mix_cols(round_state);) From patchwork Fri Mar 10 09:11:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E990C64EC4 for ; Fri, 10 Mar 2023 09:17:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229471AbjCJJRk (ORCPT ); Fri, 10 Mar 2023 04:17:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230232AbjCJJQt (ORCPT ); Fri, 10 Mar 2023 04:16:49 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7242010A4C2 for ; Fri, 10 Mar 2023 01:12:47 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnd-00GpVx-9a; Fri, 10 Mar 2023 09:12:29 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 22/45] target/riscv: Add vaesdm.vs decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:52 +0000 Message-Id: <20230310091215.931644-23-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 4 ++++ 4 files changed, 7 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 9b04f90240..ab0d2a4225 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1193,3 +1193,4 @@ DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index cdaa320f19..61f6b81644 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -933,3 +933,4 @@ vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 +vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 4f1105ce1d..93f0df6b78 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -93,3 +93,4 @@ GEN_V_UNMASKED_TRANS(vaesef_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdf_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdf_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdm_vv, vaes_check_vv) +GEN_V_UNMASKED_TRANS(vaesdm_vs, vaes_check_vs) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index f3b9070768..04d5ce5dc0 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -323,3 +323,7 @@ GEN_ZVKNED_HELPER_VV(vaesdm_vv, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key); aes_inv_mix_cols(round_state);) +GEN_ZVKNED_HELPER_VS(vaesdm_vs, aes_inv_shift_bytes(round_state); + aes_inv_sub_bytes(round_state); + xor_round_key(round_state, (uint8_t *)round_key); + aes_inv_mix_cols(round_state);) From patchwork Fri Mar 10 09:11:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E2C2C6FD19 for ; Fri, 10 Mar 2023 09:17:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229598AbjCJJRm (ORCPT ); Fri, 10 Mar 2023 04:17:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230428AbjCJJQt (ORCPT ); Fri, 10 Mar 2023 04:16:49 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AC8A910A4C3 for ; Fri, 10 Mar 2023 01:12:47 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnd-00GpVx-Ha; Fri, 10 Mar 2023 09:12:29 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 23/45] target/riscv: Add vaesz.vs decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:53 +0000 Message-Id: <20230310091215.931644-24-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 2 ++ 4 files changed, 5 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index ab0d2a4225..58121ba8ad 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1194,3 +1194,4 @@ DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 61f6b81644..22059ef95b 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -934,3 +934,4 @@ vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 +vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 93f0df6b78..1f59dbcc68 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -94,3 +94,4 @@ GEN_V_UNMASKED_TRANS(vaesdf_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdf_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdm_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdm_vs, vaes_check_vs) +GEN_V_UNMASKED_TRANS(vaesz_vs, vaes_check_vs) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 04d5ce5dc0..41e138ece5 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -327,3 +327,5 @@ GEN_ZVKNED_HELPER_VS(vaesdm_vs, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key); aes_inv_mix_cols(round_state);) +GEN_ZVKNED_HELPER_VS(vaesz_vs, + xor_round_key(round_state, (uint8_t *)round_key);) From patchwork Fri Mar 10 09:11:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168938 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CD26C6FD1C for ; Fri, 10 Mar 2023 09:17:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231436AbjCJJRX (ORCPT ); Fri, 10 Mar 2023 04:17:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230381AbjCJJQq (ORCPT ); Fri, 10 Mar 2023 04:16:46 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B15A910A2B5 for ; Fri, 10 Mar 2023 01:12:45 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnd-00GpVx-PV; Fri, 10 Mar 2023 09:12:29 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, William Salmon Subject: [PATCH 24/45] target/riscv: Add vaesem.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:54 +0000 Message-Id: <20230310091215.931644-25-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: William Salmon Signed-off-by: William Salmon --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 17 +++++++++++++++++ 4 files changed, 20 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 58121ba8ad..c55d59dc5e 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1192,6 +1192,7 @@ DEF_HELPER_4(vaesef_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesem_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 22059ef95b..f03b41f9e2 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -932,6 +932,7 @@ vaesef_vv 101000 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 +vaesem_vv 101000 1 ..... 00010 010 ..... 1110111 @r2_vm_1 vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 1f59dbcc68..0b54d6e9d3 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -95,3 +95,4 @@ GEN_V_UNMASKED_TRANS(vaesdf_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesdm_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdm_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesz_vs, vaes_check_vs) +GEN_V_UNMASKED_TRANS(vaesem_vv, vaes_check_vv) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 41e138ece5..beef7699db 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -241,6 +241,20 @@ static inline void aes_inv_mix_cols(uint8_t round_state[4][4]) } } +static inline void aes_mix_cols(uint8_t round_state[4][4]) +{ + uint8_t a, b; + for (int j = 0; j < 4; ++j) { + a = round_state[j][0]; + b = round_state[j][0] ^ round_state[j][1] ^ round_state[j][2] ^ + round_state[j][3]; + round_state[j][0] ^= xtime(round_state[j][0] ^ round_state[j][1]) ^ b; + round_state[j][1] ^= xtime(round_state[j][1] ^ round_state[j][2]) ^ b; + round_state[j][2] ^= xtime(round_state[j][2] ^ round_state[j][3]) ^ b; + round_state[j][3] ^= xtime(round_state[j][3] ^ a) ^ b; + } +} + #define GEN_ZVKNED_HELPER_VV(NAME, ...) \ void HELPER(NAME)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, \ uint32_t desc) \ @@ -319,6 +333,9 @@ GEN_ZVKNED_HELPER_VV(vaesdf_vv, aes_inv_shift_bytes(round_state); GEN_ZVKNED_HELPER_VS(vaesdf_vs, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VV(vaesem_vv, aes_shift_bytes(round_state); + aes_sub_bytes(round_state); aes_mix_cols(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) GEN_ZVKNED_HELPER_VV(vaesdm_vv, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key); From patchwork Fri Mar 10 09:11:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6034C64EC4 for ; Fri, 10 Mar 2023 09:17:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230420AbjCJJRb (ORCPT ); Fri, 10 Mar 2023 04:17:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35480 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230410AbjCJJQs (ORCPT ); Fri, 10 Mar 2023 04:16:48 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6BFB710A2BA for ; Fri, 10 Mar 2023 01:12:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYne-00GpVx-1B; Fri, 10 Mar 2023 09:12:30 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, William Salmon Subject: [PATCH 25/45] target/riscv: Add vaesem.vs decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:55 +0000 Message-Id: <20230310091215.931644-26-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: William Salmon Signed-off-by: William Salmon --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 1 + target/riscv/vcrypto_helper.c | 3 +++ 4 files changed, 6 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index c55d59dc5e..946ae8c51d 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1193,6 +1193,7 @@ DEF_HELPER_4(vaesef_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdf_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesem_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vaesem_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index f03b41f9e2..3187c5cc64 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -933,6 +933,7 @@ vaesef_vs 101001 1 ..... 00011 010 ..... 1110111 @r2_vm_1 vaesdf_vv 101000 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesdf_vs 101001 1 ..... 00001 010 ..... 1110111 @r2_vm_1 vaesem_vv 101000 1 ..... 00010 010 ..... 1110111 @r2_vm_1 +vaesem_vs 101001 1 ..... 00010 010 ..... 1110111 @r2_vm_1 vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 0b54d6e9d3..028f04a4d7 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -96,3 +96,4 @@ GEN_V_UNMASKED_TRANS(vaesdm_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesdm_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesz_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesem_vv, vaes_check_vv) +GEN_V_UNMASKED_TRANS(vaesem_vs, vaes_check_vs) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index beef7699db..600069adb1 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -336,6 +336,9 @@ GEN_ZVKNED_HELPER_VS(vaesdf_vs, aes_inv_shift_bytes(round_state); GEN_ZVKNED_HELPER_VV(vaesem_vv, aes_shift_bytes(round_state); aes_sub_bytes(round_state); aes_mix_cols(round_state); xor_round_key(round_state, (uint8_t *)round_key);) +GEN_ZVKNED_HELPER_VS(vaesem_vs, aes_shift_bytes(round_state); + aes_sub_bytes(round_state); aes_mix_cols(round_state); + xor_round_key(round_state, (uint8_t *)round_key);) GEN_ZVKNED_HELPER_VV(vaesdm_vv, aes_inv_shift_bytes(round_state); aes_inv_sub_bytes(round_state); xor_round_key(round_state, (uint8_t *)round_key); From patchwork Fri Mar 10 09:11:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168952 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79D7CC64EC4 for ; Fri, 10 Mar 2023 09:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230391AbjCJJRx (ORCPT ); Fri, 10 Mar 2023 04:17:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230459AbjCJJQy (ORCPT ); Fri, 10 Mar 2023 04:16:54 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8F32F186F for ; Fri, 10 Mar 2023 01:12:50 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYne-00GpVx-8i; Fri, 10 Mar 2023 09:12:30 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 26/45] target/riscv: Add vaeskf1.vi decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:56 +0000 Message-Id: <20230310091215.931644-27-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 58 ++++++++++++++++++++ target/riscv/vcrypto_helper.c | 44 +++++++++++++++ 4 files changed, 104 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 946ae8c51d..e68ced7796 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1197,3 +1197,4 @@ DEF_HELPER_4(vaesem_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) +DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 3187c5cc64..0b3146c4f4 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -937,3 +937,4 @@ vaesem_vs 101001 1 ..... 00010 010 ..... 1110111 @r2_vm_1 vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 +vaeskf1_vi 100010 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index 028f04a4d7..c97780f468 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -97,3 +97,61 @@ GEN_V_UNMASKED_TRANS(vaesdm_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesz_vs, vaes_check_vs) GEN_V_UNMASKED_TRANS(vaesem_vv, vaes_check_vv) GEN_V_UNMASKED_TRANS(vaesem_vs, vaes_check_vs) + +#define GEN_VI_UNMASKED_TRANS(NAME, CHECK, VL_MULTIPLE) \ +static bool trans_##NAME(DisasContext *s, arg_##NAME * a) \ +{ \ + if (CHECK(s, a)) { \ + TCGv_ptr rd_v, rs2_v; \ + TCGv_i32 uimm_v, desc; \ + uint32_t data = 0; \ + TCGLabel *over = gen_new_label(); \ + TCGLabel *vl_ok = gen_new_label(); \ + TCGv_i32 tmp = tcg_temp_new_i32(); \ + \ + /* save opcode for unwinding in case we throw an exception */ \ + decode_save_opc(s); \ + \ + /* check (vl % VL_MULTIPLE == 0) assuming it's power of 2 */ \ + tcg_gen_trunc_tl_i32(tmp, cpu_vl); \ + tcg_gen_andi_i32(tmp, tmp, VL_MULTIPLE - 1); \ + tcg_gen_brcondi_i32(TCG_COND_EQ, tmp, 0, vl_ok); \ + gen_helper_restore_cpu_and_raise_exception(cpu_env, \ + tcg_constant_i32(RISCV_EXCP_ILLEGAL_INST)); \ + gen_set_label(vl_ok); \ + \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + \ + rd_v = tcg_temp_new_ptr(); \ + rs2_v = tcg_temp_new_ptr(); \ + uimm_v = tcg_constant_i32(a->rs1); \ + desc = tcg_constant_i32(simd_desc(s->cfg_ptr->vlen / 8, \ + s->cfg_ptr->vlen / 8, data)); \ + tcg_gen_addi_ptr(rd_v, cpu_env, vreg_ofs(s, a->rd)); \ + tcg_gen_addi_ptr(rs2_v, cpu_env, vreg_ofs(s, a->rs2)); \ + gen_helper_##NAME(rd_v, rs2_v, uimm_v, cpu_env, desc); \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ +} + +static bool vaeskf1_check(DisasContext *s, arg_vaeskf1_vi * a) +{ + return s->cfg_ptr->ext_zvkned == true && + require_rvv(s) && + vext_check_isa_ill(s) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + s->vstart % 4 == 0 && + s->sew == MO_32 && + require_align(a->rd, s->lmul) && + require_align(a->rs2, s->lmul); +} + +GEN_VI_UNMASKED_TRANS(vaeskf1_vi, vaeskf1_check, 4) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 600069adb1..619e7df0fc 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -349,3 +349,47 @@ GEN_ZVKNED_HELPER_VS(vaesdm_vs, aes_inv_shift_bytes(round_state); aes_inv_mix_cols(round_state);) GEN_ZVKNED_HELPER_VS(vaesz_vs, xor_round_key(round_state, (uint8_t *)round_key);) + +void HELPER(vaeskf1_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, + CPURISCVState *env, uint32_t desc) +{ + uint32_t *vd = vd_vptr; + uint32_t *vs2 = vs2_vptr; + uint32_t vl = env->vl; + uint32_t total_elems = vext_get_total_elems(env, desc, 4); + uint32_t vta = vext_vta(desc); + + uimm &= 0b1111; + if (uimm > 10 || uimm == 0) { + uimm ^= 0b1000; + } + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + uint32_t rk[8]; + static const uint32_t rcon[] = { + 0x01000000, 0x02000000, 0x04000000, 0x08000000, 0x10000000, + 0x20000000, 0x40000000, 0x80000000, 0x1B000000, 0x36000000, + }; + + rk[0] = bswap32(vs2[i * 4 + H4(0)]); + rk[1] = bswap32(vs2[i * 4 + H4(1)]); + rk[2] = bswap32(vs2[i * 4 + H4(2)]); + rk[3] = bswap32(vs2[i * 4 + H4(3)]); + + rk[4] = rk[0] ^ (AES_Te4[(rk[3] >> 16) & 0xff] & 0xff000000) ^ + (AES_Te4[(rk[3] >> 8) & 0xff] & 0x00ff0000) ^ + (AES_Te4[(rk[3] >> 0) & 0xff] & 0x0000ff00) ^ + (AES_Te4[(rk[3] >> 24) & 0xff] & 0x000000ff) ^ rcon[uimm - 1]; + rk[5] = rk[1] ^ rk[4]; + rk[6] = rk[2] ^ rk[5]; + rk[7] = rk[3] ^ rk[6]; + + vd[i * 4 + H4(0)] = bswap32(rk[4]); + vd[i * 4 + H4(1)] = bswap32(rk[5]); + vd[i * 4 + H4(2)] = bswap32(rk[6]); + vd[i * 4 + H4(3)] = bswap32(rk[7]); + } + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); +} From patchwork Fri Mar 10 09:11:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53544C6FD19 for ; Fri, 10 Mar 2023 09:17:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231418AbjCJJRr (ORCPT ); Fri, 10 Mar 2023 04:17:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230085AbjCJJQv (ORCPT ); Fri, 10 Mar 2023 04:16:51 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00E7D10A4DB for ; Fri, 10 Mar 2023 01:12:49 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYne-00GpVx-Fq; Fri, 10 Mar 2023 09:12:30 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 27/45] target/riscv: Add vaeskf2.vi decoding, translation and execution support Date: Fri, 10 Mar 2023 09:11:57 +0000 Message-Id: <20230310091215.931644-28-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkned.c.inc | 13 +++++ target/riscv/vcrypto_helper.c | 59 ++++++++++++++++++++ 4 files changed, 74 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index e68ced7796..f07f261b7b 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1198,3 +1198,4 @@ DEF_HELPER_4(vaesdm_vv, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) +DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 0b3146c4f4..43dfd63e0d 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -938,3 +938,4 @@ vaesdm_vv 101000 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 vaeskf1_vi 100010 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkned.c.inc b/target/riscv/insn_trans/trans_rvzvkned.c.inc index c97780f468..c135b8f768 100644 --- a/target/riscv/insn_trans/trans_rvzvkned.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkned.c.inc @@ -154,4 +154,17 @@ static bool vaeskf1_check(DisasContext *s, arg_vaeskf1_vi * a) require_align(a->rs2, s->lmul); } +static bool vaeskf2_check(DisasContext *s, arg_vaeskf2_vi *a) +{ + return s->cfg_ptr->ext_zvkned == true && + require_rvv(s) && + vext_check_isa_ill(s) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + s->vstart % 4 == 0 && + s->sew == MO_32 && + require_align(a->rd, s->lmul) && + require_align(a->rs2, s->lmul); +} + GEN_VI_UNMASKED_TRANS(vaeskf1_vi, vaeskf1_check, 4) +GEN_VI_UNMASKED_TRANS(vaeskf2_vi, vaeskf2_check, 4) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 619e7df0fc..4a50178676 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -393,3 +393,62 @@ void HELPER(vaeskf1_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, /* set tail elements to 1s */ vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); } + +void HELPER(vaeskf2_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, + CPURISCVState *env, uint32_t desc) +{ + uint32_t *vd = vd_vptr; + uint32_t *vs2 = vs2_vptr; + uint32_t vl = env->vl; + uint32_t total_elems = vext_get_total_elems(env, desc, 4); + uint32_t vta = vext_vta(desc); + + uimm &= 0b1111; + if (uimm > 14 || uimm < 2) { + uimm ^= 0b1000; + } + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + uint32_t rk[12]; + static const uint32_t rcon[] = { + 0x01000000, 0x02000000, 0x04000000, 0x08000000, 0x10000000, + 0x20000000, 0x40000000, 0x80000000, 0x1B000000, 0x36000000, + }; + + rk[0] = bswap32(vd[i * 4 + H4(0)]); + rk[1] = bswap32(vd[i * 4 + H4(1)]); + rk[2] = bswap32(vd[i * 4 + H4(2)]); + rk[3] = bswap32(vd[i * 4 + H4(3)]); + rk[4] = bswap32(vs2[i * 4 + H4(0)]); + rk[5] = bswap32(vs2[i * 4 + H4(1)]); + rk[6] = bswap32(vs2[i * 4 + H4(2)]); + rk[7] = bswap32(vs2[i * 4 + H4(3)]); + + if (uimm % 2 == 0) { + rk[8] = rk[0] ^ (AES_Te4[(rk[7] >> 16) & 0xff] & 0xff000000) ^ + (AES_Te4[(rk[7] >> 8) & 0xff] & 0x00ff0000) ^ + (AES_Te4[(rk[7] >> 0) & 0xff] & 0x0000ff00) ^ + (AES_Te4[(rk[7] >> 24) & 0xff] & 0x000000ff) ^ + rcon[(uimm - 1) / 2]; + rk[9] = rk[1] ^ rk[8]; + rk[10] = rk[2] ^ rk[9]; + rk[11] = rk[3] ^ rk[10]; + } else { + rk[8] = rk[0] ^ (AES_Te4[(rk[7] >> 24) & 0xff] & 0xff000000) ^ + (AES_Te4[(rk[7] >> 16) & 0xff] & 0x00ff0000) ^ + (AES_Te4[(rk[7] >> 8) & 0xff] & 0x0000ff00) ^ + (AES_Te4[(rk[7] >> 0) & 0xff] & 0x000000ff); + rk[9] = rk[1] ^ rk[8]; + rk[10] = rk[2] ^ rk[9]; + rk[11] = rk[3] ^ rk[10]; + } + + vd[i * 4 + H4(0)] = bswap32(rk[8]); + vd[i * 4 + H4(1)] = bswap32(rk[9]); + vd[i * 4 + H4(2)] = bswap32(rk[10]); + vd[i * 4 + H4(3)] = bswap32(rk[11]); + } + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); +} From patchwork Fri Mar 10 09:11:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57395C64EC4 for ; Fri, 10 Mar 2023 09:17:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230240AbjCJJRt (ORCPT ); Fri, 10 Mar 2023 04:17:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230494AbjCJJQv (ORCPT ); Fri, 10 Mar 2023 04:16:51 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01F947DF81 for ; Fri, 10 Mar 2023 01:12:53 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYne-00GpVx-NG; Fri, 10 Mar 2023 09:12:30 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 28/45] target/riscv: Expose zvkned cpu property Date: Fri, 10 Mar 2023 09:11:58 +0000 Message-Id: <20230310091215.931644-29-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 00e1d007a4..cd87eec919 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1472,6 +1472,7 @@ static Property riscv_cpu_extensions[] = { /* Vector cryptography extensions */ DEFINE_PROP_BOOL("x-zvkb", RISCVCPU, cfg.ext_zvkb, false), + DEFINE_PROP_BOOL("x-zvkned", RISCVCPU, cfg.ext_zvkned, false), DEFINE_PROP_END_OF_LIST(), }; From patchwork Fri Mar 10 09:11:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13168939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 078A6C64EC4 for ; Fri, 10 Mar 2023 09:17:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231218AbjCJJRZ (ORCPT ); Fri, 10 Mar 2023 04:17:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230383AbjCJJQr (ORCPT ); Fri, 10 Mar 2023 04:16:47 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1327910A2B8 for ; Fri, 10 Mar 2023 01:12:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYne-00GpVx-V4; Fri, 10 Mar 2023 09:12:30 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 29/45] target/riscv: Add zvknh cpu properties Date: Fri, 10 Mar 2023 09:11:59 +0000 Message-Id: <20230310091215.931644-30-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 11 ++++++++++- target/riscv/cpu.h | 2 ++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index cd87eec919..3ffbdd53cc 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -111,6 +111,8 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zvfhmin, true, PRIV_VERSION_1_12_0, ext_zvfhmin), ISA_EXT_DATA_ENTRY(zvkb, true, PRIV_VERSION_1_12_0, ext_zvkb), ISA_EXT_DATA_ENTRY(zvkned, true, PRIV_VERSION_1_12_0, ext_zvkned), + ISA_EXT_DATA_ENTRY(zvknha, true, PRIV_VERSION_1_12_0, ext_zvknha), + ISA_EXT_DATA_ENTRY(zvknhb, true, PRIV_VERSION_1_12_0, ext_zvknhb), ISA_EXT_DATA_ENTRY(zhinx, true, PRIV_VERSION_1_12_0, ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia), @@ -1217,7 +1219,7 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) * In principle zve*x would also suffice here, were they supported * in qemu */ - if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned) && + if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg( @@ -1225,6 +1227,13 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) return; } + if (cpu->cfg.ext_zvknhb && + !(cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { + error_setg(errp, + "Zvknhb extension requires V or Zve64{f,d} extensions"); + return; + } + #ifndef CONFIG_USER_ONLY if (cpu->cfg.pmu_num) { if (!riscv_pmu_init(cpu, cpu->cfg.pmu_num) && cpu->cfg.ext_sscofpmf) { diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 4f3b97e0f1..5d101fc405 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -472,6 +472,8 @@ struct RISCVCPUConfig { bool ext_zve64d; bool ext_zvkb; bool ext_zvkned; + bool ext_zvknha; + bool ext_zvknhb; bool ext_zmmul; bool ext_zvfh; bool ext_zvfhmin; From patchwork Fri Mar 10 09:12:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BA85C6FD1C for ; Fri, 10 Mar 2023 09:46:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229577AbjCJJqT (ORCPT ); Fri, 10 Mar 2023 04:46:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230187AbjCJJqA (ORCPT ); Fri, 10 Mar 2023 04:46:00 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B02E21B318 for ; Fri, 10 Mar 2023 01:45:52 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnf-00GpVx-6e; Fri, 10 Mar 2023 09:12:31 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 30/45] target/riscv: Add vsha2ms.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:00 +0000 Message-Id: <20230310091215.931644-31-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Kiran Ostrolenk --- target/riscv/helper.h | 2 + target/riscv/insn32.decode | 3 + target/riscv/insn_trans/trans_rvzvknh.c.inc | 82 +++++++++++++++++++++ target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 74 +++++++++++++++++++ 5 files changed, 162 insertions(+) create mode 100644 target/riscv/insn_trans/trans_rvzvknh.c.inc diff --git a/target/riscv/helper.h b/target/riscv/helper.h index f07f261b7b..76ea2ff49b 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1199,3 +1199,5 @@ DEF_HELPER_4(vaesdm_vs, void, ptr, ptr, env, i32) DEF_HELPER_4(vaesz_vs, void, ptr, ptr, env, i32) DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) + +DEF_HELPER_5(vsha2ms_vv, void, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 43dfd63e0d..aef4d0b476 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -939,3 +939,6 @@ vaesdm_vs 101001 1 ..... 00000 010 ..... 1110111 @r2_vm_1 vaesz_vs 101001 1 ..... 00111 010 ..... 1110111 @r2_vm_1 vaeskf1_vi 100010 1 ..... ..... 010 ..... 1110111 @r_vm_1 vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** RV64 Zvknh vector crypto extension *** +vsha2ms_vv 101101 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvknh.c.inc b/target/riscv/insn_trans/trans_rvzvknh.c.inc new file mode 100644 index 0000000000..97bdf4d72f --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvknh.c.inc @@ -0,0 +1,82 @@ +/* + * RISC-V translation routines for the Zvknh Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#define GEN_VV_UNMASKED_TRANS(NAME, CHECK, VL_MULTIPLE) \ +static bool trans_##NAME(DisasContext *s, arg_rmrr * a) \ +{ \ + if (CHECK(s, a)) { \ + uint32_t data = 0; \ + TCGLabel *over = gen_new_label(); \ + TCGLabel *vl_ok = gen_new_label(); \ + TCGv_i32 tmp = tcg_temp_new_i32(); \ + \ + /* save opcode for unwinding in case we throw an exception */ \ + decode_save_opc(s); \ + \ + /* check (vl % VL_MULTIPLE == 0) assuming it's power of 2 */ \ + tcg_gen_trunc_tl_i32(tmp, cpu_vl); \ + tcg_gen_andi_i32(tmp, tmp, VL_MULTIPLE - 1); \ + tcg_gen_brcondi_i32(TCG_COND_EQ, tmp, 0, vl_ok); \ + gen_helper_restore_cpu_and_raise_exception(cpu_env, \ + tcg_constant_i32(RISCV_EXCP_ILLEGAL_INST)); \ + gen_set_label(vl_ok); \ + \ + tcg_gen_brcond_tl(TCG_COND_GEU, cpu_vstart, cpu_vl, over); \ + \ + data = FIELD_DP32(data, VDATA, VM, a->vm); \ + data = FIELD_DP32(data, VDATA, LMUL, s->lmul); \ + data = FIELD_DP32(data, VDATA, VTA, s->vta); \ + data = FIELD_DP32(data, VDATA, VTA_ALL_1S, s->cfg_vta_all_1s); \ + data = FIELD_DP32(data, VDATA, VMA, s->vma); \ + \ + tcg_gen_gvec_3_ptr(vreg_ofs(s, a->rd), \ + vreg_ofs(s, a->rs1), \ + vreg_ofs(s, a->rs2), cpu_env, \ + s->cfg_ptr->vlen / 8, \ + s->cfg_ptr->vlen / 8, data, \ + gen_helper_##NAME); \ + \ + mark_vs_dirty(s); \ + gen_set_label(over); \ + return true; \ + } \ + return false; \ +} + +static bool vsha_check_sew(DisasContext *s) +{ + return (s->cfg_ptr->ext_zvknha == true && s->sew == MO_32) || + (s->cfg_ptr->ext_zvknhb == true && + (s->sew == MO_32 || s->sew == MO_64)); +} + +static bool vsha_check(DisasContext *s, arg_rmrr *a) +{ + int egw_bytes = 4 << s->sew; + int mult = 1 << MAX(s->lmul, 0); + return opivv_check(s, a) && + vsha_check_sew(s) && + MAXSZ(s) >= egw_bytes && + !is_overlapped(a->rd, mult, a->rs1, mult) && + !is_overlapped(a->rd, mult, a->rs2, mult) && + s->vstart % 4 == 0 && + s->lmul >= 0; +} + +GEN_VV_UNMASKED_TRANS(vsha2ms_vv, vsha_check, 4) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index e3561b0bbd..1c1e36c10a 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1085,6 +1085,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvk.c.inc" #include "insn_trans/trans_rvzvkb.c.inc" #include "insn_trans/trans_rvzvkned.c.inc" +#include "insn_trans/trans_rvzvknh.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 4a50178676..ae253b3357 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -452,3 +452,77 @@ void HELPER(vaeskf2_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, /* set tail elements to 1s */ vext_set_elems_1s(vd, vta, vl * 4, total_elems * 4); } + +static inline uint32_t sig0_sha256(uint32_t x) +{ + return ror32(x, 7) ^ ror32(x, 18) ^ (x >> 3); +} + +static inline uint32_t sig1_sha256(uint32_t x) +{ + return ror32(x, 17) ^ ror32(x, 19) ^ (x >> 10); +} + +static inline uint64_t sig0_sha512(uint64_t x) +{ + return ror64(x, 1) ^ ror64(x, 8) ^ (x >> 7); +} + +static inline uint64_t sig1_sha512(uint64_t x) +{ + return ror64(x, 19) ^ ror64(x, 61) ^ (x >> 6); +} + +static inline void vsha2ms_e32(uint32_t *vd, uint32_t *vs1, uint32_t *vs2) +{ + uint32_t res[4]; + res[0] = sig1_sha256(vs1[H4(2)]) + vs2[H4(1)] + sig0_sha256(vd[H4(1)]) + + vd[H4(0)]; + res[1] = sig1_sha256(vs1[H4(3)]) + vs2[H4(2)] + sig0_sha256(vd[H4(2)]) + + vd[H4(1)]; + res[2] = sig1_sha256(res[0]) + vs2[H4(3)] + sig0_sha256(vd[H4(3)]) + + vd[H4(2)]; + res[3] = sig1_sha256(res[1]) + vs1[H4(0)] + sig0_sha256(vs2[H4(0)]) + + vd[H4(3)]; + vd[H4(3)] = res[3]; + vd[H4(2)] = res[2]; + vd[H4(1)] = res[1]; + vd[H4(0)] = res[0]; +} + +static inline void vsha2ms_e64(uint64_t *vd, uint64_t *vs1, uint64_t *vs2) +{ + uint64_t res[4]; + res[0] = sig1_sha512(vs1[2]) + vs2[1] + sig0_sha512(vd[1]) + vd[0]; + res[1] = sig1_sha512(vs1[3]) + vs2[2] + sig0_sha512(vd[2]) + vd[1]; + res[2] = sig1_sha512(res[0]) + vs2[3] + sig0_sha512(vd[3]) + vd[2]; + res[3] = sig1_sha512(res[1]) + vs1[0] + sig0_sha512(vs2[0]) + vd[3]; + vd[3] = res[3]; + vd[2] = res[2]; + vd[1] = res[1]; + vd[0] = res[0]; +} + +void HELPER(vsha2ms_vv)(void *vd, void *vs1, void *vs2, CPURISCVState *env, + uint32_t desc) +{ + uint32_t sew = FIELD_EX64(env->vtype, VTYPE, VSEW); + uint32_t esz = sew == MO_32 ? 4 : 8; + uint32_t total_elems; + uint32_t vta = vext_vta(desc); + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + if (sew == MO_32) { + vsha2ms_e32(((uint32_t *)vd) + i * 4, ((uint32_t *)vs1) + i * 4, + ((uint32_t *)vs2) + i * 4); + } else { + /* If not 32 then SEW should be 64 */ + vsha2ms_e64(((uint64_t *)vd) + i * 4, ((uint64_t *)vs1) + i * 4, + ((uint64_t *)vs2) + i * 4); + } + } + /* set tail elements to 1s */ + total_elems = vext_get_total_elems(env, desc, esz); + vext_set_elems_1s(vd, vta, env->vl * esz, total_elems * esz); + env->vstart = 0; +} From patchwork Fri Mar 10 09:12:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4FD9C74A44 for ; Fri, 10 Mar 2023 09:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229929AbjCJJlN (ORCPT ); Fri, 10 Mar 2023 04:41:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231583AbjCJJkP (ORCPT ); Fri, 10 Mar 2023 04:40:15 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B75A1517D for ; Fri, 10 Mar 2023 01:39:57 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnf-00GpVx-F2; Fri, 10 Mar 2023 09:12:31 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 31/45] target/riscv: Add vsha2c[hl].vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:01 +0000 Message-Id: <20230310091215.931644-32-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 2 + target/riscv/insn32.decode | 2 + target/riscv/insn_trans/trans_rvzvknh.c.inc | 2 + target/riscv/vcrypto_helper.c | 140 ++++++++++++++++++++ 4 files changed, 146 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 76ea2ff49b..77bbd9db56 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1201,3 +1201,5 @@ DEF_HELPER_5(vaeskf1_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vsha2ms_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vsha2ch_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index aef4d0b476..c95886040b 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -942,3 +942,5 @@ vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** RV64 Zvknh vector crypto extension *** vsha2ms_vv 101101 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsha2ch_vv 101110 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvknh.c.inc b/target/riscv/insn_trans/trans_rvzvknh.c.inc index 97bdf4d72f..3cf3ceaf3a 100644 --- a/target/riscv/insn_trans/trans_rvzvknh.c.inc +++ b/target/riscv/insn_trans/trans_rvzvknh.c.inc @@ -80,3 +80,5 @@ static bool vsha_check(DisasContext *s, arg_rmrr *a) } GEN_VV_UNMASKED_TRANS(vsha2ms_vv, vsha_check, 4) +GEN_VV_UNMASKED_TRANS(vsha2cl_vv, vsha_check, 4) +GEN_VV_UNMASKED_TRANS(vsha2ch_vv, vsha_check, 4) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index ae253b3357..bf0455f8e0 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -526,3 +526,143 @@ void HELPER(vsha2ms_vv)(void *vd, void *vs1, void *vs2, CPURISCVState *env, vext_set_elems_1s(vd, vta, env->vl * esz, total_elems * esz); env->vstart = 0; } + +static inline uint64_t sum0_64(uint64_t x) +{ + return ror64(x, 28) ^ ror64(x, 34) ^ ror64(x, 39); +} + +static inline uint32_t sum0_32(uint32_t x) +{ + return ror32(x, 2) ^ ror32(x, 13) ^ ror32(x, 22); +} + +static inline uint64_t sum1_64(uint64_t x) +{ + return ror64(x, 14) ^ ror64(x, 18) ^ ror64(x, 41); +} + +static inline uint32_t sum1_32(uint32_t x) +{ + return ror32(x, 6) ^ ror32(x, 11) ^ ror32(x, 25); +} + +#define ch(x, y, z) ((x & y) ^ ((~x) & z)) + +#define maj(x, y, z) ((x & y) ^ (x & z) ^ (y & z)) + +static void vsha2c_64(uint64_t *vs2, uint64_t *vd, uint64_t *vs1) +{ + uint64_t a = vs2[3], b = vs2[2], e = vs2[1], f = vs2[0]; + uint64_t c = vd[3], d = vd[2], g = vd[1], h = vd[0]; + uint64_t W0 = vs1[0], W1 = vs1[1]; + uint64_t T1 = h + sum1_64(e) + ch(e, f, g) + W0; + uint64_t T2 = sum0_64(a) + maj(a, b, c); + + h = g; + g = f; + f = e; + e = d + T1; + d = c; + c = b; + b = a; + a = T1 + T2; + + T1 = h + sum1_64(e) + ch(e, f, g) + W1; + T2 = sum0_64(a) + maj(a, b, c); + h = g; + g = f; + f = e; + e = d + T1; + d = c; + c = b; + b = a; + a = T1 + T2; + + vd[0] = f; + vd[1] = e; + vd[2] = b; + vd[3] = a; +} + +static void vsha2c_32(uint32_t *vs2, uint32_t *vd, uint32_t *vs1) +{ + uint32_t a = vs2[H4(3)], b = vs2[H4(2)], e = vs2[H4(1)], f = vs2[H4(0)]; + uint32_t c = vd[H4(3)], d = vd[H4(2)], g = vd[H4(1)], h = vd[H4(0)]; + uint32_t W0 = vs1[H4(0)], W1 = vs1[H4(1)]; + uint32_t T1 = h + sum1_32(e) + ch(e, f, g) + W0; + uint32_t T2 = sum0_32(a) + maj(a, b, c); + + h = g; + g = f; + f = e; + e = d + T1; + d = c; + c = b; + b = a; + a = T1 + T2; + + T1 = h + sum1_32(e) + ch(e, f, g) + W1; + T2 = sum0_32(a) + maj(a, b, c); + h = g; + g = f; + f = e; + e = d + T1; + d = c; + c = b; + b = a; + a = T1 + T2; + + vd[H4(0)] = f; + vd[H4(1)] = e; + vd[H4(2)] = b; + vd[H4(3)] = a; +} + +void HELPER(vsha2ch_vv)(void *vd, void *vs1, void *vs2, CPURISCVState *env, + uint32_t desc) +{ + uint32_t sew = FIELD_EX64(env->vtype, VTYPE, VSEW); + uint32_t esz = sew == MO_64 ? 8 : 4; + uint32_t total_elems; + uint32_t vta = vext_vta(desc); + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + if (sew == MO_64) { + vsha2c_64(((uint64_t *)vs2) + 4 * i, ((uint64_t *)vd) + 4 * i, + ((uint64_t *)vs1) + 4 * i + 2); + } else { + vsha2c_32(((uint32_t *)vs2) + 4 * i, ((uint32_t *)vd) + 4 * i, + ((uint32_t *)vs1) + 4 * i + 2); + } + } + + /* set tail elements to 1s */ + total_elems = vext_get_total_elems(env, desc, esz); + vext_set_elems_1s(vd, vta, env->vl * esz, total_elems * esz); + env->vstart = 0; +} + +void HELPER(vsha2cl_vv)(void *vd, void *vs1, void *vs2, CPURISCVState *env, + uint32_t desc) +{ + uint32_t sew = FIELD_EX64(env->vtype, VTYPE, VSEW); + uint32_t esz = sew == MO_64 ? 8 : 4; + uint32_t total_elems; + uint32_t vta = vext_vta(desc); + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + if (sew == MO_64) { + vsha2c_64(((uint64_t *)vs2) + 4 * i, ((uint64_t *)vd) + 4 * i, + (((uint64_t *)vs1) + 4 * i)); + } else { + vsha2c_32(((uint32_t *)vs2) + 4 * i, ((uint32_t *)vd) + 4 * i, + (((uint32_t *)vs1) + 4 * i)); + } + } + + /* set tail elements to 1s */ + total_elems = vext_get_total_elems(env, desc, esz); + vext_set_elems_1s(vd, vta, env->vl * esz, total_elems * esz); + env->vstart = 0; +} From patchwork Fri Mar 10 09:12:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7BF3C64EC4 for ; Fri, 10 Mar 2023 09:41:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231303AbjCJJk7 (ORCPT ); Fri, 10 Mar 2023 04:40:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231563AbjCJJkM (ORCPT ); Fri, 10 Mar 2023 04:40:12 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F31E46A63 for ; Fri, 10 Mar 2023 01:39:29 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnf-00GpVx-MA; Fri, 10 Mar 2023 09:12:31 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 32/45] target/riscv: Expose zvknh cpu properties Date: Fri, 10 Mar 2023 09:12:02 +0000 Message-Id: <20230310091215.931644-33-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nazar Kazakov Signed-off-by: Nazar Kazakov --- target/riscv/cpu.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 3ffbdd53cc..b3f9638067 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1482,6 +1482,8 @@ static Property riscv_cpu_extensions[] = { /* Vector cryptography extensions */ DEFINE_PROP_BOOL("x-zvkb", RISCVCPU, cfg.ext_zvkb, false), DEFINE_PROP_BOOL("x-zvkned", RISCVCPU, cfg.ext_zvkned, false), + DEFINE_PROP_BOOL("x-zvknha", RISCVCPU, cfg.ext_zvknha, false), + DEFINE_PROP_BOOL("x-zvknhb", RISCVCPU, cfg.ext_zvknhb, false), DEFINE_PROP_END_OF_LIST(), }; From patchwork Fri Mar 10 09:12:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC03BC64EC4 for ; Fri, 10 Mar 2023 09:45:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231398AbjCJJp5 (ORCPT ); Fri, 10 Mar 2023 04:45:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229844AbjCJJpk (ORCPT ); Fri, 10 Mar 2023 04:45:40 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36FE4101100 for ; Fri, 10 Mar 2023 01:45:22 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnf-00GpVx-U5; Fri, 10 Mar 2023 09:12:31 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 33/45] target/riscv: Add zvksh cpu property Date: Fri, 10 Mar 2023 09:12:03 +0000 Message-Id: <20230310091215.931644-34-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk Signed-off-by: Kiran Ostrolenk --- target/riscv/cpu.c | 4 +++- target/riscv/cpu.h | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index b3f9638067..e218a00a2d 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -113,6 +113,7 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zvkned, true, PRIV_VERSION_1_12_0, ext_zvkned), ISA_EXT_DATA_ENTRY(zvknha, true, PRIV_VERSION_1_12_0, ext_zvknha), ISA_EXT_DATA_ENTRY(zvknhb, true, PRIV_VERSION_1_12_0, ext_zvknhb), + ISA_EXT_DATA_ENTRY(zvksh, true, PRIV_VERSION_1_12_0, ext_zvksh), ISA_EXT_DATA_ENTRY(zhinx, true, PRIV_VERSION_1_12_0, ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin), ISA_EXT_DATA_ENTRY(smaia, true, PRIV_VERSION_1_12_0, ext_smaia), @@ -1219,7 +1220,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) * In principle zve*x would also suffice here, were they supported * in qemu */ - if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha) && + if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha || + cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg( diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 5d101fc405..3b0f322f3e 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -474,6 +474,7 @@ struct RISCVCPUConfig { bool ext_zvkned; bool ext_zvknha; bool ext_zvknhb; + bool ext_zvksh; bool ext_zmmul; bool ext_zvfh; bool ext_zvfhmin; From patchwork Fri Mar 10 09:12:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF788C64EC4 for ; Fri, 10 Mar 2023 09:41:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231414AbjCJJlo (ORCPT ); Fri, 10 Mar 2023 04:41:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231404AbjCJJlO (ORCPT ); Fri, 10 Mar 2023 04:41:14 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B763CE6824 for ; Fri, 10 Mar 2023 01:40:39 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYng-00GpVx-5x; Fri, 10 Mar 2023 09:12:32 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 34/45] target/riscv: Add vsm3me.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:04 +0000 Message-Id: <20230310091215.931644-35-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Kiran Ostrolenk Signed-off-by: Lawrence Hunter Signed-off-by: Kiran Ostrolenk --- target/riscv/helper.h | 2 ++ target/riscv/insn32.decode | 3 ++ target/riscv/insn_trans/trans_rvzvksh.c.inc | 37 +++++++++++++++++++ target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 39 +++++++++++++++++++++ 5 files changed, 82 insertions(+) create mode 100644 target/riscv/insn_trans/trans_rvzvksh.c.inc diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 77bbd9db56..d8f67b924e 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1203,3 +1203,5 @@ DEF_HELPER_5(vaeskf2_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vsha2ms_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsha2ch_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index c95886040b..588907dd4d 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -944,3 +944,6 @@ vaeskf2_vi 101010 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2ms_vv 101101 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2ch_vv 101110 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** RV64 Zvksh vector crypto extension *** +vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvksh.c.inc b/target/riscv/insn_trans/trans_rvzvksh.c.inc new file mode 100644 index 0000000000..a0b3de1b21 --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvksh.c.inc @@ -0,0 +1,37 @@ +/* + * RISC-V translation routines for the Zvksh Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +static inline bool vsm3_check(DisasContext *s, arg_rmrr *a) +{ + int mult = 1 << MAX(s->lmul, 0); + return s->cfg_ptr->ext_zvksh == true && + require_rvv(s) && + vext_check_isa_ill(s) && + !is_overlapped(a->rd, mult, a->rs2, mult) && + MAXSZ(s) >= (256 / 8) && /* EGW in bytes */ + s->vstart % 8 == 0 && + s->sew == MO_32; +} + +static inline bool vsm3me_check(DisasContext *s, arg_rmrr *a) +{ + return vsm3_check(s, a) && vext_check_sss(s, a->rd, a->rs1, a->rs2, a->vm); +} + +GEN_VV_UNMASKED_TRANS(vsm3me_vv, vsm3me_check, 8) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index 1c1e36c10a..256872ec28 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1086,6 +1086,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvzvkb.c.inc" #include "insn_trans/trans_rvzvkned.c.inc" #include "insn_trans/trans_rvzvknh.c.inc" +#include "insn_trans/trans_rvzvksh.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index bf0455f8e0..20c4ed8c4a 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -666,3 +666,42 @@ void HELPER(vsha2cl_vv)(void *vd, void *vs1, void *vs2, CPURISCVState *env, vext_set_elems_1s(vd, vta, env->vl * esz, total_elems * esz); env->vstart = 0; } + +static inline uint32_t p1(uint32_t x) +{ + return x ^ rol32(x, 15) ^ rol32(x, 23); +} + +static inline uint32_t zvksh_w(uint32_t m16, uint32_t m9, uint32_t m3, + uint32_t m13, uint32_t m6) +{ + return p1(m16 ^ m9 ^ rol32(m3, 15)) ^ rol32(m13, 7) ^ m6; +} + +void HELPER(vsm3me_vv)(void *vd_vptr, void *vs1_vptr, void *vs2_vptr, + CPURISCVState *env, uint32_t desc) +{ + uint32_t esz = memop_size(FIELD_EX64(env->vtype, VTYPE, VSEW)); + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + uint32_t vta = vext_vta(desc); + uint32_t *vd = vd_vptr; + uint32_t *vs1 = vs1_vptr; + uint32_t *vs2 = vs2_vptr; + + for (int i = env->vstart / 8; i < env->vl / 8; i++) { + uint32_t w[24]; + for (int j = 0; j < 8; j++) { + w[j] = bswap32(vs1[H4((i * 8) + j)]); + w[j + 8] = bswap32(vs2[H4((i * 8) + j)]); + } + for (int j = 0; j < 8; j++) { + w[j + 16] = + zvksh_w(w[j], w[j + 7], w[j + 13], w[j + 3], w[j + 10]); + } + for (int j = 0; j < 8; j++) { + vd[(i * 8) + j] = bswap32(w[H4(j + 16)]); + } + } + vext_set_elems_1s(vd_vptr, vta, env->vl * esz, total_elems * esz); + env->vstart = 0; +} From patchwork Fri Mar 10 09:12:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F598C64EC4 for ; Fri, 10 Mar 2023 09:41:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229996AbjCJJlP (ORCPT ); Fri, 10 Mar 2023 04:41:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231591AbjCJJkR (ORCPT ); Fri, 10 Mar 2023 04:40:17 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0972313D7B for ; Fri, 10 Mar 2023 01:40:08 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYng-00GpVx-Dw; Fri, 10 Mar 2023 09:12:32 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 35/45] target/riscv: Add vsm3c.vi decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:05 +0000 Message-Id: <20230310091215.931644-36-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk Signed-off-by: Kiran Ostrolenk --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvksh.c.inc | 6 ++ target/riscv/vcrypto_helper.c | 95 +++++++++++++++++++++ 4 files changed, 103 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index d8f67b924e..494ebf8cda 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1205,3 +1205,4 @@ DEF_HELPER_5(vsha2ch_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 588907dd4d..76802f37db 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -947,3 +947,4 @@ vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** RV64 Zvksh vector crypto extension *** vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvksh.c.inc b/target/riscv/insn_trans/trans_rvzvksh.c.inc index a0b3de1b21..e15cb828b7 100644 --- a/target/riscv/insn_trans/trans_rvzvksh.c.inc +++ b/target/riscv/insn_trans/trans_rvzvksh.c.inc @@ -34,4 +34,10 @@ static inline bool vsm3me_check(DisasContext *s, arg_rmrr *a) return vsm3_check(s, a) && vext_check_sss(s, a->rd, a->rs1, a->rs2, a->vm); } +static inline bool vsm3c_check(DisasContext *s, arg_rmrr *a) +{ + return vsm3_check(s, a) && vext_check_ss(s, a->rd, a->rs2, a->vm); +} + GEN_VV_UNMASKED_TRANS(vsm3me_vv, vsm3me_check, 8) +GEN_VI_UNMASKED_TRANS(vsm3c_vi, vsm3c_check, 8) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 20c4ed8c4a..2d8db45740 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -705,3 +705,98 @@ void HELPER(vsm3me_vv)(void *vd_vptr, void *vs1_vptr, void *vs2_vptr, vext_set_elems_1s(vd_vptr, vta, env->vl * esz, total_elems * esz); env->vstart = 0; } + +static inline uint32_t ff1(uint32_t x, uint32_t y, uint32_t z) +{ + return x ^ y ^ z; +} + +static inline uint32_t ff2(uint32_t x, uint32_t y, uint32_t z) +{ + return (x & y) | (x & z) | (y & z); +} + +static inline uint32_t ff_j(uint32_t x, uint32_t y, uint32_t z, uint32_t j) +{ + return (j <= 15) ? ff1(x, y, z) : ff2(x, y, z); +} + +static inline uint32_t gg1(uint32_t x, uint32_t y, uint32_t z) +{ + return x ^ y ^ z; +} + +static inline uint32_t gg2(uint32_t x, uint32_t y, uint32_t z) +{ + return (x & y) | (~x & z); +} + +static inline uint32_t gg_j(uint32_t x, uint32_t y, uint32_t z, uint32_t j) +{ + return (j <= 15) ? gg1(x, y, z) : gg2(x, y, z); +} + +static inline uint32_t t_j(uint32_t j) +{ + return (j <= 15) ? 0x79cc4519 : 0x7a879d8a; +} + +static inline uint32_t p_0(uint32_t x) +{ + return x ^ rol32(x, 9) ^ rol32(x, 17); +} + +static void sm3c(uint32_t *vd, uint32_t *vs1, uint32_t *vs2, uint32_t uimm) +{ + uint32_t x0, x1; + uint32_t j; + uint32_t ss1, ss2, tt1, tt2; + x0 = vs2[0] ^ vs2[4]; + x1 = vs2[1] ^ vs2[5]; + j = 2 * uimm; + ss1 = rol32(rol32(vs1[0], 12) + vs1[4] + rol32(t_j(j), j % 32), 7); + ss2 = ss1 ^ rol32(vs1[0], 12); + tt1 = ff_j(vs1[0], vs1[1], vs1[2], j) + vs1[3] + ss2 + x0; + tt2 = gg_j(vs1[4], vs1[5], vs1[6], j) + vs1[7] + ss1 + vs2[0]; + vs1[3] = vs1[2]; + vd[3] = rol32(vs1[1], 9); + vs1[1] = vs1[0]; + vd[1] = tt1; + vs1[7] = vs1[6]; + vd[7] = rol32(vs1[5], 19); + vs1[5] = vs1[4]; + vd[5] = p_0(tt2); + j = 2 * uimm + 1; + ss1 = rol32(rol32(vd[1], 12) + vd[5] + rol32(t_j(j), j % 32), 7); + ss2 = ss1 ^ rol32(vd[1], 12); + tt1 = ff_j(vd[1], vs1[1], vd[3], j) + vs1[3] + ss2 + x1; + tt2 = gg_j(vd[5], vs1[5], vd[7], j) + vs1[7] + ss1 + vs2[1]; + vd[2] = rol32(vs1[1], 9); + vd[0] = tt1; + vd[6] = rol32(vs1[5], 19); + vd[4] = p_0(tt2); +} + +void HELPER(vsm3c_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, + CPURISCVState *env, uint32_t desc) +{ + uint32_t esz = memop_size(FIELD_EX64(env->vtype, VTYPE, VSEW)); + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + uint32_t vta = vext_vta(desc); + uint32_t *vd = vd_vptr; + uint32_t *vs2 = vs2_vptr; + uint32_t v1[8], v2[8], v3[8]; + + for (int i = env->vstart / 8; i < env->vl / 8; i++) { + for (int k = 0; k < 8; k++) { + v2[k] = bswap32(vd[H4(i * 8 + k)]); + v3[k] = bswap32(vs2[H4(i * 8 + k)]); + } + sm3c(v1, v2, v3, uimm); + for (int k = 0; k < 8; k++) { + vd[i * 8 + k] = bswap32(v1[H4(k)]); + } + } + vext_set_elems_1s(vd_vptr, vta, env->vl * esz, total_elems * esz); + env->vstart = 0; +} From patchwork Fri Mar 10 09:12:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4AF5C64EC4 for ; Fri, 10 Mar 2023 09:44:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230387AbjCJJo0 (ORCPT ); Fri, 10 Mar 2023 04:44:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231352AbjCJJoC (ORCPT ); Fri, 10 Mar 2023 04:44:02 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 646574EFF for ; Fri, 10 Mar 2023 01:43:17 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYng-00GpVx-Ls; Fri, 10 Mar 2023 09:12:32 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org Subject: [PATCH 36/45] target/riscv: Expose zvksh cpu property Date: Fri, 10 Mar 2023 09:12:06 +0000 Message-Id: <20230310091215.931644-37-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kiran Ostrolenk Signed-off-by: Kiran Ostrolenk --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index e218a00a2d..c136a17112 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1486,6 +1486,7 @@ static Property riscv_cpu_extensions[] = { DEFINE_PROP_BOOL("x-zvkned", RISCVCPU, cfg.ext_zvkned, false), DEFINE_PROP_BOOL("x-zvknha", RISCVCPU, cfg.ext_zvknha, false), DEFINE_PROP_BOOL("x-zvknhb", RISCVCPU, cfg.ext_zvknhb, false), + DEFINE_PROP_BOOL("x-zvksh", RISCVCPU, cfg.ext_zvksh, false), DEFINE_PROP_END_OF_LIST(), }; From patchwork Fri Mar 10 09:12:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7BC1C6FD1E for ; Fri, 10 Mar 2023 09:42:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230464AbjCJJmR (ORCPT ); Fri, 10 Mar 2023 04:42:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230448AbjCJJl6 (ORCPT ); Fri, 10 Mar 2023 04:41:58 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4DC4F2798A for ; Fri, 10 Mar 2023 01:41:18 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYng-00GpVx-Tk; Fri, 10 Mar 2023 09:12:32 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 37/45] target/riscv: Add zvkg cpu property Date: Fri, 10 Mar 2023 09:12:07 +0000 Message-Id: <20230310091215.931644-38-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/cpu.c | 5 +++-- target/riscv/cpu.h | 1 + 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index c136a17112..79079d517d 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -110,6 +110,7 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zvfh, true, PRIV_VERSION_1_12_0, ext_zvfh), ISA_EXT_DATA_ENTRY(zvfhmin, true, PRIV_VERSION_1_12_0, ext_zvfhmin), ISA_EXT_DATA_ENTRY(zvkb, true, PRIV_VERSION_1_12_0, ext_zvkb), + ISA_EXT_DATA_ENTRY(zvkg, true, PRIV_VERSION_1_12_0, ext_zvkg), ISA_EXT_DATA_ENTRY(zvkned, true, PRIV_VERSION_1_12_0, ext_zvkned), ISA_EXT_DATA_ENTRY(zvknha, true, PRIV_VERSION_1_12_0, ext_zvknha), ISA_EXT_DATA_ENTRY(zvknhb, true, PRIV_VERSION_1_12_0, ext_zvknhb), @@ -1220,8 +1221,8 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) * In principle zve*x would also suffice here, were they supported * in qemu */ - if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkned || cpu->cfg.ext_zvknha || - cpu->cfg.ext_zvksh) && + if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkg || cpu->cfg.ext_zvkned || + cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg( diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 3b0f322f3e..40c4e23209 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -471,6 +471,7 @@ struct RISCVCPUConfig { bool ext_zve64f; bool ext_zve64d; bool ext_zvkb; + bool ext_zvkg; bool ext_zvkned; bool ext_zvknha; bool ext_zvknhb; From patchwork Fri Mar 10 09:12:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D332C64EC4 for ; Fri, 10 Mar 2023 09:45:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231448AbjCJJpl (ORCPT ); Fri, 10 Mar 2023 04:45:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231461AbjCJJpS (ORCPT ); Fri, 10 Mar 2023 04:45:18 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2280665C5B for ; Fri, 10 Mar 2023 01:44:52 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnh-00GpVx-5Q; Fri, 10 Mar 2023 09:12:33 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 38/45] target/riscv: Add vgmul.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:08 +0000 Message-Id: <20230310091215.931644-39-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Co-authored-by: Nazar Kazakov Signed-off-by: Nazar Kazakov Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 2 ++ target/riscv/insn32.decode | 3 ++ target/riscv/insn_trans/trans_rvzvkg.c.inc | 30 +++++++++++++++++++ target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 34 ++++++++++++++++++++++ 5 files changed, 70 insertions(+) create mode 100644 target/riscv/insn_trans/trans_rvzvkg.c.inc diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 494ebf8cda..680f695e75 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1206,3 +1206,5 @@ DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) + +DEF_HELPER_4(vgmul_vv, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 76802f37db..fdb535906f 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -948,3 +948,6 @@ vsha2cl_vv 101111 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** RV64 Zvksh vector crypto extension *** vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 + +# *** RV64 Zvkg vector crypto extension *** +vgmul_vv 101000 1 ..... 10001 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkg.c.inc b/target/riscv/insn_trans/trans_rvzvkg.c.inc new file mode 100644 index 0000000000..f1e4ea1381 --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvkg.c.inc @@ -0,0 +1,30 @@ +/* + * RISC-V translation routines for the Zvkg Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +static bool vgmul_check(DisasContext *s, arg_rmr *a) +{ + return s->cfg_ptr->ext_zvkg == true && + vext_check_isa_ill(s) && + require_rvv(s) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + vext_check_ss(s, a->rd, a->rs2, a->vm) && + s->vstart % 4 == 0 && s->sew == MO_32; +} + +GEN_V_UNMASKED_TRANS(vgmul_vv, vgmul_check) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index 256872ec28..fdb5c3364e 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1087,6 +1087,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvzvkned.c.inc" #include "insn_trans/trans_rvzvknh.c.inc" #include "insn_trans/trans_rvzvksh.c.inc" +#include "insn_trans/trans_rvzvkg.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index 2d8db45740..eb70e2e26c 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -800,3 +800,37 @@ void HELPER(vsm3c_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, vext_set_elems_1s(vd_vptr, vta, env->vl * esz, total_elems * esz); env->vstart = 0; } + +void HELPER(vgmul_vv)(void *vd_vptr, void *vs2_vptr, + CPURISCVState *env, uint32_t desc) +{ + uint64_t *vd = vd_vptr; + uint64_t *vs2 = vs2_vptr; + uint32_t vta = vext_vta(desc); + uint32_t total_elems = vext_get_total_elems(env, desc, 4); + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + uint64_t Y[2] = {brev8(vd[i * 2 + 0]), brev8(vd[i * 2 + 1])}; + uint64_t H[2] = {brev8(vs2[i * 2 + 0]), brev8(vs2[i * 2 + 1])}; + uint64_t Z[2] = {0, 0}; + + for (uint j = 0; j < 128; j++) { + if ((Y[j / 64] >> (j % 64)) & 1) { + Z[0] ^= H[0]; + Z[1] ^= H[1]; + } + bool reduce = ((H[1] >> 63) & 1); + H[1] = H[1] << 1 | H[0] >> 63; + H[0] = H[0] << 1; + if (reduce) { + H[0] ^= 0x87; + } + } + + vd[i * 2 + 0] = brev8(Z[0]); + vd[i * 2 + 1] = brev8(Z[1]); + } + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, env->vl * 4, total_elems * 4); + env->vstart = 0; +} From patchwork Fri Mar 10 09:12:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87FC8C6FD1C for ; Fri, 10 Mar 2023 09:45:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230174AbjCJJpF (ORCPT ); Fri, 10 Mar 2023 04:45:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231458AbjCJJor (ORCPT ); Fri, 10 Mar 2023 04:44:47 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 156B4101117 for ; Fri, 10 Mar 2023 01:44:11 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnh-00GpVx-DN; Fri, 10 Mar 2023 09:12:33 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 39/45] target/riscv: Add vghsh.vv decoding, translation and execution support Date: Fri, 10 Mar 2023 09:12:09 +0000 Message-Id: <20230310091215.931644-40-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/helper.h | 1 + target/riscv/insn32.decode | 1 + target/riscv/insn_trans/trans_rvzvkg.c.inc | 10 ++++++ target/riscv/vcrypto_helper.c | 38 ++++++++++++++++++++++ 4 files changed, 50 insertions(+) diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 680f695e75..3c4aa4b5df 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1207,4 +1207,5 @@ DEF_HELPER_5(vsha2cl_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3me_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) +DEF_HELPER_5(vghsh_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_4(vgmul_vv, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index fdb535906f..856e088bad 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -950,4 +950,5 @@ vsm3me_vv 100000 1 ..... ..... 010 ..... 1110111 @r_vm_1 vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** RV64 Zvkg vector crypto extension *** +vghsh_vv 101100 1 ..... ..... 010 ..... 1110111 @r_vm_1 vgmul_vv 101000 1 ..... 10001 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvkg.c.inc b/target/riscv/insn_trans/trans_rvzvkg.c.inc index f1e4ea1381..9280300ce0 100644 --- a/target/riscv/insn_trans/trans_rvzvkg.c.inc +++ b/target/riscv/insn_trans/trans_rvzvkg.c.inc @@ -28,3 +28,13 @@ static bool vgmul_check(DisasContext *s, arg_rmr *a) } GEN_V_UNMASKED_TRANS(vgmul_vv, vgmul_check) + +static bool vghsh_check(DisasContext *s, arg_rmrr *a) +{ + return s->cfg_ptr->ext_zvkg == true && + opivv_check(s, a) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + s->vstart % 4 == 0 && s->sew == MO_32; +} + +GEN_VV_UNMASKED_TRANS(vghsh_vv, vghsh_check, 4) diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index eb70e2e26c..fe9b05253d 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -801,6 +801,44 @@ void HELPER(vsm3c_vi)(void *vd_vptr, void *vs2_vptr, uint32_t uimm, env->vstart = 0; } +void HELPER(vghsh_vv)(void *vd_vptr, void *vs1_vptr, void *vs2_vptr, + CPURISCVState *env, uint32_t desc) +{ + uint64_t *vd = vd_vptr; + uint64_t *vs1 = vs1_vptr; + uint64_t *vs2 = vs2_vptr; + uint32_t vta = vext_vta(desc); + uint32_t total_elems = vext_get_total_elems(env, desc, 4); + + for (uint32_t i = env->vstart / 4; i < env->vl / 4; i++) { + uint64_t Y[2] = {vd[i * 2 + 0], vd[i * 2 + 1]}; + uint64_t H[2] = {brev8(vs2[i * 2 + 0]), brev8(vs2[i * 2 + 1])}; + uint64_t X[2] = {vs1[i * 2 + 0], vs1[i * 2 + 1]}; + uint64_t Z[2] = {0, 0}; + + uint64_t S[2] = {brev8(Y[0] ^ X[0]), brev8(Y[1] ^ X[1])}; + + for (uint j = 0; j < 128; j++) { + if ((S[j / 64] >> (j % 64)) & 1) { + Z[0] ^= H[0]; + Z[1] ^= H[1]; + } + bool reduce = ((H[1] >> 63) & 1); + H[1] = H[1] << 1 | H[0] >> 63; + H[0] = H[0] << 1; + if (reduce) { + H[0] ^= 0x87; + } + } + + vd[i * 2 + 0] = brev8(Z[0]); + vd[i * 2 + 1] = brev8(Z[1]); + } + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vta, env->vl * 4, total_elems * 4); + env->vstart = 0; +} + void HELPER(vgmul_vv)(void *vd_vptr, void *vs2_vptr, CPURISCVState *env, uint32_t desc) { From patchwork Fri Mar 10 09:12:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58002C6FD1C for ; Fri, 10 Mar 2023 09:43:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230426AbjCJJnU (ORCPT ); Fri, 10 Mar 2023 04:43:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230410AbjCJJmh (ORCPT ); Fri, 10 Mar 2023 04:42:37 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 507DC19F0B for ; Fri, 10 Mar 2023 01:42:08 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnh-00GpVx-LX; Fri, 10 Mar 2023 09:12:33 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Lawrence Hunter Subject: [PATCH 40/45] target/riscv: Expose zvkg cpu property Date: Fri, 10 Mar 2023 09:12:10 +0000 Message-Id: <20230310091215.931644-41-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Signed-off-by: Lawrence Hunter --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 79079d517d..323e0c462b 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1484,6 +1484,7 @@ static Property riscv_cpu_extensions[] = { /* Vector cryptography extensions */ DEFINE_PROP_BOOL("x-zvkb", RISCVCPU, cfg.ext_zvkb, false), + DEFINE_PROP_BOOL("x-zvkg", RISCVCPU, cfg.ext_zvkg, false), DEFINE_PROP_BOOL("x-zvkned", RISCVCPU, cfg.ext_zvkned, false), DEFINE_PROP_BOOL("x-zvknha", RISCVCPU, cfg.ext_zvknha, false), DEFINE_PROP_BOOL("x-zvknhb", RISCVCPU, cfg.ext_zvknhb, false), From patchwork Fri Mar 10 09:12:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55065C64EC4 for ; Fri, 10 Mar 2023 09:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229846AbjCJJpD (ORCPT ); Fri, 10 Mar 2023 04:45:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230440AbjCJJoe (ORCPT ); Fri, 10 Mar 2023 04:44:34 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9D7D4FA8B for ; Fri, 10 Mar 2023 01:43:57 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYnh-00GpVx-TN; Fri, 10 Mar 2023 09:12:33 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Max Chou Subject: [PATCH 41/45] crypto: Create sm4_subword Date: Fri, 10 Mar 2023 09:12:11 +0000 Message-Id: <20230310091215.931644-42-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Max Chou - Share sm4_subword between different targets. Signed-off-by: Max Chou Reviewed-by: Frank Chang --- include/crypto/sm4.h | 8 ++++++++ target/arm/tcg/crypto_helper.c | 10 ++-------- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/crypto/sm4.h b/include/crypto/sm4.h index 9bd3ebc62e..de8245d8a7 100644 --- a/include/crypto/sm4.h +++ b/include/crypto/sm4.h @@ -3,4 +3,12 @@ extern const uint8_t sm4_sbox[256]; +static inline uint32_t sm4_subword(uint32_t word) +{ + return sm4_sbox[word & 0xff] | + sm4_sbox[(word >> 8) & 0xff] << 8 | + sm4_sbox[(word >> 16) & 0xff] << 16 | + sm4_sbox[(word >> 24) & 0xff] << 24; +} + #endif diff --git a/target/arm/tcg/crypto_helper.c b/target/arm/tcg/crypto_helper.c index d28690321f..58e6c4f779 100644 --- a/target/arm/tcg/crypto_helper.c +++ b/target/arm/tcg/crypto_helper.c @@ -707,10 +707,7 @@ static void do_crypto_sm4e(uint64_t *rd, uint64_t *rn, uint64_t *rm) CR_ST_WORD(d, (i + 3) % 4) ^ CR_ST_WORD(n, i); - t = sm4_sbox[t & 0xff] | - sm4_sbox[(t >> 8) & 0xff] << 8 | - sm4_sbox[(t >> 16) & 0xff] << 16 | - sm4_sbox[(t >> 24) & 0xff] << 24; + t = sm4_subword(t); CR_ST_WORD(d, i) ^= t ^ rol32(t, 2) ^ rol32(t, 10) ^ rol32(t, 18) ^ rol32(t, 24); @@ -744,10 +741,7 @@ static void do_crypto_sm4ekey(uint64_t *rd, uint64_t *rn, uint64_t *rm) CR_ST_WORD(d, (i + 3) % 4) ^ CR_ST_WORD(m, i); - t = sm4_sbox[t & 0xff] | - sm4_sbox[(t >> 8) & 0xff] << 8 | - sm4_sbox[(t >> 16) & 0xff] << 16 | - sm4_sbox[(t >> 24) & 0xff] << 24; + t = sm4_subword(t); CR_ST_WORD(d, i) ^= t ^ rol32(t, 13) ^ rol32(t, 23); } From patchwork Fri Mar 10 09:12:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C61B2C6FD19 for ; Fri, 10 Mar 2023 09:45:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229751AbjCJJpO (ORCPT ); Fri, 10 Mar 2023 04:45:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231579AbjCJJo6 (ORCPT ); Fri, 10 Mar 2023 04:44:58 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A38F033465 for ; Fri, 10 Mar 2023 01:44:27 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYni-00GpVx-5H; Fri, 10 Mar 2023 09:12:34 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Max Chou Subject: [PATCH 42/45] crypto: Add SM4 constant parameter CK Date: Fri, 10 Mar 2023 09:12:12 +0000 Message-Id: <20230310091215.931644-43-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Max Chou Signed-off-by: Max Chou Reviewed-by: Frank Chang --- crypto/sm4.c | 10 ++++++++++ include/crypto/sm4.h | 1 + 2 files changed, 11 insertions(+) diff --git a/crypto/sm4.c b/crypto/sm4.c index 9f0cd452c7..2987306cf7 100644 --- a/crypto/sm4.c +++ b/crypto/sm4.c @@ -47,3 +47,13 @@ uint8_t const sm4_sbox[] = { 0x79, 0xee, 0x5f, 0x3e, 0xd7, 0xcb, 0x39, 0x48, }; +uint32_t const sm4_ck[] = { + 0x00070e15, 0x1c232a31, 0x383f464d, 0x545b6269, + 0x70777e85, 0x8c939aa1, 0xa8afb6bd, 0xc4cbd2d9, + 0xe0e7eef5, 0xfc030a11, 0x181f262d, 0x343b4249, + 0x50575e65, 0x6c737a81, 0x888f969d, 0xa4abb2b9, + 0xc0c7ced5, 0xdce3eaf1, 0xf8ff060d, 0x141b2229, + 0x30373e45, 0x4c535a61, 0x686f767d, 0x848b9299, + 0xa0a7aeb5, 0xbcc3cad1, 0xd8dfe6ed, 0xf4fb0209, + 0x10171e25, 0x2c333a41, 0x484f565d, 0x646b7279 +}; diff --git a/include/crypto/sm4.h b/include/crypto/sm4.h index de8245d8a7..382b26d922 100644 --- a/include/crypto/sm4.h +++ b/include/crypto/sm4.h @@ -2,6 +2,7 @@ #define QEMU_SM4_H extern const uint8_t sm4_sbox[256]; +extern const uint32_t sm4_ck[32]; static inline uint32_t sm4_subword(uint32_t word) { From patchwork Fri Mar 10 09:12:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A43DC76186 for ; Fri, 10 Mar 2023 09:41:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231388AbjCJJlI (ORCPT ); Fri, 10 Mar 2023 04:41:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231580AbjCJJkP (ORCPT ); Fri, 10 Mar 2023 04:40:15 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 740715FFA for ; Fri, 10 Mar 2023 01:39:46 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYni-00GpVx-DH; Fri, 10 Mar 2023 09:12:34 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Max Chou Subject: [PATCH 43/45] target/riscv: Add zvksed cfg property Date: Fri, 10 Mar 2023 09:12:13 +0000 Message-Id: <20230310091215.931644-44-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Max Chou Signed-off-by: Max Chou Reviewed-by: Frank Chang --- target/riscv/cpu.c | 3 ++- target/riscv/cpu.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 323e0c462b..84a225bf5f 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -114,6 +114,7 @@ static const struct isa_ext_data isa_edata_arr[] = { ISA_EXT_DATA_ENTRY(zvkned, true, PRIV_VERSION_1_12_0, ext_zvkned), ISA_EXT_DATA_ENTRY(zvknha, true, PRIV_VERSION_1_12_0, ext_zvknha), ISA_EXT_DATA_ENTRY(zvknhb, true, PRIV_VERSION_1_12_0, ext_zvknhb), + ISA_EXT_DATA_ENTRY(zvksed, true, PRIV_VERSION_1_12_0, ext_zvksed), ISA_EXT_DATA_ENTRY(zvksh, true, PRIV_VERSION_1_12_0, ext_zvksh), ISA_EXT_DATA_ENTRY(zhinx, true, PRIV_VERSION_1_12_0, ext_zhinx), ISA_EXT_DATA_ENTRY(zhinxmin, true, PRIV_VERSION_1_12_0, ext_zhinxmin), @@ -1222,7 +1223,7 @@ static void riscv_cpu_realize(DeviceState *dev, Error **errp) * in qemu */ if ((cpu->cfg.ext_zvkb || cpu->cfg.ext_zvkg || cpu->cfg.ext_zvkned || - cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksh) && + cpu->cfg.ext_zvknha || cpu->cfg.ext_zvksed || cpu->cfg.ext_zvksh) && !(cpu->cfg.ext_zve32f || cpu->cfg.ext_zve64f || cpu->cfg.ext_zve64d || cpu->cfg.ext_v)) { error_setg( diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 40c4e23209..55bbc4375a 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -475,6 +475,7 @@ struct RISCVCPUConfig { bool ext_zvkned; bool ext_zvknha; bool ext_zvknhb; + bool ext_zvksed; bool ext_zvksh; bool ext_zmmul; bool ext_zvfh; From patchwork Fri Mar 10 09:12:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E69D1C64EC4 for ; Fri, 10 Mar 2023 09:44:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231310AbjCJJoj (ORCPT ); Fri, 10 Mar 2023 04:44:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230174AbjCJJoY (ORCPT ); Fri, 10 Mar 2023 04:44:24 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A60A1E28A for ; Fri, 10 Mar 2023 01:43:38 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYni-00GpVx-LU; Fri, 10 Mar 2023 09:12:34 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Max Chou Subject: [PATCH 44/45] target/riscv: Add Zvksed support Date: Fri, 10 Mar 2023 09:12:14 +0000 Message-Id: <20230310091215.931644-45-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Max Chou - add vsm4k, vsm4r instructions Signed-off-by: Max Chou Reviewed-by: Frank Chang [lawrence.hunter@codethink.co.uk: Moved SM4 functions from crypto_helper.c to vcrypto_helper.c] [nazar.kazakov@codethink.co.uk: Added alignment checks, refactored code to use macros, and minor style changes] --- target/riscv/helper.h | 4 + target/riscv/insn32.decode | 5 + target/riscv/insn_trans/trans_rvzvksed.c.inc | 57 +++++++++ target/riscv/translate.c | 1 + target/riscv/vcrypto_helper.c | 127 +++++++++++++++++++ 5 files changed, 194 insertions(+) create mode 100644 target/riscv/insn_trans/trans_rvzvksed.c.inc diff --git a/target/riscv/helper.h b/target/riscv/helper.h index 3c4aa4b5df..4e71738b38 100644 --- a/target/riscv/helper.h +++ b/target/riscv/helper.h @@ -1209,3 +1209,7 @@ DEF_HELPER_5(vsm3c_vi, void, ptr, ptr, i32, env, i32) DEF_HELPER_5(vghsh_vv, void, ptr, ptr, ptr, env, i32) DEF_HELPER_4(vgmul_vv, void, ptr, ptr, env, i32) + +DEF_HELPER_5(vsm4k_vi, void, ptr, ptr, i32, env, i32) +DEF_HELPER_4(vsm4r_vv, void, ptr, ptr, env, i32) +DEF_HELPER_4(vsm4r_vs, void, ptr, ptr, env, i32) diff --git a/target/riscv/insn32.decode b/target/riscv/insn32.decode index 856e088bad..543e58ef18 100644 --- a/target/riscv/insn32.decode +++ b/target/riscv/insn32.decode @@ -952,3 +952,8 @@ vsm3c_vi 101011 1 ..... ..... 010 ..... 1110111 @r_vm_1 # *** RV64 Zvkg vector crypto extension *** vghsh_vv 101100 1 ..... ..... 010 ..... 1110111 @r_vm_1 vgmul_vv 101000 1 ..... 10001 010 ..... 1110111 @r2_vm_1 + +# *** RV64 Zvksed vector crypto extension *** +vsm4k_vi 100001 1 ..... ..... 010 ..... 1110111 @r_vm_1 +vsm4r_vv 101000 1 ..... 10000 010 ..... 1110111 @r2_vm_1 +vsm4r_vs 101001 1 ..... 10000 010 ..... 1110111 @r2_vm_1 diff --git a/target/riscv/insn_trans/trans_rvzvksed.c.inc b/target/riscv/insn_trans/trans_rvzvksed.c.inc new file mode 100644 index 0000000000..0025919fdb --- /dev/null +++ b/target/riscv/insn_trans/trans_rvzvksed.c.inc @@ -0,0 +1,57 @@ +/* + * RISC-V translation routines for the Zvksed Extension. + * + * Copyright (C) 2023 SiFive, Inc. + * Written by Codethink Ltd and SiFive. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#define ZVKSED_EGS 4 + +static bool zvksed_check(DisasContext *s) +{ + return s->cfg_ptr->ext_zvksed == true && + require_rvv(s) && + vext_check_isa_ill(s) && + MAXSZ(s) >= (128 / 8) && /* EGW in bytes */ + s->vstart % ZVKSED_EGS == 0 && + s->sew == MO_32; +} + +static bool vsm4k_vi_check(DisasContext *s, arg_rmrr *a) +{ + return zvksed_check(s) && + require_align(a->rd, s->lmul) && + require_align(a->rs2, s->lmul); +} + +GEN_VI_UNMASKED_TRANS(vsm4k_vi, vsm4k_vi_check, ZVKSED_EGS) + +static bool vsm4r_vv_check(DisasContext *s, arg_rmr *a) +{ + return zvksed_check(s) && + require_align(a->rd, s->lmul) && + require_align(a->rs2, s->lmul); +} + +GEN_V_UNMASKED_TRANS(vsm4r_vv, vsm4r_vv_check) + +static bool vsm4r_vs_check(DisasContext *s, arg_rmr *a) +{ + return zvksed_check(s) && + !is_overlapped(a->rd, 1 << MAX(s->lmul, 0), a->rs2, 1) && + require_align(a->rd, s->lmul); +} + +GEN_V_UNMASKED_TRANS(vsm4r_vs, vsm4r_vs_check) diff --git a/target/riscv/translate.c b/target/riscv/translate.c index fdb5c3364e..521bc2e3a9 100644 --- a/target/riscv/translate.c +++ b/target/riscv/translate.c @@ -1088,6 +1088,7 @@ static uint32_t opcode_at(DisasContextBase *dcbase, target_ulong pc) #include "insn_trans/trans_rvzvknh.c.inc" #include "insn_trans/trans_rvzvksh.c.inc" #include "insn_trans/trans_rvzvkg.c.inc" +#include "insn_trans/trans_rvzvksed.c.inc" #include "insn_trans/trans_privileged.c.inc" #include "insn_trans/trans_svinval.c.inc" #include "decode-xthead.c.inc" diff --git a/target/riscv/vcrypto_helper.c b/target/riscv/vcrypto_helper.c index fe9b05253d..63af768e2e 100644 --- a/target/riscv/vcrypto_helper.c +++ b/target/riscv/vcrypto_helper.c @@ -23,6 +23,7 @@ #include "qemu/bswap.h" #include "cpu.h" #include "crypto/aes.h" +#include "crypto/sm4.h" #include "exec/memop.h" #include "exec/exec-all.h" #include "exec/helper-proto.h" @@ -872,3 +873,129 @@ void HELPER(vgmul_vv)(void *vd_vptr, void *vs2_vptr, vext_set_elems_1s(vd, vta, env->vl * 4, total_elems * 4); env->vstart = 0; } + +void HELPER(vsm4k_vi)(void *vd, void *vs2, uint32_t uimm5, + CPURISCVState *env, uint32_t desc) +{ + const uint32_t egs = 4; + uint32_t rnd = uimm5 & 0x7; + uint32_t group_start = env->vstart / egs; + uint32_t group_end = env->vl / egs; + uint32_t esz = sizeof(uint32_t); + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + + for (uint32_t i = group_start; i < group_end; ++i) { + uint32_t vstart = i * egs; + uint32_t vend = (i + 1) * egs; + uint32_t rk[4] = {0}; + uint32_t tmp[8] = {0}; + + for (uint32_t j = vstart; j < vend; ++j) { + rk[j - vstart] = *((uint32_t *)vs2 + H4(j)); + } + + for (uint32_t j = 0; j < egs; ++j) { + tmp[j] = rk[j]; + } + + for (uint32_t j = 0; j < egs; ++j) { + uint32_t b, s; + b = tmp[j + 1] ^ tmp[j + 2] ^ tmp[j + 3] ^ sm4_ck[rnd * 4 + j]; + + s = sm4_subword(b); + + tmp[j + 4] = tmp[j] ^ (s ^ rol32(s, 13) ^ rol32(s, 23)); + } + + for (uint32_t j = vstart; j < vend; ++j) { + *((uint32_t *)vd + H4(j)) = tmp[egs + (j - vstart)]; + } + } + + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vext_vta(desc), env->vl * esz, total_elems * esz); +} + +static void do_sm4_round(uint32_t *rk, uint32_t *buf) +{ + const uint32_t egs = 4; + uint32_t s, b; + + for (uint32_t j = egs; j < egs * 2; ++j) { + b = buf[j - 3] ^ buf[j - 2] ^ buf[j - 1] ^ rk[j - 4]; + + s = sm4_subword(b); + + buf[j] = buf[j - 4] ^ (s ^ rol32(s, 2) ^ rol32(s, 10) ^ + rol32(s, 18) ^ rol32(s, 24)); + } +} + +void HELPER(vsm4r_vv)(void *vd, void *vs2, CPURISCVState *env, uint32_t desc) +{ + const uint32_t egs = 4; + uint32_t group_start = env->vstart / egs; + uint32_t group_end = env->vl / egs; + uint32_t esz = sizeof(uint32_t); + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + + for (uint32_t i = group_start; i < group_end; ++i) { + uint32_t vstart = i * egs; + uint32_t vend = (i + 1) * egs; + uint32_t rk[4] = {0}; + uint32_t tmp[8] = {0}; + + for (uint32_t j = vstart; j < vend; ++j) { + rk[j - vstart] = *((uint32_t *)vs2 + H4(j)); + } + + for (uint32_t j = vstart; j < vend; ++j) { + tmp[j - vstart] = *((uint32_t *)vd + H4(j)); + } + + do_sm4_round(rk, tmp); + + for (uint32_t j = vstart; j < vend; ++j) { + *((uint32_t *)vd + H4(j)) = tmp[egs + (j - vstart)]; + } + } + + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vext_vta(desc), env->vl * esz, total_elems * esz); +} + +void HELPER(vsm4r_vs)(void *vd, void *vs2, CPURISCVState *env, uint32_t desc) +{ + const uint32_t egs = 4; + uint32_t group_start = env->vstart / egs; + uint32_t group_end = env->vl / egs; + uint32_t esz = sizeof(uint32_t); + uint32_t total_elems = vext_get_total_elems(env, desc, esz); + + for (uint32_t i = group_start; i < group_end; ++i) { + uint32_t vstart = i * egs; + uint32_t vend = (i + 1) * egs; + uint32_t rk[4] = {0}; + uint32_t tmp[8] = {0}; + + for (uint32_t j = 0; j < egs; ++j) { + rk[j] = *((uint32_t *)vs2 + H4(j)); + } + + for (uint32_t j = vstart; j < vend; ++j) { + tmp[j - vstart] = *((uint32_t *)vd + H4(j)); + } + + do_sm4_round(rk, tmp); + + for (uint32_t j = vstart; j < vend; ++j) { + *((uint32_t *)vd + H4(j)) = tmp[egs + (j - vstart)]; + } + } + + env->vstart = 0; + /* set tail elements to 1s */ + vext_set_elems_1s(vd, vext_vta(desc), env->vl * esz, total_elems * esz); +} From patchwork Fri Mar 10 09:12:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lawrence Hunter X-Patchwork-Id: 13169047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE586C64EC4 for ; Fri, 10 Mar 2023 09:44:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230098AbjCJJoD (ORCPT ); Fri, 10 Mar 2023 04:44:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230022AbjCJJn1 (ORCPT ); Fri, 10 Mar 2023 04:43:27 -0500 Received: from imap5.colo.codethink.co.uk (imap5.colo.codethink.co.uk [78.40.148.171]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49511FE0A3 for ; Fri, 10 Mar 2023 01:42:48 -0800 (PST) Received: from [167.98.27.226] (helo=lawrence-thinkpad.office.codethink.co.uk) by imap5.colo.codethink.co.uk with esmtpsa (Exim 4.94.2 #2 (Debian)) id 1paYni-00GpVx-UQ; Fri, 10 Mar 2023 09:12:34 +0000 From: Lawrence Hunter To: qemu-devel@nongnu.org Cc: dickon.hood@codethink.co.uk, nazar.kazakov@codethink.co.uk, kiran.ostrolenk@codethink.co.uk, frank.chang@sifive.com, palmer@dabbelt.com, alistair.francis@wdc.com, bin.meng@windriver.com, pbonzini@redhat.com, philipp.tomsich@vrull.eu, kvm@vger.kernel.org, Max Chou Subject: [PATCH 45/45] target/riscv: Expose Zvksed property Date: Fri, 10 Mar 2023 09:12:15 +0000 Message-Id: <20230310091215.931644-46-lawrence.hunter@codethink.co.uk> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> References: <20230310091215.931644-1-lawrence.hunter@codethink.co.uk> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Max Chou Signed-off-by: Max Chou Reviewed-by: Frank Chang --- target/riscv/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c index 84a225bf5f..8caa485f28 100644 --- a/target/riscv/cpu.c +++ b/target/riscv/cpu.c @@ -1489,6 +1489,7 @@ static Property riscv_cpu_extensions[] = { DEFINE_PROP_BOOL("x-zvkned", RISCVCPU, cfg.ext_zvkned, false), DEFINE_PROP_BOOL("x-zvknha", RISCVCPU, cfg.ext_zvknha, false), DEFINE_PROP_BOOL("x-zvknhb", RISCVCPU, cfg.ext_zvknhb, false), + DEFINE_PROP_BOOL("x-zvksed", RISCVCPU, cfg.ext_zvksed, false), DEFINE_PROP_BOOL("x-zvksh", RISCVCPU, cfg.ext_zvksh, false), DEFINE_PROP_END_OF_LIST(),