From patchwork Wed Jul 17 06:37:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11047313 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A97F1398 for ; Wed, 17 Jul 2019 06:39:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3964728708 for ; Wed, 17 Jul 2019 06:39:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2DBC12870F; Wed, 17 Jul 2019 06:39:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6CEFF28708 for ; Wed, 17 Jul 2019 06:39:17 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hndZN-00058Z-6v; Wed, 17 Jul 2019 06:37:41 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1hndZL-000586-UI for xen-devel@lists.xenproject.org; Wed, 17 Jul 2019 06:37:39 +0000 X-Inumbo-ID: 5e1ffe24-a85d-11e9-9811-630cb90a3fb8 Received: from m4a0041g.houston.softwaregrp.com (unknown [15.124.2.87]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 5e1ffe24-a85d-11e9-9811-630cb90a3fb8; Wed, 17 Jul 2019 06:37:38 +0000 (UTC) Received: FROM m4a0041g.houston.softwaregrp.com (15.120.17.146) BY m4a0041g.houston.softwaregrp.com WITH ESMTP; Wed, 17 Jul 2019 06:37:36 +0000 Received: from M4W0335.microfocus.com (2002:f78:1193::f78:1193) by M4W0334.microfocus.com (2002:f78:1192::f78:1192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10; Wed, 17 Jul 2019 06:37:25 +0000 Received: from NAM03-BY2-obe.outbound.protection.outlook.com (15.124.8.10) by M4W0335.microfocus.com (15.120.17.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1591.10 via Frontend Transport; Wed, 17 Jul 2019 06:37:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bMlynq02ZduOGDc8EeBeseQzsAvN5moMcE0ae8uMIxXVEWZd5+YVHTRZOs81Y7gHV/4rZqVACBDZZ3od6KV98xhSkAynEWkXGWno9gkHL8GtUnbvT8nB7slAl7qah1r5tb9luXwset6q/Cc5yPxENH+5pyWzfFctehbXAwQYY515Sd4mgNt6JMgPr917OnlyR16ZEyiV30RtGBjezZN7uoIPm+jAag8o612CTW+KyWVfjZpB6CvkdqE9iT1qxl/DVlmTMDejEP8IxO77x50afQ5OKL9FvXELfVtfE2GQYT7563YuLY9r7HdV5QPJa4n+8tj70Lm5Amu+omh2vjUnsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bkHPImUKln0NGFON7nF+nAhyiz2ofptMVoocVrkOtgk=; b=HYifRtzfwkjDLSvasTpIo7f0g8cyphxEKBZeGws2TMQUEP4mZydhjUY27bIalOS3OdJdHupjnXnda/Jx4yT9uywZYdBngWPHSowv4K0PlqNgPRZrq/AATjZO2d9UwsyEAhsQYGQHphbaD1Ryfo6TT/rWsuViMErFliKXWN2fNNkN8fUf7taS9zmZ37KrmgiKuYWOukZqepSdZqU8yybhZbWYKjM4Ptqk8v6+51jyzsrhhe7NtqunCfa0SQdCnQ6K4JIIqcPpGZndq2mtnUrTJGrgQvVd/lvK1qeIAZh9Zg+9YNpnb7dl1UpK13p5RxxEihuCq/dXnrBOJxPmp2rgwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=suse.com;dmarc=pass action=none header.from=suse.com;dkim=pass header.d=suse.com;arc=none Received: from DM6PR18MB3401.namprd18.prod.outlook.com (10.255.174.218) by DM6PR18MB2841.namprd18.prod.outlook.com (20.179.51.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.14; Wed, 17 Jul 2019 06:37:24 +0000 Received: from DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7]) by DM6PR18MB3401.namprd18.prod.outlook.com ([fe80::1fe:35f6:faf3:78c7%7]) with mapi id 15.20.2073.012; Wed, 17 Jul 2019 06:37:24 +0000 From: Jan Beulich To: "xen-devel@lists.xenproject.org" Thread-Topic: [PATCH v10 11/13] x86emul: add an AES/VAES test case to the harness Thread-Index: AQHVPGoXhUQJVQyqgk2N7axY+XXRwQ== Date: Wed, 17 Jul 2019 06:37:24 +0000 Message-ID: <20cf6419-f852-fe8a-7169-d5c127ab2a30@suse.com> References: <0ccca19e-7bbb-ab1e-c0bb-a568b02874e0@suse.com> In-Reply-To: <0ccca19e-7bbb-ab1e-c0bb-a568b02874e0@suse.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DB6P189CA0004.EURP189.PROD.OUTLOOK.COM (2603:10a6:6:2e::17) To DM6PR18MB3401.namprd18.prod.outlook.com (2603:10b6:5:1cc::26) authentication-results: spf=none (sender IP is ) smtp.mailfrom=JBeulich@suse.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [87.234.252.170] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9b005bcf-6ced-4e64-852f-08d70a813a4c x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:DM6PR18MB2841; x-ms-traffictypediagnostic: DM6PR18MB2841: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:854; x-forefront-prvs: 01018CB5B3 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(4636009)(376002)(346002)(396003)(39860400002)(366004)(136003)(199004)(189003)(8676002)(6116002)(5660300002)(25786009)(53936002)(256004)(66946007)(8936002)(81166006)(446003)(476003)(6512007)(76176011)(99286004)(2616005)(11346002)(486006)(81156014)(66066001)(31686004)(478600001)(52116002)(186003)(4326008)(54906003)(86362001)(6436002)(316002)(7736002)(102836004)(3846002)(6506007)(31696002)(6916009)(80792005)(68736007)(2501003)(6486002)(26005)(386003)(36756003)(66476007)(66446008)(66556008)(64756008)(2906002)(305945005)(71190400001)(14454004)(5640700003)(2351001)(71200400001); DIR:OUT; SFP:1102; SCL:1; SRVR:DM6PR18MB2841; H:DM6PR18MB3401.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: suse.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: uX048BLLdlpsjKguaZnbx1OvXrGHp3YoZfQPRMNXEd8yeHsx1zK+CO9H6cHO0uoluUkaGBBsfzyKCPRxjIl5AJS+wBM0gGAxFxeHhzgnu+m9+0HJvyecBoRWut7FFDk+c4sMuurM7UEy/kN6clFBjGwHuOuTPfYIqHrbw1IQtQqHO5qOq4RAO3oSE5C0oXSXI3GRXG+674CqvMe9xBpg8aLUo/j4ML9u1G2FVMg4ms+McK7I7SVn0+2uGdFHUViDvj2M7ikSWiIU5M5seOX9R+RJLW1QnbbAS6uRnFjYmgE0OZU4hqtJvJViiZ9A/mbPUQ0yvNCW41aRFtk+mIklYF8spdIi0ZoGPnJZrc5r48QwylsPUkgJGB28GsWVLZ+UlxFLxXfZ8HjqS+fSeRIVk3gddx/Q+AhZcLW5cHesZ3U= Content-ID: <0E3DAFE48A61CB44B0212213E4DF6553@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 9b005bcf-6ced-4e64-852f-08d70a813a4c X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jul 2019 06:37:24.2816 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 856b813c-16e5-49a5-85ec-6f081e13b527 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: JBeulich@suse.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR18MB2841 X-OriginatorOrg: suse.com Subject: [Xen-devel] [PATCH v10 11/13] x86emul: add an AES/VAES test case to the harness X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , RogerPau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Jan Beulich Acked-by: Andrew Cooper --- v8: New. --- a/tools/tests/x86_emulator/Makefile +++ b/tools/tests/x86_emulator/Makefile @@ -19,8 +19,9 @@ CFLAGS += $(CFLAGS_xeninclude) SIMD := 3dnow sse sse2 sse4 avx avx2 xop avx512f avx512bw avx512dq avx512er avx512vbmi FMA := fma4 fma SG := avx2-sg avx512f-sg avx512vl-sg +AES := ssse3-aes avx-aes avx2-vaes avx512bw-vaes GF := sse2-gf avx2-gf avx512bw-gf -TESTCASES := blowfish $(SIMD) $(FMA) $(SG) $(GF) +TESTCASES := blowfish $(SIMD) $(FMA) $(SG) $(AES) $(GF) OPMASK := avx512f avx512dq avx512bw @@ -143,6 +144,10 @@ $(1)-cflags := \ $(foreach flt,$($(1)-flts), \ "-D_$(vec)x$(idx)f$(flt) -m$(1:-sg=) $(call non-sse,$(1)) -Os -DVEC_MAX=$(vec) -DIDX_SIZE=$(idx) -DFLOAT_SIZE=$(flt)"))) endef +define simd-aes-defs +$(1)-cflags := $(foreach vec,$($(patsubst %-aes,sse,$(1))-vecs) $($(patsubst %-vaes,%,$(1))-vecs), \ + "-D_$(vec) -maes $(addprefix -m,$(subst -,$(space),$(1))) $(call non-sse,$(1)) -Os -DVEC_SIZE=$(vec)") +endef define simd-gf-defs $(1)-cflags := $(foreach vec,$($(1:-gf=)-vecs), \ "-D_$(vec) -mgfni -m$(1:-gf=) $(call non-sse,$(1)) -Os -DVEC_SIZE=$(vec)") @@ -153,6 +158,7 @@ endef $(foreach flavor,$(SIMD) $(FMA),$(eval $(call simd-defs,$(flavor)))) $(foreach flavor,$(SG),$(eval $(call simd-sg-defs,$(flavor)))) +$(foreach flavor,$(AES),$(eval $(call simd-aes-defs,$(flavor)))) $(foreach flavor,$(GF),$(eval $(call simd-gf-defs,$(flavor)))) $(foreach flavor,$(OPMASK),$(eval $(call opmask-defs,$(flavor)))) @@ -203,10 +209,13 @@ $(addsuffix .c,$(FMA)): $(addsuffix .c,$(SG)): ln -sf simd-sg.c $@ +$(addsuffix .c,$(AES)): + ln -sf simd-aes.c $@ + $(addsuffix .c,$(GF)): ln -sf simd-gf.c $@ -$(addsuffix .h,$(SIMD) $(FMA) $(SG) $(GF)): simd.h +$(addsuffix .h,$(SIMD) $(FMA) $(SG) $(AES) $(GF)): simd.h xop.h avx512f.h: simd-fma.c --- /dev/null +++ b/tools/tests/x86_emulator/simd-aes.c @@ -0,0 +1,102 @@ +#define UINT_SIZE 1 + +#include "simd.h" +ENTRY(aes_test); + +#if VEC_SIZE == 16 +# define AES(op, a...) __builtin_ia32_vaes ## op ## _v16qi(a) +# define imc(x) ((vec_t)__builtin_ia32_aesimc128((vdi_t)(x))) +#elif VEC_SIZE == 32 +# define AES(op, a...) __builtin_ia32_vaes ## op ## _v32qi(a) +# define imc(x) ({ \ + vec_t r_; \ + unsigned char __attribute__((vector_size(16))) t_; \ + asm ( "vaesimc (%3), %x0\n\t" \ + "vaesimc 16(%3), %1\n\t" \ + "vinserti128 $1, %1, %0, %0" \ + : "=&v" (r_), "=&v" (t_) \ + : "m" (x), "r" (&(x)) ); \ + r_; \ +}) +#elif VEC_SIZE == 64 +# define AES(op, a...) __builtin_ia32_vaes ## op ## _v64qi(a) +# define imc(x) ({ \ + vec_t r_; \ + unsigned char __attribute__((vector_size(16))) t_; \ + asm ( "vaesimc (%3), %x0\n\t" \ + "vaesimc 1*16(%3), %1\n\t" \ + "vinserti32x4 $1, %1, %0, %0\n\t" \ + "vaesimc 2*16(%3), %1\n\t" \ + "vinserti32x4 $2, %1, %0, %0\n\t" \ + "vaesimc 3*16(%3), %1\n\t" \ + "vinserti32x4 $3, %1, %0, %0" \ + : "=&v" (r_), "=&v" (t_) \ + : "m" (x), "r" (&(x)) ); \ + r_; \ +}) +#endif + +#ifdef __AVX512BW__ +# define ALL_TRUE (~0ULL >> (64 - ELEM_COUNT)) +# define eq(x, y) (B(pcmpeqb, _mask, (vqi_t)(x), (vqi_t)(y), -1) == ALL_TRUE) +# define aes(op, x, y) ((vec_t)AES(op, (vqi_t)(x), (vqi_t)(y))) +#else +# if defined(__AVX2__) && VEC_SIZE == 32 +# define to_bool(cmp) B(ptestc, , cmp, (vdi_t){} == 0) +# define aes(op, x, y) ((vec_t)AES(op, (vqi_t)(x), (vqi_t)(y))) +# else +# define to_bool(cmp) (__builtin_ia32_pmovmskb128(cmp) == 0xffff) +# define aes(op, x, y) ((vec_t)__builtin_ia32_aes ## op ## 128((vdi_t)(x), (vdi_t)(y))) +# endif +# define eq(x, y) to_bool((x) == (y)) +#endif + +int aes_test(void) +{ + unsigned int i; + vec_t src, zero = {}; + + for ( i = 0; i < ELEM_COUNT; ++i ) + src[i] = i; + + do { + vec_t x, y; + + touch(src); + x = imc(src); + touch(src); + + touch(zero); + y = aes(enclast, src, zero); + touch(zero); + y = aes(dec, y, zero); + + if ( !eq(x, y) ) return __LINE__; + + touch(zero); + x = aes(declast, src, zero); + touch(zero); + y = aes(enc, x, zero); + touch(y); + x = imc(y); + + if ( !eq(x, src) ) return __LINE__; + +#if VEC_SIZE == 16 + touch(src); + x = (vec_t)__builtin_ia32_aeskeygenassist128((vdi_t)src, 0); + touch(src); + y = (vec_t)__builtin_ia32_pshufb128((vqi_t)x, + (vqi_t){ 7, 4, 5, 6, + 1, 2, 3, 0, + 15, 12, 13, 14, + 9, 10, 11, 8 }); + if ( !eq(x, y) ) return __LINE__; +#endif + + src += ELEM_COUNT; + i += ELEM_COUNT; + } while ( i <= 256 ); + + return 0; +} --- a/tools/tests/x86_emulator/simd.h +++ b/tools/tests/x86_emulator/simd.h @@ -340,6 +340,10 @@ REN(pandn, , d); REN(por, , d); REN(pxor, , d); # endif +OVR(aesdec); +OVR(aesdeclast); +OVR(aesenc); +OVR(aesenclast); OVR(cvtpd2dqx); OVR(cvtpd2dqy); OVR(cvtpd2psx); --- a/tools/tests/x86_emulator/test_x86_emulator.c +++ b/tools/tests/x86_emulator/test_x86_emulator.c @@ -12,12 +12,15 @@ asm ( ".pushsection .test, \"ax\", @prog #include "sse.h" #include "sse2.h" #include "sse2-gf.h" +#include "ssse3-aes.h" #include "sse4.h" #include "avx.h" +#include "avx-aes.h" #include "fma4.h" #include "fma.h" #include "avx2.h" #include "avx2-sg.h" +#include "avx2-vaes.h" #include "avx2-gf.h" #include "xop.h" #include "avx512f-opmask.h" @@ -27,6 +30,7 @@ asm ( ".pushsection .test, \"ax\", @prog #include "avx512f-sg.h" #include "avx512vl-sg.h" #include "avx512bw.h" +#include "avx512bw-vaes.h" #include "avx512bw-gf.h" #include "avx512dq.h" #include "avx512er.h" @@ -91,6 +95,16 @@ static bool simd_check_xop(void) return cpu_has_xop; } +static bool simd_check_ssse3_aes(void) +{ + return cpu_has_aesni && cpu_has_ssse3; +} + +static bool simd_check_avx_aes(void) +{ + return cpu_has_aesni && cpu_has_avx; +} + static bool simd_check_avx512f(void) { return cpu_has_avx512f; @@ -141,6 +155,22 @@ static bool simd_check_avx512vbmi_vl(voi return cpu_has_avx512_vbmi && cpu_has_avx512vl; } +static bool simd_check_avx2_vaes(void) +{ + return cpu_has_aesni && cpu_has_vaes && cpu_has_avx2; +} + +static bool simd_check_avx512bw_vaes(void) +{ + return cpu_has_aesni && cpu_has_vaes && cpu_has_avx512bw; +} + +static bool simd_check_avx512bw_vaes_vl(void) +{ + return cpu_has_aesni && cpu_has_vaes && + cpu_has_avx512bw && cpu_has_avx512vl; +} + static bool simd_check_sse2_gf(void) { return cpu_has_gfni && cpu_has_sse2; @@ -319,6 +349,8 @@ static const struct { SIMD(XOP i16x16, xop, 32i2), SIMD(XOP i32x8, xop, 32i4), SIMD(XOP i64x4, xop, 32i8), + SIMD(AES (legacy), ssse3_aes, 16), + SIMD(AES (VEX/x16), avx_aes, 16), SIMD(OPMASK/w, avx512f_opmask, 2), SIMD(OPMASK+DQ/b, avx512dq_opmask, 1), SIMD(OPMASK+DQ/w, avx512dq_opmask, 2), @@ -418,6 +450,10 @@ static const struct { AVX512VL(_VBMI+VL u16x8, avx512vbmi, 16u2), AVX512VL(_VBMI+VL s16x16, avx512vbmi, 32i2), AVX512VL(_VBMI+VL u16x16, avx512vbmi, 32u2), + SIMD(VAES (VEX/x32), avx2_vaes, 32), + SIMD(VAES (EVEX/x64), avx512bw_vaes, 64), + AVX512VL(VL+VAES (x16), avx512bw_vaes, 16), + AVX512VL(VL+VAES (x32), avx512bw_vaes, 32), SIMD(GFNI (legacy), sse2_gf, 16), SIMD(GFNI (VEX/x16), avx2_gf, 16), SIMD(GFNI (VEX/x32), avx2_gf, 32), --- a/tools/tests/x86_emulator/x86-emulate.h +++ b/tools/tests/x86_emulator/x86-emulate.h @@ -125,10 +125,12 @@ static inline bool xcr0_mask(uint64_t ma #define cpu_has_sse cp.basic.sse #define cpu_has_sse2 cp.basic.sse2 #define cpu_has_sse3 cp.basic.sse3 +#define cpu_has_ssse3 cp.basic.ssse3 #define cpu_has_fma (cp.basic.fma && xcr0_mask(6)) #define cpu_has_sse4_1 cp.basic.sse4_1 #define cpu_has_sse4_2 cp.basic.sse4_2 #define cpu_has_popcnt cp.basic.popcnt +#define cpu_has_aesni cp.basic.aesni #define cpu_has_avx (cp.basic.avx && xcr0_mask(6)) #define cpu_has_f16c (cp.basic.f16c && xcr0_mask(6))