From patchwork Fri Jul 1 17:04:36 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 9210275 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E0C94607D6 for ; Fri, 1 Jul 2016 17:23:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CDF2128672 for ; Fri, 1 Jul 2016 17:23:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C1FDA286C9; Fri, 1 Jul 2016 17:23:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 02A7F28672 for ; Fri, 1 Jul 2016 17:23:18 +0000 (UTC) Received: from localhost ([::1]:34710 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ29t-0008Eo-6R for patchwork-qemu-devel@patchwork.kernel.org; Fri, 01 Jul 2016 13:23:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53596) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ1sI-0002EO-HY for qemu-devel@nongnu.org; Fri, 01 Jul 2016 13:05:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bJ1sG-0005Mw-Ql for qemu-devel@nongnu.org; Fri, 01 Jul 2016 13:05:06 -0400 Received: from mail-pa0-x242.google.com ([2607:f8b0:400e:c03::242]:34321) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJ1sG-0005Mc-G2 for qemu-devel@nongnu.org; Fri, 01 Jul 2016 13:05:04 -0400 Received: by mail-pa0-x242.google.com with SMTP id us13so10132887pab.1 for ; Fri, 01 Jul 2016 10:05:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=lq7hlT4u2QxrbPZRwr9WqA24/G/Tm4+KSSR+L9og6YA=; b=yH9GrQjDr/netcizMzxHM85wFuJ/c86oxjqEZo/uCZ5l20Bfpub9nwMpUT1bvyplZj Cb4mXFbQZUs9kAHLHo1aTun3UQsClXeg62LHkQA7XhEVfqnGU9Tos3xjETZ8LnmnLock mankuMaARm6lAlTkQ/XIPxzOxqNT3O+8CTH9n71y+jnVb7WgAtZmg7a2cJkYYwIIF+ML CamEZaN+9XXSu3XNx9YodjirTGITEqICNGyOGN/9iJTcMsGeNqxixwDYwLjz9hebbJLU UeyhixS5PE3qRCZNLkg3wXYLGnOAuuI4VcXfGZy+ord5Bz0AVHmiZwDSpKK6/4OCcNXu W4FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=lq7hlT4u2QxrbPZRwr9WqA24/G/Tm4+KSSR+L9og6YA=; b=XINQSIvttrntS51D+kr5CUr/zrm6QUDL+ZpX9487fi5nHScf87PCCweCsjhvsVUca+ 7JyxT+WR6FJoMVuAetku0IES3eBSOANO5q/1KdqbW0Mja+LqdNcAO8+s8OUXjvzn+C7S Tgt+Upfh9i1Dyodx6/ePblFq/IMXv4ptYtVjshfwd/ouMano9qxsTYovSLENfZkePDGx GYImgMFklSUcQ62UR5+e1+2bkMnLOWyxDgcNcZe9EZQfgzBHmpnGS2hJTPNLBE5d1R1i xR2PJKQBvONNd9NSCP3Sm0yCKtnco+wzeMWXXGZ2ZYwMUcltcqmRJuL8qMYyHl70HDdE FjKA== X-Gm-Message-State: ALyK8tLpRZPe4NezlYrqJalAD2gZny73+MIbPajCW/FxxibyEYVOt3EIwrsufSFsCmIMXg== X-Received: by 10.66.196.203 with SMTP id io11mr33107535pac.107.1467392703595; Fri, 01 Jul 2016 10:05:03 -0700 (PDT) Received: from bigtime.twiddle.net (71-37-54-227.tukw.qwest.net. [71.37.54.227]) by smtp.gmail.com with ESMTPSA id ff9sm2652229pac.5.2016.07.01.10.05.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Jul 2016 10:05:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 1 Jul 2016 10:04:36 -0700 Message-Id: <1467392693-22715-11-git-send-email-rth@twiddle.net> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1467392693-22715-1-git-send-email-rth@twiddle.net> References: <1467392693-22715-1-git-send-email-rth@twiddle.net> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400e:c03::242 Subject: [Qemu-devel] [PATCH v2 10/27] tcg: Add atomic128 helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, serge.fdrv@gmail.com, cota@braap.org, alex.bennee@linaro.org, peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Force the use of cmpxchg16b on x86_64. Wikipedia suggests that only very old AMD64 (circa 2004) did not have this instruction. Further, it's required by Windows 8 so no new cpus will ever omit it. If we truely care about these, then we could check this at startup time and then avoid executing paths that use it. Signed-off-by: Richard Henderson --- configure | 29 ++++++++++++- cputlb.c | 6 +++ include/qemu/int128.h | 6 +++ softmmu_template.h | 110 +++++++++++++++++++++++++++++++++++++------------- tcg/tcg.h | 22 ++++++++++ 5 files changed, 144 insertions(+), 29 deletions(-) diff --git a/configure b/configure index 59ea124..586abd6 100755 --- a/configure +++ b/configure @@ -1201,7 +1201,10 @@ case "$cpu" in cc_i386='$(CC) -m32' ;; x86_64) - CPU_CFLAGS="-m64" + # ??? Only extremely old AMD cpus do not have cmpxchg16b. + # If we truly care, we should simply detect this case at + # runtime and generate the fallback to serial emulation. + CPU_CFLAGS="-m64 -mcx16" LDFLAGS="-m64 $LDFLAGS" cc_i386='$(CC) -m32' ;; @@ -4434,6 +4437,26 @@ if compile_prog "" "" ; then int128=yes fi +######################################### +# See if 128-bit atomic operations are supported. + +atomic128=no +if test "$int128" = "yes"; then + cat > $TMPC << EOF +int main(void) +{ + unsigned __int128 x = 0, y = 0; + y = __atomic_load_16(&x, 0); + __atomic_store_16(&x, y, 0); + __atomic_compare_exchange_16(&x, &y, x, 0, 0, 0); + return 0; +} +EOF + if compile_prog "" "" ; then + atomic128=yes + fi +fi + ######################################## # check if getauxval is available. @@ -5383,6 +5406,10 @@ if test "$int128" = "yes" ; then echo "CONFIG_INT128=y" >> $config_host_mak fi +if test "$atomic128" = "yes" ; then + echo "CONFIG_ATOMIC128=y" >> $config_host_mak +fi + if test "$getauxval" = "yes" ; then echo "CONFIG_GETAUXVAL=y" >> $config_host_mak fi diff --git a/cputlb.c b/cputlb.c index 5272456..660f824 100644 --- a/cputlb.c +++ b/cputlb.c @@ -510,6 +510,12 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr) #define SHIFT 3 #include "softmmu_template.h" + +#ifdef CONFIG_ATOMIC128 +#define SHIFT 4 +#include "softmmu_template.h" +#endif + #undef MMUSUFFIX #define MMUSUFFIX _cmmu diff --git a/include/qemu/int128.h b/include/qemu/int128.h index ab67275..5819da4 100644 --- a/include/qemu/int128.h +++ b/include/qemu/int128.h @@ -2,6 +2,7 @@ #define INT128_H #ifdef CONFIG_INT128 +#include "qemu/bswap.h" typedef __int128 Int128; @@ -137,6 +138,11 @@ static inline void int128_subfrom(Int128 *a, Int128 b) *a -= b; } +static inline Int128 bswap128(Int128 a) +{ + return int128_make128(bswap64(int128_gethi(a)), bswap64(int128_getlo(a))); +} + #else /* !CONFIG_INT128 */ /* Here we are catering to the ABI of the host. If the host returns diff --git a/softmmu_template.h b/softmmu_template.h index 76712b9..0a9f49b 100644 --- a/softmmu_template.h +++ b/softmmu_template.h @@ -27,25 +27,30 @@ #define DATA_SIZE (1 << SHIFT) -#if DATA_SIZE == 8 -#define SUFFIX q -#define LSUFFIX q -#define SDATA_TYPE int64_t +#if DATA_SIZE == 16 +#define SUFFIX o +#define LSUFFIX o +#define SDATA_TYPE Int128 +#define DATA_TYPE Int128 +#elif DATA_SIZE == 8 +#define SUFFIX q +#define LSUFFIX q +#define SDATA_TYPE int64_t #define DATA_TYPE uint64_t #elif DATA_SIZE == 4 -#define SUFFIX l -#define LSUFFIX l -#define SDATA_TYPE int32_t +#define SUFFIX l +#define LSUFFIX l +#define SDATA_TYPE int32_t #define DATA_TYPE uint32_t #elif DATA_SIZE == 2 -#define SUFFIX w -#define LSUFFIX uw -#define SDATA_TYPE int16_t +#define SUFFIX w +#define LSUFFIX uw +#define SDATA_TYPE int16_t #define DATA_TYPE uint16_t #elif DATA_SIZE == 1 -#define SUFFIX b -#define LSUFFIX ub -#define SDATA_TYPE int8_t +#define SUFFIX b +#define LSUFFIX ub +#define SDATA_TYPE int8_t #define DATA_TYPE uint8_t #else #error unsupported data size @@ -56,7 +61,7 @@ to the register size of the host. This is tcg_target_long, except in the case of a 32-bit host and 64-bit data, and for that we always have uint64_t. Don't bother with this widened value for SOFTMMU_CODE_ACCESS. */ -#if defined(SOFTMMU_CODE_ACCESS) || DATA_SIZE == 8 +#if defined(SOFTMMU_CODE_ACCESS) || DATA_SIZE >= 8 # define WORD_TYPE DATA_TYPE # define USUFFIX SUFFIX #else @@ -73,7 +78,9 @@ #define ADDR_READ addr_read #endif -#if DATA_SIZE == 8 +#if DATA_SIZE == 16 +# define BSWAP(X) bswap128(X) +#elif DATA_SIZE == 8 # define BSWAP(X) bswap64(X) #elif DATA_SIZE == 4 # define BSWAP(X) bswap32(X) @@ -140,6 +147,7 @@ vidx >= 0; \ }) +#if DATA_SIZE < 16 #ifndef SOFTMMU_CODE_ACCESS static inline DATA_TYPE glue(io_read, SUFFIX)(CPUArchState *env, CPUIOTLBEntry *iotlbentry, @@ -307,9 +315,10 @@ WORD_TYPE helper_be_ld_name(CPUArchState *env, target_ulong addr, return res; } #endif /* DATA_SIZE > 1 */ +#endif /* DATA_SIZE < 16 */ #ifndef SOFTMMU_CODE_ACCESS - +#if DATA_SIZE < 16 /* Provide signed versions of the load routines as well. We can of course avoid this for 64-bit data, or for 32-bit data on 32-bit host. */ #if DATA_SIZE * 8 < TCG_TARGET_REG_BITS @@ -507,6 +516,7 @@ void probe_write(CPUArchState *env, target_ulong addr, int mmu_idx, } } #endif +#endif /* DATA_SIZE < 16 */ #if DATA_SIZE == 1 # define HE_SUFFIX _mmu @@ -573,9 +583,30 @@ DATA_TYPE glue(glue(helper_atomic_cmpxchg, SUFFIX), HE_SUFFIX) TCGMemOpIdx oi, uintptr_t retaddr) { ATOMIC_MMU_BODY; +#if DATA_SIZE < 16 return atomic_cmpxchg(haddr, cmpv, newv); +#else + __atomic_compare_exchange(haddr, &cmpv, &newv, false, + __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); + return cmpv; +#endif } +#if DATA_SIZE > 1 +DATA_TYPE glue(glue(helper_atomic_cmpxchg, SUFFIX), RE_SUFFIX) + (CPUArchState *env, target_ulong addr, DATA_TYPE cmpv, DATA_TYPE newv, + TCGMemOpIdx oi, uintptr_t retaddr) +{ + DATA_TYPE retv; + cmpv = BSWAP(cmpv); + newv = BSWAP(newv); + retv = (glue(glue(helper_atomic_cmpxchg, SUFFIX), HE_SUFFIX) + (env, addr, cmpv, newv, oi, retaddr)); + return BSWAP(retv); +} +#endif + +#if DATA_SIZE < 16 #define GEN_ATOMIC_HELPER(NAME) \ DATA_TYPE glue(glue(glue(helper_atomic_, NAME), SUFFIX), HE_SUFFIX) \ (CPUArchState *env, target_ulong addr, DATA_TYPE val, \ @@ -600,18 +631,6 @@ GEN_ATOMIC_HELPER(xchg) #undef GEN_ATOMIC_HELPER #if DATA_SIZE > 1 -DATA_TYPE glue(glue(helper_atomic_cmpxchg, SUFFIX), RE_SUFFIX) - (CPUArchState *env, target_ulong addr, DATA_TYPE cmpv, DATA_TYPE newv, - TCGMemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE retv; - cmpv = BSWAP(cmpv); - newv = BSWAP(newv); - retv = (glue(glue(helper_atomic_cmpxchg, SUFFIX), HE_SUFFIX) - (env, addr, cmpv, newv, oi, retaddr)); - return BSWAP(retv); -} - #define GEN_ATOMIC_HELPER(NAME) \ DATA_TYPE glue(glue(glue(helper_atomic_, NAME), SUFFIX), RE_SUFFIX) \ (CPUArchState *env, target_ulong addr, DATA_TYPE val, \ @@ -676,6 +695,41 @@ DATA_TYPE glue(glue(helper_atomic_add_fetch, SUFFIX), RE_SUFFIX) } } #endif /* DATA_SIZE > 1 */ +#else /* DATA_SIZE >= 16 */ +DATA_TYPE glue(glue(helper_atomic_ld, SUFFIX), HE_SUFFIX) + (CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) +{ + DATA_TYPE res; + ATOMIC_MMU_BODY; + __atomic_load(haddr, &res, __ATOMIC_RELAXED); + return res; +} + +DATA_TYPE glue(glue(helper_atomic_ld, SUFFIX), RE_SUFFIX) + (CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) +{ + DATA_TYPE res; + res = (glue(glue(helper_atomic_ld, SUFFIX), HE_SUFFIX) + (env, addr, oi, retaddr)); + return BSWAP(res); +} + +void glue(glue(helper_atomic_st, SUFFIX), HE_SUFFIX) + (CPUArchState *env, target_ulong addr, DATA_TYPE val, + TCGMemOpIdx oi, uintptr_t retaddr) +{ + ATOMIC_MMU_BODY; + __atomic_store(haddr, &val, __ATOMIC_RELAXED); +} + +void glue(glue(helper_atomic_st, SUFFIX), RE_SUFFIX) + (CPUArchState *env, target_ulong addr, DATA_TYPE val, + TCGMemOpIdx oi, uintptr_t retaddr) +{ + (glue(glue(helper_atomic_st, SUFFIX), HE_SUFFIX) + (env, addr, BSWAP(val), oi, retaddr)); +} +#endif /* DATA_SIZE < 16 */ #undef ATOMIC_MMU_BODY diff --git a/tcg/tcg.h b/tcg/tcg.h index 4e60498..1304a42 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -1216,6 +1216,28 @@ GEN_ATOMIC_HELPER_ALL(xchg) #undef GEN_ATOMIC_HELPER_ALL #undef GEN_ATOMIC_HELPER +#ifdef CONFIG_ATOMIC128 +#include "qemu/int128.h" + +/* These aren't really a "proper" helpers because TCG cannot manage Int128. + However, use the same format as the others, for use by the backends. */ +Int128 helper_atomic_cmpxchgo_le_mmu(CPUArchState *env, target_ulong addr, + Int128 cmpv, Int128 newv, + TCGMemOpIdx oi, uintptr_t retaddr); +Int128 helper_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, + Int128 cmpv, Int128 newv, + TCGMemOpIdx oi, uintptr_t retaddr); + +Int128 helper_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +Int128 helper_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +void helper_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, + TCGMemOpIdx oi, uintptr_t retaddr); +void helper_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, + TCGMemOpIdx oi, uintptr_t retaddr); + +#endif /* CONFIG_ATOMIC128 */ #endif /* CONFIG_SOFTMMU */ #endif /* TCG_H */