From patchwork Tue Jul 23 03:34:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C76EC3DA63 for ; Tue, 23 Jul 2024 03:35:51 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6Iz-0002Fr-WC; Mon, 22 Jul 2024 23:35:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6Iv-0002Bo-6R for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:11 -0400 Received: from mail-yw1-x1132.google.com ([2607:f8b0:4864:20::1132]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6Ip-0001kc-WE for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:08 -0400 Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-6659e81bc68so52243477b3.0 for ; Mon, 22 Jul 2024 20:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705700; x=1722310500; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=afR5Mh7AuJ+l97gjVcHTy2tuYfmOnsauTdcz7zi2zaI=; b=imMlddUMOwjwA6Pm1fYb3YtpGYemVnbZEbt1q5b2kglFugItnitziWgbDsYNBuy2ka SMzHaNzmooHxgjDazPOD+QxUqyStvdmX2VH76B144S9z5YCFGfDqx3Yu59C+MevhxRXB pxFPba10jBmVv325TY15NsWLaZPZM0j/IW9gxU9khIRsonwDlGKwlhQCOdX9ZlQQ5jEe 2zXb4RJaL0Xq9s0oM0RR+MdQCCCktirrDzGOZqTuOpcEoX2WYaZi9dACXTdybtR+LRzd 1KtGv806WzLERdSAkbhb0zjQqq9YboE1WXBsh4rKF2vGXUikv5tQ+FH0QZaT3M1kCkG1 KFag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705700; x=1722310500; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=afR5Mh7AuJ+l97gjVcHTy2tuYfmOnsauTdcz7zi2zaI=; b=jL/v61mcookhWoHxjLZF6NVkJNgAwh6CXQZtNJ2hyn/e1lZgtWPuJjwIK2qA36g+ZT /Pl6st+FHByhdkM4UH0i5gHGUyvFoCs1wyfU3GELslQkAuDSJqgi5jxm/TodYQZz83yT NUCKIyyR6fD1XWmj1DWmqze/fVUvPPujTtFVWJ5oxar4VDoXsPgY2sklZKcKt2grljlA qlYQb/6kFQ+0/g+5+bxKSxtJ2vv0ADpjhZ9VQ6ifqbm8MgzsgOtkOJXtvoAucVgw0eAW vlUgiyqHg/mcPtN4EYf0Pqmrl+wB8H8dgW6jr+lhWyt4RqVdwktFF3PXr9+kPx95Oi2q TvmQ== X-Gm-Message-State: AOJu0YwIx2zPWPpDZobH5n+bz4cRR86M2pXW0h20vMMfHfwrh/RJm1tT 1FK/q9KvkkLx4PsQNCL+gaiAAPXehlg/1MaPe2PrKQxst8o8CKK4FW2XiB6PQ3GTVVzYUi6RDFn xlZB1XQ== X-Google-Smtp-Source: AGHT+IGg0Bi0Og+aYHlptA18vLJmrCKVPQ6whU19jrHq1QqppRZqExUNw56Zb7IyfkRYlW7cEOipHw== X-Received: by 2002:a05:690c:3208:b0:62f:2553:d3b3 with SMTP id 00721157ae682-66ada1fc141mr105800277b3.29.1721705700204; Mon, 22 Jul 2024 20:35:00 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:34:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 01/12] accel/tcg: Move {set, clear}_helper_retaddr to cpu_ldst.h Date: Tue, 23 Jul 2024 13:34:40 +1000 Message-ID: <20240723033451.546151-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1132; envelope-from=richard.henderson@linaro.org; helo=mail-yw1-x1132.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Use of these in helpers goes hand-in-hand with tlb_vaddr_to_host and other probing functions. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- accel/tcg/user-retaddr.h | 28 ---------------------------- include/exec/cpu_ldst.h | 34 ++++++++++++++++++++++++++++++++++ accel/tcg/cpu-exec.c | 3 --- accel/tcg/user-exec.c | 1 - 4 files changed, 34 insertions(+), 32 deletions(-) delete mode 100644 accel/tcg/user-retaddr.h diff --git a/accel/tcg/user-retaddr.h b/accel/tcg/user-retaddr.h deleted file mode 100644 index e0f57e1994..0000000000 --- a/accel/tcg/user-retaddr.h +++ /dev/null @@ -1,28 +0,0 @@ -#ifndef ACCEL_TCG_USER_RETADDR_H -#define ACCEL_TCG_USER_RETADDR_H - -#include "qemu/atomic.h" - -extern __thread uintptr_t helper_retaddr; - -static inline void set_helper_retaddr(uintptr_t ra) -{ - helper_retaddr = ra; - /* - * Ensure that this write is visible to the SIGSEGV handler that - * may be invoked due to a subsequent invalid memory operation. - */ - signal_barrier(); -} - -static inline void clear_helper_retaddr(void) -{ - /* - * Ensure that previous memory operations have succeeded before - * removing the data visible to the signal handler. - */ - signal_barrier(); - helper_retaddr = 0; -} - -#endif diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 71009f84f5..dac12bd8eb 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -379,4 +379,38 @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, MMUAccessType access_type, int mmu_idx); #endif +/* + * For user-only, helpers that use guest to host address translation + * must protect the actual host memory access by recording 'retaddr' + * for the signal handler. This is required for a race condition in + * which another thread unmaps the page between a probe and the + * actual access. + */ +#ifdef CONFIG_USER_ONLY +extern __thread uintptr_t helper_retaddr; + +static inline void set_helper_retaddr(uintptr_t ra) +{ + helper_retaddr = ra; + /* + * Ensure that this write is visible to the SIGSEGV handler that + * may be invoked due to a subsequent invalid memory operation. + */ + signal_barrier(); +} + +static inline void clear_helper_retaddr(void) +{ + /* + * Ensure that previous memory operations have succeeded before + * removing the data visible to the signal handler. + */ + signal_barrier(); + helper_retaddr = 0; +} +#else +#define set_helper_retaddr(ra) do { } while (0) +#define clear_helper_retaddr() do { } while (0) +#endif + #endif /* CPU_LDST_H */ diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index 9010dad073..8163295f34 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -41,9 +41,6 @@ #include "tb-context.h" #include "internal-common.h" #include "internal-target.h" -#if defined(CONFIG_USER_ONLY) -#include "user-retaddr.h" -#endif /* -icount align implementation. */ diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 80d24540ed..7ddc47b0ba 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -33,7 +33,6 @@ #include "tcg/tcg-ldst.h" #include "internal-common.h" #include "internal-target.h" -#include "user-retaddr.h" __thread uintptr_t helper_retaddr; From patchwork Tue Jul 23 03:34:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C2A4C3DA49 for ; Tue, 23 Jul 2024 03:37:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J1-0002M3-HO; Mon, 22 Jul 2024 23:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6Iz-0002Ev-9u for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:13 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6Iv-0001tm-HO for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:13 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-70d316f0060so778375b3a.1 for ; Mon, 22 Jul 2024 20:35:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705703; x=1722310503; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lffTC77hfuBCBlcm53ysSfFxUjHJ9OXDuax++3VtjJ0=; b=vbOd8PDnk5mdA6+TL3HVqNnvHY80cUcfFdzM0ttwh8YN98R60y9NbQMpvEUmjgnYj3 lRYEY4m9ecR6+/cih1n0/o64xR2zT0dN07Vxkjcu1m7uueZyKtXnewPtBOZyFkuijp1z 9yn0/Pk3RwY6qfTSEsHDHqfE/qH2pkQ26hAVzr5tGt8Sa1nR5Kqp76pZKR1z+2ZyAnV+ QdPPnxOdg1ba/nsKhCG9jqI/WnBXia6nwvENt9WYYul2Cpe5Pih93VzMYDrPtLb/BPSn NdNHL3zLnKZ4VbrX0BhrMLZf7rTGFiWy/yEnhvnUW7fPAaC0n7AI8ieotERrTYIAv85Q X7zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705703; x=1722310503; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lffTC77hfuBCBlcm53ysSfFxUjHJ9OXDuax++3VtjJ0=; b=u4PQDT/lW86UtHF38iCsdslev5waa2Hn4+ijCO9jsqhMbz7avQyxlkd1yOeJ+40Yw2 IFcI4TZX2mO4vSOcwNpEeBw0vB4suNIux2xaUS6WbMmC38gKvdbI1UDcKjah3SwN4oon 67z2GWzuwEVBAgyGZfBWCdYEgWKGZ+Q8kmaB3yP7GlSt0yw66fywb3ASGQ+++lSvwj4a CXXQ0YjeDJjBj7DOtZ2tfGr8qINqlIgY7+BZH0I4J7yPn5bZFvEXxXy59CZo9RawOs6+ pibVe/y6HZi24xI0zwQuOuhW/1nTt97fFjLNPoV8LxfLHANhPoWbrCYG/Iy+4xzB3FK8 2FDQ== X-Gm-Message-State: AOJu0YxAYnAFL+8qr+CHlJkJ3Vk4srXM/kBJE7O81hN3Vu2ohp5Z9J5u /ZDQ6ZbUkWpjDE6+Fuftapu/Nn5Y1lnxwJP+ZtV5okkq27/HQKnr9/+cDJruZ3Vehkb93dVn1w6 lXtHLJA== X-Google-Smtp-Source: AGHT+IGC9k+zrXEsLm5UlcnnwNaYbgv5BXDI4j2WEbdKAiHhKg7KvJURFdQttciock8OgzM3UjCS+w== X-Received: by 2002:a05:6a20:6725:b0:1c0:f09c:1b98 with SMTP id adf61e73a8af0-1c4521737afmr958698637.16.1721705702570; Mon, 22 Jul 2024 20:35:02 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 02/12] target/arm: Use set/clear_helper_retaddr in helper-a64.c Date: Tue, 23 Jul 2024 13:34:41 +1000 Message-ID: <20240723033451.546151-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Use these in helper_dc_dva and the FEAT_MOPS routines to avoid a race condition with munmap in another thread. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/tcg/helper-a64.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index 0ea8668ab4..c60d2a7ec9 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -928,6 +928,8 @@ uint32_t HELPER(sqrt_f16)(uint32_t a, void *fpstp) void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) { + uintptr_t ra = GETPC(); + /* * Implement DC ZVA, which zeroes a fixed-length block of memory. * Note that we do not implement the (architecturally mandated) @@ -948,8 +950,6 @@ void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) #ifndef CONFIG_USER_ONLY if (unlikely(!mem)) { - uintptr_t ra = GETPC(); - /* * Trap if accessing an invalid page. DC_ZVA requires that we supply * the original pointer for an invalid page. But watchpoints require @@ -971,7 +971,9 @@ void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) } #endif + set_helper_retaddr(ra); memset(mem, 0, blocklen); + clear_helper_retaddr(); } void HELPER(unaligned_access)(CPUARMState *env, uint64_t addr, @@ -1120,7 +1122,9 @@ static uint64_t set_step(CPUARMState *env, uint64_t toaddr, } #endif /* Easy case: just memset the host memory */ + set_helper_retaddr(ra); memset(mem, data, setsize); + clear_helper_retaddr(); return setsize; } @@ -1163,7 +1167,9 @@ static uint64_t set_step_tags(CPUARMState *env, uint64_t toaddr, } #endif /* Easy case: just memset the host memory */ + set_helper_retaddr(ra); memset(mem, data, setsize); + clear_helper_retaddr(); mte_mops_set_tags(env, toaddr, setsize, *mtedesc); return setsize; } @@ -1497,7 +1503,9 @@ static uint64_t copy_step(CPUARMState *env, uint64_t toaddr, uint64_t fromaddr, } #endif /* Easy case: just memmove the host memory */ + set_helper_retaddr(ra); memmove(wmem, rmem, copysize); + clear_helper_retaddr(); return copysize; } @@ -1572,7 +1580,9 @@ static uint64_t copy_step_rev(CPUARMState *env, uint64_t toaddr, * Easy case: just memmove the host memory. Note that wmem and * rmem here point to the *last* byte to copy. */ + set_helper_retaddr(ra); memmove(wmem - (copysize - 1), rmem - (copysize - 1), copysize); + clear_helper_retaddr(); return copysize; } From patchwork Tue Jul 23 03:34:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0F5BC3DA49 for ; Tue, 23 Jul 2024 03:36:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J2-0002Mg-41; Mon, 22 Jul 2024 23:35:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6Iy-0002Et-Rs for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:13 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6Iv-0001w2-Ph for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:12 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-70d18112b60so1452291b3a.1 for ; Mon, 22 Jul 2024 20:35:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705705; x=1722310505; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KQAUd555e3r5NFz67ssul86cWhr7fC1FabETxGwgt/Y=; b=U72q1J2cA+8gzf2lgJ4l59n7y7HgqHNs6th48mHGIyVlElmq/pdwRF4bWI5/Cld1ty 8YSQw8AJXeeFeLqeCj6LcBCWar/0C1gGRFgmglhegCysCW3QaNwxvA6e8mUWBmcpNthL wqAFRGIkUym4MkzsedufPrhECHK+XAjv45KogKyOBAnlEdB9+NByCr+juanT7vjP6XO6 /FfNddGwazbvsqjly/vfl4nuPcno77eHqV4J1KXDaTqzfghXHK/Xy48dQ1eJ0qy9e48D mkf3+soBuDNI73g9Md3DOl72xm/C9OKCjl7UjRL2+WvBOx6zArHKHRK8EvlcUFmLc7DC jPoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705705; x=1722310505; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KQAUd555e3r5NFz67ssul86cWhr7fC1FabETxGwgt/Y=; b=oFFmtG2ohjEbLsT2drnQJJ1IP7/KaUdBDYv+Nsp9h8EUEhAlLBQyx/Sh7asxHBavJ2 QYltV7AuuIsG/YaQ6YtMJ1b1oeUiaJHJ5OJwQioeVeYCPFPO+9WFe4mAMZI/NrWZEkiq LGdNR9uH7jxaplrIU6SIvjasCnQ8Cc9j788scunCcuva3BeaHh115QCmXITplkHF1qMA 9AUFJ+UibCguMy4t6f8jvQmXTp/IVR9h14MhM7C8diBICjj0eL2UQi/gt51IaeOL+sSw S66mGGjhD8Lu8hPOhA0vXBi96Z+0HEnVbLHm3DspL1TWtJVtIH0mkII9isyStyifV/Dp gjng== X-Gm-Message-State: AOJu0YxGDPS3CMy/t/YNtcwvw091rOaP4cY/Fj6PC/YgpolFoVDkbW6C znmOQJwPLNT3TaLZKliNXFCm84nw1ktOrmM0WN7yKBAZAVXsCpFxw3WuuCgA7631B70V1aKasQF 23zDrsg== X-Google-Smtp-Source: AGHT+IFoNDkAkCk2ChAzPkyVK1x6FJ9JCUV+Pi8RUeLIii2lHg4JuggKTOz+5BK5OVIYgsuoUgRp4A== X-Received: by 2002:a05:6a00:179d:b0:70a:ee81:2fa2 with SMTP id d2e1a72fcca58-70e80753650mr1827789b3a.2.1721705704816; Mon, 22 Jul 2024 20:35:04 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 03/12] target/arm: Use set/clear_helper_retaddr in SVE and SME helpers Date: Tue, 23 Jul 2024 13:34:42 +1000 Message-ID: <20240723033451.546151-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Avoid a race condition with munmap in another thread. Use around blocks that exclusively use "host_fn". Keep the blocks as small as possible, but without setting and clearing for every operation on one page. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/arm/tcg/sme_helper.c | 16 ++++++++++++++ target/arm/tcg/sve_helper.c | 42 +++++++++++++++++++++++++++++-------- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index 5a6dd76489..50bb088d04 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -517,6 +517,8 @@ void sme_ld1(CPUARMState *env, void *za, uint64_t *vg, clr_fn(za, 0, reg_off); } + set_helper_retaddr(ra); + while (reg_off <= reg_last) { uint64_t pg = vg[reg_off >> 6]; do { @@ -529,6 +531,8 @@ void sme_ld1(CPUARMState *env, void *za, uint64_t *vg, } while (reg_off <= reg_last && (reg_off & 63)); } + clear_helper_retaddr(); + /* * Use the slow path to manage the cross-page misalignment. * But we know this is RAM and cannot trap. @@ -543,6 +547,8 @@ void sme_ld1(CPUARMState *env, void *za, uint64_t *vg, reg_last = info.reg_off_last[1]; host = info.page[1].host; + set_helper_retaddr(ra); + do { uint64_t pg = vg[reg_off >> 6]; do { @@ -554,6 +560,8 @@ void sme_ld1(CPUARMState *env, void *za, uint64_t *vg, reg_off += esize; } while (reg_off & 63); } while (reg_off <= reg_last); + + clear_helper_retaddr(); } } @@ -701,6 +709,8 @@ void sme_st1(CPUARMState *env, void *za, uint64_t *vg, reg_last = info.reg_off_last[0]; host = info.page[0].host; + set_helper_retaddr(ra); + while (reg_off <= reg_last) { uint64_t pg = vg[reg_off >> 6]; do { @@ -711,6 +721,8 @@ void sme_st1(CPUARMState *env, void *za, uint64_t *vg, } while (reg_off <= reg_last && (reg_off & 63)); } + clear_helper_retaddr(); + /* * Use the slow path to manage the cross-page misalignment. * But we know this is RAM and cannot trap. @@ -725,6 +737,8 @@ void sme_st1(CPUARMState *env, void *za, uint64_t *vg, reg_last = info.reg_off_last[1]; host = info.page[1].host; + set_helper_retaddr(ra); + do { uint64_t pg = vg[reg_off >> 6]; do { @@ -734,6 +748,8 @@ void sme_st1(CPUARMState *env, void *za, uint64_t *vg, reg_off += 1 << esz; } while (reg_off & 63); } while (reg_off <= reg_last); + + clear_helper_retaddr(); } } diff --git a/target/arm/tcg/sve_helper.c b/target/arm/tcg/sve_helper.c index dd49e67d7a..f1ee0e060f 100644 --- a/target/arm/tcg/sve_helper.c +++ b/target/arm/tcg/sve_helper.c @@ -5738,6 +5738,8 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, reg_last = info.reg_off_last[0]; host = info.page[0].host; + set_helper_retaddr(retaddr); + while (reg_off <= reg_last) { uint64_t pg = vg[reg_off >> 6]; do { @@ -5752,6 +5754,8 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, } while (reg_off <= reg_last && (reg_off & 63)); } + clear_helper_retaddr(); + /* * Use the slow path to manage the cross-page misalignment. * But we know this is RAM and cannot trap. @@ -5771,6 +5775,8 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, reg_last = info.reg_off_last[1]; host = info.page[1].host; + set_helper_retaddr(retaddr); + do { uint64_t pg = vg[reg_off >> 6]; do { @@ -5784,6 +5790,8 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, mem_off += N << msz; } while (reg_off & 63); } while (reg_off <= reg_last); + + clear_helper_retaddr(); } } @@ -5934,15 +5942,11 @@ DO_LDN_2(4, dd, MO_64) /* * Load contiguous data, first-fault and no-fault. * - * For user-only, one could argue that we should hold the mmap_lock during - * the operation so that there is no race between page_check_range and the - * load operation. However, unmapping pages out from under a running thread - * is extraordinarily unlikely. This theoretical race condition also affects - * linux-user/ in its get_user/put_user macros. - * - * TODO: Construct some helpers, written in assembly, that interact with - * host_signal_handler to produce memory ops which can properly report errors - * without racing. + * For user-only, we control the race between page_check_range and + * another thread's munmap by using set/clear_helper_retaddr. Any + * SEGV that occurs between those markers is assumed to be because + * the guest page vanished. Keep that block as small as possible + * so that unrelated QEMU bugs are not blamed on the guest. */ /* Fault on byte I. All bits in FFR from I are cleared. The vector @@ -6093,6 +6097,8 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, reg_last = info.reg_off_last[0]; host = info.page[0].host; + set_helper_retaddr(retaddr); + do { uint64_t pg = *(uint64_t *)(vg + (reg_off >> 3)); do { @@ -6101,9 +6107,11 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, (cpu_watchpoint_address_matches (env_cpu(env), addr + mem_off, 1 << msz) & BP_MEM_READ)) { + clear_helper_retaddr(); goto do_fault; } if (mtedesc && !mte_probe(env, mtedesc, addr + mem_off)) { + clear_helper_retaddr(); goto do_fault; } host_fn(vd, reg_off, host + mem_off); @@ -6113,6 +6121,8 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, } while (reg_off <= reg_last && (reg_off & 63)); } while (reg_off <= reg_last); + clear_helper_retaddr(); + /* * MemSingleNF is allowed to fail for any reason. We have special * code above to handle the first element crossing a page boundary. @@ -6348,6 +6358,8 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, reg_last = info.reg_off_last[0]; host = info.page[0].host; + set_helper_retaddr(retaddr); + while (reg_off <= reg_last) { uint64_t pg = vg[reg_off >> 6]; do { @@ -6362,6 +6374,8 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, } while (reg_off <= reg_last && (reg_off & 63)); } + clear_helper_retaddr(); + /* * Use the slow path to manage the cross-page misalignment. * But we know this is RAM and cannot trap. @@ -6381,6 +6395,8 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, reg_last = info.reg_off_last[1]; host = info.page[1].host; + set_helper_retaddr(retaddr); + do { uint64_t pg = vg[reg_off >> 6]; do { @@ -6394,6 +6410,8 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, mem_off += N << msz; } while (reg_off & 63); } while (reg_off <= reg_last); + + clear_helper_retaddr(); } } @@ -6560,7 +6578,9 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, if (unlikely(info.flags & TLB_MMIO)) { tlb_fn(env, &scratch, reg_off, addr, retaddr); } else { + set_helper_retaddr(retaddr); host_fn(&scratch, reg_off, info.host); + clear_helper_retaddr(); } } else { /* Element crosses the page boundary. */ @@ -6782,7 +6802,9 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, goto fault; } + set_helper_retaddr(retaddr); host_fn(vd, reg_off, info.host); + clear_helper_retaddr(); } reg_off += esize; } while (reg_off & 63); @@ -6986,7 +7008,9 @@ void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, do { void *h = host[i]; if (likely(h != NULL)) { + set_helper_retaddr(retaddr); host_fn(vd, reg_off, h); + clear_helper_retaddr(); } else if ((vg[reg_off >> 6] >> (reg_off & 63)) & 1) { target_ulong addr = base + (off_fn(vm, reg_off) << scale); tlb_fn(env, vd, reg_off, addr, retaddr); From patchwork Tue Jul 23 03:34:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 77869C3DA49 for ; Tue, 23 Jul 2024 03:38:01 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J1-0002Ll-Gk; Mon, 22 Jul 2024 23:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6Iy-0002EX-BI for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:12 -0400 Received: from mail-io1-xd32.google.com ([2607:f8b0:4864:20::d32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6Iw-0001wM-Cz for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:12 -0400 Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-807687b6652so242365539f.0 for ; Mon, 22 Jul 2024 20:35:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705708; x=1722310508; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T/SUfCnTpRQkNLbtLFhLzUNUB0u06MHsZx7MB+lBLUc=; b=AH2miezvy5+w2UaNmgmONX/dkTIGZ/197PKQxEB790pQ7RJi7XEtrKeGK74Kpr1+/m QLQ6HfdeCebJWAAi6ClfrsMpjY0jw+cQU1EUabtmyTH5oU2oLJlc9gJ/XyD/0PCpRU0q l/FVR2SY0HaOltlWuA2S4FKDj9QbjO145EQbOZFxTNqRDWGp5FgZPmQTW8T6wtxeMC02 zZUkbGyTsNyaZaU0OdBeSfQ3cDVCVZhQ4wMT190FsGwcHyvn2zonauouSDdyLwUVFj00 coQLOBH/qUrcKEEoH40kQ6CIYgNOJ5EVNjCeS6vH3Zh2l1sJgOO32Q1gYoymxFOY3gvA RnIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705708; x=1722310508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T/SUfCnTpRQkNLbtLFhLzUNUB0u06MHsZx7MB+lBLUc=; b=REAkXsLC8bFZyXX1hh45fHFmlu/RFpUuYzYj1hDyHiLNL02dObQpBwKUjGn5DXXM1o ZcWwvk3ITJdVKv48RUi1jx2NGWPjNJ8To42nZc2Cll8LNkorL+jiFV553GlxDLxxd3+c IizBze5J7BBux2wgF/CMCBJioavzdp4HY6WPOSdkI1lsTbkHrXgORmUifhQJ2dCQ+JmK 3XslZ6rHCxKOAc7hLud5SyAdB2GS2BDwOVW8ANGpwqmtIkX2kdHooCpd4Bgcyx7487CU l23j1hkx2NBLWoE6S1PV27iLpm8j9BkLr+Az0Qn9bSeEdswjfXrNeKs9SvhDubZv/U+U tOnw== X-Gm-Message-State: AOJu0Yy7LJrUxsGDf/uwhIXG3B2C4Gtp2tQDrgYjAjqprR181Ky72e/+ 5WorYKe/RAkqL0XT93Umz9k6HetdPlI1mbA5a1Z4+bxfHAxrLBS7mWI2C3fVH+8gmfjN7Cu9Os5 Ko11QDQ== X-Google-Smtp-Source: AGHT+IEJ7kzoscIvOIXojIF7Ih7UYqdmDUS2TYRnCovMCgWCcKsSd1uXJS0QeR1cFYz1mYsL28aK4w== X-Received: by 2002:a05:6e02:1feb:b0:383:284:113 with SMTP id e9e14a558f8ab-3993ff97b4emr122662745ab.4.1721705707696; Mon, 22 Jul 2024 20:35:07 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: BALATON Zoltan , Nicholas Piggin Subject: [PATCH 04/12] target/ppc/mem_helper.c: Remove a conditional from dcbz_common() Date: Tue, 23 Jul 2024 13:34:43 +1000 Message-ID: <20240723033451.546151-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::d32; envelope-from=richard.henderson@linaro.org; helo=mail-io1-xd32.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: BALATON Zoltan Instead of passing a bool and select a value within dcbz_common() let the callers pass in the right value to avoid this conditional statement. On PPC dcbz is often used to zero memory and some code uses it a lot. This change improves the run time of a test case that copies memory with a dcbz call in every iteration from 6.23 to 5.83 seconds. Signed-off-by: BALATON Zoltan Message-Id: <20240622204833.5F7C74E6000@zero.eik.bme.hu> Reviewed-by: Richard Henderson Signed-off-by: Richard Henderson Reviewed-by: Nicholas Piggin --- target/ppc/mem_helper.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index f88155ad45..361fd72226 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -271,12 +271,11 @@ void helper_stsw(CPUPPCState *env, target_ulong addr, uint32_t nb, } static void dcbz_common(CPUPPCState *env, target_ulong addr, - uint32_t opcode, bool epid, uintptr_t retaddr) + uint32_t opcode, int mmu_idx, uintptr_t retaddr) { target_ulong mask, dcbz_size = env->dcache_line_size; uint32_t i; void *haddr; - int mmu_idx = epid ? PPC_TLB_EPID_STORE : ppc_env_mmu_index(env, false); #if defined(TARGET_PPC64) /* Check for dcbz vs dcbzl on 970 */ @@ -309,12 +308,12 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, void helper_dcbz(CPUPPCState *env, target_ulong addr, uint32_t opcode) { - dcbz_common(env, addr, opcode, false, GETPC()); + dcbz_common(env, addr, opcode, ppc_env_mmu_index(env, false), GETPC()); } void helper_dcbzep(CPUPPCState *env, target_ulong addr, uint32_t opcode) { - dcbz_common(env, addr, opcode, true, GETPC()); + dcbz_common(env, addr, opcode, PPC_TLB_EPID_STORE, GETPC()); } void helper_icbi(CPUPPCState *env, target_ulong addr) From patchwork Tue Jul 23 03:34:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AEF48C3DA49 for ; Tue, 23 Jul 2024 03:37:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J2-0002R5-V2; Mon, 22 Jul 2024 23:35:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6J0-0002IZ-JT for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:14 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6Iz-0001x1-0R for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:14 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-70d18112b60so1452342b3a.1 for ; Mon, 22 Jul 2024 20:35:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705710; x=1722310510; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qQ06f1POln3X8Sb8fDa8/CDLulrLL54olCBz3brWMPA=; b=aao1tx64iHeJ5dRihmsiRGT8WTyUKfMorDTjPha1eX0Hf6XwWW8UVU/2Hj1WPATIrX 2aRdDTJbLCN2j3bl6ZMYeF/4jaRsN5ocHwuS6aOLoVdQe39BbWjAL/GXKjph56aaU4Cw dtx1uODqE5UMZqHDT93z0zA7Y1knt+LLOTq6Lr8Ja1knXkmQOOjAJ6iy72RV4GuryAUi Fh8tTDuT0J5OAFaX0MSkx6KWO9zr9VRhLa6ZTlu/+XVjbzF96Yd3BmoCv0BN2al9XooR Ld03CKePt1Of1yyLXJ9MduHLZYk9FHj6RCpdETMl1ACYu5GuClzHn2hGWZN1OCyUaReI eNjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705710; x=1722310510; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qQ06f1POln3X8Sb8fDa8/CDLulrLL54olCBz3brWMPA=; b=XnHj72vn9OVK8uCjW8c4aRslUDQ67PkTHhQ+EkCwZ8Mn9tdeLMIZQxzqF689ZVZ2Fa kq8FCHwNvE4xUWNIrT6wQerD1yEVl2Wn/+Vw4bvDIu0xQPK2e1w+azI4WF0mLcoJkx3o AmRb8Q77YfMelZZFL+NnNe6TvKELoNW0sHw1EziwjFg8eGbI3VqBAOh+YgpXFqGztiGw DMse9nW/vrYMuFNRKpZyd2EiX55CXRBtTmBvAMhaIBZJQKXS4ez71QpauZ4Wgzy9B8Tf J+LNHpRoqxtQSYbt6cSPRw2EL4lrjuWJ9k/6SUEKrLliGpSkU08nft/OiVYhAplJqwf4 nPIw== X-Gm-Message-State: AOJu0Yx2tP8GcFlzvD77z6oKd9v9/fWuBJs84dYtmQLE7iq4ti2yrmwO FWEpINfBXIWznmNJyjaIEbCdRvUCl04NnqQvUPvSoSLwqt+IoZ2dIObz+6c3KZdpqqnRzUzhAdz mHZ8G0Q== X-Google-Smtp-Source: AGHT+IF7Z2lbnp9CXecA00v0CI0UH7DfLhBQiAPO5S0ntBDAA3cLFNI02NviW0TQWSw6s/3lfb/DwQ== X-Received: by 2002:aa7:8457:0:b0:70c:e1ec:4233 with SMTP id d2e1a72fcca58-70e80779778mr1312903b3a.6.1721705710365; Mon, 22 Jul 2024 20:35:10 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Nicholas Piggin , BALATON Zoltan Subject: [PATCH 05/12] target/ppc: Hoist dcbz_size out of dcbz_common Date: Tue, 23 Jul 2024 13:34:44 +1000 Message-ID: <20240723033451.546151-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The 970 logic does not apply to dcbzep, which is an e500 insn. Reviewed-by: Nicholas Piggin Reviewed-by: BALATON Zoltan Signed-off-by: Richard Henderson --- target/ppc/mem_helper.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 361fd72226..5067919ff8 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -271,22 +271,12 @@ void helper_stsw(CPUPPCState *env, target_ulong addr, uint32_t nb, } static void dcbz_common(CPUPPCState *env, target_ulong addr, - uint32_t opcode, int mmu_idx, uintptr_t retaddr) + int dcbz_size, int mmu_idx, uintptr_t retaddr) { - target_ulong mask, dcbz_size = env->dcache_line_size; - uint32_t i; + target_ulong mask = ~(target_ulong)(dcbz_size - 1); void *haddr; -#if defined(TARGET_PPC64) - /* Check for dcbz vs dcbzl on 970 */ - if (env->excp_model == POWERPC_EXCP_970 && - !(opcode & 0x00200000) && ((env->spr[SPR_970_HID5] >> 7) & 0x3) == 1) { - dcbz_size = 32; - } -#endif - /* Align address */ - mask = ~(dcbz_size - 1); addr &= mask; /* Check reservation */ @@ -300,7 +290,7 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, memset(haddr, 0, dcbz_size); } else { /* Slow path */ - for (i = 0; i < dcbz_size; i += 8) { + for (int i = 0; i < dcbz_size; i += 8) { cpu_stq_mmuidx_ra(env, addr + i, 0, mmu_idx, retaddr); } } @@ -308,12 +298,22 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, void helper_dcbz(CPUPPCState *env, target_ulong addr, uint32_t opcode) { - dcbz_common(env, addr, opcode, ppc_env_mmu_index(env, false), GETPC()); + int dcbz_size = env->dcache_line_size; + +#if defined(TARGET_PPC64) + /* Check for dcbz vs dcbzl on 970 */ + if (env->excp_model == POWERPC_EXCP_970 && + !(opcode & 0x00200000) && ((env->spr[SPR_970_HID5] >> 7) & 0x3) == 1) { + dcbz_size = 32; + } +#endif + + dcbz_common(env, addr, dcbz_size, ppc_env_mmu_index(env, false), GETPC()); } void helper_dcbzep(CPUPPCState *env, target_ulong addr, uint32_t opcode) { - dcbz_common(env, addr, opcode, PPC_TLB_EPID_STORE, GETPC()); + dcbz_common(env, addr, env->dcache_line_size, PPC_TLB_EPID_STORE, GETPC()); } void helper_icbi(CPUPPCState *env, target_ulong addr) From patchwork Tue Jul 23 03:34:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 18941C3DA63 for ; Tue, 23 Jul 2024 03:37:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J6-0002eW-BG; Mon, 22 Jul 2024 23:35:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6J2-0002RL-TV for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:16 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6J0-0001xq-Mc for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:16 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-70d1d818c42so1122102b3a.1 for ; Mon, 22 Jul 2024 20:35:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705713; x=1722310513; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vMEEORHFY/5AJeKs7AXO2kip576OxvLWKFUcuNYKW6c=; b=aQn+7Z0fIEZjXKw0cu+5kpbaqSsSJaEV3l8j4h6auVlEIb985VZjQ4oHYIDdUhmQnF Jm3U7Cui7gKUdQlKt9vspacptjiznJZlJNoxSD+pkda45e0DHLXqp56lEHt+R6Ncet8g 91t3ckWuBxKK4q/Hu1mPsY4Fd2wfImpWcrewb9O1HwcuTaW52nj2MpK6ORH2YHrV35L3 R6R0GgTBug0h6C0uKKrViND34yKslC5yViM0agaUBnTI2W3m3yacy5wn4NMezxpX7lz1 LB1ennyb0egTkSGFpa3dr+iNuKPvFAIbuXlchX7pBZ+EX3gSh6W0xg+mUCGgC5Jj83xD Wobw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705713; x=1722310513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vMEEORHFY/5AJeKs7AXO2kip576OxvLWKFUcuNYKW6c=; b=YmU8un3UYjujYya0dds6GOhIzurILPWUT4/6Ffbv+D/iLWKN8S/NCbvk9ymXARMa/y KHqmnd3hkyieb7Z41wFhqNUnZ3s128aI18NBtyy/wum7aE8cFIJq49B3gdbsoHB7n+Np 3gKxPIlp7eIej5/4dX3Vkuz3fOv1ZZqzYFciDXiYtqVWnQ6IXLRXIsekv9vq6BlPVur6 ZL0r2G6MhJ92RERiLzb6AWmDeWrO+vMwiMJTKJI9p1YKOXgxtxFJD9WHLo76ECAfIAQm EtGQONDAVwyesUWuEQtHiL+Ug96LlaoTumKz9q1JNgYMMcNihjxW252+hOmpwxyL5lsc hm3Q== X-Gm-Message-State: AOJu0Yy8Z/QykNrQtnrWWyuMDEs+sJSwxmHo6ufKiEvJ3K1qSnca8feL g81bL5he6nKl9yyrP9nSWwiMSY3huSrr6XjqDvvjEmCOe1HYh3hdTTM5hvOVXf4Lu5y+5np0CLO D7WUbAw== X-Google-Smtp-Source: AGHT+IHCY5N9BikBxn26COk7IchKhQs1QcxMpm5TJwv/kJwd1gzCDrnDpV2TihUTgoSzXUVFmwoC4w== X-Received: by 2002:a05:6a00:2e03:b0:706:5dab:83c4 with SMTP id d2e1a72fcca58-70d3a8b177amr2736644b3a.14.1721705713035; Mon, 22 Jul 2024 20:35:13 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Nicholas Piggin , BALATON Zoltan Subject: [PATCH 06/12] target/ppc: Split out helper_dbczl for 970 Date: Tue, 23 Jul 2024 13:34:45 +1000 Message-ID: <20240723033451.546151-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We can determine at translation time whether the insn is or is not dbczl. We must retain a runtime check against the HID5 register, but we can move that to a separate function that never affects other ppc models. Reviewed-by: Nicholas Piggin Reviewed-by: BALATON Zoltan Signed-off-by: Richard Henderson --- target/ppc/helper.h | 7 +++++-- target/ppc/mem_helper.c | 34 +++++++++++++++++++++------------- target/ppc/translate.c | 24 ++++++++++++++---------- 3 files changed, 40 insertions(+), 25 deletions(-) diff --git a/target/ppc/helper.h b/target/ppc/helper.h index 76b8f25c77..afc56855ff 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -46,8 +46,11 @@ DEF_HELPER_FLAGS_3(stmw, TCG_CALL_NO_WG, void, env, tl, i32) DEF_HELPER_4(lsw, void, env, tl, i32, i32) DEF_HELPER_5(lswx, void, env, tl, i32, i32, i32) DEF_HELPER_FLAGS_4(stsw, TCG_CALL_NO_WG, void, env, tl, i32, i32) -DEF_HELPER_FLAGS_3(dcbz, TCG_CALL_NO_WG, void, env, tl, i32) -DEF_HELPER_FLAGS_3(dcbzep, TCG_CALL_NO_WG, void, env, tl, i32) +DEF_HELPER_FLAGS_2(dcbz, TCG_CALL_NO_WG, void, env, tl) +DEF_HELPER_FLAGS_2(dcbzep, TCG_CALL_NO_WG, void, env, tl) +#ifdef TARGET_PPC64 +DEF_HELPER_FLAGS_2(dcbzl, TCG_CALL_NO_WG, void, env, tl) +#endif DEF_HELPER_FLAGS_2(icbi, TCG_CALL_NO_WG, void, env, tl) DEF_HELPER_FLAGS_2(icbiep, TCG_CALL_NO_WG, void, env, tl) DEF_HELPER_5(lscbx, tl, env, tl, i32, i32, i32) diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 5067919ff8..d4957efd6e 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -296,26 +296,34 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, } } -void helper_dcbz(CPUPPCState *env, target_ulong addr, uint32_t opcode) +void helper_dcbz(CPUPPCState *env, target_ulong addr) { - int dcbz_size = env->dcache_line_size; - -#if defined(TARGET_PPC64) - /* Check for dcbz vs dcbzl on 970 */ - if (env->excp_model == POWERPC_EXCP_970 && - !(opcode & 0x00200000) && ((env->spr[SPR_970_HID5] >> 7) & 0x3) == 1) { - dcbz_size = 32; - } -#endif - - dcbz_common(env, addr, dcbz_size, ppc_env_mmu_index(env, false), GETPC()); + dcbz_common(env, addr, env->dcache_line_size, + ppc_env_mmu_index(env, false), GETPC()); } -void helper_dcbzep(CPUPPCState *env, target_ulong addr, uint32_t opcode) +void helper_dcbzep(CPUPPCState *env, target_ulong addr) { dcbz_common(env, addr, env->dcache_line_size, PPC_TLB_EPID_STORE, GETPC()); } +#ifdef TARGET_PPC64 +void helper_dcbzl(CPUPPCState *env, target_ulong addr) +{ + int dcbz_size = env->dcache_line_size; + + /* + * The translator checked for POWERPC_EXCP_970. + * All that's left is to check HID5. + */ + if (((env->spr[SPR_970_HID5] >> 7) & 0x3) == 1) { + dcbz_size = 32; + } + + dcbz_common(env, addr, dcbz_size, ppc_env_mmu_index(env, false), GETPC()); +} +#endif + void helper_icbi(CPUPPCState *env, target_ulong addr) { addr &= ~(env->dcache_line_size - 1); diff --git a/target/ppc/translate.c b/target/ppc/translate.c index 0bc16d7251..9e472ab7ef 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -178,6 +178,7 @@ struct DisasContext { /* Translation flags */ MemOp default_tcg_memop_mask; #if defined(TARGET_PPC64) + powerpc_excp_t excp_model; bool sf_mode; bool has_cfar; bool has_bhrb; @@ -4445,27 +4446,29 @@ static void gen_dcblc(DisasContext *ctx) /* dcbz */ static void gen_dcbz(DisasContext *ctx) { - TCGv tcgv_addr; - TCGv_i32 tcgv_op; + TCGv tcgv_addr = tcg_temp_new(); gen_set_access_type(ctx, ACCESS_CACHE); - tcgv_addr = tcg_temp_new(); - tcgv_op = tcg_constant_i32(ctx->opcode & 0x03FF000); gen_addr_reg_index(ctx, tcgv_addr); - gen_helper_dcbz(tcg_env, tcgv_addr, tcgv_op); + +#ifdef TARGET_PPC64 + if (ctx->excp_model == POWERPC_EXCP_970 && !(ctx->opcode & 0x00200000)) { + gen_helper_dcbzl(tcg_env, tcgv_addr); + return; + } +#endif + + gen_helper_dcbz(tcg_env, tcgv_addr); } /* dcbzep */ static void gen_dcbzep(DisasContext *ctx) { - TCGv tcgv_addr; - TCGv_i32 tcgv_op; + TCGv tcgv_addr = tcg_temp_new(); gen_set_access_type(ctx, ACCESS_CACHE); - tcgv_addr = tcg_temp_new(); - tcgv_op = tcg_constant_i32(ctx->opcode & 0x03FF000); gen_addr_reg_index(ctx, tcgv_addr); - gen_helper_dcbzep(tcg_env, tcgv_addr, tcgv_op); + gen_helper_dcbzep(tcg_env, tcgv_addr); } /* dst / dstt */ @@ -6486,6 +6489,7 @@ static void ppc_tr_init_disas_context(DisasContextBase *dcbase, CPUState *cs) ctx->default_tcg_memop_mask = ctx->le_mode ? MO_LE : MO_BE; ctx->flags = env->flags; #if defined(TARGET_PPC64) + ctx->excp_model = env->excp_model; ctx->sf_mode = (hflags >> HFLAGS_64) & 1; ctx->has_cfar = !!(env->flags & POWERPC_FLAG_CFAR); ctx->has_bhrb = !!(env->flags & POWERPC_FLAG_BHRB); From patchwork Tue Jul 23 03:34:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 865A2C3DA63 for ; Tue, 23 Jul 2024 03:36:06 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J9-0002qz-0W; Mon, 22 Jul 2024 23:35:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6J5-0002b1-8t for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:19 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6J3-0001yL-DF for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:18 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-70d24ee678cso1266765b3a.0 for ; Mon, 22 Jul 2024 20:35:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705716; x=1722310516; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qvZQYGxfhFbqPHPcUgtNXtbdH2OSgyvIGjZjBPS5VAI=; b=ZA8HimR2ev5/mALl9DogQZ8cfGNteOWSGqthzfTo++9LOVEK48AsoNhqnEo+Q4zW56 UMuHynzVyOojQYtM5hHv5+9L6VMoxGqMrW+d4ce0HQUPcre5hDpiSEv1sLRnv28I/WYy bXUQJltfJZpvnYUJRlYaR2mvbk8V9w92fp/Ukwf3QE70zCPlKmK5CLRij+R4TIF6XWfL FAk8ywGLEfLpUs2auhzl3L39gdZJgCvjnLrG/LSC7+QH+p3NAf+/Ers5l8DNv461bNPj W9OCLsy8TmCAXv8cLzOUbSCjxV1EZTMSGjPM3NpZiWEp3QwmxSRl2laZ5aTMFadnvBhG aSqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705716; x=1722310516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qvZQYGxfhFbqPHPcUgtNXtbdH2OSgyvIGjZjBPS5VAI=; b=FJY8LZDnN6uG0TN08mhDHb7E+hCj50s/FgfxmVJWhMmb06w3Pd32fYhZyhc92rJ3Wq NwuT2xavJ1I7zoJdinYfEcEhFfKKSl172RF/wYSxviG6ZlDpuHEXRgtr2T4MRR/O75rt m4UVR4hIFpe40abIbd4EGvS+an1LzEmFPp7pPlpaNOa+C7SgElbRhtfoF9jHbg2p+bDd kTHbZqtSH2Gejo+ZKKytl8zkOBJsSExyw9xK5M13Gv2l0AttLBYGyCK1t/nMdFz8IID3 4wkWOR4jKlcO0xvqavNruDBUnD+SzCl+p0nFzrCoBHihh4Xk3xMBVyAiyOgajN4f2dXz sR2g== X-Gm-Message-State: AOJu0Yxu678z8J9KIa80yxBP+z+Ic81XgTb76PwYdWC/9C2vXViDk7Nj 1Xt/hzC7U6MUohUGEp9yGmledoVa5aB/O0/IYh41cjo4/vmP4yml0FWodSSH5eopzrbLyVAHryW G7Sxh1g== X-Google-Smtp-Source: AGHT+IHqxQg7bcdbbJXYOsARf99xSp0swsxEGbO+JuNANLa0Gc32UM6509PM6wsFywuIyT2+ewjwGw== X-Received: by 2002:a05:6a00:2e86:b0:70d:2497:383 with SMTP id d2e1a72fcca58-70d24970528mr4861457b3a.18.1721705715463; Mon, 22 Jul 2024 20:35:15 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:15 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Nicholas Piggin Subject: [PATCH 07/12] target/ppc: Merge helper_{dcbz,dcbzep} Date: Tue, 23 Jul 2024 13:34:46 +1000 Message-ID: <20240723033451.546151-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Merge the two and pass the mmu_idx directly from translation. Swap the argument order in dcbz_common to avoid extra swaps. Reviewed-by: Nicholas Piggin Signed-off-by: Richard Henderson --- target/ppc/helper.h | 3 +-- target/ppc/mem_helper.c | 14 ++++---------- target/ppc/translate.c | 4 ++-- 3 files changed, 7 insertions(+), 14 deletions(-) diff --git a/target/ppc/helper.h b/target/ppc/helper.h index afc56855ff..4fa089cbf9 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -46,8 +46,7 @@ DEF_HELPER_FLAGS_3(stmw, TCG_CALL_NO_WG, void, env, tl, i32) DEF_HELPER_4(lsw, void, env, tl, i32, i32) DEF_HELPER_5(lswx, void, env, tl, i32, i32, i32) DEF_HELPER_FLAGS_4(stsw, TCG_CALL_NO_WG, void, env, tl, i32, i32) -DEF_HELPER_FLAGS_2(dcbz, TCG_CALL_NO_WG, void, env, tl) -DEF_HELPER_FLAGS_2(dcbzep, TCG_CALL_NO_WG, void, env, tl) +DEF_HELPER_FLAGS_3(dcbz, TCG_CALL_NO_WG, void, env, tl, int) #ifdef TARGET_PPC64 DEF_HELPER_FLAGS_2(dcbzl, TCG_CALL_NO_WG, void, env, tl) #endif diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index d4957efd6e..24bae3b80c 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -271,7 +271,7 @@ void helper_stsw(CPUPPCState *env, target_ulong addr, uint32_t nb, } static void dcbz_common(CPUPPCState *env, target_ulong addr, - int dcbz_size, int mmu_idx, uintptr_t retaddr) + int mmu_idx, int dcbz_size, uintptr_t retaddr) { target_ulong mask = ~(target_ulong)(dcbz_size - 1); void *haddr; @@ -296,15 +296,9 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, } } -void helper_dcbz(CPUPPCState *env, target_ulong addr) +void helper_dcbz(CPUPPCState *env, target_ulong addr, int mmu_idx) { - dcbz_common(env, addr, env->dcache_line_size, - ppc_env_mmu_index(env, false), GETPC()); -} - -void helper_dcbzep(CPUPPCState *env, target_ulong addr) -{ - dcbz_common(env, addr, env->dcache_line_size, PPC_TLB_EPID_STORE, GETPC()); + dcbz_common(env, addr, mmu_idx, env->dcache_line_size, GETPC()); } #ifdef TARGET_PPC64 @@ -320,7 +314,7 @@ void helper_dcbzl(CPUPPCState *env, target_ulong addr) dcbz_size = 32; } - dcbz_common(env, addr, dcbz_size, ppc_env_mmu_index(env, false), GETPC()); + dcbz_common(env, addr, ppc_env_mmu_index(env, false), dcbz_size, GETPC()); } #endif diff --git a/target/ppc/translate.c b/target/ppc/translate.c index 9e472ab7ef..cba943a49d 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -4458,7 +4458,7 @@ static void gen_dcbz(DisasContext *ctx) } #endif - gen_helper_dcbz(tcg_env, tcgv_addr); + gen_helper_dcbz(tcg_env, tcgv_addr, tcg_constant_i32(ctx->mem_idx)); } /* dcbzep */ @@ -4468,7 +4468,7 @@ static void gen_dcbzep(DisasContext *ctx) gen_set_access_type(ctx, ACCESS_CACHE); gen_addr_reg_index(ctx, tcgv_addr); - gen_helper_dcbzep(tcg_env, tcgv_addr); + gen_helper_dcbz(tcg_env, tcgv_addr, tcg_constant_i32(PPC_TLB_EPID_STORE)); } /* dst / dstt */ From patchwork Tue Jul 23 03:34:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739304 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F167CC3DA63 for ; Tue, 23 Jul 2024 03:36:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6J8-0002oN-Di; Mon, 22 Jul 2024 23:35:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6J7-0002jr-Ai for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:21 -0400 Received: from mail-oi1-x22b.google.com ([2607:f8b0:4864:20::22b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6J5-0001yh-Gh for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:21 -0400 Received: by mail-oi1-x22b.google.com with SMTP id 5614622812f47-3daf0e73a5bso1294812b6e.0 for ; Mon, 22 Jul 2024 20:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705718; x=1722310518; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uh4gAqAS6TTZXCMO8wLgp4N1p8byuXj34zq25Ai1zrU=; b=p52uYPIWwO8lYKuwnYun/0BYYakmQHATaLXmUawm5weeT8PmF9PTPPNRrec8jt1PNO NwWtWxmDWAFbgaYTLXOv8h8ylS1u95gNzRD6vLNgRT/t6Y7nbsqY9YAZDQ7Yu18zYLvr lARkZ2hIMD/iwPsQUY/FtxDmKZgrlJOq5WWuIpIFIfPVhGVZ5+w1rWgdDja7eztMI7NB AdqC2kbD4RubhadRWgJpBpu6sDT/OTamlGUM5vuiepnSHWwPz8nzEBexZ38VmwImnw5K oJfaydXdG9HZwRIGI3M04NRSI7WDFmPLWT1BEJ/le0lVukiAUdeFj1/2ZXOPyWJJDEKY krQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705718; x=1722310518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uh4gAqAS6TTZXCMO8wLgp4N1p8byuXj34zq25Ai1zrU=; b=Wu91vOMgr4YJkFgNGe2PMCaEE9ThYC5f/C/7rlj+a9DF/c/gUd29/4WcEkTvpM/f9O SvBKMICJkfXfa3Qs3Lp/4ROwE4Y3rb44SkddIGm7ofidzT5CL+Lbtafi2X75bODwDbFM eG5T518GIzu1vURsi48r6+gmO+W5ZlLp9lBvpQ6NpJ7wsBFoAEIu5bbDblDWAgyGJ45O xoGIr7mU6IxQQohttfXGLZHxEMljuFobQMpO+shh0pZbp+zkXHkFPmq6MtQQA3yuOehB uJwMV59tkKg3dKNLHHIlC0jAEqKtGBkaHpf3ENx8GNDMYSoELUwNBoo7qtu5kyxK4dm4 VvJw== X-Gm-Message-State: AOJu0YyyUQy6RK0TuEKC436EPJQ+YKxBsC0TI2aycZX9o1aiDHlfZp0g xvYUAKoUEWQkb1FM4rMnV9K+S+F7KSaONtmb8x+gq5c1G3aL57aRGAiMod4e+76Vx1qdzlZNSVL W/UYfnA== X-Google-Smtp-Source: AGHT+IFTbnn0KiQQBtdkxHYkTF/rWc2DJKfdPF68qi95FX4DlU7NJXy3c3SWRaBZ12mB2ThQQRzlvA== X-Received: by 2002:a05:6808:1514:b0:3da:a0a5:a249 with SMTP id 5614622812f47-3dae5fb0fbdmr11785351b6e.23.1721705718052; Mon, 22 Jul 2024 20:35:18 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 08/12] target/ppc: Improve helper_dcbz for user-only Date: Tue, 23 Jul 2024 13:34:47 +1000 Message-ID: <20240723033451.546151-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22b; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Mark the reserve_addr check unlikely. Use tlb_vaddr_to_host instead of probe_write, relying on the memset itself to test for page writability. Use set/clear_helper_retaddr so that we can properly unwind on segfault. With this, a trivial loop around guest memset will no longer spend nearly 25% of runtime within page_get_flags. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/ppc/mem_helper.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 24bae3b80c..953dd08d5d 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -280,20 +280,27 @@ static void dcbz_common(CPUPPCState *env, target_ulong addr, addr &= mask; /* Check reservation */ - if ((env->reserve_addr & mask) == addr) { + if (unlikely((env->reserve_addr & mask) == addr)) { env->reserve_addr = (target_ulong)-1ULL; } /* Try fast path translate */ +#ifdef CONFIG_USER_ONLY + haddr = tlb_vaddr_to_host(env, addr, MMU_DATA_STORE, mmu_idx); +#else haddr = probe_write(env, addr, dcbz_size, mmu_idx, retaddr); - if (haddr) { - memset(haddr, 0, dcbz_size); - } else { + if (unlikely(!haddr)) { /* Slow path */ for (int i = 0; i < dcbz_size; i += 8) { cpu_stq_mmuidx_ra(env, addr + i, 0, mmu_idx, retaddr); } + return; } +#endif + + set_helper_retaddr(retaddr); + memset(haddr, 0, dcbz_size); + clear_helper_retaddr(); } void helper_dcbz(CPUPPCState *env, target_ulong addr, int mmu_idx) From patchwork Tue Jul 23 03:34:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 919F1C3DA49 for ; Tue, 23 Jul 2024 03:36:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6JB-00031a-4a; Mon, 22 Jul 2024 23:35:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6J9-0002uS-Ho for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:23 -0400 Received: from mail-oa1-x35.google.com ([2001:4860:4864:20::35]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6J7-0001yu-VG for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:23 -0400 Received: by mail-oa1-x35.google.com with SMTP id 586e51a60fabf-260f057aa9bso2815508fac.2 for ; Mon, 22 Jul 2024 20:35:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705720; x=1722310520; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0Aw83kLLuGjCQHx2MdR2LGZJ7cF9lloOkwjusYN+Kq0=; b=a2hNxd9okMXPrPsJgNol1bFaRISM+BTqi7Kr8Q37cGkAir4tges/a0aqXcPp4hW2XG gPFY6gilsf9DfB5d3KvNt6gbTZwyCpazx8xOXqyXEtvqn/OVA7uQmLPKqljVnZyGfEWg IRpRI+sR44zsVn8fe9A5USym8jb1fW/lK7bl5wK6zATUo3+2Cn1QJcUmzNlY9S/DlYJt vQBO97i6DjE1DC0rXZsFHOcUgst2M+FgFXKvIMMerFy0b5alc2eATIUPJdJwtiC0pg2E cFu0HQIc2ATgQg2VVvXS9mR/ME5JRAQYa4n3gHeoBD0CTr1VbYwBCRYLD/MM14kaJC2g U2hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705720; x=1722310520; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0Aw83kLLuGjCQHx2MdR2LGZJ7cF9lloOkwjusYN+Kq0=; b=cbjatLq+tI4ra2G/lH2kPIclMelM5BmQ/hzFXDOryaPfihUN6K/9HQaCoWp5qQiRi6 PTjir6n/90pyHG5r/zgZdc3G120nNRxVpbiC5gtpIuJWi/D7WhO5k2Z6M21VqHgvfnAW 2GCOG4rUixXokTkH7NZ4JodTJCuGqZQYNFW1Gmqn+JUYoq5PqRdEE8+6oAyfuIukqCWk mlcTcGJH799DNJIZcX/0/iYrejnRY2JGExPPQocGKElGr+JW3npP0lFYWdlLPKy05shN KZKs0ZrkJH/Xzu4dxYzfYnryvsGZ9+4VlD/nLSPCRCvqV6QtN/423/HsNf8w9eiR1BjG RYfw== X-Gm-Message-State: AOJu0YzhHd3uwCDgmRphs4ryC4izEk8RDySvx0O97Gqo2HKf1sf51bHf N1YUzBEizQPqZ94fyNUD8mlRA2ITJrxOrjzftlhd8mRSaz9/UbTIoeGO4vEvZ3ZsKQ4xmVkXGcl +/FRmhg== X-Google-Smtp-Source: AGHT+IF0V+oohgp57Ifix300JeB1ZCupg0XVrWSm/A3RxXNMWdZKzLnEzZRNKorC2Xq+sV/IOqRbUw== X-Received: by 2002:a05:6870:d627:b0:260:fbc0:96f2 with SMTP id 586e51a60fabf-264692d60bdmr1545848fac.34.1721705720471; Mon, 22 Jul 2024 20:35:20 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 09/12] target/s390x: Use user_or_likely in do_access_memset Date: Tue, 23 Jul 2024 13:34:48 +1000 Message-ID: <20240723033451.546151-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::35; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Eliminate the ifdef by using a predicate that is always true with CONFIG_USER_ONLY. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/s390x/tcg/mem_helper.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 6cdbc34178..5311a15a09 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -225,10 +225,7 @@ static void do_access_memset(CPUS390XState *env, vaddr vaddr, char *haddr, uint8_t byte, uint16_t size, int mmu_idx, uintptr_t ra) { -#ifdef CONFIG_USER_ONLY - memset(haddr, byte, size); -#else - if (likely(haddr)) { + if (user_or_likely(haddr)) { memset(haddr, byte, size); } else { MemOpIdx oi = make_memop_idx(MO_UB, mmu_idx); @@ -236,7 +233,6 @@ static void do_access_memset(CPUS390XState *env, vaddr vaddr, char *haddr, cpu_stb_mmu(env, vaddr + i, byte, oi, ra); } } -#endif } static void access_memset(CPUS390XState *env, S390Access *desta, From patchwork Tue Jul 23 03:34:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A081C3DA49 for ; Tue, 23 Jul 2024 03:36:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6JD-0003Cg-Vl; Mon, 22 Jul 2024 23:35:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6JC-00035n-6b for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:26 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6JA-0001zD-9F for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:25 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-70d138e06e3so1921708b3a.2 for ; Mon, 22 Jul 2024 20:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705723; x=1722310523; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bg9qUEQyPB3zLUZtLfqXq0g2/mBlvx+VDk4J1iPFK7Y=; b=vV8wlmQajOqaKAwZXENLOCvbcOX8SmTEJZO9a8E/3vhyxCnZG7TTMJvgpffAW0UJKe FuuHgd2LoHI7oJnC+XGi258kzTCWjso6NWazivYVL6z2AXtvNtaeSQa8BZx58gyD6kE+ tc3awAcXUKTf3VPrQx/U2x6yoExIr+PVYP2AAUTaQUH5gd1ddXnFyPGMk9ZPjRIXDtvn My1Lt8pkDZHyQ66s6PAhy/XC+EXp+NLKc50gPnGEwVFSUvj0uyQQnwdU3G8jpkh1SnWG sRyJNii7ukM3MxyLxjm9vdvSNDutKQiLBb5iBVuIYum5+qnQ85db6FS1fIbuvpOcqsa5 SLDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705723; x=1722310523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bg9qUEQyPB3zLUZtLfqXq0g2/mBlvx+VDk4J1iPFK7Y=; b=VZ8GgrKRpPifGZTm/z2/TH7mi3CIrnynO2dy7zg29MM3FizYey8kegkufW+cnONsFw +WDx+Fper0A43ZRLlGQeCPx5EqBsxJDuMDLsy7hsPE/+XEM0nSNeI+ufCGm7DlAU/ZiV abE/ImB/yrpio72393Jz5A6HO7Q0EcVig8CNPbP3Hu1MSfS0BEOB1D4hOjNUki3CD7s7 xMbr/PGAFIzgC5HLKhPbhxPHUCRTnKZSgdEeC+DkaWvbBRKUv8dlu+3FVSseO20rNnkx WzDjriMQaWXOg2aWaoSg35VlO+rGZ0APES4p/rY+bMsoWjUp6O6xCYfQg0YfI3bmenkK IqwQ== X-Gm-Message-State: AOJu0YxFoHP6W46lRLKwV46oKwFKa2edalOg5KBsIdJlbPXeTS28ODZO aMLbzk5ZE+lNKiYMF3jmDOkbnwxA0iCjpU1H7XZ8/t4LBdT89HCEjv01SG0bcIJFaHgBat8QGnp 0LJOFLA== X-Google-Smtp-Source: AGHT+IGvciqi8cV0n+WG1q0Mb6FyyWyOw1eaxsFtweQ0HlZc2gP1bOyntZvczofZRoIR+h0e6UROCQ== X-Received: by 2002:a05:6a00:3991:b0:706:750c:8dda with SMTP id d2e1a72fcca58-70d0847c8e3mr12793630b3a.6.1721705722751; Mon, 22 Jul 2024 20:35:22 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 10/12] target/s390x: Use user_or_likely in access_memmove Date: Tue, 23 Jul 2024 13:34:49 +1000 Message-ID: <20240723033451.546151-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Invert the conditional, indent the block, and use the macro that expands to true for user-only. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/s390x/tcg/mem_helper.c | 54 +++++++++++++++++------------------ 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 5311a15a09..331a35b2e5 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -296,41 +296,39 @@ static void access_memmove(CPUS390XState *env, S390Access *desta, S390Access *srca, uintptr_t ra) { int len = desta->size1 + desta->size2; - int diff; assert(len == srca->size1 + srca->size2); /* Fallback to slow access in case we don't have access to all host pages */ - if (unlikely(!desta->haddr1 || (desta->size2 && !desta->haddr2) || - !srca->haddr1 || (srca->size2 && !srca->haddr2))) { - int i; + if (user_or_likely(desta->haddr1 && + srca->haddr1 && + (!desta->size2 || desta->haddr2) && + (!srca->size2 || srca->haddr2))) { + int diff = desta->size1 - srca->size1; - for (i = 0; i < len; i++) { - uint8_t byte = access_get_byte(env, srca, i, ra); - - access_set_byte(env, desta, i, byte, ra); - } - return; - } - - diff = desta->size1 - srca->size1; - if (likely(diff == 0)) { - memmove(desta->haddr1, srca->haddr1, srca->size1); - if (unlikely(srca->size2)) { - memmove(desta->haddr2, srca->haddr2, srca->size2); - } - } else if (diff > 0) { - memmove(desta->haddr1, srca->haddr1, srca->size1); - memmove(desta->haddr1 + srca->size1, srca->haddr2, diff); - if (likely(desta->size2)) { - memmove(desta->haddr2, srca->haddr2 + diff, desta->size2); + if (likely(diff == 0)) { + memmove(desta->haddr1, srca->haddr1, srca->size1); + if (unlikely(srca->size2)) { + memmove(desta->haddr2, srca->haddr2, srca->size2); + } + } else if (diff > 0) { + memmove(desta->haddr1, srca->haddr1, srca->size1); + memmove(desta->haddr1 + srca->size1, srca->haddr2, diff); + if (likely(desta->size2)) { + memmove(desta->haddr2, srca->haddr2 + diff, desta->size2); + } + } else { + diff = -diff; + memmove(desta->haddr1, srca->haddr1, desta->size1); + memmove(desta->haddr2, srca->haddr1 + desta->size1, diff); + if (likely(srca->size2)) { + memmove(desta->haddr2 + diff, srca->haddr2, srca->size2); + } } } else { - diff = -diff; - memmove(desta->haddr1, srca->haddr1, desta->size1); - memmove(desta->haddr2, srca->haddr1 + desta->size1, diff); - if (likely(srca->size2)) { - memmove(desta->haddr2 + diff, srca->haddr2, srca->size2); + for (int i = 0; i < len; i++) { + uint8_t byte = access_get_byte(env, srca, i, ra); + access_set_byte(env, desta, i, byte, ra); } } } From patchwork Tue Jul 23 03:34:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10139C3DA63 for ; Tue, 23 Jul 2024 03:37:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6JG-0003OR-QK; Mon, 22 Jul 2024 23:35:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6JF-0003KQ-Ry for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:29 -0400 Received: from mail-yw1-x1134.google.com ([2607:f8b0:4864:20::1134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6JC-0001zO-Mr for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:29 -0400 Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-66599ca3470so50993797b3.2 for ; Mon, 22 Jul 2024 20:35:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705725; x=1722310525; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FmWwVxVRnF8pwGo5KrCOQQuxqHCEbEF4oIcvEOO+M5Y=; b=o/NEWhbdG8ukSDHmEFhntdyuTMpLXCrFgEztfvH2PjNhEbIHL7yus6q5c1wF8RB0FA ZN2N/4p38EXfY9qiNWibfy7fL+pKgYS+5cFKSks6Dv09c7L8JICzv7xfKjuzv5VBQo6F tb77bDnp42N6/84kQF+mA4ma9uIltn7y0EpAnTI3VXwP4saS9CCmrEyYsHrqlTMTTRAQ 5r9mYbM7ZElGdm133RE2GEZGjrUA1WCPCpHBmt0LlSytpjYp2hM/diaAeLryevgsgAuk yCqzZ6gR3o/sze/+FbMHtf4fJCzS0YHmg/fKZKl+Kvm413zBMZCu46ytcKHk9oT5s+CR kXVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705725; x=1722310525; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FmWwVxVRnF8pwGo5KrCOQQuxqHCEbEF4oIcvEOO+M5Y=; b=iH4mL5mMdz2nQ7hN1b3Qn9f1n0sljQFpgzUFl5Lgdrj4VLPJFQTHHbMd6fOUSc6SZA Y3wJ56qC0FmuXgu2ZTqG8mSMaY+EMO0vYbNJ3SgWz4xxyUrZ3kGTiLCqIbNRe88aqmtO KbiCeuTy6jU4bxwDaU88cEtR1mwOBnLNZLmc4903U7A+pCjAbdsfuKat8fllw0/+7LX2 xt8c/tzTj8/PlLtXXkR0IoPhiYZvpRvr7ZJdKUYZWp2oBt8R02t2EbFDRsHw6vJgGXTH JDvkE8oeHfUqpXHBrta7jBuAk/I/cCvNRZbLaIHJ9uQ8R4UH2dJSV9YVrCIPU/eU8nsV em4A== X-Gm-Message-State: AOJu0YwX7j9uRAoKyAEJUbMVkvofwtpvctfLONov+wLY5jV2vyNVUn4a pHbTEi+qUSRnLSCbClG4+TZcwFEtnhS3Db1rhp164XUSmFufyYL9yvB3KDFpYYmyo+yQ+jmPNjb NFQR9gA== X-Google-Smtp-Source: AGHT+IF8wmvPPHJCiQjtx3s2tL5D+m8K4ezqmu5knTi2aD/51pzHqxXa6MTIuLhi8a8ntHlIqzK2Yg== X-Received: by 2002:a05:690c:290b:b0:62c:c684:b1e1 with SMTP id 00721157ae682-66a682ce848mr107265187b3.29.1721705725132; Mon, 22 Jul 2024 20:35:25 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Peter Maydell Subject: [PATCH 11/12] target/s390x: Use set/clear_helper_retaddr in mem_helper.c Date: Tue, 23 Jul 2024 13:34:50 +1000 Message-ID: <20240723033451.546151-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1134; envelope-from=richard.henderson@linaro.org; helo=mail-yw1-x1134.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Avoid a race condition with munmap in another thread. For access_memset and access_memmove, manage the value within the helper. For uses of access_{get,set}_byte, manage the value across the for loops. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- target/s390x/tcg/mem_helper.c | 43 ++++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 6 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 331a35b2e5..0e12dae2aa 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -238,14 +238,14 @@ static void do_access_memset(CPUS390XState *env, vaddr vaddr, char *haddr, static void access_memset(CPUS390XState *env, S390Access *desta, uint8_t byte, uintptr_t ra) { - + set_helper_retaddr(ra); do_access_memset(env, desta->vaddr1, desta->haddr1, byte, desta->size1, desta->mmu_idx, ra); - if (likely(!desta->size2)) { - return; + if (unlikely(desta->size2)) { + do_access_memset(env, desta->vaddr2, desta->haddr2, byte, + desta->size2, desta->mmu_idx, ra); } - do_access_memset(env, desta->vaddr2, desta->haddr2, byte, desta->size2, - desta->mmu_idx, ra); + clear_helper_retaddr(); } static uint8_t access_get_byte(CPUS390XState *env, S390Access *access, @@ -366,6 +366,8 @@ static uint32_t do_helper_nc(CPUS390XState *env, uint32_t l, uint64_t dest, access_prepare(&srca1, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&srca2, env, dest, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + set_helper_retaddr(ra); + for (i = 0; i < l; i++) { const uint8_t x = access_get_byte(env, &srca1, i, ra) & access_get_byte(env, &srca2, i, ra); @@ -373,6 +375,8 @@ static uint32_t do_helper_nc(CPUS390XState *env, uint32_t l, uint64_t dest, c |= x; access_set_byte(env, &desta, i, x, ra); } + + clear_helper_retaddr(); return c != 0; } @@ -407,6 +411,7 @@ static uint32_t do_helper_xc(CPUS390XState *env, uint32_t l, uint64_t dest, return 0; } + set_helper_retaddr(ra); for (i = 0; i < l; i++) { const uint8_t x = access_get_byte(env, &srca1, i, ra) ^ access_get_byte(env, &srca2, i, ra); @@ -414,6 +419,7 @@ static uint32_t do_helper_xc(CPUS390XState *env, uint32_t l, uint64_t dest, c |= x; access_set_byte(env, &desta, i, x, ra); } + clear_helper_retaddr(); return c != 0; } @@ -441,6 +447,8 @@ static uint32_t do_helper_oc(CPUS390XState *env, uint32_t l, uint64_t dest, access_prepare(&srca1, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&srca2, env, dest, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + set_helper_retaddr(ra); + for (i = 0; i < l; i++) { const uint8_t x = access_get_byte(env, &srca1, i, ra) | access_get_byte(env, &srca2, i, ra); @@ -448,6 +456,8 @@ static uint32_t do_helper_oc(CPUS390XState *env, uint32_t l, uint64_t dest, c |= x; access_set_byte(env, &desta, i, x, ra); } + + clear_helper_retaddr(); return c != 0; } @@ -484,11 +494,13 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, } else if (!is_destructive_overlap(env, dest, src, l)) { access_memmove(env, &desta, &srca, ra); } else { + set_helper_retaddr(ra); for (i = 0; i < l; i++) { uint8_t byte = access_get_byte(env, &srca, i, ra); access_set_byte(env, &desta, i, byte, ra); } + clear_helper_retaddr(); } return env->cc_op; @@ -514,10 +526,12 @@ void HELPER(mvcrl)(CPUS390XState *env, uint64_t l, uint64_t dest, uint64_t src) access_prepare(&srca, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + set_helper_retaddr(ra); for (i = l - 1; i >= 0; i--) { uint8_t byte = access_get_byte(env, &srca, i, ra); access_set_byte(env, &desta, i, byte, ra); } + clear_helper_retaddr(); } /* move inverse */ @@ -534,11 +548,13 @@ void HELPER(mvcin)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src) src = wrap_address(env, src - l + 1); access_prepare(&srca, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + + set_helper_retaddr(ra); for (i = 0; i < l; i++) { const uint8_t x = access_get_byte(env, &srca, l - i - 1, ra); - access_set_byte(env, &desta, i, x, ra); } + clear_helper_retaddr(); } /* move numerics */ @@ -555,12 +571,15 @@ void HELPER(mvn)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src) access_prepare(&srca1, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&srca2, env, dest, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + + set_helper_retaddr(ra); for (i = 0; i < l; i++) { const uint8_t x = (access_get_byte(env, &srca1, i, ra) & 0x0f) | (access_get_byte(env, &srca2, i, ra) & 0xf0); access_set_byte(env, &desta, i, x, ra); } + clear_helper_retaddr(); } /* move with offset */ @@ -580,6 +599,8 @@ void HELPER(mvo)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src) /* Handle rightmost byte */ byte_dest = cpu_ldub_data_ra(env, dest + len_dest - 1, ra); + + set_helper_retaddr(ra); byte_src = access_get_byte(env, &srca, len_src - 1, ra); byte_dest = (byte_dest & 0x0f) | (byte_src << 4); access_set_byte(env, &desta, len_dest - 1, byte_dest, ra); @@ -595,6 +616,7 @@ void HELPER(mvo)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src) byte_dest |= byte_src << 4; access_set_byte(env, &desta, i, byte_dest, ra); } + clear_helper_retaddr(); } /* move zones */ @@ -611,12 +633,15 @@ void HELPER(mvz)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src) access_prepare(&srca1, env, src, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&srca2, env, dest, l, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, dest, l, MMU_DATA_STORE, mmu_idx, ra); + + set_helper_retaddr(ra); for (i = 0; i < l; i++) { const uint8_t x = (access_get_byte(env, &srca1, i, ra) & 0xf0) | (access_get_byte(env, &srca2, i, ra) & 0x0f); access_set_byte(env, &desta, i, x, ra); } + clear_helper_retaddr(); } /* compare unsigned byte arrays */ @@ -961,15 +986,19 @@ uint32_t HELPER(mvst)(CPUS390XState *env, uint32_t r1, uint32_t r2) */ access_prepare(&srca, env, s, len, MMU_DATA_LOAD, mmu_idx, ra); access_prepare(&desta, env, d, len, MMU_DATA_STORE, mmu_idx, ra); + + set_helper_retaddr(ra); for (i = 0; i < len; i++) { const uint8_t v = access_get_byte(env, &srca, i, ra); access_set_byte(env, &desta, i, v, ra); if (v == c) { + clear_helper_retaddr(); set_address_zero(env, r1, d + i); return 1; } } + clear_helper_retaddr(); set_address_zero(env, r1, d + len); set_address_zero(env, r2, s + len); return 3; @@ -1060,6 +1089,7 @@ static inline uint32_t do_mvcl(CPUS390XState *env, *dest = wrap_address(env, *dest + len); } else { access_prepare(&desta, env, *dest, len, MMU_DATA_STORE, mmu_idx, ra); + set_helper_retaddr(ra); /* The remaining length selects the padding byte. */ for (i = 0; i < len; (*destlen)--, i++) { @@ -1069,6 +1099,7 @@ static inline uint32_t do_mvcl(CPUS390XState *env, access_set_byte(env, &desta, i, pad >> 8, ra); } } + clear_helper_retaddr(); *dest = wrap_address(env, *dest + len); } From patchwork Tue Jul 23 03:34:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13739302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AB852C3DA49 for ; Tue, 23 Jul 2024 03:36:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sW6JI-0003Vs-NM; Mon, 22 Jul 2024 23:35:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sW6JH-0003Pm-4a for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:31 -0400 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sW6JF-0001za-8p for qemu-devel@nongnu.org; Mon, 22 Jul 2024 23:35:30 -0400 Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3d9bcb47182so2979608b6e.3 for ; Mon, 22 Jul 2024 20:35:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1721705728; x=1722310528; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LQzguNqg29abBGKUeSrC6IOQ1lzCyJlmW94ab8MjDZo=; b=wcnUPsoExxpvoiIYJY583a5bwQzgtXSE54eU8ndkmv/5H+vhHYyDqckAiWYUk6zaVQ 5rYGFoNNZX16oMyMwu625Ydb6RGSLnNJR4VoP1KOD5vrD0eI0aLb9cfh2Y4o8ZS6d6Et OOylX99ERSJ9RjEMzKYh0tKqIM/vSSVwnfAFG9tJ6hKNlFcsDpVOopIyxH+LMhvWSK+l CE3RfQp2WTyhDLwOGN4uT0LOJkLMPDtAYc4tvwTkmqDlmwEdbdEiz0hvqW2mQ/uSavTz OC4Vvb9/ZzbECRP/SANU3XMn2obUp1qh3YIKvNlfEFJ7JHyZ41sD6KL/vGrWK3u75Lmr 06Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721705728; x=1722310528; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LQzguNqg29abBGKUeSrC6IOQ1lzCyJlmW94ab8MjDZo=; b=fSc8bu7hEY+mctBNt9Pj+XOmJUberTr6EqMfb8s5s5En6dsNDmRFsvalR+y/OLEZsN m7ej/jNYpv5l7gzhsRgT23VEJukclTsiMD/Vxt6rhiBkZ09ji1eSuZjinq9FQb3DNGIJ b9ivgNdy+DE9uN34n7Kc3ytBESx+2/r+aP34SVal5oz6oUPVfOQTu2LoqkWdj5TQtclu dLA8Ztcw6NqURmej2tRtvOlVdZ9rc4Bl/s0qLxMrvqKzSbozkWbeHUlg8MqUkwS4o/Ra 6Lzg3u8ITCUFvyb9NmLW/upRQPfQe7TSh5FZ373GHJkRqOx+wKsOZjPPg9Oc9mK3Amfl 5xvQ== X-Gm-Message-State: AOJu0Yy1/DmQlTkLRRM1WI2TnMGs+3aIbk1s4rP3empv+ne9mK+pZfq7 EtRtzdM+Ymp0v/gjKlv8iLKYClyTWRm0d87RLaaIfU5SNZ6U/dAG/HsofOsWf0XnuZkkIBWHurO shKGLQg== X-Google-Smtp-Source: AGHT+IERWCpGPSUH4boHPNMrzuCxGv6Fu0/iiUrqA7MKU+Y29WjKhCTonB0SOHfQu+bs2yL+Fo/YeQ== X-Received: by 2002:a05:6808:189e:b0:3d6:3000:9ab3 with SMTP id 5614622812f47-3dae97dfbfdmr12513773b6e.40.1721705727691; Mon, 22 Jul 2024 20:35:27 -0700 (PDT) Received: from stoup.. ([203.56.128.103]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70d2b96bfcasm2171380b3a.87.2024.07.22.20.35.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Jul 2024 20:35:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Max Chou , Alistair Francis Subject: [PATCH 12/12] target/riscv: Simplify probing in vext_ldff Date: Tue, 23 Jul 2024 13:34:51 +1000 Message-ID: <20240723033451.546151-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240723033451.546151-1-richard.henderson@linaro.org> References: <20240723033451.546151-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x235.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The current pairing of tlb_vaddr_to_host with extra is either inefficient (user-only, with page_check_range) or incorrect (system, with probe_pages). For proper non-fault behaviour, use probe_access_flags with its nonfault parameter set to true. Reviewed-by: Max Chou Acked-by: Alistair Francis Signed-off-by: Richard Henderson --- target/riscv/vector_helper.c | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/target/riscv/vector_helper.c b/target/riscv/vector_helper.c index 1b4d5a8e37..10a52ceb5b 100644 --- a/target/riscv/vector_helper.c +++ b/target/riscv/vector_helper.c @@ -474,7 +474,6 @@ vext_ldff(void *vd, void *v0, target_ulong base, vext_ldst_elem_fn *ldst_elem, uint32_t log2_esz, uintptr_t ra) { - void *host; uint32_t i, k, vl = 0; uint32_t nf = vext_nf(desc); uint32_t vm = vext_vm(desc); @@ -493,27 +492,31 @@ vext_ldff(void *vd, void *v0, target_ulong base, } addr = adjust_addr(env, base + i * (nf << log2_esz)); if (i == 0) { + /* Allow fault on first element. */ probe_pages(env, addr, nf << log2_esz, ra, MMU_DATA_LOAD); } else { - /* if it triggers an exception, no need to check watchpoint */ remain = nf << log2_esz; while (remain > 0) { + void *host; + int flags; + offset = -(addr | TARGET_PAGE_MASK); - host = tlb_vaddr_to_host(env, addr, MMU_DATA_LOAD, mmu_index); - if (host) { -#ifdef CONFIG_USER_ONLY - if (!page_check_range(addr, offset, PAGE_READ)) { - vl = i; - goto ProbeSuccess; - } -#else - probe_pages(env, addr, offset, ra, MMU_DATA_LOAD); -#endif - } else { + + /* Probe nonfault on subsequent elements. */ + flags = probe_access_flags(env, addr, offset, MMU_DATA_LOAD, + mmu_index, true, &host, 0); + + /* + * Stop if invalid (unmapped) or mmio (transaction may fail). + * Do not stop if watchpoint, as the spec says that + * first-fault should continue to access the same + * elements regardless of any watchpoint. + */ + if (flags & ~TLB_WATCHPOINT) { vl = i; goto ProbeSuccess; } - if (remain <= offset) { + if (remain <= offset) { break; } remain -= offset;