From patchwork Tue Dec 6 04:16:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BCBA2C3A5A7 for ; Tue, 6 Dec 2022 04:18:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POc-0003Ld-6J; Mon, 05 Dec 2022 23:17:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POZ-0003Kv-1v for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:27 -0500 Received: from mail-oa1-x2c.google.com ([2001:4860:4864:20::2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POT-0006vg-Uz for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:24 -0500 Received: by mail-oa1-x2c.google.com with SMTP id 586e51a60fabf-1322d768ba7so16082190fac.5 for ; Mon, 05 Dec 2022 20:17:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A+8aUePn9jH0yH6KjYmTxIPOGpU5cf4JO1tUTLjFLKY=; b=o/2kP2CaKlVWohGcwXspQwjW9o+boJ0T9KbqS55TQ4EzQGlE2eHhw/QTZLquQIQ5ov Uy6rN0xShpnlRxZhPqch86dfIZ9da49o9PgthtVuNpGQ1qeKHaX+2K5dmLAlYVR95jue luhH0j7izDh2fAUBWJJr9+aykKG24pwznmWdqAMaih37aHPQyJHxv4w7c10rGs8UPh6u 6BmVxdCiq29fUknacPDPjmfcuiBhOyuxMCnXS2YqBX4c6lumGznFK2jxT1jKCrEsXz6+ bxa38Kph+GaZx5xgZB/Makewu6WgGyRA47J6A1nmlKaQgbzZjDiS3rOuKndogd7Wpqd9 YfZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A+8aUePn9jH0yH6KjYmTxIPOGpU5cf4JO1tUTLjFLKY=; b=NDNO//oyekWcQH70IGnfnYojv1NHCzA7oMhm06Bs/xrI1eLGRmXB15XJqlq/exBpPB SX++kqiuVi7RiTWENkGbK0JQmVbIMoKQFOTCkviHWf0Mmhuw1+yG/7eKNJlzbEGpv23R pCwOkjOaAC7xGj+kl8F450fUSB2pxJ6Lv2J3f4RN1T9riowq5zPyIdbJSU4789AaaHtM ujObvvPoMPl79JqDY0WhuZUji8UrszXgosy2oZdvMunnYdoguXYzy9163ETA8Y2XVs6A QX1nl7IsNCKOT93A/vTK6w1gGhzPydJziGXyM+Y163uhISlKt9inCrsrNpt3Ud2+HiyL Af9Q== X-Gm-Message-State: ANoB5pmqnGhIkhEt0+oD0RR5lmLSMMCrNRkdBjrL8C8R0XkiO4a01ros VeFRB6fZBj7cNDiXsdSCVVZa/rDFKqUDaHiR2sU= X-Google-Smtp-Source: AA0mqf7vYjxbwZNxunPl9RKkZL2PTKigK+1UmabnxWq0idUiMHF9Pjb8aHTkW6Jdw7SltsITvmP8AA== X-Received: by 2002:a05:6870:d78a:b0:142:d310:2c9e with SMTP id bd10-20020a056870d78a00b00142d3102c9emr42517986oab.71.1670300240059; Mon, 05 Dec 2022 20:17:20 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:19 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 01/22] tcg: Split out tcg_out_exit_tb Date: Mon, 5 Dec 2022 22:16:54 -0600 Message-Id: <20221206041715.314209-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::2c; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The INDEX_op_exit_tb opcode needs no register allocation. Split out a dedicated helper function for it. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/tcg.c | 4 ++++ tcg/aarch64/tcg-target.c.inc | 22 ++++++++++-------- tcg/arm/tcg-target.c.inc | 11 +++++---- tcg/i386/tcg-target.c.inc | 21 +++++++++-------- tcg/loongarch64/tcg-target.c.inc | 22 ++++++++++-------- tcg/mips/tcg-target.c.inc | 33 +++++++++++++-------------- tcg/ppc/tcg-target.c.inc | 11 +++++---- tcg/riscv/tcg-target.c.inc | 22 ++++++++++-------- tcg/s390x/tcg-target.c.inc | 23 ++++++++++--------- tcg/sparc64/tcg-target.c.inc | 39 +++++++++++++++++--------------- tcg/tci/tcg-target.c.inc | 10 ++++---- 11 files changed, 121 insertions(+), 97 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 436fcf6ebd..b53961baf7 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -107,6 +107,7 @@ static void tcg_out_ld(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg1, static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg); static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg ret, tcg_target_long arg); +static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg); static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]); @@ -4302,6 +4303,9 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) case INDEX_op_call: tcg_reg_alloc_call(s, op); break; + case INDEX_op_exit_tb: + tcg_out_exit_tb(s, op->args[0]); + break; case INDEX_op_dup2_vec: if (tcg_reg_alloc_dup2(s, op)) { break; diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 344b63e20f..16c5e33b69 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1886,6 +1886,17 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, static const tcg_insn_unit *tb_ret_addr; +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + /* Reuse the zeroing that exists for goto_ptr. */ + if (a0 == 0) { + tcg_out_goto_long(s, tcg_code_gen_epilogue); + } else { + tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_X0, a0); + tcg_out_goto_long(s, tb_ret_addr); + } +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1905,16 +1916,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, #define REG0(I) (const_args[I] ? TCG_REG_XZR : (TCGReg)args[I]) switch (opc) { - case INDEX_op_exit_tb: - /* Reuse the zeroing that exists for goto_ptr. */ - if (a0 == 0) { - tcg_out_goto_long(s, tcg_code_gen_epilogue); - } else { - tcg_out_movi(s, TCG_TYPE_I64, TCG_REG_X0, a0); - tcg_out_goto_long(s, tb_ret_addr); - } - break; - case INDEX_op_goto_tb: tcg_debug_assert(s->tb_jmp_insn_offset != NULL); /* @@ -2304,6 +2305,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 2c6c353eea..e5c2eae5a5 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1927,6 +1927,12 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) static void tcg_out_epilogue(TCGContext *s); +static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) +{ + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R0, arg); + tcg_out_epilogue(s); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1935,10 +1941,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c; switch (opc) { - case INDEX_op_exit_tb: - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R0, args[0]); - tcg_out_epilogue(s); - break; case INDEX_op_goto_tb: { /* Indirect jump method */ @@ -2250,6 +2252,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index cb04e4b3ad..5c20bedd20 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2346,6 +2346,17 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) #endif } +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + /* Reuse the zeroing that exists for goto_ptr. */ + if (a0 == 0) { + tcg_out_jmp(s, tcg_code_gen_epilogue); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_EAX, a0); + tcg_out_jmp(s, tb_ret_addr); + } +} + static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -2370,15 +2381,6 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const_a2 = const_args[2]; switch (opc) { - case INDEX_op_exit_tb: - /* Reuse the zeroing that exists for goto_ptr. */ - if (a0 == 0) { - tcg_out_jmp(s, tcg_code_gen_epilogue); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_EAX, a0); - tcg_out_jmp(s, tb_ret_addr); - } - break; case INDEX_op_goto_tb: if (s->tb_jmp_insn_offset) { /* direct jump method */ @@ -2793,6 +2795,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index d326e28740..41fc5ffa91 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1067,6 +1067,17 @@ void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, static const tcg_insn_unit *tb_ret_addr; +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + /* Reuse the zeroing that exists for goto_ptr. */ + if (a0 == 0) { + tcg_out_call_int(s, tcg_code_gen_epilogue, true); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A0, a0); + tcg_out_call_int(s, tb_ret_addr, true); + } +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1077,16 +1088,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c2 = const_args[2]; switch (opc) { - case INDEX_op_exit_tb: - /* Reuse the zeroing that exists for goto_ptr. */ - if (a0 == 0) { - tcg_out_call_int(s, tcg_code_gen_epilogue, true); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A0, a0); - tcg_out_call_int(s, tb_ret_addr, true); - } - break; - case INDEX_op_goto_tb: tcg_debug_assert(s->tb_jmp_insn_offset != NULL); /* @@ -1499,6 +1500,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index bd76f0c97f..819648b100 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1950,6 +1950,21 @@ static void tcg_out_clz(TCGContext *s, MIPSInsn opcv2, MIPSInsn opcv6, } } +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + TCGReg b0 = TCG_REG_ZERO; + + if (a0 & ~0xffff) { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_V0, a0 & ~0xffff); + b0 = TCG_REG_V0; + } + if (!tcg_out_opc_jmp(s, OPC_J, tb_ret_addr)) { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_TMP0, (uintptr_t)tb_ret_addr); + tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); + } + tcg_out_opc_imm(s, OPC_ORI, TCG_REG_V0, b0, a0 & 0xffff); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1969,23 +1984,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, c2 = const_args[2]; switch (opc) { - case INDEX_op_exit_tb: - { - TCGReg b0 = TCG_REG_ZERO; - - a0 = (intptr_t)a0; - if (a0 & ~0xffff) { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_V0, a0 & ~0xffff); - b0 = TCG_REG_V0; - } - if (!tcg_out_opc_jmp(s, OPC_J, tb_ret_addr)) { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_TMP0, - (uintptr_t)tb_ret_addr); - tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); - } - tcg_out_opc_imm(s, OPC_ORI, TCG_REG_V0, b0, a0 & 0xffff); - } - break; case INDEX_op_goto_tb: /* indirect jump method */ tcg_debug_assert(s->tb_jmp_insn_offset == 0); @@ -2402,6 +2400,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index e3dba47697..186599dae0 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2616,6 +2616,12 @@ static void tcg_target_qemu_prologue(TCGContext *s) tcg_out32(s, BCLR | BO_ALWAYS); } +static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) +{ + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R3, arg); + tcg_out_b(s, 0, tcg_code_gen_epilogue); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -2623,10 +2629,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGArg a0, a1, a2; switch (opc) { - case INDEX_op_exit_tb: - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R3, args[0]); - tcg_out_b(s, 0, tcg_code_gen_epilogue); - break; case INDEX_op_goto_tb: if (s->tb_jmp_insn_offset) { /* Direct jump. */ @@ -3185,6 +3187,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 81a83e45b1..1381c835af 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1264,6 +1264,17 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is_64) static const tcg_insn_unit *tb_ret_addr; +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + /* Reuse the zeroing that exists for goto_ptr. */ + if (a0 == 0) { + tcg_out_call_int(s, tcg_code_gen_epilogue, true); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A0, a0); + tcg_out_call_int(s, tb_ret_addr, true); + } +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1274,16 +1285,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c2 = const_args[2]; switch (opc) { - case INDEX_op_exit_tb: - /* Reuse the zeroing that exists for goto_ptr. */ - if (a0 == 0) { - tcg_out_call_int(s, tcg_code_gen_epilogue, true); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_A0, a0); - tcg_out_call_int(s, tb_ret_addr, true); - } - break; - case INDEX_op_goto_tb: assert(s->tb_jmp_insn_offset == 0); /* indirect jump method */ @@ -1592,6 +1593,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 33becd7694..9568452773 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2070,6 +2070,17 @@ static void tcg_out_qemu_st(TCGContext* s, TCGReg data_reg, TCGReg addr_reg, #endif } +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + /* Reuse the zeroing that exists for goto_ptr. */ + if (a0 == 0) { + tgen_gotoi(s, S390_CC_ALWAYS, tcg_code_gen_epilogue); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R2, a0); + tgen_gotoi(s, S390_CC_ALWAYS, tb_ret_addr); + } +} + # define OP_32_64(x) \ case glue(glue(INDEX_op_,x),_i32): \ case glue(glue(INDEX_op_,x),_i64) @@ -2082,17 +2093,6 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGArg a0, a1, a2; switch (opc) { - case INDEX_op_exit_tb: - /* Reuse the zeroing that exists for goto_ptr. */ - a0 = args[0]; - if (a0 == 0) { - tgen_gotoi(s, S390_CC_ALWAYS, tcg_code_gen_epilogue); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_R2, a0); - tgen_gotoi(s, S390_CC_ALWAYS, tb_ret_addr); - } - break; - case INDEX_op_goto_tb: a0 = args[0]; if (s->tb_jmp_insn_offset) { @@ -2661,6 +2661,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index cb9453efdd..1e3351a4e8 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1427,6 +1427,26 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data, TCGReg addr, #endif /* CONFIG_SOFTMMU */ } +static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) +{ + if (check_fit_ptr(a0, 13)) { + tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); + tcg_out_movi_imm13(s, TCG_REG_O0, a0); + return; + } else if (USE_REG_TB) { + intptr_t tb_diff = tcg_tbrel_diff(s, (void *)a0); + if (check_fit_ptr(tb_diff, 13)) { + tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); + /* Note that TCG_REG_TB has been unwound to O1. */ + tcg_out_arithi(s, TCG_REG_O0, TCG_REG_O1, tb_diff, ARITH_ADD); + return; + } + } + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_I0, a0 & ~0x3ff); + tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); + tcg_out_arithi(s, TCG_REG_O0, TCG_REG_O0, a0 & 0x3ff, ARITH_OR); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1441,24 +1461,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, c2 = const_args[2]; switch (opc) { - case INDEX_op_exit_tb: - if (check_fit_ptr(a0, 13)) { - tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); - tcg_out_movi_imm13(s, TCG_REG_O0, a0); - break; - } else if (USE_REG_TB) { - intptr_t tb_diff = tcg_tbrel_diff(s, (void *)a0); - if (check_fit_ptr(tb_diff, 13)) { - tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); - /* Note that TCG_REG_TB has been unwound to O1. */ - tcg_out_arithi(s, TCG_REG_O0, TCG_REG_O1, tb_diff, ARITH_ADD); - break; - } - } - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_I0, a0 & ~0x3ff); - tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); - tcg_out_arithi(s, TCG_REG_O0, TCG_REG_O0, a0 & 0x3ff, ARITH_OR); - break; case INDEX_op_goto_tb: if (s->tb_jmp_insn_offset) { /* direct jump method */ @@ -1715,6 +1717,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index f3d7441e06..34583a3499 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -593,6 +593,11 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *func, # define CASE_64(x) #endif +static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) +{ + tcg_out_op_p(s, INDEX_op_exit_tb, (void *)arg); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -600,10 +605,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGOpcode exts; switch (opc) { - case INDEX_op_exit_tb: - tcg_out_op_p(s, opc, (void *)args[0]); - break; - case INDEX_op_goto_tb: tcg_debug_assert(s->tb_jmp_insn_offset == 0); /* indirect jump method. */ @@ -782,6 +783,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ + case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ default: tcg_abort(); } From patchwork Tue Dec 6 04:16:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A7CAC3A5A7 for ; Tue, 6 Dec 2022 04:19:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POe-0003M7-4W; Mon, 05 Dec 2022 23:17:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POZ-0003Kw-3d for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:27 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POU-0006vs-Qn for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:25 -0500 Received: by mail-oi1-x229.google.com with SMTP id e205so15552669oif.11 for ; Mon, 05 Dec 2022 20:17:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dv7mKrORCdRv24tX0CJVaTCQqf4CXOzJEBkmuES7f0o=; b=V7LpB+dxdXWtOicbSSJkuIp5h/aGqKf9G7biuJ4534vKqSTaVho4iXcPLEqT2qnH8E ChYLUlA/emNCwF24tScx9Y34WEHOgfpFTpeCAp424nZxJv5BDiXfiZTCnv5NfNhFNFcG wjW23Zfwo+0IUG7E+6/RZ1HQUOpWzQFb2QTQ0z4A/JqS/Jlw6Z6/Kr1w0qwr6NK2WZk5 aO3VJDJ/T577R8VcrpQK09kdiulF/XiOaCtjU8mSRlHjbitGUczetGFnv2fLSB0baobp xtFHGnjpOrPodDFkesvJUNtOyttc636vdTZYO/sxbLgHRrmU5V9wozRW0+4SEJD8rgsg qveg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dv7mKrORCdRv24tX0CJVaTCQqf4CXOzJEBkmuES7f0o=; b=3AUYh2P4RslAIEJjmdOzfcxTznA7yB8XVjPaIzm/9R7UJ+zdfACRZE5EyO5RpcmQad NIHcyU5laqEdP/zsE85J8Iy2vKOWkrYKt0Z8Gc9FiqnL7cs9SnJwYm8DY8N2ov9eQSWj eQTu9V1bFD4HdMGMCS2oIzrN4l5eL05V795jjvNim10MG7x5Y58Z5uFAkei3i9Ld3/KX cJKQcGNdPSnOPfa4Hhc+JfR/sgEMwt0Ol4PKs46KPeim4E9ZxOEYd6/zTW2iHiiEuGbz 9jG8o5tlWDPL4EcLaMM6PpFLNWFR58QVvvP/VELB/DSYjQ2odNvNIaKBoEErhl9gF9eb M7vA== X-Gm-Message-State: ANoB5pnBpNDyU4FCFYfZexN7RKOoo0i2ZxmwXu5a9qQbuEAslB2xikGK Ycg6I4lW+PVLlveb7JJmpIdaBEG0qO1o1y5tDlo= X-Google-Smtp-Source: AA0mqf6M4YduRyz1VOx+uhFZEpBJmCrbnL+IN2BfDiVM5zkJ9n8Oil2VSgvhvzpIBJ7XpWK/qVVbXA== X-Received: by 2002:a54:4590:0:b0:35a:8ab0:70f0 with SMTP id z16-20020a544590000000b0035a8ab070f0mr32366731oib.272.1670300241259; Mon, 05 Dec 2022 20:17:21 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:20 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 02/22] tcg/i386: Remove unused goto_tb code for indirect jump Date: Mon, 5 Dec 2022 22:16:55 -0600 Message-Id: <20221206041715.314209-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.c.inc | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 5c20bedd20..f3a40fc428 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2382,23 +2382,19 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - if (s->tb_jmp_insn_offset) { - /* direct jump method */ - int gap; - /* jump displacement must be aligned for atomic patching; + qemu_build_assert(TCG_TARGET_HAS_direct_jump); + { + /* + * Jump displacement must be aligned for atomic patching; * see if we need to add extra nops before jump */ - gap = QEMU_ALIGN_PTR_UP(s->code_ptr + 1, 4) - s->code_ptr; + int gap = QEMU_ALIGN_PTR_UP(s->code_ptr + 1, 4) - s->code_ptr; if (gap != 1) { tcg_out_nopn(s, gap - 1); } tcg_out8(s, OPC_JMP_long); /* jmp im */ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); tcg_out32(s, 0); - } else { - /* indirect jump method */ - tcg_out_modrm_offset(s, OPC_GRP5, EXT5_JMPN_Ev, -1, - (intptr_t)(s->tb_jmp_target_addr + a0)); } set_jmp_reset_offset(s, a0); break; From patchwork Tue Dec 6 04:16:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDAA3C352A1 for ; Tue, 6 Dec 2022 04:19:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POe-0003MB-DP; Mon, 05 Dec 2022 23:17:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POb-0003LV-Ek for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:29 -0500 Received: from mail-oi1-x231.google.com ([2607:f8b0:4864:20::231]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POX-0006w2-LE for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:28 -0500 Received: by mail-oi1-x231.google.com with SMTP id r11so9875394oie.13 for ; Mon, 05 Dec 2022 20:17:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=20ZbmANpsxjH51EZqy28mmD7yN/n0IpPmu7xMiVxfng=; b=ZpTUFOg1a5TDtdktbLIGnWzLi7tNkldpbUpitZitmYPh2HvfOaDAosz8ju5g0t3b9u gE/6IBosOZ87X0h1b8hYbN3lLi3UKpVaS6BKLBz2m8hh+x09Z36go0GSxM54THMbG8TA 0Om3MdJz8aD9gD2ME2UGUZOl7uPfMPDDs1iag2VUf7e+1oLLPp2qe/g6oFck/2MBNEVh GhmWPU8kkJY17b0m5PYaUuvGGNz6FuYisIGnIo9wabn7Ju4SehPsUVv1xaKm0r+SU3s8 PZ/oHb0KUsTSM6XqLjbNHY+/oeVdfwGU8xACoYGH9LHLHu02aN8bNbMphv/O2f3M2SXY /Tjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=20ZbmANpsxjH51EZqy28mmD7yN/n0IpPmu7xMiVxfng=; b=KoPb0nEelUZ9pE7KRnqOJR4sYWfLqMvKhPYjUhq/8KfL5/hH7IvxSvaLKMdyB0S+qu YiD8QZy4DEIPOOH3gFQ0VFWF7MKtTHY9y5TGkzb00sSygYaGm2Fax1ad6r0vMovb+XVP EuXxfSTj5Q97PgmQ0fPPdVnYvQEVlFuAQTJxV04tXAsIkADEpNC+KjDVWTCnuuXy2Yon Wg/33+eLvbBE44BTI0fhZwpefIIsN5IX2HsKaEboT+9QnF02g9aiLw3HVaNM6a8rYSRR ydEYyCNCaI6LPhTqQhPlZJayIYnMOX8693+obs2dMG64JD6fujgEuQnhZpwRKTXtSoRR xsQw== X-Gm-Message-State: ANoB5pknEeTVEi7DZrjDK1j4HsQjRIfvs/rwwpzIV/VEUEaRM8fNjR+W msY98kutFEKO0RPiEIqI3tAVaN8NM4xnu4FI2eo= X-Google-Smtp-Source: AA0mqf4zi0wAh7tfS8ZWHAxOrMm63Yav0t1AffnupjtkMjMj33U7ph48vLyM09CHzMdj4/QrVMBlVw== X-Received: by 2002:a05:6808:189d:b0:35a:ec1:8350 with SMTP id bi29-20020a056808189d00b0035a0ec18350mr34868426oib.71.1670300242405; Mon, 05 Dec 2022 20:17:22 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:21 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 03/22] tcg/ppc: Remove unused goto_tb code for indirect jump Date: Mon, 5 Dec 2022 22:16:56 -0600 Message-Id: <20221206041715.314209-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::231; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x231.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.c.inc | 32 +++++++++++++------------------- 1 file changed, 13 insertions(+), 19 deletions(-) diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 186599dae0..3f9ee4b39a 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2630,27 +2630,21 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - if (s->tb_jmp_insn_offset) { - /* Direct jump. */ - if (TCG_TARGET_REG_BITS == 64) { - /* Ensure the next insns are 8 or 16-byte aligned. */ - while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { - tcg_out32(s, NOP); - } - s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); - tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); - tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); - } else { - s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); - tcg_out32(s, B); - s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s); - break; + qemu_build_assert(TCG_TARGET_HAS_direct_jump); + /* Direct jump. */ + if (TCG_TARGET_REG_BITS == 64) { + /* Ensure the next insns are 8 or 16-byte aligned. */ + while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { + tcg_out32(s, NOP); } + s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); + tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); + tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); } else { - /* Indirect jump. */ - tcg_debug_assert(s->tb_jmp_insn_offset == NULL); - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TB, 0, - (intptr_t)(s->tb_jmp_insn_offset + args[0])); + s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); + tcg_out32(s, B); + s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s); + break; } tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); tcg_out32(s, BCCTR | BO_ALWAYS); From patchwork Tue Dec 6 04:16:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9EDB3C352A1 for ; Tue, 6 Dec 2022 04:19:16 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POg-0003Na-QN; Mon, 05 Dec 2022 23:17:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POa-0003LO-RL for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:29 -0500 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POY-0006wN-MQ for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:28 -0500 Received: by mail-oi1-x22e.google.com with SMTP id h132so15591079oif.2 for ; Mon, 05 Dec 2022 20:17:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1QmeEoNvi3/gFzSY8V7j6nN4NuJzUHgEhMQK1/FSFS4=; b=WFMlXzRZ+4rhrGTbHRrz4XXMPapbLw6vyVV+5hK6v5/RaFyMn6okaKQNo1PPCYQb1b PpNJYIqXTXMr5W5ApKunRkXxjChvTE48AH5TcHzWMOi2JsJ/K7Tpy/uyO9E7nWBF+N3n DA5P7X0lQP9t4awt7kp8P6TxPVgWw84uxTT1BrD1L8Tu1tPV54V2kCWL/h1WAMVnDi0E vv5xBfqPfuA0mKF+oIpZnQgYmXmx/9b6MIC2T7Hjd35pFYI51FSESBkI/xrHx8Tl4r5F myPfYa2e1TGXQLEcjUXP8FTGZ9+8M8Ieqx1d7YRScpzPUIvZPk2t7tHAIo2RZnYvdnH+ 71Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1QmeEoNvi3/gFzSY8V7j6nN4NuJzUHgEhMQK1/FSFS4=; b=DjCc9UYlzq2zDPBIMagIpj5mxEEda5xbNfu1PS06XSAiSJgETzGLTPc5tiLI+DiRGB LTWPI4VIPyDYA0ywAOlybBInHJTjbCgiZ0oPFOMpxkY4mb2rjD5fm87VF+Wq5MfOPKsK rZUADlP3/gkTq/dLNjJXurNg/DTai/0cMCDsqzofiZDzTfLtl8TdWjDuCOLSq7yhAhce VT3mmpKiwsceRkQDaYS23AS4Dcn61IYabAHrdbDbEdhDoI+4p1JFLbs1i3uOpm7H3+5o V9frEGBndVmKRREEHUTRDQ1cWn5M4nKDCF4jwh72axeyjeQ4/KFw+Wbgua9Rs6AzJHoX iwIg== X-Gm-Message-State: ANoB5pk5aelqWqZUMUXCq7fcbKhKKYZvE27xqhrokI9i1UOuaqO7rG7F u7KHAnkj9/qN4HBL5PNd/QPoYHnhRGaH96YYSqc= X-Google-Smtp-Source: AA0mqf5XLukppLEps5bo41OpOmSCTs2JyhSrX8METgVd4rBIa9KfTvojytZvEN5A4W4vjUZwc7EJkg== X-Received: by 2002:aca:a815:0:b0:35c:dee:db96 with SMTP id r21-20020acaa815000000b0035c0deedb96mr5988905oie.235.1670300243665; Mon, 05 Dec 2022 20:17:23 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:23 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 04/22] tcg/sparc64: Remove unused goto_tb code for indirect jump Date: Mon, 5 Dec 2022 22:16:57 -0600 Message-Id: <20221206041715.314209-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22e; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 41 +++++++++++------------------------- 1 file changed, 12 insertions(+), 29 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 1e3351a4e8..f035bf7dd2 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -537,17 +537,6 @@ static bool tcg_out_sti(TCGContext *s, TCGType type, TCGArg val, return false; } -static void tcg_out_ld_ptr(TCGContext *s, TCGReg ret, const void *arg) -{ - intptr_t diff = tcg_tbrel_diff(s, arg); - if (USE_REG_TB && check_fit_ptr(diff, 13)) { - tcg_out_ld(s, TCG_TYPE_PTR, ret, TCG_REG_TB, diff); - return; - } - tcg_out_movi(s, TCG_TYPE_PTR, ret, (uintptr_t)arg & ~0x3ff); - tcg_out_ld(s, TCG_TYPE_PTR, ret, ret, (uintptr_t)arg & 0x3ff); -} - static void tcg_out_sety(TCGContext *s, TCGReg rs) { tcg_out32(s, WRY | INSN_RS1(TCG_REG_G0) | INSN_RS2(rs)); @@ -1462,27 +1451,21 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - if (s->tb_jmp_insn_offset) { - /* direct jump method */ - if (USE_REG_TB) { - /* make sure the patch is 8-byte aligned. */ - if ((intptr_t)s->code_ptr & 4) { - tcg_out_nop(s); - } - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); - tcg_out_sethi(s, TCG_REG_T1, 0); - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); - tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); - } else { - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); - tcg_out32(s, CALL); + qemu_build_assert(TCG_TARGET_HAS_direct_jump); + /* Direct jump. */ + if (USE_REG_TB) { + /* make sure the patch is 8-byte aligned. */ + if ((intptr_t)s->code_ptr & 4) { tcg_out_nop(s); } + s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + tcg_out_sethi(s, TCG_REG_T1, 0); + tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); + tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); + tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); } else { - /* indirect jump method */ - tcg_out_ld_ptr(s, TCG_REG_TB, s->tb_jmp_target_addr + a0); - tcg_out_arithi(s, TCG_REG_G0, TCG_REG_TB, 0, JMPL); + s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + tcg_out32(s, CALL); tcg_out_nop(s); } set_jmp_reset_offset(s, a0); From patchwork Tue Dec 6 04:16:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A4AAFC352A1 for ; Tue, 6 Dec 2022 04:18:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POf-0003MT-BZ; Mon, 05 Dec 2022 23:17:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POc-0003Lg-6z for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:30 -0500 Received: from mail-oi1-x233.google.com ([2607:f8b0:4864:20::233]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POY-0006wU-QH for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:29 -0500 Received: by mail-oi1-x233.google.com with SMTP id q83so2083610oif.7 for ; Mon, 05 Dec 2022 20:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PS3WVtbyd0Q5s5TKsXuaE9o3KEk3XDb4ASNc/tl64so=; b=Cu/npn+Upuu7olVNS1d2qrxdBlpzS6p5foCx7DYe6gQ7I+8vqhY7AKcdHtJ1lw1OpN ZGY7A2snno4YPfXeciAXPEcvyuatOtfaGaDYeVTV9GYUuPIkocIrC39QjKbQv7nS3Ape oytEH4V6654IlhuwTYeIh2fQrOn2GU9GpIEPyy5RliWbmVpR3kBODNobCjV6DM8wJxuw SoHec3G0wyohV5897L2+qKgF0lH63Ue6y7jFMP2K2uwzG/2C4Tw3XBReHTqqUv64k31S QdDp5ptBCY6uFYM1whbo9UArLapNDyyN6ulbxwuaXkCGNPqVW7qqSctecDiVRrO9xBoT 3EXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PS3WVtbyd0Q5s5TKsXuaE9o3KEk3XDb4ASNc/tl64so=; b=0Zsqvn3cAsksbsW0/n1SYOumUMIu/xcbfwP4v5J6mgmEvve8uWHwEclaneRjcxLZhL 37aHpo4d1IgXzCarWmDqt5v6fruqVZ8hu1EkD2ntM5RktzltO61oHEi9lpGSxlLljq4W 0dvn9dNXm7AfnjpCEymRgwMOf8GnEqO1rEpYULK9cp4ejUK8He6XH8BHJyPc3xdVZf7A Z5yIu6KGITZ2TSU/+6Tvqwv1pXxvzDuU/oF74KNNu5YpqBFORMjhfpA2NCisuPPx+dm+ mU4f6EMlTMw8iHBFZFG3418HIDfDpd8CRUO+r3Ljwfey7KMlX4gBQF2dJl8jVsQ14Z7M T0vA== X-Gm-Message-State: ANoB5pl9I3gMt2JIwVGfEZSrz/TzSF+4AlL4PleVJEy/zs2HheBeBOmA 33yG6UVFb5VSMjjP6+A80b66YUyCDq2rG9x/h4U= X-Google-Smtp-Source: AA0mqf5nD7NM/PZv6hw0339B3l7TIckUAwvJ8/jKdw3ZMl5IVPdfQD/sVWmtpJiDA4yITha3LP3Pyw== X-Received: by 2002:a05:6808:3006:b0:35b:d281:9063 with SMTP id ay6-20020a056808300600b0035bd2819063mr13504209oib.252.1670300245019; Mon, 05 Dec 2022 20:17:25 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:24 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 05/22] tcg: Replace asserts on tcg_jmp_insn_offset Date: Mon, 5 Dec 2022 22:16:58 -0600 Message-Id: <20221206041715.314209-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::233; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x233.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Test TCG_TARGET_HAS_direct_jump instead of testing an implementation pointer. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/aarch64/tcg-target.c.inc | 2 +- tcg/arm/tcg-target.c.inc | 2 +- tcg/loongarch64/tcg-target.c.inc | 2 +- tcg/mips/tcg-target.c.inc | 2 +- tcg/riscv/tcg-target.c.inc | 2 +- tcg/s390x/tcg-target.c.inc | 2 +- tcg/tci/tcg-target.c.inc | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 16c5e33b69..8e97da3a39 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1917,7 +1917,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - tcg_debug_assert(s->tb_jmp_insn_offset != NULL); + qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* * Ensure that ADRP+ADD are 8-byte aligned so that an atomic * write can be used to patch the target address. diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index e5c2eae5a5..31f8c5b7a7 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1947,7 +1947,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, intptr_t ptr, dif, dil; TCGReg base = TCG_REG_PC; - tcg_debug_assert(s->tb_jmp_insn_offset == 0); + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); ptr = (intptr_t)tcg_splitwx_to_rx(s->tb_jmp_target_addr + args[0]); dif = tcg_pcrel_diff(s, (void *)ptr) - 8; dil = sextract32(dif, 0, 12); diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 41fc5ffa91..78398684cd 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1089,7 +1089,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - tcg_debug_assert(s->tb_jmp_insn_offset != NULL); + qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* * Ensure that patch area is 8-byte aligned so that an * atomic write can be used to patch the target address. diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 819648b100..5fc96e4406 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1986,7 +1986,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: /* indirect jump method */ - tcg_debug_assert(s->tb_jmp_insn_offset == 0); + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO, (uintptr_t)(s->tb_jmp_target_addr + a0)); tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 1381c835af..1e8406b8c1 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1286,7 +1286,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - assert(s->tb_jmp_insn_offset == 0); + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, (uintptr_t)(s->tb_jmp_target_addr + a0)); diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 9568452773..9498694564 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2095,7 +2095,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: a0 = args[0]; - if (s->tb_jmp_insn_offset) { + if (TCG_TARGET_HAS_direct_jump) { /* * branch displacement must be aligned for atomic patching; * see if we need to add extra nop before branch diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index 34583a3499..d1cc41261a 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -606,7 +606,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - tcg_debug_assert(s->tb_jmp_insn_offset == 0); + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method. */ tcg_out_op_p(s, opc, s->tb_jmp_target_addr + args[0]); set_jmp_reset_offset(s, args[0]); From patchwork Tue Dec 6 04:16:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6862C352A1 for ; Tue, 6 Dec 2022 04:21:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POg-0003NP-KK; Mon, 05 Dec 2022 23:17:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POc-0003Li-GK for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:30 -0500 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POZ-0006wa-ID for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:30 -0500 Received: by mail-oi1-x234.google.com with SMTP id c129so15647366oia.0 for ; Mon, 05 Dec 2022 20:17:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=416YCk0L/hcpCucqDSMqyt5y/QtKG77oPXJUoMUiAb8=; b=aLPtwFUuyTDsuR2L7VtWG/Uak8/iVCXU/nw+BNOIGTG2yORaEPlv2j47Y+Hm4tBkAX DBorUIeT0Dsjr1MPrLzfHrUR7bMqNCPNf4UdX9uIlF+kXum8utFgswq8G1p7nIBIXV7s yk+XIyrTnGh+zAf1lYDmsGOYPuXz/oGd5Z6Rmt+dOUzvA6KokPjp1Xo0f+Pxqa+7ZKlG 0nD9o6jK0PNnh8az+mBg4L+hGgRiQCBcTqMwNgfSOBNTuh7hcyQ0jidzGtujS02P+4R7 8D3LtHlWSOQZIfvNpsDvjEtq1xX05e/IRZIyEa3+X5mzpADDj5rifq8p/c+CeUVdL05n T4fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=416YCk0L/hcpCucqDSMqyt5y/QtKG77oPXJUoMUiAb8=; b=bjC+KwtREfW/cXWXNZiS+bNJVoQL+hZOdzo5/q5yrsauji5bkYy/LnRD5ouOOcn6wM HsW6AbAlCxnAwWPD9dY9SHySai9AIPZfxti3NFCu4bFRjgpqQI/USr/DjVP5apyxyJLd ad/ooLpfkKT+OnA1bnqDlJpVMa9O01odJfeGx3838BTcPkbemy2nqIqNg3YLn6LMi2Aq TKCKHIaOg++T76aM+lzxsrFp8xA40Sr6fzCPdtbo4lr2CPQ25jLhMlRdfQwEhvSuqbw/ 4YKvrGOqYMTAXWQUjImiQDPnkRgx/B0Qd8zow8gkZdBaP0GXcnqCnsbOcEtgJN53TCNe JvOw== X-Gm-Message-State: ANoB5pnYiCAaZ5+qaKZgO9tDHQZgZ8tU8IULRHXUuz5iahhKli+HuzQ/ LDewIMIeQYt5W11j/Hmzu9OKdvFsUv7NK6eEfto= X-Google-Smtp-Source: AA0mqf76wzNywhCHzYFszPsawI5b2KJaE6/KCuYFsTGKfih0pbUvGtL3Ni5bmb3jDnSYMKrEm4IwDA== X-Received: by 2002:aca:c28b:0:b0:35a:581e:f643 with SMTP id s133-20020acac28b000000b0035a581ef643mr45002531oif.151.1670300246207; Mon, 05 Dec 2022 20:17:26 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:25 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 06/22] tcg: Introduce set_jmp_insn_offset Date: Mon, 5 Dec 2022 22:16:59 -0600 Message-Id: <20221206041715.314209-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::234; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x234.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Similar to the existing set_jmp_reset_offset. Move any assert for TCG_TARGET_HAS_direct_jump into the new function (which now cannot be build-time). Will be unused if TCG_TARGET_HAS_direct_jump is constant 0, but we can't test for constant in the preprocessor, so just mark it G_GNUC_UNUSED. Signed-off-by: Richard Henderson --- tcg/tcg.c | 10 ++++++++++ tcg/aarch64/tcg-target.c.inc | 3 +-- tcg/i386/tcg-target.c.inc | 3 +-- tcg/loongarch64/tcg-target.c.inc | 3 +-- tcg/ppc/tcg-target.c.inc | 7 +++---- tcg/s390x/tcg-target.c.inc | 2 +- tcg/sparc64/tcg-target.c.inc | 5 ++--- 7 files changed, 19 insertions(+), 14 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index b53961baf7..df5a6cedf0 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -320,6 +320,16 @@ static void set_jmp_reset_offset(TCGContext *s, int which) s->tb_jmp_reset_offset[which] = tcg_current_code_size(s); } +static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) +{ + /* + * We will check for overflow at the end of the opcode loop in + * tcg_gen_code, where we bound tcg_current_code_size to UINT16_MAX. + */ + tcg_debug_assert(TCG_TARGET_HAS_direct_jump); + s->tb_jmp_insn_offset[which] = tcg_current_code_size(s); +} + /* Signal overflow, starting over with fewer guest insns. */ static G_NORETURN void tcg_raise_tb_overflow(TCGContext *s) diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 8e97da3a39..3aa89d6060 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1917,7 +1917,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* * Ensure that ADRP+ADD are 8-byte aligned so that an atomic * write can be used to patch the target address. @@ -1925,7 +1924,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, if ((uintptr_t)s->code_ptr & 7) { tcg_out32(s, NOP); } - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); /* * actual branch destination will be patched by * tb_target_set_jmp_target later diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index f3a40fc428..3d4cf71552 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2382,7 +2382,6 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - qemu_build_assert(TCG_TARGET_HAS_direct_jump); { /* * Jump displacement must be aligned for atomic patching; @@ -2393,7 +2392,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_nopn(s, gap - 1); } tcg_out8(s, OPC_JMP_long); /* jmp im */ - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); tcg_out32(s, 0); } set_jmp_reset_offset(s, a0); diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index 78398684cd..dd4934d6d5 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1089,7 +1089,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* * Ensure that patch area is 8-byte aligned so that an * atomic write can be used to patch the target address. @@ -1097,7 +1096,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, if ((uintptr_t)s->code_ptr & 7) { tcg_out_nop(s); } - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); /* * actual branch destination will be patched by * tb_target_set_jmp_target later diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 3f9ee4b39a..86684e1c84 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2630,20 +2630,19 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* Direct jump. */ if (TCG_TARGET_REG_BITS == 64) { /* Ensure the next insns are 8 or 16-byte aligned. */ while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { tcg_out32(s, NOP); } - s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); + set_jmp_insn_offset(s, args[0]); tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); } else { - s->tb_jmp_insn_offset[args[0]] = tcg_current_code_size(s); + set_jmp_insn_offset(s, args[0]); tcg_out32(s, B); - s->tb_jmp_reset_offset[args[0]] = tcg_current_code_size(s); + set_jmp_reset_offset(s, args[0]); break; } tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 9498694564..ad6433fcf2 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2105,7 +2105,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, } tcg_debug_assert(!USE_REG_TB); tcg_out16(s, RIL_BRCL | (S390_CC_ALWAYS << 4)); - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); s->code_ptr += 2; } else { /* load address stored at s->tb_jmp_target_addr + a0 */ diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index f035bf7dd2..f0c6332ef1 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1451,20 +1451,19 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_tb: - qemu_build_assert(TCG_TARGET_HAS_direct_jump); /* Direct jump. */ if (USE_REG_TB) { /* make sure the patch is 8-byte aligned. */ if ((intptr_t)s->code_ptr & 4) { tcg_out_nop(s); } - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); tcg_out_sethi(s, TCG_REG_T1, 0); tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); } else { - s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s); + set_jmp_insn_offset(s, a0); tcg_out32(s, CALL); tcg_out_nop(s); } From patchwork Tue Dec 6 04:17:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68F5BC3A5A7 for ; Tue, 6 Dec 2022 04:22:21 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POg-0003Nj-VP; Mon, 05 Dec 2022 23:17:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POc-0003Lj-OF for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:31 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POa-0006vs-19 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:30 -0500 Received: by mail-oi1-x229.google.com with SMTP id e205so15552858oif.11 for ; Mon, 05 Dec 2022 20:17:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Uzpy7hdVQnCFhgoilIBmji9svrZHcvZGnbGwuPjB4fQ=; b=dUZih/nV71gDz+EWXNNJZjssonPiG52ttjilS9hwgYGeuiToPMnQJxV0AFCHpCf3yq Xy4jN3MyxS/Q0zhwHjwJGzbtIN2DQKXBaJv59ucLBgfeoS51OVJb/Gmookf8GMeWa5y0 EPKeTf1INO5qsrWVGzPvi41BtCPLQBZJb9p3AocnmOoIAF3v1xgsqzmTL8abHpYU4FWv iHids5Jkf1GpgAk7DYAumODniU54HBGOpWq4YNzfp4xFrNkBsWuwaJ3sag62sjCH4usZ XnBvm4s7rA3kbLnvaJvsKaCyOUICKn4nVgd+Smx6ZO/utF7vFh/+1Kj28bmufK4VuPTU MySg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Uzpy7hdVQnCFhgoilIBmji9svrZHcvZGnbGwuPjB4fQ=; b=35WVqO1lJNKFzYj5ryp+/daURm8Y6b4s1CF5W35IA3VvJ4vicza6TQPag1UXJmNYFe B2uyNzKjh6DAXRx13sPIDvNLmpLxCCXQFaNpFcINyZg1t4I1HK4iO3JTuTpitP0mvR8K 5kSD1hxDJZENX9Ha5YNJI0IJL37Iaq8D0FCSIrqtetYMTryG1SsO92PhH6cRA5fdcVJY gSR4fyEqnUccV8ThjOmr3tYPiiXkPJuAUeBn7K4VF7OtTpLwkSnzsiBWqI+KGDtHvris +3RLrFdGwW/67AxHtNtHfpJoyG6iS4XZq4DGIWU0X2RHSmjYnv6MU85HZ3uC+QK9yBvR d6Vg== X-Gm-Message-State: ANoB5pmH5KFnWNSNdB3JaWSpmyNlJicvd0ohkE4FnMvIosN8h7l1l2Tn NLVB4EZUf+HY5vFJqKYeBUtuQLuQ8Ud+w/FQ9q8= X-Google-Smtp-Source: AA0mqf74c2407MSKN0a8yMwiCumEeMdaAE2o3AdPIidEY3HXFrvZvLNa9uQqWeygEB/rT4lnkZWwgQ== X-Received: by 2002:a05:6808:617:b0:359:ef86:2f4f with SMTP id y23-20020a056808061700b00359ef862f4fmr41419016oih.14.1670300247342; Mon, 05 Dec 2022 20:17:27 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:26 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 07/22] tcg: Introduce get_jmp_target_addr Date: Mon, 5 Dec 2022 22:17:00 -0600 Message-Id: <20221206041715.314209-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Similar to the existing set_jmp_reset_offset. Include the rw->rx address space coversion done by arm and s390x, and forgotten by mips and riscv. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/tcg.c | 9 +++++++++ tcg/arm/tcg-target.c.inc | 2 +- tcg/mips/tcg-target.c.inc | 2 +- tcg/riscv/tcg-target.c.inc | 2 +- tcg/s390x/tcg-target.c.inc | 2 +- tcg/tci/tcg-target.c.inc | 2 +- 6 files changed, 14 insertions(+), 5 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index df5a6cedf0..8bde8eb0d4 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -330,6 +330,15 @@ static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) s->tb_jmp_insn_offset[which] = tcg_current_code_size(s); } +static uintptr_t G_GNUC_UNUSED get_jmp_target_addr(TCGContext *s, int which) +{ + /* + * Return the read-execute version of the pointer, for the benefit + * of any pc-relative addressing mode. + */ + return (uintptr_t)tcg_splitwx_to_rx(s->tb_jmp_target_addr + which); +} + /* Signal overflow, starting over with fewer guest insns. */ static G_NORETURN void tcg_raise_tb_overflow(TCGContext *s) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 31f8c5b7a7..3a545f3707 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1948,7 +1948,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGReg base = TCG_REG_PC; qemu_build_assert(!TCG_TARGET_HAS_direct_jump); - ptr = (intptr_t)tcg_splitwx_to_rx(s->tb_jmp_target_addr + args[0]); + ptr = get_jmp_target_addr(s, args[0]); dif = tcg_pcrel_diff(s, (void *)ptr) - 8; dil = sextract32(dif, 0, 12); if (dif != dil) { diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 5fc96e4406..6b9c070f6b 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1988,7 +1988,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, /* indirect jump method */ qemu_build_assert(!TCG_TARGET_HAS_direct_jump); tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO, - (uintptr_t)(s->tb_jmp_target_addr + a0)); + get_jmp_target_addr(s, a0)); tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); tcg_out_nop(s); set_jmp_reset_offset(s, a0); diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 1e8406b8c1..091c44d071 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1289,7 +1289,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, - (uintptr_t)(s->tb_jmp_target_addr + a0)); + get_jmp_target_addr(s, a0)); tcg_out_opc_imm(s, OPC_JALR, TCG_REG_ZERO, TCG_REG_TMP0, 0); set_jmp_reset_offset(s, a0); break; diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index ad6433fcf2..dba02db6ee 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2110,7 +2110,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, } else { /* load address stored at s->tb_jmp_target_addr + a0 */ tcg_out_ld_abs(s, TCG_TYPE_PTR, TCG_REG_TB, - tcg_splitwx_to_rx(s->tb_jmp_target_addr + a0)); + (const void *)get_jmp_target_addr(s, a0)); /* and go there */ tcg_out_insn(s, RR, BCR, S390_CC_ALWAYS, TCG_REG_TB); } diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index d1cc41261a..baf3480c67 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -608,7 +608,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_goto_tb: qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method. */ - tcg_out_op_p(s, opc, s->tb_jmp_target_addr + args[0]); + tcg_out_op_p(s, opc, (void *)get_jmp_target_addr(s, args[0])); set_jmp_reset_offset(s, args[0]); break; From patchwork Tue Dec 6 04:17:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC314C3A5A7 for ; Tue, 6 Dec 2022 04:19:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POi-0003OL-5Y; Mon, 05 Dec 2022 23:17:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POg-0003NN-GH for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:34 -0500 Received: from mail-oa1-x34.google.com ([2001:4860:4864:20::34]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POc-0006xq-FO for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:33 -0500 Received: by mail-oa1-x34.google.com with SMTP id 586e51a60fabf-14449b7814bso11125143fac.3 for ; Mon, 05 Dec 2022 20:17:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qv/niVBYTSEARbjCDCIjmv13QGx0DTp8Ej8mDXhbbE4=; b=HsnAbjeBqvCTA9gTU5VEC1dboIRrMTUQZ73I7SvBRaWNmzV0tSSQZZt7OIyuWDHt9Y V3QbMhgLqi5CYRuHaFhiuJ1RI71evvq8tqnBTfJABFE7WWpx6REYUcsKFuM6sKvNqW+Z I3IKYKtuo9Gm+0xes/fdEOMh1eunZTNFinMPPstq+s+rbzgpSzEd5hPc0LkofHi7UlOv pkJ8+XzkbnbVQtwhy7q7lsSXMTFuK9gwTUWAaHVnZbWGCK8uh0ZgttnloH2q0wNdSgfs 3UNQ01D6LxJazGp45J3LuVodV01/UnDLm0GNlxiyKdDy4BSdTAu8EoRQQ4cPm3FUL+r3 dkmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qv/niVBYTSEARbjCDCIjmv13QGx0DTp8Ej8mDXhbbE4=; b=sRzMo+Gmw+lLuP0oUow8dFFsKxMoNxs1JLIFtl909uQ3O0AlrglvqvOSMHqOQRpwWV Be3scYm5+KmbstDaKBIKY5kh1LyKIOlkHBB2HWFSHQxZbqTFo9zQlKm6hLUFzZmQ0k4y bMcF6Ai3XFAq6GZG//jb/SpJKq+e9boBKUF52nnD1fkqXBYRteSL/G/WzS711xy7z3th A6KAoJzCkgv8hOlPQyf94/Mir4/SirO3HyjO3ONwT+OnHkHzYoRfvu7fi6T7nueDgXb/ MQSrrfk0t7+4qW1bva9Pf5dn3dZov2IdxgaoW09mkS6B/+pHbBYszVb/85jcEAQ+kA2g edrw== X-Gm-Message-State: ANoB5pniCaegpaxLN6QDjWgcR1FXqcm4HQXO5ZcsZmQWRSi1uf+tIhMi rUIyhGJZyIgHK7r02wbJrSYYgXYkN9dAgTMWi+M= X-Google-Smtp-Source: AA0mqf7es0z03Y45amMFyQqerDJFvmn/brNoCxbQ0lTx0aqZeK0XhzoojXTQwGvdGz84iGzUTI4TRA== X-Received: by 2002:a05:6870:2404:b0:13b:a68d:6185 with SMTP id n4-20020a056870240400b0013ba68d6185mr48408569oap.177.1670300248853; Mon, 05 Dec 2022 20:17:28 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:28 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 08/22] tcg: Split out tcg_out_goto_tb Date: Mon, 5 Dec 2022 22:17:01 -0600 Message-Id: <20221206041715.314209-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::34; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x34.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The INDEX_op_goto_tb opcode needs no register allocation. Split out a dedicated helper function for it. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/tcg.c | 4 ++ tcg/aarch64/tcg-target.c.inc | 40 +++++++++--------- tcg/arm/tcg-target.c.inc | 49 +++++++++++----------- tcg/i386/tcg-target.c.inc | 33 +++++++-------- tcg/loongarch64/tcg-target.c.inc | 38 +++++++++-------- tcg/mips/tcg-target.c.inc | 21 ++++++---- tcg/ppc/tcg-target.c.inc | 52 ++++++++++++------------ tcg/riscv/tcg-target.c.inc | 20 +++++---- tcg/s390x/tcg-target.c.inc | 70 ++++++++++++++++---------------- tcg/sparc64/tcg-target.c.inc | 68 ++++++++++++++++--------------- tcg/tci/tcg-target.c.inc | 16 ++++---- 11 files changed, 219 insertions(+), 192 deletions(-) diff --git a/tcg/tcg.c b/tcg/tcg.c index 8bde8eb0d4..d334f95375 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -108,6 +108,7 @@ static bool tcg_out_mov(TCGContext *s, TCGType type, TCGReg ret, TCGReg arg); static void tcg_out_movi(TCGContext *s, TCGType type, TCGReg ret, tcg_target_long arg); static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg); +static void tcg_out_goto_tb(TCGContext *s, int which); static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]); @@ -4325,6 +4326,9 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) case INDEX_op_exit_tb: tcg_out_exit_tb(s, op->args[0]); break; + case INDEX_op_goto_tb: + tcg_out_goto_tb(s, op->args[0]); + break; case INDEX_op_dup2_vec: if (tcg_reg_alloc_dup2(s, op)) { break; diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 3aa89d6060..072d32f355 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1897,6 +1897,26 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) } } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* + * Ensure that ADRP+ADD are 8-byte aligned so that an atomic + * write can be used to patch the target address. + */ + if ((uintptr_t)s->code_ptr & 7) { + tcg_out32(s, NOP); + } + set_jmp_insn_offset(s, which); + /* + * actual branch destination will be patched by + * tb_target_set_jmp_target later + */ + tcg_out_insn(s, 3406, ADRP, TCG_REG_TMP, 0); + tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_TMP, TCG_REG_TMP, 0); + tcg_out_insn(s, 3207, BR, TCG_REG_TMP); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1916,25 +1936,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, #define REG0(I) (const_args[I] ? TCG_REG_XZR : (TCGReg)args[I]) switch (opc) { - case INDEX_op_goto_tb: - /* - * Ensure that ADRP+ADD are 8-byte aligned so that an atomic - * write can be used to patch the target address. - */ - if ((uintptr_t)s->code_ptr & 7) { - tcg_out32(s, NOP); - } - set_jmp_insn_offset(s, a0); - /* - * actual branch destination will be patched by - * tb_target_set_jmp_target later - */ - tcg_out_insn(s, 3406, ADRP, TCG_REG_TMP, 0); - tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_TMP, TCG_REG_TMP, 0); - tcg_out_insn(s, 3207, BR, TCG_REG_TMP); - set_jmp_reset_offset(s, a0); - break; - case INDEX_op_goto_ptr: tcg_out_insn(s, 3207, BR, a0); break; @@ -2305,6 +2306,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 3a545f3707..0afc286c8e 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1933,6 +1933,31 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) tcg_out_epilogue(s); } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* Indirect jump method */ + intptr_t ptr, dif, dil; + TCGReg base = TCG_REG_PC; + + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); + ptr = get_jmp_target_addr(s, which); + dif = tcg_pcrel_diff(s, (void *)ptr) - 8; + dil = sextract32(dif, 0, 12); + if (dif != dil) { + /* + * The TB is close, but outside the 12 bits addressable by + * the load. We can extend this to 20 bits with a sub of a + * shifted immediate from pc. In the vastly unlikely event + * the code requires more than 1MB, we'll use 2 insns and + * be no worse off. + */ + base = TCG_REG_R0; + tcg_out_movi32(s, COND_AL, base, ptr - dil); + } + tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, base, dil); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1941,29 +1966,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c; switch (opc) { - case INDEX_op_goto_tb: - { - /* Indirect jump method */ - intptr_t ptr, dif, dil; - TCGReg base = TCG_REG_PC; - - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); - ptr = get_jmp_target_addr(s, args[0]); - dif = tcg_pcrel_diff(s, (void *)ptr) - 8; - dil = sextract32(dif, 0, 12); - if (dif != dil) { - /* The TB is close, but outside the 12 bits addressable by - the load. We can extend this to 20 bits with a sub of a - shifted immediate from pc. In the vastly unlikely event - the code requires more than 1MB, we'll use 2 insns and - be no worse off. */ - base = TCG_REG_R0; - tcg_out_movi32(s, COND_AL, base, ptr - dil); - } - tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, base, dil); - set_jmp_reset_offset(s, args[0]); - } - break; case INDEX_op_goto_ptr: tcg_out_b_reg(s, COND_AL, args[0]); break; @@ -2253,6 +2255,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i32: /* Always emitted via tcg_out_mov. */ case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 3d4cf71552..d3cef242ef 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2357,6 +2357,22 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) } } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* + * Jump displacement must be aligned for atomic patching; + * see if we need to add extra nops before jump + */ + int gap = QEMU_ALIGN_PTR_UP(s->code_ptr + 1, 4) - s->code_ptr; + if (gap != 1) { + tcg_out_nopn(s, gap - 1); + } + tcg_out8(s, OPC_JMP_long); /* jmp im */ + set_jmp_insn_offset(s, which); + tcg_out32(s, 0); + set_jmp_reset_offset(s, which); +} + static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -2381,22 +2397,6 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const_a2 = const_args[2]; switch (opc) { - case INDEX_op_goto_tb: - { - /* - * Jump displacement must be aligned for atomic patching; - * see if we need to add extra nops before jump - */ - int gap = QEMU_ALIGN_PTR_UP(s->code_ptr + 1, 4) - s->code_ptr; - if (gap != 1) { - tcg_out_nopn(s, gap - 1); - } - tcg_out8(s, OPC_JMP_long); /* jmp im */ - set_jmp_insn_offset(s, a0); - tcg_out32(s, 0); - } - set_jmp_reset_offset(s, a0); - break; case INDEX_op_goto_ptr: /* jmp to the given host address (could be epilogue) */ tcg_out_modrm(s, OPC_GRP5, EXT5_JMPN_Ev, a0); @@ -2791,6 +2791,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index dd4934d6d5..dbe64b7d8b 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1078,6 +1078,25 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) } } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* + * Ensure that patch area is 8-byte aligned so that an + * atomic write can be used to patch the target address. + */ + if ((uintptr_t)s->code_ptr & 7) { + tcg_out_nop(s); + } + set_jmp_insn_offset(s, which); + /* + * actual branch destination will be patched by + * tb_target_set_jmp_target later + */ + tcg_out_opc_pcaddu18i(s, TCG_REG_TMP0, 0); + tcg_out_opc_jirl(s, TCG_REG_ZERO, TCG_REG_TMP0, 0); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1088,24 +1107,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c2 = const_args[2]; switch (opc) { - case INDEX_op_goto_tb: - /* - * Ensure that patch area is 8-byte aligned so that an - * atomic write can be used to patch the target address. - */ - if ((uintptr_t)s->code_ptr & 7) { - tcg_out_nop(s); - } - set_jmp_insn_offset(s, a0); - /* - * actual branch destination will be patched by - * tb_target_set_jmp_target later - */ - tcg_out_opc_pcaddu18i(s, TCG_REG_TMP0, 0); - tcg_out_opc_jirl(s, TCG_REG_ZERO, TCG_REG_TMP0, 0); - set_jmp_reset_offset(s, a0); - break; - case INDEX_op_mb: tcg_out_mb(s, a0); break; @@ -1500,6 +1501,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 6b9c070f6b..605aa7714a 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1965,6 +1965,17 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) tcg_out_opc_imm(s, OPC_ORI, TCG_REG_V0, b0, a0 & 0xffff); } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* indirect jump method */ + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO, + get_jmp_target_addr(s, which)); + tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); + tcg_out_nop(s); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1984,15 +1995,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, c2 = const_args[2]; switch (opc) { - case INDEX_op_goto_tb: - /* indirect jump method */ - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); - tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO, - get_jmp_target_addr(s, a0)); - tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); - tcg_out_nop(s); - set_jmp_reset_offset(s, a0); - break; case INDEX_op_goto_ptr: /* jmp to the given host address (could be epilogue) */ tcg_out_opc_reg(s, OPC_JR, 0, a0, 0); @@ -2401,6 +2403,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 86684e1c84..ec13ea3171 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -2622,6 +2622,32 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) tcg_out_b(s, 0, tcg_code_gen_epilogue); } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* Direct jump. */ + if (TCG_TARGET_REG_BITS == 64) { + /* Ensure the next insns are 8 or 16-byte aligned. */ + while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { + tcg_out32(s, NOP); + } + set_jmp_insn_offset(s, which); + tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); + tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); + tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); + tcg_out32(s, BCCTR | BO_ALWAYS); + set_jmp_reset_offset(s, which); + if (USE_REG_TB) { + /* For the unlinked case, need to reset TCG_REG_TB. */ + tcg_out_mem_long(s, ADDI, ADD, TCG_REG_TB, TCG_REG_TB, + -tcg_current_code_size(s)); + } + } else { + set_jmp_insn_offset(s, which); + tcg_out32(s, B); + set_jmp_reset_offset(s, which); + } +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -2629,31 +2655,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGArg a0, a1, a2; switch (opc) { - case INDEX_op_goto_tb: - /* Direct jump. */ - if (TCG_TARGET_REG_BITS == 64) { - /* Ensure the next insns are 8 or 16-byte aligned. */ - while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { - tcg_out32(s, NOP); - } - set_jmp_insn_offset(s, args[0]); - tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); - tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); - } else { - set_jmp_insn_offset(s, args[0]); - tcg_out32(s, B); - set_jmp_reset_offset(s, args[0]); - break; - } - tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); - tcg_out32(s, BCCTR | BO_ALWAYS); - set_jmp_reset_offset(s, args[0]); - if (USE_REG_TB) { - /* For the unlinked case, need to reset TCG_REG_TB. */ - tcg_out_mem_long(s, ADDI, ADD, TCG_REG_TB, TCG_REG_TB, - -tcg_current_code_size(s)); - } - break; case INDEX_op_goto_ptr: tcg_out32(s, MTSPR | RS(args[0]) | CTR); if (USE_REG_TB) { @@ -3181,6 +3182,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 091c44d071..e73dfb4cbb 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1275,6 +1275,16 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) } } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); + /* indirect jump method */ + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, + get_jmp_target_addr(s, which)); + tcg_out_opc_imm(s, OPC_JALR, TCG_REG_ZERO, TCG_REG_TMP0, 0); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1285,15 +1295,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, int c2 = const_args[2]; switch (opc) { - case INDEX_op_goto_tb: - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); - /* indirect jump method */ - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, - get_jmp_target_addr(s, a0)); - tcg_out_opc_imm(s, OPC_JALR, TCG_REG_ZERO, TCG_REG_TMP0, 0); - set_jmp_reset_offset(s, a0); - break; - case INDEX_op_goto_ptr: tcg_out_opc_imm(s, OPC_JALR, TCG_REG_ZERO, a0, 0); break; @@ -1594,6 +1595,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: g_assert_not_reached(); } diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index dba02db6ee..a60edc18cf 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2081,6 +2081,41 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) } } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + if (TCG_TARGET_HAS_direct_jump) { + /* + * Branch displacement must be aligned for atomic patching; + * see if we need to add extra nop before branch + */ + if (!QEMU_PTR_IS_ALIGNED(s->code_ptr + 1, 4)) { + tcg_out16(s, NOP); + } + tcg_debug_assert(!USE_REG_TB); + tcg_out16(s, RIL_BRCL | (S390_CC_ALWAYS << 4)); + set_jmp_insn_offset(s, which); + s->code_ptr += 2; + } else { + /* Load address stored in the TB. */ + tcg_out_ld_abs(s, TCG_TYPE_PTR, TCG_REG_TB, + (const void *)get_jmp_target_addr(s, which)); + /* and go there */ + tcg_out_insn(s, RR, BCR, S390_CC_ALWAYS, TCG_REG_TB); + } + set_jmp_reset_offset(s, which); + + /* + * For the unlinked path of goto_tb, we need to reset TCG_REG_TB + * to the beginning of this TB. + */ + if (USE_REG_TB) { + int ofs = -tcg_current_code_size(s); + /* All TB are restricted to 64KiB by unwind info. */ + tcg_debug_assert(ofs == sextract64(ofs, 0, 20)); + tcg_out_insn(s, RXY, LAY, TCG_REG_TB, TCG_REG_TB, TCG_REG_NONE, ofs); + } +} + # define OP_32_64(x) \ case glue(glue(INDEX_op_,x),_i32): \ case glue(glue(INDEX_op_,x),_i64) @@ -2093,40 +2128,6 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGArg a0, a1, a2; switch (opc) { - case INDEX_op_goto_tb: - a0 = args[0]; - if (TCG_TARGET_HAS_direct_jump) { - /* - * branch displacement must be aligned for atomic patching; - * see if we need to add extra nop before branch - */ - if (!QEMU_PTR_IS_ALIGNED(s->code_ptr + 1, 4)) { - tcg_out16(s, NOP); - } - tcg_debug_assert(!USE_REG_TB); - tcg_out16(s, RIL_BRCL | (S390_CC_ALWAYS << 4)); - set_jmp_insn_offset(s, a0); - s->code_ptr += 2; - } else { - /* load address stored at s->tb_jmp_target_addr + a0 */ - tcg_out_ld_abs(s, TCG_TYPE_PTR, TCG_REG_TB, - (const void *)get_jmp_target_addr(s, a0)); - /* and go there */ - tcg_out_insn(s, RR, BCR, S390_CC_ALWAYS, TCG_REG_TB); - } - set_jmp_reset_offset(s, a0); - - /* For the unlinked path of goto_tb, we need to reset - TCG_REG_TB to the beginning of this TB. */ - if (USE_REG_TB) { - int ofs = -tcg_current_code_size(s); - /* All TB are restricted to 64KiB by unwind info. */ - tcg_debug_assert(ofs == sextract64(ofs, 0, 20)); - tcg_out_insn(s, RXY, LAY, TCG_REG_TB, - TCG_REG_TB, TCG_REG_NONE, ofs); - } - break; - case INDEX_op_goto_ptr: a0 = args[0]; if (USE_REG_TB) { @@ -2662,6 +2663,7 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index f0c6332ef1..ed8efa413c 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1436,6 +1436,41 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) tcg_out_arithi(s, TCG_REG_O0, TCG_REG_O0, a0 & 0x3ff, ARITH_OR); } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + /* Direct jump. */ + if (USE_REG_TB) { + /* make sure the patch is 8-byte aligned. */ + if ((intptr_t)s->code_ptr & 4) { + tcg_out_nop(s); + } + set_jmp_insn_offset(s, which); + tcg_out_sethi(s, TCG_REG_T1, 0); + tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); + tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); + tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); + } else { + set_jmp_insn_offset(s, which); + tcg_out32(s, CALL); + tcg_out_nop(s); + } + set_jmp_reset_offset(s, which); + + /* + * For the unlinked path of goto_tb, we need to reset TCG_REG_TB + * to the beginning of this TB. + */ + if (USE_REG_TB) { + int c = -tcg_current_code_size(s); + if (check_fit_i32(c, 13)) { + tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, c, ARITH_ADD); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, c); + tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); + } + } +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1450,38 +1485,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, c2 = const_args[2]; switch (opc) { - case INDEX_op_goto_tb: - /* Direct jump. */ - if (USE_REG_TB) { - /* make sure the patch is 8-byte aligned. */ - if ((intptr_t)s->code_ptr & 4) { - tcg_out_nop(s); - } - set_jmp_insn_offset(s, a0); - tcg_out_sethi(s, TCG_REG_T1, 0); - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); - tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); - } else { - set_jmp_insn_offset(s, a0); - tcg_out32(s, CALL); - tcg_out_nop(s); - } - set_jmp_reset_offset(s, a0); - - /* For the unlinked path of goto_tb, we need to reset - TCG_REG_TB to the beginning of this TB. */ - if (USE_REG_TB) { - c = -tcg_current_code_size(s); - if (check_fit_i32(c, 13)) { - tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, c, ARITH_ADD); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, c); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, - TCG_REG_T1, ARITH_ADD); - } - } - break; case INDEX_op_goto_ptr: tcg_out_arithi(s, TCG_REG_G0, a0, 0, JMPL); if (USE_REG_TB) { @@ -1700,6 +1703,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index baf3480c67..cf84ff3133 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -598,6 +598,14 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) tcg_out_op_p(s, INDEX_op_exit_tb, (void *)arg); } +static void tcg_out_goto_tb(TCGContext *s, int which) +{ + qemu_build_assert(!TCG_TARGET_HAS_direct_jump); + /* indirect jump method. */ + tcg_out_op_p(s, INDEX_op_goto_tb, (void *)get_jmp_target_addr(s, which)); + set_jmp_reset_offset(s, which); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -605,13 +613,6 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, TCGOpcode exts; switch (opc) { - case INDEX_op_goto_tb: - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); - /* indirect jump method. */ - tcg_out_op_p(s, opc, (void *)get_jmp_target_addr(s, args[0])); - set_jmp_reset_offset(s, args[0]); - break; - case INDEX_op_goto_ptr: tcg_out_op_r(s, opc, args[0]); break; @@ -784,6 +785,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, case INDEX_op_mov_i64: case INDEX_op_call: /* Always emitted via tcg_out_call. */ case INDEX_op_exit_tb: /* Always emitted via tcg_out_exit_tb. */ + case INDEX_op_goto_tb: /* Always emitted via tcg_out_goto_tb. */ default: tcg_abort(); } From patchwork Tue Dec 6 04:17:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F20CDC3A5A7 for ; Tue, 6 Dec 2022 04:18:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POh-0003OB-E5; Mon, 05 Dec 2022 23:17:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POf-0003MW-1t for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:33 -0500 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POd-0006y7-Bo for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:32 -0500 Received: by mail-oi1-x235.google.com with SMTP id q186so15574603oia.9 for ; Mon, 05 Dec 2022 20:17:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vl4jY0qCeUzsoqmWnAohVecCt0XdBWnXbPzcSP7167M=; b=Telw/ooPfVPgpRTs7xr7RZx6XdWLqTS4q6pLOembtJL7pEF6VQ1Uma+PBLzv9e545H /PhNT2pdZVdpOUq+XC7rXc0LLxnuClWLU65hmUHdcr6FpcESMbc3g3rCuyg7vSO7w3sU 2dk4cMTw1VO6agPG2Bcry4NX4397Ci5YYaVAwTnF+WB7z05UkqUKMyU18uXshsxzgaBo bXjF/YimrBiZLVm2g9Toh3umWCVr0MEqhYmFRmdjnbHkLee6yUbm7Q3GcBpy8AwruMIR wMbgK6w9ZJqGdpVIC2mr5ApCpz1pmh/JBxbWvZVkoXeLpWEQmhsPQudk5csWprvTlYhS nFiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vl4jY0qCeUzsoqmWnAohVecCt0XdBWnXbPzcSP7167M=; b=RZIOwy21QnvbqIMa/cx9kqW6qB5v/qvZryCknDHl9gz7A5m0NkE/IKNWeAx0sjj4v3 4L8WLALps74cFTZmjhVqUb61+oh8QsF7mAny58+HJ+MNiT8Y11fxKnNgA5qh/o8xdnVs coxpf1lGXGXyKt2eSwm5SD/0yEs0F/9V6/cJFyTc6QeLWivczePf8+PHASfsBuGSmefE jfgU6+iNIRVgrsxSw7/DQdvsqNNmpEmt5o9pnnzoN/wYX9Oyp5P8RFOv5JiDCcUfa/0N ajhrykOnhx52Sp/SrGywvftCOwv0sGlmlH+5jlHrRjJPktRdTzkp7z7z1noNhJvUSYfU 59zA== X-Gm-Message-State: ANoB5pmp+hjhQYs0F1ks751oqbOLuCduptvCQFf3WdAtVRWfnkX3x9fQ L9B/NYaX9Ql3z5+xDglrHJC48tJ0WKJRgeavRhU= X-Google-Smtp-Source: AA0mqf48bU8VVD8GTdKLDFwWaSRo6IEaMiiM85eMpbCik0V7JJfADnq5Im1uHiB3moEbCp6iWUvy0A== X-Received: by 2002:aca:38c6:0:b0:35b:cf0f:59e7 with SMTP id f189-20020aca38c6000000b0035bcf0f59e7mr14406875oia.2.1670300250056; Mon, 05 Dec 2022 20:17:30 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:29 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 09/22] tcg: Rename TB_JMP_RESET_OFFSET_INVALID to TB_JMP_OFFSET_INVALID Date: Mon, 5 Dec 2022 22:17:02 -0600 Message-Id: <20221206041715.314209-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x235.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This will shortly be used for more than reset. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- include/exec/exec-all.h | 2 +- accel/tcg/translate-all.c | 8 ++++---- tcg/tcg.c | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 9b7bfbf09a..7566ad9e7f 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -575,8 +575,8 @@ struct TranslationBlock { * setting one of the jump targets (or patching the jump instruction). Only * two of such jumps are supported. */ +#define TB_JMP_OFFSET_INVALID 0xffff /* indicates no jump generated */ uint16_t jmp_reset_offset[2]; /* offset of original jump target */ -#define TB_JMP_RESET_OFFSET_INVALID 0xffff /* indicates no jump generated */ uintptr_t jmp_target_arg[2]; /* target address or offset */ /* diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index ac3ee3740c..9cf88da6cb 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -979,10 +979,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu, tb->jmp_dest[1] = (uintptr_t)NULL; /* init original jump addresses which have been set during tcg_gen_code() */ - if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) { + if (tb->jmp_reset_offset[0] != TB_JMP_OFFSET_INVALID) { tb_reset_jump(tb, 0); } - if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) { + if (tb->jmp_reset_offset[1] != TB_JMP_OFFSET_INVALID) { tb_reset_jump(tb, 1); } @@ -1164,9 +1164,9 @@ static gboolean tb_tree_stats_iter(gpointer key, gpointer value, gpointer data) if (tb_page_addr1(tb) != -1) { tst->cross_page++; } - if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) { + if (tb->jmp_reset_offset[0] != TB_JMP_OFFSET_INVALID) { tst->direct_jmp_count++; - if (tb->jmp_reset_offset[1] != TB_JMP_RESET_OFFSET_INVALID) { + if (tb->jmp_reset_offset[1] != TB_JMP_OFFSET_INVALID) { tst->direct_jmp2_count++; } } diff --git a/tcg/tcg.c b/tcg/tcg.c index d334f95375..37957208c7 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -4250,8 +4250,8 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) #endif /* Initialize goto_tb jump offsets. */ - tb->jmp_reset_offset[0] = TB_JMP_RESET_OFFSET_INVALID; - tb->jmp_reset_offset[1] = TB_JMP_RESET_OFFSET_INVALID; + tb->jmp_reset_offset[0] = TB_JMP_OFFSET_INVALID; + tb->jmp_reset_offset[1] = TB_JMP_OFFSET_INVALID; tcg_ctx->tb_jmp_reset_offset = tb->jmp_reset_offset; if (TCG_TARGET_HAS_direct_jump) { tcg_ctx->tb_jmp_insn_offset = tb->jmp_target_arg; From patchwork Tue Dec 6 04:17:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1089CC3A5A7 for ; Tue, 6 Dec 2022 04:22:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POj-0003Ov-0R; Mon, 05 Dec 2022 23:17:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POg-0003NQ-JS for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:34 -0500 Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POe-0006vs-Rz for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:34 -0500 Received: by mail-oi1-x229.google.com with SMTP id e205so15552993oif.11 for ; Mon, 05 Dec 2022 20:17:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E0wFR031lCo2wcrWO+fbqeHXYKNVaKCwEXleN4UiegU=; b=AuvA95lARaqKua7iS7YTh//xTIKeMJYzNEuQZ0Nw4tIByU7IA0TzhGmUTMTpK8iQB7 0EntsABga0/F/n+22r0+wkkhCm3sepwhjBl7XcCQ/bE8T/J7yfWGOKAtA5EN9nXCBfNT GDyb6+PRiw4iBgM0nnweUnb3jCSERZ0BYt8GucrrirRtwgT2XPvJ08XhPLqx07j26H+c xE+0ikjAA1dZdN+/1ENQCL64Q3xBCWa+Thr7ndzuxdUVUShB9M+byzd8GNz3LoBn6mTP mWeTPcZgjMq0Os8I+kdUWVkNtzx3OeykaVEW8laYrPqZRbBVdBNsVr6GmcURWaFMHRm2 N7eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E0wFR031lCo2wcrWO+fbqeHXYKNVaKCwEXleN4UiegU=; b=NL7yGusofYVTu4J+/l7jEWB2t6nHeYv1pRkduOtYrtb4eblFYJK6o84zi4bxpGQqw1 9blpwtw5PPMskxaw2BOuudWEQNugslphvQYdGH4kuUvADeBJQM2DshHP7HliwyopsMRf YVEoB9l9JPZKmXPKIMJVsYmyb8lkljH0QmnOv1VV8ieDmIOsgpXpoWcXY5d0V7SJ8JPL 5KYV9FFIORqukoS51s+z55il5ts3voPIaPhX7wSNdaLsVBCIF+kwvHTD7QdZVnpW0ipp zkfyg5wYM7lIUZMQDPluWURjpT5svoIbIEx8cizu0z0Wr7SfDWmwn9XkP29LqU/jqQWj maQw== X-Gm-Message-State: ANoB5plB13MPHyWob7n5+sI+AbzbGp875cBKASSrsuGMPIlxE8WTTNvm dZNecxeYyetA//1UGGKTdIZEEyyZYpS4hmKJsFU= X-Google-Smtp-Source: AA0mqf7nr42AmiSDZIdaLQ5MsbkhOwCTDiTQzI0l1ThYyXceTnLvCiDEk4Zc2uk53GOs1bqMgxKpEw== X-Received: by 2002:aca:3c82:0:b0:35c:29f8:e83a with SMTP id j124-20020aca3c82000000b0035c29f8e83amr4215111oia.25.1670300251648; Mon, 05 Dec 2022 20:17:31 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:31 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 10/22] tcg: Add gen_tb to TCGContext Date: Mon, 5 Dec 2022 22:17:03 -0600 Message-Id: <20221206041715.314209-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::229; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x229.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This can replace four other variables that are references into the TranslationBlock structure. Signed-off-by: Richard Henderson --- include/tcg/tcg.h | 11 +++-------- accel/tcg/translate-all.c | 2 +- tcg/tcg-op.c | 14 +++++++------- tcg/tcg.c | 14 +++----------- 4 files changed, 14 insertions(+), 27 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index d84bae6e3f..993aafa1a2 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -544,20 +544,15 @@ struct TCGContext { int nb_indirects; int nb_ops; - /* goto_tb support */ - tcg_insn_unit *code_buf; - uint16_t *tb_jmp_reset_offset; /* tb->jmp_reset_offset */ - uintptr_t *tb_jmp_insn_offset; /* tb->jmp_target_arg if direct_jump */ - uintptr_t *tb_jmp_target_addr; /* tb->jmp_target_arg if !direct_jump */ - TCGRegSet reserved_regs; - uint32_t tb_cflags; /* cflags of the current TB */ intptr_t current_frame_offset; intptr_t frame_start; intptr_t frame_end; TCGTemp *frame_temp; - tcg_insn_unit *code_ptr; + TranslationBlock *gen_tb; /* tb for which code is being generated */ + tcg_insn_unit *code_buf; /* pointer for start of tb */ + tcg_insn_unit *code_ptr; /* pointer for running end of tb */ #ifdef CONFIG_PROFILER TCGProfile prof; diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 9cf88da6cb..94238a1926 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -827,7 +827,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu, tb->trace_vcpu_dstate = *cpu->trace_dstate; tb_set_page_addr0(tb, phys_pc); tb_set_page_addr1(tb, -1); - tcg_ctx->tb_cflags = cflags; + tcg_ctx->gen_tb = tb; tb_overflow: #ifdef CONFIG_PROFILER diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index 019fab00cc..585f33ffaf 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -94,7 +94,7 @@ void tcg_gen_op6(TCGOpcode opc, TCGArg a1, TCGArg a2, TCGArg a3, void tcg_gen_mb(TCGBar mb_type) { - if (tcg_ctx->tb_cflags & CF_PARALLEL) { + if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { tcg_gen_op1(INDEX_op_mb, mb_type); } } @@ -2763,7 +2763,7 @@ void tcg_gen_exit_tb(const TranslationBlock *tb, unsigned idx) void tcg_gen_goto_tb(unsigned idx) { /* We tested CF_NO_GOTO_TB in translator_use_goto_tb. */ - tcg_debug_assert(!(tcg_ctx->tb_cflags & CF_NO_GOTO_TB)); + tcg_debug_assert(!(tcg_ctx->gen_tb->cflags & CF_NO_GOTO_TB)); /* We only support two chained exits. */ tcg_debug_assert(idx <= TB_EXIT_IDXMAX); #ifdef CONFIG_DEBUG_TCG @@ -2779,7 +2779,7 @@ void tcg_gen_lookup_and_goto_ptr(void) { TCGv_ptr ptr; - if (tcg_ctx->tb_cflags & CF_NO_GOTO_PTR) { + if (tcg_ctx->gen_tb->cflags & CF_NO_GOTO_PTR) { tcg_gen_exit_tb(NULL, 0); return; } @@ -3146,7 +3146,7 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, TCGv_i32 cmpv, { memop = tcg_canonicalize_memop(memop, 0, 0); - if (!(tcg_ctx->tb_cflags & CF_PARALLEL)) { + if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { TCGv_i32 t1 = tcg_temp_new_i32(); TCGv_i32 t2 = tcg_temp_new_i32(); @@ -3184,7 +3184,7 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, TCGv_i64 cmpv, { memop = tcg_canonicalize_memop(memop, 1, 0); - if (!(tcg_ctx->tb_cflags & CF_PARALLEL)) { + if (!(tcg_ctx->gen_tb->cflags & CF_PARALLEL)) { TCGv_i64 t1 = tcg_temp_new_i64(); TCGv_i64 t2 = tcg_temp_new_i64(); @@ -3345,7 +3345,7 @@ static void * const table_##NAME[(MO_SIZE | MO_BSWAP) + 1] = { \ void tcg_gen_atomic_##NAME##_i32 \ (TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, MemOp memop) \ { \ - if (tcg_ctx->tb_cflags & CF_PARALLEL) { \ + if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ do_atomic_op_i32(ret, addr, val, idx, memop, table_##NAME); \ } else { \ do_nonatomic_op_i32(ret, addr, val, idx, memop, NEW, \ @@ -3355,7 +3355,7 @@ void tcg_gen_atomic_##NAME##_i32 \ void tcg_gen_atomic_##NAME##_i64 \ (TCGv_i64 ret, TCGv addr, TCGv_i64 val, TCGArg idx, MemOp memop) \ { \ - if (tcg_ctx->tb_cflags & CF_PARALLEL) { \ + if (tcg_ctx->gen_tb->cflags & CF_PARALLEL) { \ do_atomic_op_i64(ret, addr, val, idx, memop, table_##NAME); \ } else { \ do_nonatomic_op_i64(ret, addr, val, idx, memop, NEW, \ diff --git a/tcg/tcg.c b/tcg/tcg.c index 37957208c7..c7ad46ff45 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -318,7 +318,7 @@ static void set_jmp_reset_offset(TCGContext *s, int which) * We will check for overflow at the end of the opcode loop in * tcg_gen_code, where we bound tcg_current_code_size to UINT16_MAX. */ - s->tb_jmp_reset_offset[which] = tcg_current_code_size(s); + s->gen_tb->jmp_reset_offset[which] = tcg_current_code_size(s); } static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) @@ -328,7 +328,7 @@ static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) * tcg_gen_code, where we bound tcg_current_code_size to UINT16_MAX. */ tcg_debug_assert(TCG_TARGET_HAS_direct_jump); - s->tb_jmp_insn_offset[which] = tcg_current_code_size(s); + s->gen_tb->jmp_target_arg[which] = tcg_current_code_size(s); } static uintptr_t G_GNUC_UNUSED get_jmp_target_addr(TCGContext *s, int which) @@ -337,7 +337,7 @@ static uintptr_t G_GNUC_UNUSED get_jmp_target_addr(TCGContext *s, int which) * Return the read-execute version of the pointer, for the benefit * of any pc-relative addressing mode. */ - return (uintptr_t)tcg_splitwx_to_rx(s->tb_jmp_target_addr + which); + return (uintptr_t)tcg_splitwx_to_rx(s->gen_tb->jmp_target_arg + which); } /* Signal overflow, starting over with fewer guest insns. */ @@ -4252,14 +4252,6 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) /* Initialize goto_tb jump offsets. */ tb->jmp_reset_offset[0] = TB_JMP_OFFSET_INVALID; tb->jmp_reset_offset[1] = TB_JMP_OFFSET_INVALID; - tcg_ctx->tb_jmp_reset_offset = tb->jmp_reset_offset; - if (TCG_TARGET_HAS_direct_jump) { - tcg_ctx->tb_jmp_insn_offset = tb->jmp_target_arg; - tcg_ctx->tb_jmp_target_addr = NULL; - } else { - tcg_ctx->tb_jmp_insn_offset = NULL; - tcg_ctx->tb_jmp_target_addr = tb->jmp_target_arg; - } tcg_reg_alloc_start(s); From patchwork Tue Dec 6 04:17:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E95BEC3A5A7 for ; Tue, 6 Dec 2022 04:23:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POj-0003Pl-L0; Mon, 05 Dec 2022 23:17:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POh-0003OD-Ku for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:35 -0500 Received: from mail-oi1-x22b.google.com ([2607:f8b0:4864:20::22b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POg-0006yh-1H for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:35 -0500 Received: by mail-oi1-x22b.google.com with SMTP id v70so7210210oie.3 for ; Mon, 05 Dec 2022 20:17:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MgfIKhOnV5J4W2WjGszVQvaIMC5JwI5x8TLYbia9Zpk=; b=IH8r5UiU0zZnyW2nz5tm/3qHB99mNsXtSKKGga3rnDcnAsxOpNvXxyhlYc4VqD6S5m Km6xLD609sAgZm8dD8aTsktFnN2Bu7oHUPb5L0rNbW3RggbpI2c7VGENGarW1jf8ToO2 DDHmZOYkYWwImepWG3/dgLd7zUWL2cRLxEzox1vCA+29rqjv9FWN0X2eDFYLbE/e7Fic YynVgvwqz8jZvwSApW08liFg8ZgoHIHtSi0iQYMK2elJt0EDLkDz9WwWdKV9cmYwUZMq 4ZJtZovrs3a/MYUHplpc5+HUYbvRihUoZZmilfIxq9O3yMF1U9NBHDdZbQ6nrVRgkrhC FEcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MgfIKhOnV5J4W2WjGszVQvaIMC5JwI5x8TLYbia9Zpk=; b=wAo19/3W0Tfi1CUAzBxOqb6GoM+KFtIorwpIStUUXcAKVm/0ONIYZC3uh0h59Jhgi5 fC50zbhC0h4BGcQzczv6NfPMNZqoP9NgFfbzyj7KruMC5knudXH4tBjQX9xHQ3PY3re1 OGxYnLQpgH9/XbnJaDQmqAYRekygJWNPMKnGkEaCIl6zL7K+urDGu77d8/m6oEYBUKnZ XhPJlLvv3AlKlC/2Llzt+v+B3iU1lN4C/tn8r2cNObTk90y9MCqfzsBjz90CsYrhrY6C t+26rw4LMFUNYzv1ddka1ZQoGMIEalpZdFaG63g/Mt9CGM6Bn5AAXH1UqUriDJhMv2ph Fs4w== X-Gm-Message-State: ANoB5pmeqgLsxspExOc+9DfL4TH44mvF3bDk12QnT1ofyseRhCmjheAD 2kWpKrbw+RGAs6KSK2stHrYSwVAqwxQ25bcWRQE= X-Google-Smtp-Source: AA0mqf5WLCpVjsjbn8Y1/6K0Gh8RDRlqINqAz5xw//8dF6p5pbH7ymFO0EMcQbbH1b+nSsY9e3RewQ== X-Received: by 2002:a05:6808:148c:b0:35b:da57:5d85 with SMTP id e12-20020a056808148c00b0035bda575d85mr12577959oiw.168.1670300252865; Mon, 05 Dec 2022 20:17:32 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:32 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 11/22] tcg: Add TranslationBlock.jmp_insn_offset Date: Mon, 5 Dec 2022 22:17:04 -0600 Message-Id: <20221206041715.314209-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22b; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Stop overloading jmp_target_arg for both offset and address, depending on TCG_TARGET_HAS_direct_jump. Instead, add a new field to hold the jump insn offset and always set the target address in jmp_target_addr[]. This will allow a tcg backend to use either direct or indirect depending on displacement. Signed-off-by: Richard Henderson --- include/exec/exec-all.h | 3 ++- accel/tcg/cpu-exec.c | 5 ++--- tcg/tcg.c | 6 ++++-- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 7566ad9e7f..bb01508b4b 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -577,7 +577,8 @@ struct TranslationBlock { */ #define TB_JMP_OFFSET_INVALID 0xffff /* indicates no jump generated */ uint16_t jmp_reset_offset[2]; /* offset of original jump target */ - uintptr_t jmp_target_arg[2]; /* target address or offset */ + uint16_t jmp_insn_offset[2]; /* offset of direct jump insn */ + uintptr_t jmp_target_addr[2]; /* target address */ /* * Each TB has a NULL-terminated list (jmp_list_head) of incoming jumps. diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index 356fe348de..a87fbf74f4 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -572,14 +572,13 @@ void cpu_exec_step_atomic(CPUState *cpu) void tb_set_jmp_target(TranslationBlock *tb, int n, uintptr_t addr) { + tb->jmp_target_addr[n] = addr; if (TCG_TARGET_HAS_direct_jump) { - uintptr_t offset = tb->jmp_target_arg[n]; + uintptr_t offset = tb->jmp_insn_offset[n]; uintptr_t tc_ptr = (uintptr_t)tb->tc.ptr; uintptr_t jmp_rx = tc_ptr + offset; uintptr_t jmp_rw = jmp_rx - tcg_splitwx_diff; tb_target_set_jmp_target(tc_ptr, jmp_rx, jmp_rw, addr); - } else { - tb->jmp_target_arg[n] = addr; } } diff --git a/tcg/tcg.c b/tcg/tcg.c index c7ad46ff45..c103dd0037 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -328,7 +328,7 @@ static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) * tcg_gen_code, where we bound tcg_current_code_size to UINT16_MAX. */ tcg_debug_assert(TCG_TARGET_HAS_direct_jump); - s->gen_tb->jmp_target_arg[which] = tcg_current_code_size(s); + s->gen_tb->jmp_insn_offset[which] = tcg_current_code_size(s); } static uintptr_t G_GNUC_UNUSED get_jmp_target_addr(TCGContext *s, int which) @@ -337,7 +337,7 @@ static uintptr_t G_GNUC_UNUSED get_jmp_target_addr(TCGContext *s, int which) * Return the read-execute version of the pointer, for the benefit * of any pc-relative addressing mode. */ - return (uintptr_t)tcg_splitwx_to_rx(s->gen_tb->jmp_target_arg + which); + return (uintptr_t)tcg_splitwx_to_rx(&s->gen_tb->jmp_target_addr[which]); } /* Signal overflow, starting over with fewer guest insns. */ @@ -4252,6 +4252,8 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start) /* Initialize goto_tb jump offsets. */ tb->jmp_reset_offset[0] = TB_JMP_OFFSET_INVALID; tb->jmp_reset_offset[1] = TB_JMP_OFFSET_INVALID; + tb->jmp_insn_offset[0] = TB_JMP_OFFSET_INVALID; + tb->jmp_insn_offset[1] = TB_JMP_OFFSET_INVALID; tcg_reg_alloc_start(s); From patchwork Tue Dec 6 04:17:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81A63C3A5A7 for ; Tue, 6 Dec 2022 04:22:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POl-0003QN-54; Mon, 05 Dec 2022 23:17:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POj-0003Pc-Es for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:37 -0500 Received: from mail-ot1-x331.google.com ([2607:f8b0:4864:20::331]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POh-0006z3-AD for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:37 -0500 Received: by mail-ot1-x331.google.com with SMTP id s9-20020a05683004c900b0066e7414466bso8566020otd.12 for ; Mon, 05 Dec 2022 20:17:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3NQPdkO/a3hKfT01ZxYAHQgw+2ISUzBitXvh20BqPrU=; b=rR1gFmMGhsaIi8UwLgFFqBxzbgq6C27P3ZgopOrEzaCSWMKHESH6ZVQoh5bedfBtM6 +8slplf9tkB70H4Aa2bnE371bNXSkJ+ArpsrH1uwt89H8KAvwrX5UwUr8mtzf60tF1B5 OIIwdpCcXT5iNBuYePf565VU2a5YHDpj/pyp48s1UdDfWc5O6eFnFiYs6YZdmWLODMhW kWGUnT5iXUuErQrVsR7Np91DBZCkT2Pgm/Wh8iprUNuKr5ikAxmcxUSp8Jb9zcoUAgKF EgosNK83ujQXV/e5mNOQsC8hEVlTY4ZTXnvhx09vR0cAZvEI/nybGyglUPT2Q13eORCZ zMmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3NQPdkO/a3hKfT01ZxYAHQgw+2ISUzBitXvh20BqPrU=; b=QINZ8WB7TRVamO6B35VZfPDnji9zMilfvVJS2dat7yJYXlG+Lhk/FTu21NjTbdK6lm BP/xb5bhljd7Nnewg2LZqNvu/FPV41o7bPr49Lo31BKfJ13MkCvo4dRIPSvvRpOUTFbb WjRaHNzxgUKfKVvkOQnpv6I0DkjpajnqoJ3oehMZIizsWukoqtsJbRQs7NoJ0JpqWH1+ SsIAx1Bo2nBmoQEVISJEu0vOMi3RE8d+GoFryXZ0D+JN3Q6VSJOuDDoqHLXUUlgZTHJL 8ZU+zNlvzAS5zlyKodUu67z3X+SNBIq1dBXo6YOAdZEyVsrTwSZzaOvGLrwiaBGAd6WD XAFA== X-Gm-Message-State: ANoB5pk8gdix2ssLTBXFXZwVZoyFiGeiK3PMcoDaUuSTG0rb7oS4GXHl 2RJlVe/4fvHOi+LdbN0uyeIaqZL1IyI9RLRmONI= X-Google-Smtp-Source: AA0mqf44CZx2JfcOx5Ha+9K7LmYuOgQEV21sC41auSLdNHMDOUSU8q24e3sW+sfg2vLnOuEhHL2WNA== X-Received: by 2002:a9d:64c6:0:b0:66e:a8a1:3be2 with SMTP id n6-20020a9d64c6000000b0066ea8a13be2mr5093104otl.250.1670300254018; Mon, 05 Dec 2022 20:17:34 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:33 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 12/22] tcg: Change tb_target_set_jmp_target arguments Date: Mon, 5 Dec 2022 22:17:05 -0600 Message-Id: <20221206041715.314209-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::331; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x331.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Replace 'tc_ptr' and 'addr' with 'tb' and 'n'. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 3 ++- tcg/arm/tcg-target.h | 3 ++- tcg/i386/tcg-target.h | 9 ++------- tcg/loongarch64/tcg-target.h | 3 ++- tcg/mips/tcg-target.h | 3 ++- tcg/ppc/tcg-target.h | 3 ++- tcg/riscv/tcg-target.h | 3 ++- tcg/s390x/tcg-target.h | 10 ++-------- tcg/sparc64/tcg-target.h | 3 ++- tcg/tci/tcg-target.h | 3 ++- accel/tcg/cpu-exec.c | 6 +++--- tcg/aarch64/tcg-target.c.inc | 5 +++-- tcg/i386/tcg-target.c.inc | 9 +++++++++ tcg/loongarch64/tcg-target.c.inc | 5 +++-- tcg/ppc/tcg-target.c.inc | 7 ++++--- tcg/s390x/tcg-target.c.inc | 10 ++++++++++ tcg/sparc64/tcg-target.c.inc | 7 ++++--- 17 files changed, 56 insertions(+), 36 deletions(-) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 485f685bd2..9b0927012c 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -152,7 +152,8 @@ typedef enum { #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 0 -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *, int, + uintptr_t, uintptr_t); #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 7e96495392..743a725aa7 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -152,7 +152,8 @@ extern bool use_neon_instructions; #define TCG_TARGET_HAS_MEMORY_BSWAP 0 /* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 00fcbe297d..b64317bf40 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -218,13 +218,8 @@ extern bool have_movbe; #define TCG_TARGET_extract_i64_valid(ofs, len) \ (((ofs) == 8 && (len) == 8) || ((ofs) + (len)) == 32) -static inline void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) -{ - /* patch the branch destination */ - qatomic_set((int32_t *)jmp_rw, addr - (jmp_rx + 4)); - /* no need to flush icache explicitly */ -} +void tb_target_set_jmp_target(const TranslationBlock *, int, + uintptr_t, uintptr_t); /* This defines the natural memory order supported by this * architecture before guarantees made by various barrier diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h index a659c8d6fd..6a6c8d6941 100644 --- a/tcg/loongarch64/tcg-target.h +++ b/tcg/loongarch64/tcg-target.h @@ -170,7 +170,8 @@ typedef enum { #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #define TCG_TARGET_DEFAULT_MO (0) diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 7669213175..57154ec808 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -204,7 +204,8 @@ extern bool use_mips32r2_instructions; #define TCG_TARGET_HAS_MEMORY_BSWAP 1 /* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t) +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t) QEMU_ERROR("code path is reachable"); #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index b5cd225cfa..02764c3331 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -180,7 +180,8 @@ extern bool have_vsx; #define TCG_TARGET_HAS_bitsel_vec have_vsx #define TCG_TARGET_HAS_cmpsel_vec 0 -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 11c9b3e4f4..21d455a081 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -161,7 +161,8 @@ typedef enum { #endif /* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #define TCG_TARGET_DEFAULT_MO (0) diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index 23e2063667..d9a45e20a8 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -172,14 +172,8 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) -static inline void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) -{ - /* patch the branch destination */ - intptr_t disp = addr - (jmp_rx - 2); - qatomic_set((int32_t *)jmp_rw, disp / 2); - /* no need to flush icache explicitly */ -} +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw); #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index 8655acdbe5..f70fa9e350 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -154,7 +154,8 @@ extern bool use_vis3_instructions; #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index ceb36c4f7a..5d1a77d078 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -170,6 +170,7 @@ typedef enum { #define TCG_TARGET_HAS_MEMORY_BSWAP 1 /* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t, uintptr_t); #endif /* TCG_TARGET_H */ diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index a87fbf74f4..ac5b581e52 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -574,11 +574,11 @@ void tb_set_jmp_target(TranslationBlock *tb, int n, uintptr_t addr) { tb->jmp_target_addr[n] = addr; if (TCG_TARGET_HAS_direct_jump) { + const TranslationBlock *c_tb = tcg_splitwx_to_rx(tb); uintptr_t offset = tb->jmp_insn_offset[n]; - uintptr_t tc_ptr = (uintptr_t)tb->tc.ptr; - uintptr_t jmp_rx = tc_ptr + offset; + uintptr_t jmp_rx = (uintptr_t)tb->tc.ptr + offset; uintptr_t jmp_rw = jmp_rx - tcg_splitwx_diff; - tb_target_set_jmp_target(tc_ptr, jmp_rx, jmp_rw, addr); + tb_target_set_jmp_target(c_tb, n, jmp_rx, jmp_rw); } } diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index 072d32f355..af30f9db69 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1352,9 +1352,10 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) } } -void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) { + uintptr_t addr = tb->jmp_target_addr[n]; tcg_insn_unit i1, i2; TCGType rt = TCG_TYPE_I64; TCGReg rd = TCG_REG_TMP; diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index d3cef242ef..6650e2eda3 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -2373,6 +2373,15 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* patch the branch destination */ + uintptr_t addr = tb->jmp_target_addr[n]; + qatomic_set((int32_t *)jmp_rw, addr - (jmp_rx + 4)); + /* no need to flush icache explicitly */ +} + static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc index dbe64b7d8b..c05b19a084 100644 --- a/tcg/loongarch64/tcg-target.c.inc +++ b/tcg/loongarch64/tcg-target.c.inc @@ -1038,11 +1038,12 @@ static void tcg_out_nop(TCGContext *s) tcg_out32(s, NOP); } -void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) { tcg_insn_unit i1, i2; ptrdiff_t upper, lower; + uintptr_t addr = tb->jmp_target_addr[n]; ptrdiff_t offset = (ptrdiff_t)(addr - jmp_rx) >> 2; if (offset == sextreg(offset, 0, 26)) { diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index ec13ea3171..592b8d6498 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -1886,11 +1886,12 @@ static inline void ppc64_replace4(uintptr_t rx, uintptr_t rw, flush_idcache_range(rx, rw, 16); } -void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) { tcg_insn_unit i0, i1, i2, i3; - intptr_t tb_diff = addr - tc_ptr; + uintptr_t addr = tb->jmp_target_addr[n]; + intptr_t tb_diff = addr - (uintptr_t)tb->tc.ptr; intptr_t br_diff = addr - (jmp_rx + 4); intptr_t lo, hi; diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index a60edc18cf..82435ecadb 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2116,6 +2116,16 @@ static void tcg_out_goto_tb(TCGContext *s, int which) } } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* patch the branch destination */ + uintptr_t addr = tb->jmp_target_addr[n]; + intptr_t disp = addr - (jmp_rx - 2); + qatomic_set((int32_t *)jmp_rw, disp / 2); + /* no need to flush icache explicitly */ +} + # define OP_32_64(x) \ case glue(glue(INDEX_op_,x),_i32): \ case glue(glue(INDEX_op_,x),_i64) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index ed8efa413c..96d58f30b1 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1884,10 +1884,11 @@ void tcg_register_jit(const void *buf, size_t buf_size) tcg_register_jit_int(buf, buf_size, &debug_frame, sizeof(debug_frame)); } -void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx, - uintptr_t jmp_rw, uintptr_t addr) +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) { - intptr_t tb_disp = addr - tc_ptr; + uintptr_t addr = tb->jmp_target_addr[n]; + intptr_t tb_disp = addr - (uintptr_t)tb->tc.ptr; intptr_t br_disp = addr - jmp_rx; tcg_insn_unit i1, i2; From patchwork Tue Dec 6 04:17:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F0F7C352A1 for ; Tue, 6 Dec 2022 04:22:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POm-0003RT-QW; Mon, 05 Dec 2022 23:17:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POl-0003QT-Cq for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:39 -0500 Received: from mail-oa1-x29.google.com ([2001:4860:4864:20::29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POi-0006zf-Gd for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:38 -0500 Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-1443a16b71cso12564303fac.13 for ; Mon, 05 Dec 2022 20:17:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v17zjlYfVEAhqtn7LQ/etK4ihaKCJvMvqmpjlspRWPw=; b=iv5dHE8d5wXuoWx7ZnbxoIKLmGIciA91X4GfSMtQpYvUGiLkesxyoQBUMBsEDA35D8 Rgda0MXqHgY6qduxy4Y7mZTEbshG8QyM5ds0dTjngqgBokx5DBwZ2POg9Y/WlMFzVM3r R1VWUE93wXBI6a/+MU0Y0TNit7pmY67sG+K33MIqCmou3RMhr5D6gEOueu8W9S3e5R+j j2g+QIHbcws9dWm0wx5TeY4usypyOKdzZtH5ljiTDj1cHTHwfHxROSzdN/ih/vRxx17L 2KOi4rolmmvDBdIWVqmAD1cN7UMeHbPR4g64JQJlScV5DPYmiqyKFj8+xQH9jMWT5eQi HksA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v17zjlYfVEAhqtn7LQ/etK4ihaKCJvMvqmpjlspRWPw=; b=yMXrIHxzgrM0sOVIcwgwVa5kDCai9t/jUgPp8y3bq087Q0k9ga7Mcm9vFZa6RTkt5Z oBotB6tQpodrfBHOj9WWDg3Jbwp2JgJ6R9SAqzp2yXcll/WWfDjCrW/PJxbNJ93RTpKe pDzNbyWXmjMwaLUIyTYuMIp0WyES1IJJmyQ8uoS+3Nw+mE3zn0u/dmWl3TbcqDtnIhNW 3suIfZH9M0CdQ3teCARlSvVV3lb1fNE6HUgeXH8yNzmiH4RM+ZhtLBmPckM8Zyct0pIS Z0+9Khu5Nk4YtR2lDPB9VF3FLez2T5yBqMlhLLBxLLJ1FZTXjfhDQQYSZZ+dCQwVDlKx nyLA== X-Gm-Message-State: ANoB5pmaDpO/5MwMcuJ4JjWdVN8JjxIavRelrTrP4Rfw2JwdSCRDg8JS WiJ49r1ubWpyL6IcktaUz3t8nqVDGGWpTye1bus= X-Google-Smtp-Source: AA0mqf6151q4fWwbe/W/O64m2/Ybiz0kcJF0myGcxd/9g88Tdw1B3cPfbvICdy7LshPvCBx/YHRrmQ== X-Received: by 2002:a05:6870:c20a:b0:144:3ed1:c66 with SMTP id z10-20020a056870c20a00b001443ed10c66mr9338456oae.2.1670300255235; Mon, 05 Dec 2022 20:17:35 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:34 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 13/22] tcg: Move tb_target_set_jmp_target declaration to tcg.h Date: Mon, 5 Dec 2022 22:17:06 -0600 Message-Id: <20221206041715.314209-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::29; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x29.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- include/tcg/tcg.h | 3 +++ tcg/aarch64/tcg-target.h | 4 ---- tcg/arm/tcg-target.h | 5 ----- tcg/i386/tcg-target.h | 3 --- tcg/loongarch64/tcg-target.h | 3 --- tcg/mips/tcg-target.h | 5 ----- tcg/ppc/tcg-target.h | 4 ---- tcg/riscv/tcg-target.h | 4 ---- tcg/s390x/tcg-target.h | 4 ---- tcg/sparc64/tcg-target.h | 4 ---- tcg/tci/tcg-target.h | 4 ---- 11 files changed, 3 insertions(+), 40 deletions(-) diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index 993aafa1a2..6f3b602564 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -837,6 +837,9 @@ void tcg_func_start(TCGContext *s); int tcg_gen_code(TCGContext *s, TranslationBlock *tb, target_ulong pc_start); +void tb_target_set_jmp_target(const TranslationBlock *, int, + uintptr_t, uintptr_t); + void tcg_set_frame(TCGContext *s, TCGReg reg, intptr_t start, intptr_t size); TCGTemp *tcg_global_mem_new_internal(TCGType, TCGv_ptr, diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 9b0927012c..dc16fd0da6 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -151,10 +151,6 @@ typedef enum { #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 0 - -void tb_target_set_jmp_target(const TranslationBlock *, int, - uintptr_t, uintptr_t); - #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 743a725aa7..13ad721438 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -150,11 +150,6 @@ extern bool use_neon_instructions; #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 0 - -/* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index b64317bf40..4b4ceacfa5 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -218,9 +218,6 @@ extern bool have_movbe; #define TCG_TARGET_extract_i64_valid(ofs, len) \ (((ofs) == 8 && (len) == 8) || ((ofs) + (len)) == 32) -void tb_target_set_jmp_target(const TranslationBlock *, int, - uintptr_t, uintptr_t); - /* This defines the natural memory order supported by this * architecture before guarantees made by various barrier * instructions. diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h index 6a6c8d6941..87f40d935c 100644 --- a/tcg/loongarch64/tcg-target.h +++ b/tcg/loongarch64/tcg-target.h @@ -170,9 +170,6 @@ typedef enum { #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 57154ec808..31236d8e81 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -203,11 +203,6 @@ extern bool use_mips32r2_instructions; #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 -/* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t) - QEMU_ERROR("code path is reachable"); - #define TCG_TARGET_NEED_LDST_LABELS #endif diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 02764c3331..5ffb41fb57 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -180,12 +180,8 @@ extern bool have_vsx; #define TCG_TARGET_HAS_bitsel_vec have_vsx #define TCG_TARGET_HAS_cmpsel_vec 0 -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 - #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 21d455a081..ff2f861e82 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -160,10 +160,6 @@ typedef enum { #define TCG_TARGET_HAS_mulsh_i64 1 #endif -/* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index d9a45e20a8..274cb3cc5d 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -171,10 +171,6 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_HAS_MEMORY_BSWAP 1 #define TCG_TARGET_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) - -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t jmp_rx, uintptr_t jmp_rw); - #define TCG_TARGET_NEED_LDST_LABELS #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index f70fa9e350..ca7e4da6d3 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -153,10 +153,6 @@ extern bool use_vis3_instructions; #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 - -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #define TCG_TARGET_NEED_POOL_LABELS #endif diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 5d1a77d078..4b4b26a89f 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -169,8 +169,4 @@ typedef enum { #define TCG_TARGET_HAS_MEMORY_BSWAP 1 -/* not defined -- call should be eliminated at compile time */ -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t, uintptr_t); - #endif /* TCG_TARGET_H */ From patchwork Tue Dec 6 04:17:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 24C4AC352A1 for ; Tue, 6 Dec 2022 04:21:45 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POm-0003RG-Km; Mon, 05 Dec 2022 23:17:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POl-0003QO-7H for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:39 -0500 Received: from mail-ot1-x32c.google.com ([2607:f8b0:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POj-000719-If for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:38 -0500 Received: by mail-ot1-x32c.google.com with SMTP id v19-20020a9d5a13000000b0066e82a3872dso6548222oth.5 for ; Mon, 05 Dec 2022 20:17:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VsbLI93FZVgakuE1ah+UYlgKlpOz2uEL9CaUUy2CR+0=; b=jObHoPF//9ZmJjOl3ak8MzivBzG9ZesvH5Sz3yVk9cCWS9Igxb0IvmP9aHKCxsdGJo S4fRt0xwHWrHqLkpcwxfNUHAvAjkoSh3Cd0c1SbRvTDBl7mBl/tdCkEOVPcMqdJcEwx2 CWqwynzw1lVlr9PUbJCY86rOZqW5Wd5DX4WSiSo8V/u27xj6+GAKO97hqgm/X8MM2A/i XxNfX5kHeXR8D1JSeRn8ctuib2/g6DXSO2eusydaVjWRZnV6/6mAYHkpR5eb31BL17uh aTow5xAddmj8Rq9cCGVHKNS5nWqYwj8kTGsajqtCZwruyAfC7J19huipl+JWFKK2KEiX Gw0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VsbLI93FZVgakuE1ah+UYlgKlpOz2uEL9CaUUy2CR+0=; b=NPFNOgVkNyu0QT8rGdOYzdc905RCN0z2QnZWGu709ntZGkkTyFsAIiPLAkhG0r6TVz SA1Wwq5KycFiKKwh71SYB1G7llJUEDLT79tjPsrsRowucF+xIcvQniU3T5yclY7r1UGG SUflnkuUtfakyaGTmcCb6GSeTG32oebCHtEzio7za4q/FNyv+V0Wee9ZTiWPbzhME7JH MwaCmSsyJOjc2GtQlUnVHUqgHaL/1lrkd5NcgP+slwjiajnonUox6NWQ15I8u5MrfdYO tY+CDXqt1c3QtHeA5k9ejos7fo1/Z7T//GzBmK0sXR1/eoH2KZnu8cj7vtfxU53zO3kU DeQw== X-Gm-Message-State: ANoB5plng6qDI5WJ4HmSjZyz5Xe0dyJOCeFV2l04FP0sEdsdG/Bu4BxS udOqVad042eDCk3LsEs+Te+kWnAstzr4cXxFNS0= X-Google-Smtp-Source: AA0mqf5D6nA/NxRKGmTwYjJN89gneUzIWNO/1wdXYcUh0jiHvsaOPKBQRY3guRAt8H/XQ0siBNkLOQ== X-Received: by 2002:a05:6830:91f:b0:66e:824b:e48d with SMTP id v31-20020a056830091f00b0066e824be48dmr7992752ott.212.1670300256343; Mon, 05 Dec 2022 20:17:36 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:35 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 14/22] tcg: Always define tb_target_set_jmp_target Date: Mon, 5 Dec 2022 22:17:07 -0600 Message-Id: <20221206041715.314209-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32c; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Install empty versions for !TCG_TARGET_HAS_direct_jump hosts. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/arm/tcg-target.c.inc | 6 ++++++ tcg/mips/tcg-target.c.inc | 6 ++++++ tcg/riscv/tcg-target.c.inc | 6 ++++++ tcg/tci/tcg-target.c.inc | 6 ++++++ 4 files changed, 24 insertions(+) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index 0afc286c8e..f5103dcc6d 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1958,6 +1958,12 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* Always indirect, nothing to do */ +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index 605aa7714a..dfb31a17bb 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1976,6 +1976,12 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* Always indirect, nothing to do */ +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index e73dfb4cbb..06308c5243 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1285,6 +1285,12 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* Always indirect, nothing to do */ +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index cf84ff3133..31094067b7 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -606,6 +606,12 @@ static void tcg_out_goto_tb(TCGContext *s, int which) set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + /* Always indirect, nothing to do */ +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) From patchwork Tue Dec 6 04:17:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D312CC352A1 for ; Tue, 6 Dec 2022 04:19:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POp-0003SH-Bk; Mon, 05 Dec 2022 23:17:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POn-0003RX-7j for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:41 -0500 Received: from mail-oa1-x32.google.com ([2001:4860:4864:20::32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POl-00071n-3J for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:41 -0500 Received: by mail-oa1-x32.google.com with SMTP id 586e51a60fabf-1447c7aa004so7929005fac.11 for ; Mon, 05 Dec 2022 20:17:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WwJxbSoVGL5mD629YidrzGLXu4Bi0Swxh1yfXQO3WsY=; b=m0uMhHU7rOC09ZSPqCEH3Me8k5etZjogDlRoCSzHlaG3B9okHIxt4weSBTSNbd71Gq TPKqpsvxH9+UzpOdOn/qZ5NA0IvikFyFFpvlDi1pj+MqxE9eymumBxMNVBfyIctlV4VD 2SzKzTttWoTxKLVsl97NIlkHTfxmoXNHvhMubCpDsOU9NP6WO2sS4hZHTph3hefo2Hg/ SVqOzLqwC7ldAgwjcw8u0kBegcaBnu+xru8b/BxcpEYmGc+PcXxflDQS4OH9WLN9QdK6 QgyxaDR3RGwaoApNjvHH0yCp1eNC8NIOrisXZCwtDeoCWQMlilzgyGkz/77FnttazGmb FFEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WwJxbSoVGL5mD629YidrzGLXu4Bi0Swxh1yfXQO3WsY=; b=JG9fiC6VaEF0iTU79I2o/ki65vD1aZQSCTvZy6nnyRtH9Ej2FBOloRnwfcCLQCMck3 4Ogw0JRorJW8QX+7QBMAQtoswLsEEzCZdcAy96Hqv5FyXXI0VlMNWZjCydB3ieySQVY9 gyixEdZ1XGs2gA7vF+qZ/fBBeM2H1LjDobBZr4ohpdk1BMSZ02qksww15/iybY44go5O qSSX4Yg3TommWKwwoR7t/cuVJxIsk4MQA9dbbVmmdhEVIOlbAmhPOMqvBtuIyuPXCId0 Bc85jFWnpQgkgeICxfhuU3W7Be2PQ51DqMYmVHQ5mnLKqRQWVwepzaxSfyNvvo3jlOVz 3AOg== X-Gm-Message-State: ANoB5pkg+oZyScZoTSk724j02r4vINpHwbC9DCAAcaEL3OyPiqCF5Fo9 gu/qJ2XtFMViOZ1g6n3TnVttkmbdCQPsf3ei1h4= X-Google-Smtp-Source: AA0mqf6wYdfF+QtePU1MZah6+EStznjZ5UiE/kY/LEogTNYG4aNzp8M3N6nxFMivaccR7jjCFKyXJw== X-Received: by 2002:a05:6870:d3ca:b0:144:922e:337d with SMTP id l10-20020a056870d3ca00b00144922e337dmr4632985oag.25.1670300257464; Mon, 05 Dec 2022 20:17:37 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:36 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 15/22] tcg: Remove TCG_TARGET_HAS_direct_jump Date: Mon, 5 Dec 2022 22:17:08 -0600 Message-Id: <20221206041715.314209-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::32; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x32.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We now have the option to generate direct or indirect goto_tb depending on the dynamic displacement, thus the define is no longer necessary or completely accurate. Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 1 - tcg/arm/tcg-target.h | 1 - tcg/i386/tcg-target.h | 1 - tcg/loongarch64/tcg-target.h | 1 - tcg/mips/tcg-target.h | 1 - tcg/ppc/tcg-target.h | 1 - tcg/riscv/tcg-target.h | 1 - tcg/s390x/tcg-target.h | 1 - tcg/sparc64/tcg-target.h | 1 - tcg/tci/tcg-target.h | 1 - accel/tcg/cpu-exec.c | 13 ++++++------- tcg/tcg.c | 1 - tcg/arm/tcg-target.c.inc | 1 - tcg/mips/tcg-target.c.inc | 1 - tcg/riscv/tcg-target.c.inc | 1 - tcg/s390x/tcg-target.c.inc | 5 ++++- tcg/tci/tcg-target.c.inc | 1 - 17 files changed, 10 insertions(+), 23 deletions(-) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index dc16fd0da6..c8202e4bc5 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -123,7 +123,6 @@ typedef enum { #define TCG_TARGET_HAS_muls2_i64 0 #define TCG_TARGET_HAS_muluh_i64 1 #define TCG_TARGET_HAS_mulsh_i64 1 -#define TCG_TARGET_HAS_direct_jump 1 #define TCG_TARGET_HAS_v64 1 #define TCG_TARGET_HAS_v128 1 diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 13ad721438..879eecd93d 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -121,7 +121,6 @@ extern bool use_neon_instructions; #define TCG_TARGET_HAS_mulsh_i32 0 #define TCG_TARGET_HAS_div_i32 use_idiv_instructions #define TCG_TARGET_HAS_rem_i32 0 -#define TCG_TARGET_HAS_direct_jump 0 #define TCG_TARGET_HAS_qemu_st8_i32 0 #define TCG_TARGET_HAS_v64 use_neon_instructions diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 4b4ceacfa5..b1ffd47493 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -139,7 +139,6 @@ extern bool have_movbe; #define TCG_TARGET_HAS_muls2_i32 1 #define TCG_TARGET_HAS_muluh_i32 0 #define TCG_TARGET_HAS_mulsh_i32 0 -#define TCG_TARGET_HAS_direct_jump 1 #if TCG_TARGET_REG_BITS == 64 /* Keep target addresses zero-extended in a register. */ diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h index 87f40d935c..086c90bda3 100644 --- a/tcg/loongarch64/tcg-target.h +++ b/tcg/loongarch64/tcg-target.h @@ -127,7 +127,6 @@ typedef enum { #define TCG_TARGET_HAS_clz_i32 1 #define TCG_TARGET_HAS_ctz_i32 1 #define TCG_TARGET_HAS_ctpop_i32 0 -#define TCG_TARGET_HAS_direct_jump 1 #define TCG_TARGET_HAS_brcond2 0 #define TCG_TARGET_HAS_setcond2 0 #define TCG_TARGET_HAS_qemu_st8_i32 0 diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 31236d8e81..cc0a6f301a 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -132,7 +132,6 @@ extern bool use_mips32r2_instructions; #define TCG_TARGET_HAS_muluh_i32 1 #define TCG_TARGET_HAS_mulsh_i32 1 #define TCG_TARGET_HAS_bswap32_i32 1 -#define TCG_TARGET_HAS_direct_jump 0 #if TCG_TARGET_REG_BITS == 64 #define TCG_TARGET_HAS_add2_i32 0 diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 5ffb41fb57..f253184915 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -108,7 +108,6 @@ extern bool have_vsx; #define TCG_TARGET_HAS_muls2_i32 0 #define TCG_TARGET_HAS_muluh_i32 1 #define TCG_TARGET_HAS_mulsh_i32 1 -#define TCG_TARGET_HAS_direct_jump 1 #define TCG_TARGET_HAS_qemu_st8_i32 0 #if TCG_TARGET_REG_BITS == 64 diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index ff2f861e82..56f7bc3346 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -116,7 +116,6 @@ typedef enum { #define TCG_TARGET_HAS_clz_i32 0 #define TCG_TARGET_HAS_ctz_i32 0 #define TCG_TARGET_HAS_ctpop_i32 0 -#define TCG_TARGET_HAS_direct_jump 0 #define TCG_TARGET_HAS_brcond2 1 #define TCG_TARGET_HAS_setcond2 1 #define TCG_TARGET_HAS_qemu_st8_i32 0 diff --git a/tcg/s390x/tcg-target.h b/tcg/s390x/tcg-target.h index 274cb3cc5d..2c7c14055b 100644 --- a/tcg/s390x/tcg-target.h +++ b/tcg/s390x/tcg-target.h @@ -103,7 +103,6 @@ extern uint64_t s390_facilities[3]; #define TCG_TARGET_HAS_mulsh_i32 0 #define TCG_TARGET_HAS_extrl_i64_i32 0 #define TCG_TARGET_HAS_extrh_i64_i32 0 -#define TCG_TARGET_HAS_direct_jump HAVE_FACILITY(GEN_INST_EXT) #define TCG_TARGET_HAS_qemu_st8_i32 0 #define TCG_TARGET_HAS_div2_i64 1 diff --git a/tcg/sparc64/tcg-target.h b/tcg/sparc64/tcg-target.h index ca7e4da6d3..21d3c59bd7 100644 --- a/tcg/sparc64/tcg-target.h +++ b/tcg/sparc64/tcg-target.h @@ -110,7 +110,6 @@ extern bool use_vis3_instructions; #define TCG_TARGET_HAS_muls2_i32 1 #define TCG_TARGET_HAS_muluh_i32 0 #define TCG_TARGET_HAS_mulsh_i32 0 -#define TCG_TARGET_HAS_direct_jump 1 #define TCG_TARGET_HAS_qemu_st8_i32 0 #define TCG_TARGET_HAS_extrl_i64_i32 1 diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 4b4b26a89f..c9d2fe5b94 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -82,7 +82,6 @@ #define TCG_TARGET_HAS_muls2_i32 1 #define TCG_TARGET_HAS_muluh_i32 0 #define TCG_TARGET_HAS_mulsh_i32 0 -#define TCG_TARGET_HAS_direct_jump 0 #define TCG_TARGET_HAS_qemu_st8_i32 0 #if TCG_TARGET_REG_BITS == 64 diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c index ac5b581e52..0892c6534a 100644 --- a/accel/tcg/cpu-exec.c +++ b/accel/tcg/cpu-exec.c @@ -572,14 +572,13 @@ void cpu_exec_step_atomic(CPUState *cpu) void tb_set_jmp_target(TranslationBlock *tb, int n, uintptr_t addr) { + const TranslationBlock *c_tb = tcg_splitwx_to_rx(tb); + uintptr_t offset = tb->jmp_insn_offset[n]; + uintptr_t jmp_rx = (uintptr_t)tb->tc.ptr + offset; + uintptr_t jmp_rw = jmp_rx - tcg_splitwx_diff; + tb->jmp_target_addr[n] = addr; - if (TCG_TARGET_HAS_direct_jump) { - const TranslationBlock *c_tb = tcg_splitwx_to_rx(tb); - uintptr_t offset = tb->jmp_insn_offset[n]; - uintptr_t jmp_rx = (uintptr_t)tb->tc.ptr + offset; - uintptr_t jmp_rw = jmp_rx - tcg_splitwx_diff; - tb_target_set_jmp_target(c_tb, n, jmp_rx, jmp_rw); - } + tb_target_set_jmp_target(c_tb, n, jmp_rx, jmp_rw); } static inline void tb_add_jump(TranslationBlock *tb, int n, diff --git a/tcg/tcg.c b/tcg/tcg.c index c103dd0037..b09500ffb1 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -327,7 +327,6 @@ static void G_GNUC_UNUSED set_jmp_insn_offset(TCGContext *s, int which) * We will check for overflow at the end of the opcode loop in * tcg_gen_code, where we bound tcg_current_code_size to UINT16_MAX. */ - tcg_debug_assert(TCG_TARGET_HAS_direct_jump); s->gen_tb->jmp_insn_offset[which] = tcg_current_code_size(s); } diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index f5103dcc6d..a44d1d969f 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -1939,7 +1939,6 @@ static void tcg_out_goto_tb(TCGContext *s, int which) intptr_t ptr, dif, dil; TCGReg base = TCG_REG_PC; - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); ptr = get_jmp_target_addr(s, which); dif = tcg_pcrel_diff(s, (void *)ptr) - 8; dil = sextract32(dif, 0, 12); diff --git a/tcg/mips/tcg-target.c.inc b/tcg/mips/tcg-target.c.inc index dfb31a17bb..d52b6a902b 100644 --- a/tcg/mips/tcg-target.c.inc +++ b/tcg/mips/tcg-target.c.inc @@ -1968,7 +1968,6 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { /* indirect jump method */ - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, TCG_REG_ZERO, get_jmp_target_addr(s, which)); tcg_out_opc_reg(s, OPC_JR, 0, TCG_TMP0, 0); diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 06308c5243..37baae9cda 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1277,7 +1277,6 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, get_jmp_target_addr(s, which)); diff --git a/tcg/s390x/tcg-target.c.inc b/tcg/s390x/tcg-target.c.inc index 82435ecadb..cf96ccb053 100644 --- a/tcg/s390x/tcg-target.c.inc +++ b/tcg/s390x/tcg-target.c.inc @@ -2083,7 +2083,7 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { - if (TCG_TARGET_HAS_direct_jump) { + if (HAVE_FACILITY(GEN_INST_EXT)) { /* * Branch displacement must be aligned for atomic patching; * see if we need to add extra nop before branch @@ -2119,6 +2119,9 @@ static void tcg_out_goto_tb(TCGContext *s, int which) void tb_target_set_jmp_target(const TranslationBlock *tb, int n, uintptr_t jmp_rx, uintptr_t jmp_rw) { + if (!HAVE_FACILITY(GEN_INST_EXT)) { + return; + } /* patch the branch destination */ uintptr_t addr = tb->jmp_target_addr[n]; intptr_t disp = addr - (jmp_rx - 2); diff --git a/tcg/tci/tcg-target.c.inc b/tcg/tci/tcg-target.c.inc index 31094067b7..434fcc74dc 100644 --- a/tcg/tci/tcg-target.c.inc +++ b/tcg/tci/tcg-target.c.inc @@ -600,7 +600,6 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) static void tcg_out_goto_tb(TCGContext *s, int which) { - qemu_build_assert(!TCG_TARGET_HAS_direct_jump); /* indirect jump method. */ tcg_out_op_p(s, INDEX_op_goto_tb, (void *)get_jmp_target_addr(s, which)); set_jmp_reset_offset(s, which); From patchwork Tue Dec 6 04:17:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4B23FC3A5A7 for ; Tue, 6 Dec 2022 04:18:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POp-0003SU-UQ; Mon, 05 Dec 2022 23:17:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POo-0003Rh-9U for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:42 -0500 Received: from mail-ot1-x330.google.com ([2607:f8b0:4864:20::330]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POl-00072B-Ts for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:42 -0500 Received: by mail-ot1-x330.google.com with SMTP id p24-20020a0568301d5800b0066e6dc09be5so8575461oth.8 for ; Mon, 05 Dec 2022 20:17:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PQXI8Tic8Aq/NBUY33irNWjhT2v3z3OgYZUNuPxrg78=; b=oD33QhDWZfze4VyiOZyxWx4g+0MWolL8M7AqOGX92HHc2IsAZXEs+v2cO+VLORzHvq 6vfVUp18Umk474wfIj9F8aV31cYkACB9ttPhkuXWVpoPnqIfTSShDbjEO3lJ39dIFZ3m VtNX+Q6SKJ3ljXVXzDjQEyrLM86Ent8o6oqbWYpuA33kbgBDzok7mOEm8fiAUOFmB5e4 GcSlBbMUQ5QQiuvb0AKb05QlxI3hT8AxQdh0LF3eZ8CFpP4qljsd1hekFQM9vHuDSgi7 jpRxhdXsvAc2USd9bKRFMRrcFUncb756XMq1DiHD9suvX8dGKK5KUx34Av6KAsBCiOfd HofQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PQXI8Tic8Aq/NBUY33irNWjhT2v3z3OgYZUNuPxrg78=; b=NWLyu+S1sb3YM77vXFf4oMwr8I6mqHUgtVbocllEwsJs1yWrdCJw6UNThFmDkVP71v /LDYDWkAKTvF14YbWoZOpdUaQaAarvnt1OMuNqTZiuRUFGAsw+xQZKrzxYAufQ5ANxt3 GseByrYxb45nYD7Z93D4icnWauBDXI0lK9GlbwbP8d/kGYrs10/icYMzgyfQ623Peuw5 LSeCXzJZXA6F58aZkkC8r5Rbt5uaFiw3pZekxFqLInojrFE/VmwOJhYHD9IuJ7yxCBPa pyAesyZXRhlHhauRnufuDjuZBFekJNvRT26v/CQ9c9yAquAiJYNb2c0VNsybQpqLmNr2 CH9w== X-Gm-Message-State: ANoB5pnYwAW3BtUP0AyEfqnnmsgmNshyrblQAXeDGSIcknrbHUYHOL8p Pa7uyTf2f2GGS9GpxcCf0H8oEx5LO0Eq5Y5Z9VE= X-Google-Smtp-Source: AA0mqf5DVowDBShAeT+YnfGsEK8lZ8CKMZuc1ler0vRXLF4TQb7z2UGEGLrCbXNNvSqLxwWyHoXEPg== X-Received: by 2002:a9d:7dca:0:b0:66c:6cf1:7967 with SMTP id k10-20020a9d7dca000000b0066c6cf17967mr42520737otn.139.1670300258525; Mon, 05 Dec 2022 20:17:38 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:38 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 16/22] tcg/aarch64: Reorg goto_tb implementation Date: Mon, 5 Dec 2022 22:17:09 -0600 Message-Id: <20221206041715.314209-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::330; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x330.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The old implementation replaces two insns, swapping between b nop br x30 and adrp x30, addi x30, x30, lo12: br x30 There is a race condition in which a thread could be stopped at the PC of the second insn, and when restarted does not see the complete address computation and branches to nowhere. The new implemetation replaces only one insn, swapping between b br tmp and ldr tmp, br tmp Reported-by: hev Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.c.inc | 64 +++++++++++++++--------------------- 2 files changed, 27 insertions(+), 39 deletions(-) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index c8202e4bc5..9b8835eaa3 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -15,7 +15,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 -#define MAX_CODE_GEN_BUFFER_SIZE (2 * GiB) +#define MAX_CODE_GEN_BUFFER_SIZE ((size_t)-1) #undef TCG_TARGET_STACK_GROWSUP typedef enum { diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index af30f9db69..fe8e8636aa 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -1352,33 +1352,6 @@ static void tcg_out_call(TCGContext *s, const tcg_insn_unit *target) } } -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t jmp_rx, uintptr_t jmp_rw) -{ - uintptr_t addr = tb->jmp_target_addr[n]; - tcg_insn_unit i1, i2; - TCGType rt = TCG_TYPE_I64; - TCGReg rd = TCG_REG_TMP; - uint64_t pair; - - ptrdiff_t offset = addr - jmp_rx; - - if (offset == sextract64(offset, 0, 26)) { - i1 = I3206_B | ((offset >> 2) & 0x3ffffff); - i2 = NOP; - } else { - offset = (addr >> 12) - (jmp_rx >> 12); - - /* patch ADRP */ - i1 = I3406_ADRP | (offset & 3) << 29 | (offset & 0x1ffffc) << (5 - 2) | rd; - /* patch ADDI */ - i2 = I3401_ADDI | rt << 31 | (addr & 0xfff) << 10 | rd << 5 | rd; - } - pair = (uint64_t)i2 << 32 | i1; - qatomic_set((uint64_t *)jmp_rw, pair); - flush_idcache_range(jmp_rx, jmp_rw, 8); -} - static inline void tcg_out_goto_label(TCGContext *s, TCGLabel *l) { if (!l->has_value) { @@ -1901,23 +1874,38 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { /* - * Ensure that ADRP+ADD are 8-byte aligned so that an atomic - * write can be used to patch the target address. + * Direct branch, or indirect address load, will be patched + * by tb_target_set_jmp_target. Assert indirect load offset + * in range early, regardless of direct branch distance. */ - if ((uintptr_t)s->code_ptr & 7) { - tcg_out32(s, NOP); - } + intptr_t i_off = tcg_pcrel_diff(s, (void *)get_jmp_target_addr(s, which)); + tcg_debug_assert(i_off == sextract64(i_off, 0, 21)); + set_jmp_insn_offset(s, which); - /* - * actual branch destination will be patched by - * tb_target_set_jmp_target later - */ - tcg_out_insn(s, 3406, ADRP, TCG_REG_TMP, 0); - tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_TMP, TCG_REG_TMP, 0); + tcg_out32(s, I3206_B); tcg_out_insn(s, 3207, BR, TCG_REG_TMP); set_jmp_reset_offset(s, which); } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + uintptr_t d_addr = tb->jmp_target_addr[n]; + uintptr_t i_addr = (uintptr_t)&tb->jmp_target_addr[n]; + ptrdiff_t d_offset = d_addr - jmp_rx; + ptrdiff_t i_offset = i_addr - jmp_rx; + tcg_insn_unit insn; + + /* Either directly branch, or indirect branch load. */ + if (d_offset == sextract64(d_offset, 0, 26)) { + insn = I3206_B | ((d_offset >> 2) & 0x3ffffff); + } else { + insn = I3305_LDR | TCG_REG_TMP | (((i_offset >> 2) & 0x7ffff) << 5); + } + qatomic_set((uint32_t *)jmp_rw, insn); + flush_idcache_range(jmp_rx, jmp_rw, 4); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) From patchwork Tue Dec 6 04:17:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6FA65C3A5A7 for ; Tue, 6 Dec 2022 04:24:18 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POr-0003T7-Ea; Mon, 05 Dec 2022 23:17:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POp-0003S9-49 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:43 -0500 Received: from mail-ot1-x32f.google.com ([2607:f8b0:4864:20::32f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POn-00072b-3r for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:42 -0500 Received: by mail-ot1-x32f.google.com with SMTP id q18-20020a056830441200b006704633f258so2220324otv.0 for ; Mon, 05 Dec 2022 20:17:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZmoxygvVTEq2d+IOQdC2EOT2LDl9LC/v0r7RV1PzJPs=; b=dH5WQuCUyab/nVKvhFh5Xf0N8oApBGkfYpGfC6liSrFOArBMpkQKjt2O3v0X6vytvA ZIWuK3nkFx1EJbrqZlfEkWVt3/pgxa9Zc3uem+A+mwdzmqq0NeXxVP3HnCzmDW02nwFe CcBcwqnB8AJKaQXjDAD0U5ctKqg5l6X8VcL71ysQTJXFvGntRHGzyxfwGzvF7IIRNdij LXqFNLqRmRppYtc0fR3L2X3HkXT6+mqYvmTUS9Sv6AAw4jkKdyMjNIfC/jfSAdo4KTuP Gjwt/78Q2EjZH/9EZV7Q1G8hg7Ga8wvpsa6FRH4QRzkNbNkSYRjQzBXdxvnmKyy3+Vjq Rz4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZmoxygvVTEq2d+IOQdC2EOT2LDl9LC/v0r7RV1PzJPs=; b=EEgEIgHnS5kuXoFty+qgDaQ8O0Vvf7iMU01HWbjL/SXDCy4uQmR98ZZ1Zn0z/7rOor JjDKUt6VC2L7FGc18q0cjhgJXgsEBniZPVKSFz3hNLGwl2E2mNnZgQeIsaQxYaLhygFN awbb6HJQVcFZpqgmyHLRDq1d/w89KixDpnkyUowJMdG7J9uz1gZjOaMVHTs3HYrbUf6K Oot4COMP9hmIfDsxEuDo3wGafLOWXMUNuEsWqNRBOYSoG8NJR55Hx2xsUJEtBDwHOXIL czRsocknUkGFOWBaeLo7VzZ0xabo0tg/2aYf66vpwDiJYnCpNQfjE2167a9QD59esZDI 4J8Q== X-Gm-Message-State: ANoB5pkND8v949rVyE6krguT6OHFTwtFmj+LqmHxBjkAJnNAhyZImddp lY/EXQwbrzAzDMeAUl0FEN715zXptty0hoaAqso= X-Google-Smtp-Source: AA0mqf6jXhd4z1qRZQn/oB2I6fyAZYvUR6WGrD2sWAG+DfvyDH2NtqzEIDyzy0f9WAstSkSEQsBraw== X-Received: by 2002:a05:6830:1bec:b0:66e:7deb:b5b with SMTP id k12-20020a0568301bec00b0066e7deb0b5bmr9059669otb.295.1670300259868; Mon, 05 Dec 2022 20:17:39 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:39 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 17/22] tcg/ppc: Reorg goto_tb implementation Date: Mon, 5 Dec 2022 22:17:10 -0600 Message-Id: <20221206041715.314209-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::32f; envelope-from=richard.henderson@linaro.org; helo=mail-ot1-x32f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The old ppc64 implementation replaces 2 or 4 insns, which leaves a race condition in which a thread could be stopped at a PC in the middle of the sequence, and when restarted does not see the complete address computation and branches to nowhere. The new implemetation replaces only one insn, swapping between b and mtctr r31 falling through to a general-case indirect branch. Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.h | 3 +- tcg/ppc/tcg-target.c.inc | 158 +++++++++++---------------------------- 2 files changed, 44 insertions(+), 117 deletions(-) diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index f253184915..af81c5a57f 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -27,11 +27,10 @@ #ifdef _ARCH_PPC64 # define TCG_TARGET_REG_BITS 64 -# define MAX_CODE_GEN_BUFFER_SIZE (2 * GiB) #else # define TCG_TARGET_REG_BITS 32 -# define MAX_CODE_GEN_BUFFER_SIZE (32 * MiB) #endif +#define MAX_CODE_GEN_BUFFER_SIZE ((size_t)-1) #define TCG_TARGET_NB_REGS 64 #define TCG_TARGET_INSN_UNIT_SIZE 4 diff --git a/tcg/ppc/tcg-target.c.inc b/tcg/ppc/tcg-target.c.inc index 592b8d6498..755b954447 100644 --- a/tcg/ppc/tcg-target.c.inc +++ b/tcg/ppc/tcg-target.c.inc @@ -1847,104 +1847,6 @@ static void tcg_out_mb(TCGContext *s, TCGArg a0) tcg_out32(s, insn); } -static inline uint64_t make_pair(tcg_insn_unit i1, tcg_insn_unit i2) -{ - if (HOST_BIG_ENDIAN) { - return (uint64_t)i1 << 32 | i2; - } - return (uint64_t)i2 << 32 | i1; -} - -static inline void ppc64_replace2(uintptr_t rx, uintptr_t rw, - tcg_insn_unit i0, tcg_insn_unit i1) -{ -#if TCG_TARGET_REG_BITS == 64 - qatomic_set((uint64_t *)rw, make_pair(i0, i1)); - flush_idcache_range(rx, rw, 8); -#else - qemu_build_not_reached(); -#endif -} - -static inline void ppc64_replace4(uintptr_t rx, uintptr_t rw, - tcg_insn_unit i0, tcg_insn_unit i1, - tcg_insn_unit i2, tcg_insn_unit i3) -{ - uint64_t p[2]; - - p[!HOST_BIG_ENDIAN] = make_pair(i0, i1); - p[HOST_BIG_ENDIAN] = make_pair(i2, i3); - - /* - * There's no convenient way to get the compiler to allocate a pair - * of registers at an even index, so copy into r6/r7 and clobber. - */ - asm("mr %%r6, %1\n\t" - "mr %%r7, %2\n\t" - "stq %%r6, %0" - : "=Q"(*(__int128 *)rw) : "r"(p[0]), "r"(p[1]) : "r6", "r7"); - flush_idcache_range(rx, rw, 16); -} - -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t jmp_rx, uintptr_t jmp_rw) -{ - tcg_insn_unit i0, i1, i2, i3; - uintptr_t addr = tb->jmp_target_addr[n]; - intptr_t tb_diff = addr - (uintptr_t)tb->tc.ptr; - intptr_t br_diff = addr - (jmp_rx + 4); - intptr_t lo, hi; - - if (TCG_TARGET_REG_BITS == 32) { - intptr_t diff = addr - jmp_rx; - tcg_debug_assert(in_range_b(diff)); - qatomic_set((uint32_t *)jmp_rw, B | (diff & 0x3fffffc)); - flush_idcache_range(jmp_rx, jmp_rw, 4); - return; - } - - /* - * For 16-bit displacements, we can use a single add + branch. - * This happens quite often. - */ - if (tb_diff == (int16_t)tb_diff) { - i0 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, tb_diff); - i1 = B | (br_diff & 0x3fffffc); - ppc64_replace2(jmp_rx, jmp_rw, i0, i1); - return; - } - - lo = (int16_t)tb_diff; - hi = (int32_t)(tb_diff - lo); - assert(tb_diff == hi + lo); - i0 = ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, hi >> 16); - i1 = ADDI | TAI(TCG_REG_TB, TCG_REG_TB, lo); - - /* - * Without stq from 2.07, we can only update two insns, - * and those must be the ones that load the target address. - */ - if (!have_isa_2_07) { - ppc64_replace2(jmp_rx, jmp_rw, i0, i1); - return; - } - - /* - * For 26-bit displacements, we can use a direct branch. - * Otherwise we still need the indirect branch, which we - * must restore after a potential direct branch write. - */ - br_diff -= 4; - if (in_range_b(br_diff)) { - i2 = B | (br_diff & 0x3fffffc); - i3 = NOP; - } else { - i2 = MTSPR | RS(TCG_REG_TB) | CTR; - i3 = BCCTR | BO_ALWAYS; - } - ppc64_replace4(jmp_rx, jmp_rw, i0, i1, i2, i3); -} - static void tcg_out_call_int(TCGContext *s, int lk, const tcg_insn_unit *target) { @@ -2625,30 +2527,56 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) static void tcg_out_goto_tb(TCGContext *s, int which) { - /* Direct jump. */ - if (TCG_TARGET_REG_BITS == 64) { - /* Ensure the next insns are 8 or 16-byte aligned. */ - while ((uintptr_t)s->code_ptr & (have_isa_2_07 ? 15 : 7)) { - tcg_out32(s, NOP); - } + uintptr_t ptr = get_jmp_target_addr(s, which); + + if (USE_REG_TB) { + ptrdiff_t offset = tcg_tbrel_diff(s, (void *)ptr); + tcg_out_mem_long(s, LD, LDX, TCG_REG_TB, TCG_REG_TB, offset); + + /* Direct branch will be patched by tb_target_set_jmp_target. */ set_jmp_insn_offset(s, which); - tcg_out32(s, ADDIS | TAI(TCG_REG_TB, TCG_REG_TB, 0)); - tcg_out32(s, ADDI | TAI(TCG_REG_TB, TCG_REG_TB, 0)); tcg_out32(s, MTSPR | RS(TCG_REG_TB) | CTR); + + /* When branch is out of range, fall through to indirect. */ + tcg_out32(s, BCCTR | BO_ALWAYS); + + /* For the unlinked case, need to reset TCG_REG_TB. */ + set_jmp_reset_offset(s, which); + tcg_out_mem_long(s, ADDI, ADD, TCG_REG_TB, TCG_REG_TB, + -tcg_current_code_size(s)); + } else { + /* Direct branch will be patched by tb_target_set_jmp_target. */ + set_jmp_insn_offset(s, which); + tcg_out32(s, NOP); + + /* When branch is out of range, fall through to indirect. */ + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_TMP1, ptr - (int16_t)ptr); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, TCG_REG_TMP1, (int16_t)ptr); + tcg_out32(s, MTSPR | RS(TCG_REG_TMP1) | CTR); tcg_out32(s, BCCTR | BO_ALWAYS); set_jmp_reset_offset(s, which); - if (USE_REG_TB) { - /* For the unlinked case, need to reset TCG_REG_TB. */ - tcg_out_mem_long(s, ADDI, ADD, TCG_REG_TB, TCG_REG_TB, - -tcg_current_code_size(s)); - } - } else { - set_jmp_insn_offset(s, which); - tcg_out32(s, B); - set_jmp_reset_offset(s, which); } } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + uintptr_t addr = tb->jmp_target_addr[n]; + intptr_t diff = addr - jmp_rx; + tcg_insn_unit insn; + + if (in_range_b(diff)) { + insn = B | (diff & 0x3fffffc); + } else if (USE_REG_TB) { + insn = MTSPR | RS(TCG_REG_TB) | CTR; + } else { + insn = NOP; + } + + qatomic_set((uint32_t *)jmp_rw, insn); + flush_idcache_range(jmp_rx, jmp_rw, 4); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) From patchwork Tue Dec 6 04:17:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9574EC352A1 for ; Tue, 6 Dec 2022 04:23:44 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POr-0003T8-WD; Mon, 05 Dec 2022 23:17:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POp-0003SJ-Fz for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:43 -0500 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POn-0006wa-Os for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:43 -0500 Received: by mail-oi1-x234.google.com with SMTP id c129so15647967oia.0 for ; Mon, 05 Dec 2022 20:17:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eHbWZNc2e37AaQzsCP2Rz1slFzFOy+tMFTM0BqvuSPQ=; b=gRu/lVnyATgEVBYnNYc02ey5JDwMPFgHnGt8TuaeAv6/RETNF1KSVoujWhjt9cafZ6 WsErLl/qWG4xPeCapSxI4UuimM1j9q4LYzs6UeIqnrOP4BRfPL5NWNqDJlhr6xGHM7bp oSq9x+7aMKPutnO/IqT9vknRGCM2lOmtLTrHIQZqxwVyTWmjipvvEUyUnwVECLaa6JqS TCjCKq+JH3zN/P/vBqmMKD3ONNOfzNGan74qU4puDFZMnrzmKDXr/5cJeRh63GsWXT6g iRM37otQigdLFcR/ZVgiPB9mZpPTRBC8InwLJNwkpYWEMk8FmEeK/zDb8wtLxmVGY/Fl U6JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eHbWZNc2e37AaQzsCP2Rz1slFzFOy+tMFTM0BqvuSPQ=; b=e+40tpcl02jSq35iDrQ5HJWxqPpXFxpglWBjBuRHkYBIp/GgES/ENKbwmkAn2L0wmH vOSTp+3X0nHYIh/xK3+rn/HuGjhPt9Adlh9eWMfPYPdAuSz7aLriX8JITUdv+MKEcqyx RKUjzfytGZI4ApSvLTqJ8KwyTVTO9bWU6KUL9bArJyCCRBDkUhY7wd+rwaCRqrIDXLPX RgA/nt+dM3LXwuLcPo5uR//2VYYyxLdI2Hb8tF7n40GRWU8SaqG5v/oUKdOZ5xSPVPRi +hvtbdt/bOPve7/x9cENvEeONhDybL/8Qk3VX8iNapa0nhp8Y5x52catdQs67fu+cUXF VEOQ== X-Gm-Message-State: ANoB5plFjZLsi0/Ko8Adn1zie9FWdEC6HCzHGY67HPUBifp+8/qrqAjX iFO9ndgWkbUSuVLwQXjTecjP4vRMxxB92KDw7IY= X-Google-Smtp-Source: AA0mqf7xp3Ij50GoWEI7tsbb+tfsR2tJwWH+Q6e/8nxcfY04Z1t/tb9rJsPIkDikZjv/kaY3R7NLQw== X-Received: by 2002:aca:b504:0:b0:35b:4b8c:9f87 with SMTP id e4-20020acab504000000b0035b4b8c9f87mr31281939oif.188.1670300261017; Mon, 05 Dec 2022 20:17:41 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:40 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 18/22] tcg/sparc64: Remove USE_REG_TB Date: Mon, 5 Dec 2022 22:17:11 -0600 Message-Id: <20221206041715.314209-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::234; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x234.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is always true for sparc64, so this is dead since 3a5f6805c7ca. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/sparc64/tcg-target.c.inc | 57 ++++++++++++++---------------------- 1 file changed, 22 insertions(+), 35 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 96d58f30b1..1ae9615ef0 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -92,7 +92,6 @@ static const char * const tcg_target_reg_names[TCG_TARGET_NB_REGS] = { #endif #define TCG_REG_TB TCG_REG_I1 -#define USE_REG_TB (sizeof(void *) > 4) static const int tcg_target_reg_alloc_order[] = { TCG_REG_L0, @@ -439,7 +438,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, } /* A 13-bit constant relative to the TB. */ - if (!in_prologue && USE_REG_TB) { + if (!in_prologue) { test = tcg_tbrel_diff(s, (void *)arg); if (check_fit_ptr(test, 13)) { tcg_out_arithi(s, ret, TCG_REG_TB, test, ARITH_ADD); @@ -468,7 +467,7 @@ static void tcg_out_movi_int(TCGContext *s, TCGType type, TCGReg ret, } /* Use the constant pool, if possible. */ - if (!in_prologue && USE_REG_TB) { + if (!in_prologue) { new_pool_label(s, arg, R_SPARC_13, s->code_ptr, tcg_tbrel_diff(s, NULL)); tcg_out32(s, LDX | INSN_RD(ret) | INSN_RS1(TCG_REG_TB)); @@ -1014,10 +1013,8 @@ static void tcg_target_qemu_prologue(TCGContext *s) #endif /* We choose TCG_REG_TB such that no move is required. */ - if (USE_REG_TB) { - QEMU_BUILD_BUG_ON(TCG_REG_TB != TCG_REG_I1); - tcg_regset_set_reg(s->reserved_regs, TCG_REG_TB); - } + QEMU_BUILD_BUG_ON(TCG_REG_TB != TCG_REG_I1); + tcg_regset_set_reg(s->reserved_regs, TCG_REG_TB); tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I1, 0, JMPL); /* delay slot */ @@ -1422,7 +1419,7 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); tcg_out_movi_imm13(s, TCG_REG_O0, a0); return; - } else if (USE_REG_TB) { + } else { intptr_t tb_diff = tcg_tbrel_diff(s, (void *)a0); if (check_fit_ptr(tb_diff, 13)) { tcg_out_arithi(s, TCG_REG_G0, TCG_REG_I7, 8, RETURN); @@ -1438,36 +1435,30 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { + int c; + /* Direct jump. */ - if (USE_REG_TB) { - /* make sure the patch is 8-byte aligned. */ - if ((intptr_t)s->code_ptr & 4) { - tcg_out_nop(s); - } - set_jmp_insn_offset(s, which); - tcg_out_sethi(s, TCG_REG_T1, 0); - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); - tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); - } else { - set_jmp_insn_offset(s, which); - tcg_out32(s, CALL); + /* make sure the patch is 8-byte aligned. */ + if ((intptr_t)s->code_ptr & 4) { tcg_out_nop(s); } + set_jmp_insn_offset(s, which); + tcg_out_sethi(s, TCG_REG_T1, 0); + tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); + tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); + tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); set_jmp_reset_offset(s, which); /* * For the unlinked path of goto_tb, we need to reset TCG_REG_TB * to the beginning of this TB. */ - if (USE_REG_TB) { - int c = -tcg_current_code_size(s); - if (check_fit_i32(c, 13)) { - tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, c, ARITH_ADD); - } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, c); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); - } + c = -tcg_current_code_size(s); + if (check_fit_i32(c, 13)) { + tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, c, ARITH_ADD); + } else { + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, c); + tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); } } @@ -1487,11 +1478,7 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, switch (opc) { case INDEX_op_goto_ptr: tcg_out_arithi(s, TCG_REG_G0, a0, 0, JMPL); - if (USE_REG_TB) { - tcg_out_mov_delay(s, TCG_REG_TB, a0); - } else { - tcg_out_nop(s); - } + tcg_out_mov_delay(s, TCG_REG_TB, a0); break; case INDEX_op_br: tcg_out_bpcc(s, COND_A, BPCC_PT, arg_label(a0)); @@ -1897,7 +1884,7 @@ void tb_target_set_jmp_target(const TranslationBlock *tb, int n, tcg_debug_assert(tb_disp == (int32_t)tb_disp); tcg_debug_assert(br_disp == (int32_t)br_disp); - if (!USE_REG_TB) { + if (0) { qatomic_set((uint32_t *)jmp_rw, deposit32(CALL, 0, 30, br_disp >> 2)); flush_idcache_range(jmp_rx, jmp_rw, 4); From patchwork Tue Dec 6 04:17:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 975E4C3A5A7 for ; Tue, 6 Dec 2022 04:23:40 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POs-0003TC-7O; Mon, 05 Dec 2022 23:17:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POr-0003Sm-0A for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:45 -0500 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POp-0006vR-B1 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:44 -0500 Received: by mail-oi1-x235.google.com with SMTP id m204so15582340oib.6 for ; Mon, 05 Dec 2022 20:17:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LtPujVcPKsyMDklXnZcNahng/PTvSULlrispS/3dEi4=; b=Fv852m95R/RPyh4rktyqL4Brke3qboQBdEYluHpmjley2Ge+UQgQODGGbo7s4KSSs0 dPfirMTzlo/3Ur+GbNaqylamf0CGfld3Vn3ILWPXZRvX/Z7APAF7b7Vbuvr2yHvx4twu 93+V9P8F3StA+djHI+glVe8rsdMEckAzuRH/VwPfeckGpaz/N8aEnUTkaR2CIoWPESPz IiaaNFEehBx7qWHCsBpEsl1LgYuBxVyzwByRcUzMQDxY2Q/M6Zcdr94iTR4HCgxknaCe gKwfUuETsQ05ETnJNx2ysZSGhfunm2JE+qjUpeMop2qC/qrroLHypgTWajCgwiLlMPTp jrZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LtPujVcPKsyMDklXnZcNahng/PTvSULlrispS/3dEi4=; b=gW9EqJLBk9UktRUrQY35YJD+s6oGb/XNHJ6wLX8FWcMhNKS1u+WGQt2t3k9BzdVueF afbeZotYVfUwUUJkAj59bbD7mFC3LtHCe8uaMFuPdr9j8IL432CB2J25oB7+UJI//N/T 7ZHs68tTrAdh/Bs/DShTnRdq4eWxSDzJ/I9DvnfsQel1hCZ9Cu7+aaWPE41dh5M/QG1r yMZhgm56Sp/I7hkmptx38PGFrZrAebG1aIdu8tsoDUl9MHVZs9a8DnKdOHblWo91OBHQ x0xgLtnGUU1+64CTFSE0JQYyIefFJg81wvdYv0/jCnuTX7qTeEDxwFRWNwiQbSQTnHXJ vW0A== X-Gm-Message-State: ANoB5pkyfck+ml02dIPVCePeMVi+Vk7XQgjORx0cZTwTXmtCiv0fG4hA hHvG6LSS0+YLEjdMwGGUx/SVx4BFb5C8Ck1g/qc= X-Google-Smtp-Source: AA0mqf7PI8vFAuRcS1NwVBkfk42i6wjDhyrz7n6G5E7okzQRq+t17S8hHsGn6D9BKkpdCGBwTsDwCA== X-Received: by 2002:a05:6808:2012:b0:35b:bcb3:46f with SMTP id q18-20020a056808201200b0035bbcb3046fmr18108268oiw.86.1670300262555; Mon, 05 Dec 2022 20:17:42 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:41 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 19/22] tcg/sparc64: Reorg goto_tb implementation Date: Mon, 5 Dec 2022 22:17:12 -0600 Message-Id: <20221206041715.314209-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x235.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The old sparc64 implementation may replace two insns, which leaves a race condition in which a thread could be stopped at a PC in the middle of the sequence, and when restarted does not see the complete address computation and branches to nowhere. The new implemetation replaces only one insn, swapping between a direct branch and a direct call. The TCG_REG_TB register is loaded from tb->jmp_target_addr[] in the delay slot. Signed-off-by: Richard Henderson --- tcg/sparc64/tcg-target.c.inc | 93 ++++++++++++++---------------------- 1 file changed, 37 insertions(+), 56 deletions(-) diff --git a/tcg/sparc64/tcg-target.c.inc b/tcg/sparc64/tcg-target.c.inc index 1ae9615ef0..07ebea7a6e 100644 --- a/tcg/sparc64/tcg-target.c.inc +++ b/tcg/sparc64/tcg-target.c.inc @@ -1435,33 +1435,56 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { - int c; + ptrdiff_t off = tcg_tbrel_diff(s, (void *)get_jmp_target_addr(s, which)); - /* Direct jump. */ - /* make sure the patch is 8-byte aligned. */ - if ((intptr_t)s->code_ptr & 4) { - tcg_out_nop(s); - } + /* Direct branch will be patched by tb_target_set_jmp_target. */ set_jmp_insn_offset(s, which); - tcg_out_sethi(s, TCG_REG_T1, 0); - tcg_out_arithi(s, TCG_REG_T1, TCG_REG_T1, 0, ARITH_OR); - tcg_out_arith(s, TCG_REG_G0, TCG_REG_TB, TCG_REG_T1, JMPL); - tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); + tcg_out32(s, CALL); + /* delay slot */ + tcg_debug_assert(check_fit_ptr(off, 13)); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TB, TCG_REG_TB, off); set_jmp_reset_offset(s, which); /* * For the unlinked path of goto_tb, we need to reset TCG_REG_TB * to the beginning of this TB. */ - c = -tcg_current_code_size(s); - if (check_fit_i32(c, 13)) { - tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, c, ARITH_ADD); + off = -tcg_current_code_size(s); + if (check_fit_i32(off, 13)) { + tcg_out_arithi(s, TCG_REG_TB, TCG_REG_TB, off, ARITH_ADD); } else { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, c); + tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, off); tcg_out_arith(s, TCG_REG_TB, TCG_REG_TB, TCG_REG_T1, ARITH_ADD); } } +void tb_target_set_jmp_target(const TranslationBlock *tb, int n, + uintptr_t jmp_rx, uintptr_t jmp_rw) +{ + uintptr_t addr = tb->jmp_target_addr[n]; + intptr_t br_disp = (intptr_t)(addr - jmp_rx) >> 2; + tcg_insn_unit insn; + + br_disp >>= 2; + if (check_fit_ptr(br_disp, 19)) { + /* ba,pt %icc, addr */ + insn = deposit32(INSN_OP(0) | INSN_OP2(1) | INSN_COND(COND_A) + | BPCC_ICC | BPCC_PT, 0, 19, br_disp); + } else if (check_fit_ptr(br_disp, 22)) { + /* ba addr */ + insn = deposit32(INSN_OP(0) | INSN_OP2(2) | INSN_COND(COND_A), + 0, 22, br_disp); + } else { + /* The code_gen_buffer can't be larger than 2GB. */ + tcg_debug_assert(check_fit_ptr(br_disp, 30)); + /* call addr */ + insn = deposit32(CALL, 0, 30, br_disp); + } + + qatomic_set((uint32_t *)jmp_rw, insn); + flush_idcache_range(jmp_rx, jmp_rw, 4); +} + static void tcg_out_op(TCGContext *s, TCGOpcode opc, const TCGArg args[TCG_MAX_OP_ARGS], const int const_args[TCG_MAX_OP_ARGS]) @@ -1871,45 +1894,3 @@ void tcg_register_jit(const void *buf, size_t buf_size) tcg_register_jit_int(buf, buf_size, &debug_frame, sizeof(debug_frame)); } -void tb_target_set_jmp_target(const TranslationBlock *tb, int n, - uintptr_t jmp_rx, uintptr_t jmp_rw) -{ - uintptr_t addr = tb->jmp_target_addr[n]; - intptr_t tb_disp = addr - (uintptr_t)tb->tc.ptr; - intptr_t br_disp = addr - jmp_rx; - tcg_insn_unit i1, i2; - - /* We can reach the entire address space for ILP32. - For LP64, the code_gen_buffer can't be larger than 2GB. */ - tcg_debug_assert(tb_disp == (int32_t)tb_disp); - tcg_debug_assert(br_disp == (int32_t)br_disp); - - if (0) { - qatomic_set((uint32_t *)jmp_rw, - deposit32(CALL, 0, 30, br_disp >> 2)); - flush_idcache_range(jmp_rx, jmp_rw, 4); - return; - } - - /* This does not exercise the range of the branch, but we do - still need to be able to load the new value of TCG_REG_TB. - But this does still happen quite often. */ - if (check_fit_ptr(tb_disp, 13)) { - /* ba,pt %icc, addr */ - i1 = (INSN_OP(0) | INSN_OP2(1) | INSN_COND(COND_A) - | BPCC_ICC | BPCC_PT | INSN_OFF19(br_disp)); - i2 = (ARITH_ADD | INSN_RD(TCG_REG_TB) | INSN_RS1(TCG_REG_TB) - | INSN_IMM13(tb_disp)); - } else if (tb_disp >= 0) { - i1 = SETHI | INSN_RD(TCG_REG_T1) | ((tb_disp & 0xfffffc00) >> 10); - i2 = (ARITH_OR | INSN_RD(TCG_REG_T1) | INSN_RS1(TCG_REG_T1) - | INSN_IMM13(tb_disp & 0x3ff)); - } else { - i1 = SETHI | INSN_RD(TCG_REG_T1) | ((~tb_disp & 0xfffffc00) >> 10); - i2 = (ARITH_XOR | INSN_RD(TCG_REG_T1) | INSN_RS1(TCG_REG_T1) - | INSN_IMM13((tb_disp & 0x3ff) | -0x400)); - } - - qatomic_set((uint64_t *)jmp_rw, deposit64(i2, 32, 32, i1)); - flush_idcache_range(jmp_rx, jmp_rw, 8); -} From patchwork Tue Dec 6 04:17:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97519C3A5A7 for ; Tue, 6 Dec 2022 04:23:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POv-0003Uf-He; Mon, 05 Dec 2022 23:17:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POt-0003Tk-G7 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:47 -0500 Received: from mail-oi1-x234.google.com ([2607:f8b0:4864:20::234]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POq-00073o-Vx for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:47 -0500 Received: by mail-oi1-x234.google.com with SMTP id m204so15582379oib.6 for ; Mon, 05 Dec 2022 20:17:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tPgT65Py6HacN6BGNUnYknCGAA+AMkPn1v8W/SDINWU=; b=HyNK+whjGKnZE/zg/V0/vjv8sp+hX7dRguNM1rzL9WzqjdFrpnzEv2ZYBfEYJu++6W AWxyu2bEAqjgxXUyvL/ImcccaBMWXpxFm9ZOSdW0ziGFvotD0fxJ+kk/5A8FI0RDoka3 aHb81EP0afqt9VqZDliqIgw1jzQl0ZRFOMYL681FH+JCu6rje1IHk7BtPLljwmGyRBoT 7024Gsaxevh9ycwtNNIczBKNvHIyvRY1zTKnCmPyUux8+7tR7YoOgodTYk7UmFVXZC59 1x3IDpqPzRbGzjAlYhieIFMlUUkto89Ifgcdp7jxSV6SoVmN+4AiZHUfGogqyjI3uTBq WvBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tPgT65Py6HacN6BGNUnYknCGAA+AMkPn1v8W/SDINWU=; b=nmeiox+Ox03z8jEOQ2uqlq+5hDTaD/zB69e02j3FFfrbr/RK7ut3Y5EOjX6xlBtwGr scfs4JvaXGIF+uapzeQgKMYR0A3jOfyTvsnpENpvWC86ku+2mrVzUOYVlBdQHf7buX5m B+xj9LTFnoxABUFCAwOeY49V0Vof9/H7DoBMGdOUujZCSP9Hev6qJHCx/1+ohGnIKkV7 GAxIZdyz0MIvBUaUNL0iq4Qdx14hfvOKN65JaO3fRa39lQY4SyP+BIrW8muQS2XzdXdu RIHqPUzwK2QmYy5TIbKX8VPQvftPfj2MMCXxRpjQtcvlMTux71ehwt78kL/04CiOqlMO Yx5w== X-Gm-Message-State: ANoB5pn/nDPZQhA7xxGEXg+OrNZ/ysroxZNWWfdkrPj7J4Qekfckp1qN xde+aCsPoR9QH3dC/avbPcjqXiup702C83DlO7A= X-Google-Smtp-Source: AA0mqf64hYrQ8jyNqEEXpzWYq5FBCUUBdtwxascsxQIEKaK1E+G6AEMOe8Ex36H6q/tSvzg7WhziEQ== X-Received: by 2002:a05:6808:1649:b0:34b:7532:2d0e with SMTP id az9-20020a056808164900b0034b75322d0emr41244968oib.109.1670300263725; Mon, 05 Dec 2022 20:17:43 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:43 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 20/22] tcg/arm: Implement direct branch for goto_tb Date: Mon, 5 Dec 2022 22:17:13 -0600 Message-Id: <20221206041715.314209-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::234; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x234.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that tcg can handle direct and indirect goto_tb simultaneously, we can optimistically leave space for a direct branch and fall back to loading the pointer from the TB for an indirect branch. Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.c.inc | 52 ++++++++++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 15 deletions(-) diff --git a/tcg/arm/tcg-target.c.inc b/tcg/arm/tcg-target.c.inc index a44d1d969f..fad2be700e 100644 --- a/tcg/arm/tcg-target.c.inc +++ b/tcg/arm/tcg-target.c.inc @@ -135,6 +135,8 @@ typedef enum { ARITH_BIC = 0xe << 21, ARITH_MVN = 0xf << 21, + INSN_B = 0x0a000000, + INSN_CLZ = 0x016f0f10, INSN_RBIT = 0x06ff0f30, @@ -546,7 +548,7 @@ static bool tcg_target_const_match(int64_t val, TCGType type, int ct) static void tcg_out_b_imm(TCGContext *s, ARMCond cond, int32_t offset) { - tcg_out32(s, (cond << 28) | 0x0a000000 | + tcg_out32(s, (cond << 28) | INSN_B | (((offset - 8) >> 2) & 0x00ffffff)); } @@ -1935,32 +1937,52 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t arg) static void tcg_out_goto_tb(TCGContext *s, int which) { - /* Indirect jump method */ - intptr_t ptr, dif, dil; - TCGReg base = TCG_REG_PC; + uintptr_t i_addr; + intptr_t i_disp; - ptr = get_jmp_target_addr(s, which); - dif = tcg_pcrel_diff(s, (void *)ptr) - 8; - dil = sextract32(dif, 0, 12); - if (dif != dil) { + /* Direct branch will be patched by tb_target_set_jmp_target. */ + set_jmp_insn_offset(s, which); + tcg_out32(s, INSN_NOP); + + /* When branch is out of range, fall through to indirect. */ + i_addr = get_jmp_target_addr(s, which); + i_disp = tcg_pcrel_diff(s, (void *)i_addr) - 8; + tcg_debug_assert(i_disp < 0); + if (i_disp >= -0xfff) { + tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, TCG_REG_PC, i_disp); + } else { /* * The TB is close, but outside the 12 bits addressable by * the load. We can extend this to 20 bits with a sub of a - * shifted immediate from pc. In the vastly unlikely event - * the code requires more than 1MB, we'll use 2 insns and - * be no worse off. + * shifted immediate from pc. */ - base = TCG_REG_R0; - tcg_out_movi32(s, COND_AL, base, ptr - dil); + int h = -i_disp; + int l = h & 0xfff; + + h = encode_imm_nofail(h - l); + tcg_out_dat_imm(s, COND_AL, ARITH_SUB, TCG_REG_R0, TCG_REG_PC, h); + tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, TCG_REG_R0, l); } - tcg_out_ld32_12(s, COND_AL, TCG_REG_PC, base, dil); set_jmp_reset_offset(s, which); } void tb_target_set_jmp_target(const TranslationBlock *tb, int n, uintptr_t jmp_rx, uintptr_t jmp_rw) { - /* Always indirect, nothing to do */ + uintptr_t addr = tb->jmp_target_addr[n]; + ptrdiff_t offset = addr - (jmp_rx + 8); + tcg_insn_unit insn; + + /* Either directly branch, or fall through to indirect branch. */ + if (offset == sextract64(offset, 0, 26)) { + /* B */ + insn = (COND_AL << 28) | INSN_B | ((offset >> 2) & 0x00ffffff); + } else { + insn = INSN_NOP; + } + + qatomic_set((uint32_t *)jmp_rw, insn); + flush_idcache_range(jmp_rx, jmp_rw, 4); } static void tcg_out_op(TCGContext *s, TCGOpcode opc, From patchwork Tue Dec 6 04:17:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65AEAC352A1 for ; Tue, 6 Dec 2022 04:23:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POv-0003Uj-Qm; Mon, 05 Dec 2022 23:17:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POt-0003Ti-CX for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:47 -0500 Received: from mail-oa1-x34.google.com ([2001:4860:4864:20::34]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POr-00074B-Um for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:47 -0500 Received: by mail-oa1-x34.google.com with SMTP id 586e51a60fabf-12c8312131fso16095148fac.4 for ; Mon, 05 Dec 2022 20:17:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6Ctn4q8UCWShe0HV3h1VrXrerV51yET9dh87n+Fb5Oc=; b=O80g0rrP97tMmLTBVm6DClNTRDd/1So5g5u/0064U9iA/vNfqOZJ6wHsriZyeANpy3 sReFB3KorNFCocmspxYuuUtlw1vnY0M9IAn/+mHiwgKTk5WLge8dDzPxKDmEM3XGI2OL 03khekSdlcuau/JDA18Ewf8XElZ8zWmx3MiMgAHcm7wKwqEfmmPDyVUH8nJxYD85Jfa/ 3Lcs54zI8PFUmv/7MMFGBqpehdAOSnPKtIV4knfdNIrH5WCvUMKo/H/BQ37xvqpr2pYz 2qw/DUqB+rtoiRTc62bYLc+qDX59ZHu30oO8htbfmjHoUR6UsvXl1l8+wyM6GECs9OIU k6FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6Ctn4q8UCWShe0HV3h1VrXrerV51yET9dh87n+Fb5Oc=; b=FSSG3gpQWrtDPVt8RgjTTT12apvpIoerSD7YiDXvALHFc3N9XOEyGfCAaPalBQg03S mGfl1n0YFefZv/ffjlcRQYWjR0b2iC1N2PNjvCqeLoPQfAY8xnC/zEiK6H8Qz0qLt0sO FCNVJ/Eyi/I1XHJqeZ8MzvrBsoNi9I984cWZHEDjQyhkycmKh5y3mi2uvz/HLz/QD//S OQVphCyEZFrA6ifBz7uuMmc/JQyQlrIVG8jubTD0AHetYLFppPFOug6HBaTYirL0VW4d Uydyv5M3yG2b90wOInu0szzmmYtD67Wrs0uIePyreouUSZRO4D+vrmmZiNxdaz1hHWBc uoeg== X-Gm-Message-State: ANoB5plS2fA/9KX4veqz4A4l21445pvUOFvhLJjC997lNiAr3XlXf11Y 4pDEpkOibeEC5fR6KYTkkfUDfP7oop6bIcedfZ4= X-Google-Smtp-Source: AA0mqf47mgHaJZNxd0ghROR5QwK+PcebL6c0hbwWMLLvJsBnagBfW8zmR6RjoUJxckktMYy2msAGew== X-Received: by 2002:a05:6870:24a5:b0:132:3de9:bdf with SMTP id s37-20020a05687024a500b001323de90bdfmr36220896oaq.188.1670300264958; Mon, 05 Dec 2022 20:17:44 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:44 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 21/22] tcg/riscv: Introduce OPC_NOP Date: Mon, 5 Dec 2022 22:17:14 -0600 Message-Id: <20221206041715.314209-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::34; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x34.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- tcg/riscv/tcg-target.c.inc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index 37baae9cda..de029d62b4 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -254,6 +254,7 @@ typedef enum { #endif OPC_FENCE = 0x0000000f, + OPC_NOP = OPC_ADDI, /* nop = addi r0,r0,0 */ } RISCVInsn; /* @@ -390,7 +391,7 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) { int i; for (i = 0; i < count; ++i) { - p[i] = encode_i(OPC_ADDI, TCG_REG_ZERO, TCG_REG_ZERO, 0); + p[i] = OPC_NOP; } } From patchwork Tue Dec 6 04:17:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13065375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86B56C3A5A7 for ; Tue, 6 Dec 2022 04:20:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p2POx-0003Um-3k; Mon, 05 Dec 2022 23:17:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p2POu-0003U3-J8 for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:48 -0500 Received: from mail-oi1-x22b.google.com ([2607:f8b0:4864:20::22b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p2POs-0006yh-RJ for qemu-devel@nongnu.org; Mon, 05 Dec 2022 23:17:48 -0500 Received: by mail-oi1-x22b.google.com with SMTP id v70so7210770oie.3 for ; Mon, 05 Dec 2022 20:17:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nViIHSFjfOIWq3NohhhvUXaF8+WP2rU2B4NkQ7rnjQQ=; b=S63mfpzvHMuIvpm82NSHqmrTKuaNaCb0E3A667FYNoZFKcrIePQ1jAG6Z2xJ4zIMSZ 57KPxVhQjcD0l5Ox/O4y3RWGk7w/uDvO/QGEXcYKGj9sF97yThdQHVyYj/92oiT3jNTB VRdKu5f/tmo1qjYcS4Qgh/WBjmZwW2gNF2FV/khUg3fV1bn+JLJJgQ/3IqJgzA1h7vG8 M9IrTAdQt+m3FDL7i8nzGfRD6wbITE7S5beWXCSrjcmmCj6QySYubuTz2PKxETwaUaUC 9Sxbrc43R0BfTK+XGFGqCN3HLUqN2/dm3GoIc9Iwjs/7XdnXTFBsu54/ULy+MgpwVrTs HhyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nViIHSFjfOIWq3NohhhvUXaF8+WP2rU2B4NkQ7rnjQQ=; b=xhMJxzooTR1MJwc0l5UN3xZDH+nvS+j1II99i1MfvasdyaiwKoyDKA+//AUZX89Bci bi0AZkqndvIg3whI7/iF9vsktma6DcahVCSRu5BqAhOM7qmeGikRt18V82iO9G0b+BoH mNZrbrgwv8BGyaPLcoWT+WwEQXcp0XibMGFubgu0PrA4TFdqA8yJKaHkuCn2VVSGa82z uPZRqk9j4gXKEiP3tcsxxs1E8EYt+VzGT+tsd5wWuQ36Az7A+ERhWnqBSlV3DN8MUdgG eHZC/mKjkoL38X3NYCwj1zomDZayH4O7KDetcH0wu3onwBY7+BGRlTcIA+aBv8+YXEh9 kphg== X-Gm-Message-State: ANoB5plpWcvSZRXN54zQzVi2icFMXtQJ/9DbyqNSdO97C0yRqao5FLKK +Ds9pCycO+5e0VqIRcQ355pnBQFPd0Av5XXHqKQ= X-Google-Smtp-Source: AA0mqf4Yjuvg7ltJAj/TaX7zUSQccNDhDthK6MtOl3RSOJIatakThl4F0d6xlfd6akJL1FIHIRJqXg== X-Received: by 2002:aca:1308:0:b0:354:94c3:2b33 with SMTP id e8-20020aca1308000000b0035494c32b33mr45247396oii.247.1670300266100; Mon, 05 Dec 2022 20:17:46 -0800 (PST) Received: from stoup.. ([2806:102e:18:70b5:d104:21c2:8c7e:77e6]) by smtp.gmail.com with ESMTPSA id o35-20020a056870912300b0014185b2b3d5sm10162018oae.18.2022.12.05.20.17.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Dec 2022 20:17:45 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: r@hev.cc Subject: [PATCH 22/22] tcg/riscv: Implement direct branch for goto_tb Date: Mon, 5 Dec 2022 22:17:15 -0600 Message-Id: <20221206041715.314209-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221206041715.314209-1-richard.henderson@linaro.org> References: <20221206041715.314209-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22b; envelope-from=richard.henderson@linaro.org; helo=mail-oi1-x22b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that tcg can handle direct and indirect goto_tb simultaneously, we can optimistically leave space for a direct branch and fall back to loading the pointer from the TB for an indirect branch. Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.h | 5 +++++ tcg/riscv/tcg-target.c.inc | 19 +++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 56f7bc3346..a75c84f2a6 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -159,6 +159,11 @@ typedef enum { #define TCG_TARGET_HAS_mulsh_i64 1 #endif +<<<<<<< HEAD +======= +void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t); + +>>>>>>> 89ab294271 (tcg/riscv: Implement TCG_TARGET_HAS_direct_jump) #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_NEED_LDST_LABELS diff --git a/tcg/riscv/tcg-target.c.inc b/tcg/riscv/tcg-target.c.inc index de029d62b4..ee269ea157 100644 --- a/tcg/riscv/tcg-target.c.inc +++ b/tcg/riscv/tcg-target.c.inc @@ -1278,7 +1278,11 @@ static void tcg_out_exit_tb(TCGContext *s, uintptr_t a0) static void tcg_out_goto_tb(TCGContext *s, int which) { - /* indirect jump method */ + /* Direct branch will be patched by tb_target_set_jmp_target. */ + set_jmp_insn_offset(s, which); + tcg_out32(s, OPC_NOP); + + /* When branch is out of range, fall through to indirect. */ tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO, get_jmp_target_addr(s, which)); tcg_out_opc_imm(s, OPC_JALR, TCG_REG_ZERO, TCG_REG_TMP0, 0); @@ -1288,7 +1292,18 @@ static void tcg_out_goto_tb(TCGContext *s, int which) void tb_target_set_jmp_target(const TranslationBlock *tb, int n, uintptr_t jmp_rx, uintptr_t jmp_rw) { - /* Always indirect, nothing to do */ + uintptr_t addr = tb->jmp_target_addr[n]; + ptrdiff_t offset = addr - jmp_rx; + tcg_insn_unit insn; + + /* Either directly branch, or fall through to indirect branch. */ + if (offset == sextreg(offset, 0, 20)) { + insn = encode_uj(OPC_JAL, TCG_REG_ZERO, offset); + } else { + insn = OPC_NOP; + } + qatomic_set((uint32_t *)jmp_rw, insn); + flush_idcache_range(jmp_rx, jmp_rw, 4); } static void tcg_out_op(TCGContext *s, TCGOpcode opc,