From patchwork Thu Nov 14 16:00:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 052EBD68B33 for ; Thu, 14 Nov 2024 16:10:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHt-0004uV-35; Thu, 14 Nov 2024 11:01:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHp-0004tr-FZ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:37 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHn-0002Av-5x for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:36 -0500 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-7206304f93aso657607b3a.0 for ; Thu, 14 Nov 2024 08:01:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600093; x=1732204893; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=gGugcia7xIHYiO2oiJjHlSRtV4JGvYxNcTYF4XQ4e4A=; b=CArrUWp3kCgUI0dpBxSBIZw8za/+WIEJFor9gLCw5ho5Glxra1/xJu2CUqSR/0WOR4 YTjsvwEPeiijbAtq/Ew9kWQh4MuDeWVuBvvoQtMNXv/SKAVskHDn2yM3AnI4PmuEkN1H n9t70VFQBhlNXajOFjH6JDtN5/E3mHE875EYOP4AVpyE2oQP27dfmdSOP9wL9gFhyKqT NIYJQ8ivK/GVkdKaVtL38JzzOtcAaJ8Ib/CiDGOEa1sJQOf1Aq554dMvoQ19GkyqQC8F BadIEj6tyDu1r2yJvEEpjDpy7SVWwFysVgjPA8H3YjCMKaZ1Bda3Hj82DozCOEnFr7et Qhpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600093; x=1732204893; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gGugcia7xIHYiO2oiJjHlSRtV4JGvYxNcTYF4XQ4e4A=; b=B4hrMj03hHh9CeSFfKOC8NMvD90HTCZfoE4YaAHh4ciurt7ls6dEBs+9Az/umAWBi2 jN3QnqTZ7Q5b7vKZDV7vzXgwvqDNE3afROTS2p+eFf5YWXLd+58mhmbMOdPmkbIfKoFS Ab43I24/ZZSlUu2I/nqlNV55T5hNehaPq7aBYwwIPSwQQ0lYv/xjwpW0sJAjH01h26g8 7/S1p0UkJh/1pKMemQ5yLJflLfLAtjuJR5jyQCca2j+f9MBWDzW4dxTVp3eTIA8KGEXg XzedLAb2iab1U5zEl6HecVAEdbIBFJOISZMGSVKGB1EjxvDYKmZx+ndrEepqthjv6UkU oj3g== X-Gm-Message-State: AOJu0YygJvhLE8u5ZYeikgVcsTXFOxyvr/Vn5G5PmgYNaSZ+cN8uKUOj WK8G9YxryV5o/cWpTyY04MEtJvzeTEYILC7zu4go3fracoLgdf9jfIQqXLLhKHJ8gqwzYc87DX9 f X-Google-Smtp-Source: AGHT+IEU0J+mWXRFECjKORFcPUCCkpmyiemTjYiQwKr6HpM7dRdd+H8jcIT/5H2by5bJyEsnm5rMog== X-Received: by 2002:a17:90b:1d81:b0:2e2:b41b:854e with SMTP id 98e67ed59e1d1-2ea0631e509mr3129501a91.20.1731600093063; Thu, 14 Nov 2024 08:01:33 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:32 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 01/54] util/interval-tree: Introduce interval_tree_free_nodes Date: Thu, 14 Nov 2024 08:00:37 -0800 Message-ID: <20241114160131.48616-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Provide a general-purpose release-all-nodes operation, that allows for the IntervalTreeNode to be embeded within a larger structure. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/qemu/interval-tree.h | 11 +++++++++++ util/interval-tree.c | 20 ++++++++++++++++++++ util/selfmap.c | 13 +------------ 3 files changed, 32 insertions(+), 12 deletions(-) diff --git a/include/qemu/interval-tree.h b/include/qemu/interval-tree.h index 25006debe8..d90ea6d17f 100644 --- a/include/qemu/interval-tree.h +++ b/include/qemu/interval-tree.h @@ -96,4 +96,15 @@ IntervalTreeNode *interval_tree_iter_first(IntervalTreeRoot *root, IntervalTreeNode *interval_tree_iter_next(IntervalTreeNode *node, uint64_t start, uint64_t last); +/** + * interval_tree_free_nodes: + * @root: root of the tree + * @it_offset: offset from outermost type to IntervalTreeNode + * + * Free, via g_free, all nodes under @root. IntervalTreeNode may + * not be the true type of the nodes allocated; @it_offset gives + * the offset from the outermost type to the IntervalTreeNode member. + */ +void interval_tree_free_nodes(IntervalTreeRoot *root, size_t it_offset); + #endif /* QEMU_INTERVAL_TREE_H */ diff --git a/util/interval-tree.c b/util/interval-tree.c index 53465182e6..663d3ec222 100644 --- a/util/interval-tree.c +++ b/util/interval-tree.c @@ -639,6 +639,16 @@ static void rb_erase_augmented_cached(RBNode *node, RBRootLeftCached *root, rb_erase_augmented(node, &root->rb_root, augment); } +static void rb_node_free(RBNode *rb, size_t rb_offset) +{ + if (rb->rb_left) { + rb_node_free(rb->rb_left, rb_offset); + } + if (rb->rb_right) { + rb_node_free(rb->rb_right, rb_offset); + } + g_free((void *)rb - rb_offset); +} /* * Interval trees. @@ -870,6 +880,16 @@ IntervalTreeNode *interval_tree_iter_next(IntervalTreeNode *node, } } +void interval_tree_free_nodes(IntervalTreeRoot *root, size_t it_offset) +{ + if (root && root->rb_root.rb_node) { + rb_node_free(root->rb_root.rb_node, + it_offset + offsetof(IntervalTreeNode, rb)); + root->rb_root.rb_node = NULL; + root->rb_leftmost = NULL; + } +} + /* Occasionally useful for calling from within the debugger. */ #if 0 static void debug_interval_tree_int(IntervalTreeNode *node, diff --git a/util/selfmap.c b/util/selfmap.c index 483cb617e2..d2b86da301 100644 --- a/util/selfmap.c +++ b/util/selfmap.c @@ -87,23 +87,12 @@ IntervalTreeRoot *read_self_maps(void) * @root: an interval tree * * Free a tree of MapInfo structures. - * Since we allocated each MapInfo in one chunk, we need not consider the - * contents and can simply free each RBNode. */ -static void free_rbnode(RBNode *n) -{ - if (n) { - free_rbnode(n->rb_left); - free_rbnode(n->rb_right); - g_free(n); - } -} - void free_self_maps(IntervalTreeRoot *root) { if (root) { - free_rbnode(root->rb_root.rb_node); + interval_tree_free_nodes(root, offsetof(MapInfo, itree)); g_free(root); } } From patchwork Thu Nov 14 16:00:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66AA6D68B34 for ; Thu, 14 Nov 2024 16:08:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHw-0004wN-3X; Thu, 14 Nov 2024 11:01:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHq-0004u4-CW for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:38 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHn-0002Ay-E0 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:38 -0500 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2ea09a033e2so583112a91.3 for ; Thu, 14 Nov 2024 08:01:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600094; x=1732204894; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=EYQWI5p9vUhtvQFXm2oStjAmXC4f4vQZGym/5HmIo8g=; b=s7RNhd0UlvgnjnMgrXc+Tmn4We2rUzhKiYfjM6vrA1hYqOfdOHrnCWQncLYH4/eCk3 5gJvA5Sn/4ZdYU8AQPWcwW5aBJPEJe/Ze3BAJG0uoaPB8v0Y0GCDU+s35f/HT3nztr8C Jz7Nm7oXzT7VO1+j4gYZZNiM3YXJOREV9QuxLeuN1Oi2PLyEOXDA4+JMytlInchE+STE q5Svco8xgzMmAEGOF7qG01Pw0R2iV+GdAd7RrEDljilb0Xw1nI7iDkeh53v5pgqm3Spy 544msGLOlwHHWm4UDsUTHGUkmHB3z+laXdcnaKluVCGYXLjkWIJLvX010soDwMMtGtZv 5lAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600094; x=1732204894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EYQWI5p9vUhtvQFXm2oStjAmXC4f4vQZGym/5HmIo8g=; b=im3Ro+VjnNIerUYkGPQp9t47Lhg1R31UarTj7xX7K9t5ZB9R6eudyYB622ElNBbc3w lXxG/3ZBiuiLZHcKNwY+LtxCKXzCLPAeIu2OgG93zSk3ZcZkxsqJyLJrqPgFa0b9seZU yUjr/Mk2D7de4rVLnraQtZ5AVGxmxUnU5yyGLQO4bGy2Xxe78q+Q52b6VxO3RN6x3ARA dy65m/TRbjFCKGiFNVALrPZV2dg+sAf51bzVZf+u+6ZspqsE5TZ6+5sreZbwbM5gZgVZ QdgykHGohSFAAeWi95Og80O1R9fm+E7y2EP3hgxTvIA8evHz8axl2X6VgSi3AJkA1ZKz dddg== X-Gm-Message-State: AOJu0YwZS9GaP5OQ7D821ItrZrUDtu7j/5u2kulj4WFPGbXfB6NsZeiP EipUBY1a61BnL/CmRgKgbvwPllJCBpsMJ74Y0bdVDkCKSfKwbehriNoszVnNJdUNPGM7OhFTVRd C X-Google-Smtp-Source: AGHT+IEqOnDNFY6IiZLjJTkd5LifxiO+3BFrt5gH9BNB8MwjceC5yDdGd+ahC08EIhbp8OS8ApEN3w== X-Received: by 2002:a17:90b:4e8d:b0:2e1:a81f:5520 with SMTP id 98e67ed59e1d1-2e9f2c4ef71mr8430900a91.1.1731600093918; Thu, 14 Nov 2024 08:01:33 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:33 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 02/54] accel/tcg: Split out tlbfast_flush_locked Date: Thu, 14 Nov 2024 08:00:38 -0800 Message-ID: <20241114160131.48616-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We will have a need to flush only the "fast" portion of the tlb, allowing re-fill from the "full" portion. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b76a4eac4e..c1838412e8 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -284,13 +284,18 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast, } } -static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) +static void tlbfast_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) { desc->n_used_entries = 0; + memset(fast->table, -1, sizeof_tlb(fast)); +} + +static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) +{ + tlbfast_flush_locked(desc, fast); desc->large_page_addr = -1; desc->large_page_mask = -1; desc->vindex = 0; - memset(fast->table, -1, sizeof_tlb(fast)); memset(desc->vtable, -1, sizeof(desc->vtable)); } From patchwork Thu Nov 14 16:00:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF3BAD68B33 for ; Thu, 14 Nov 2024 16:10:32 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHu-0004vd-6q; Thu, 14 Nov 2024 11:01:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHr-0004uW-9z for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:40 -0500 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHp-0002B9-87 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:38 -0500 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-720d01caa66so742974b3a.2 for ; Thu, 14 Nov 2024 08:01:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600095; x=1732204895; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=D/kCglcC+IjHEzmMz83kkwbAHqcDArOp+yBPf3gcvXc=; b=dTBMZ5QZd6Iiwgn+qIvOVLidhYCA9dTLgLtmU7sLAukEBopYcOzJOMaVnub/z38W3a RYJNLQH4Wj+/QsQrRHKmhdde93EB1PrH96siklbYfvbnTQ93lxcOz1j/cQwjVhL2J9mo 8oJlwOXnpSOWlOirnpjc8sua4BHY3HjKR6Ftf5Yy4n/9/v9A6BjueIVZzZC/YCYXsopG TdxapY14tYya8MqNPi/ih0E3tE79nLoTj7jSm1OeThrV926EEcXF/SsbYvYtH9RM9Ltx ka+P6mWR+TyV5i+DcmzY8uCahx9Nh3GluOcMcwuFDam07ZgnTmI+oRVTLPOu7Uj/py1F ZqgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600095; x=1732204895; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=D/kCglcC+IjHEzmMz83kkwbAHqcDArOp+yBPf3gcvXc=; b=OkV4EneVxpTju2eeuVYLvs7CHp2reGP+XUtEm8sIvCrpnHtg5855hL7BsQNtILB632 rZcHTzawYlqMKNov0haYGHsAXaM/u8mRXjIv4MOKbafv8g5c5KrPKmO2GdA/ayhQDEoF ohp+XS5P/HkQAP3YzOAh8NmZq22mGuyDnfb0aegWVU7ZXUYCdHbaWzr4Qy8oT++Yf85q 4OneCdQbMRTaUbt48tBYa2c3c+uiSWmy9OTN0/AEoUsz/dxgif5w8KRdQHjl2fT8c9kz VUpM6R28y8jiRaO0UnOoxQt/Q5RednzyoLNXxnYlpUzO3zXd380lxnnoQCUkhXotf/Ix +TIQ== X-Gm-Message-State: AOJu0YxJhu95LGAlepIt+A1iKWl0pP4lr5rTevVgQbl/cTmMAnr2548m r7IbVF28HW5XRqIrq9xWEGBxnI2RwLvQobCsw1REiw460nI1ihOZ8OWKsB7zrcaiW9FgaTGBLjX a X-Google-Smtp-Source: AGHT+IHHVQCIW8ZP7pg9m4QMeIwLQbHEjchContFqQR/igzca7bl3SocsqX180HON64phWEJ4OHlng== X-Received: by 2002:a17:90b:4f47:b0:2ea:f91:f32 with SMTP id 98e67ed59e1d1-2ea0f91108cmr1355267a91.9.1731600094719; Thu, 14 Nov 2024 08:01:34 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.34 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:34 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 03/54] accel/tcg: Split out tlbfast_{index,entry} Date: Thu, 14 Nov 2024 08:00:39 -0800 Message-ID: <20241114160131.48616-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Often we already have the CPUTLBDescFast structure pointer. Allows future code simplification. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index c1838412e8..e37af24525 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -131,20 +131,28 @@ static inline uint64_t tlb_addr_write(const CPUTLBEntry *entry) return tlb_read_idx(entry, MMU_DATA_STORE); } +static inline uintptr_t tlbfast_index(CPUTLBDescFast *fast, vaddr addr) +{ + return (addr >> TARGET_PAGE_BITS) & (fast->mask >> CPU_TLB_ENTRY_BITS); +} + +static inline CPUTLBEntry *tlbfast_entry(CPUTLBDescFast *fast, vaddr addr) +{ + return fast->table + tlbfast_index(fast, addr); +} + /* Find the TLB index corresponding to the mmu_idx + address pair. */ static inline uintptr_t tlb_index(CPUState *cpu, uintptr_t mmu_idx, vaddr addr) { - uintptr_t size_mask = cpu->neg.tlb.f[mmu_idx].mask >> CPU_TLB_ENTRY_BITS; - - return (addr >> TARGET_PAGE_BITS) & size_mask; + return tlbfast_index(&cpu->neg.tlb.f[mmu_idx], addr); } /* Find the TLB entry corresponding to the mmu_idx + address pair. */ static inline CPUTLBEntry *tlb_entry(CPUState *cpu, uintptr_t mmu_idx, vaddr addr) { - return &cpu->neg.tlb.f[mmu_idx].table[tlb_index(cpu, mmu_idx, addr)]; + return tlbfast_entry(&cpu->neg.tlb.f[mmu_idx], addr); } static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns, From patchwork Thu Nov 14 16:00:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D785D68B33 for ; Thu, 14 Nov 2024 16:02:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHx-0004xA-CD; Thu, 14 Nov 2024 11:01:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHr-0004ub-Qk for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHp-0002BG-8s for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:39 -0500 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-7e6cbf6cd1dso549443a12.3 for ; Thu, 14 Nov 2024 08:01:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600095; x=1732204895; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MqnKReoBjQRrcKu52BeUjvV/aI8mQFMQp+BRc29T3cQ=; b=KghsZxvg5cjQ/bwK7bamKa4G8kvhTtArlXw4bi7UU7OWbWMWDNsfEIPwYO5JheIkJL JPYTtpo/LSkaIFgp0Y3UTn5U+fFFqquyJBcKOh9gFBJ196lNXGoX5PbAzi+LSMD5byy3 ZanistM46CNIkIJz6psfJ4pmZaDHOXw1FssDRY/PaEFOi5tUiZ5YE8fiuVIbPYlmaURr 1twVE+rYyI9OltaqPfthMexlBNdPDdJNQkBJEk2PjXPY7tbnx2eSKOa6s3tgFjrTutQw GrONHuyDFGrwho20qHqfRYv0epqYs7tYpAh1wdY7QCl5eeYY13Vx8RuUvJ5LOhD0Bft0 NEvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600095; x=1732204895; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MqnKReoBjQRrcKu52BeUjvV/aI8mQFMQp+BRc29T3cQ=; b=DqxwvDbzh2vLagOpTApuWr3LwxnSHIT14m7c1lmO67VgBEhzz+kgr6WLm6TS8toZ+y BBTnauonTb3IloaBsWBaxX58INFk1juvENfIfdPurfOP+whMt1Lgzxg0+fge067dV+7z CkdQHSQz/bxa9gTPF7DE5RprQqKQimMI3aXNKJjj+2MAnFVNzQ6rzn1DpXpomqxiAaOB tDuSONZf8qVEJxkSYDxdDjZnWGnaadS8C6Fj2d065tOfABPUwmGWfqd7DcX1I+Ch5l9s 6mRsNfAF11MjXMYbnVxXQFEt/auS+1Wez33eMlii6ibS6hanwoj20goPdCqopCIkJFrs 6l0A== X-Gm-Message-State: AOJu0Yz90LmY4+cVqUeVDLhJpq9gHk3jfRGTSCkZW32rlCctsgukUO9R DLnIBJJNOGHZ4WLLysUCkOSNI8EgW5xsVwOZK4/79WKzv3b9TWDkTAiesTRo3K9yxZd5d5x/FdR E X-Google-Smtp-Source: AGHT+IFoQto2KZJ9KfoG6yGpxSDOUHwph0+XFkztELwNnaAm8ESL0lU+tqb8FitL8JoOLXVb4ZFkAw== X-Received: by 2002:a17:90b:4cc1:b0:2d8:e7db:9996 with SMTP id 98e67ed59e1d1-2e9e4add8c1mr13447495a91.13.1731600095486; Thu, 14 Nov 2024 08:01:35 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:35 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Pierrick Bouvier Subject: [PATCH v2 04/54] accel/tcg: Split out tlbfast_flush_range_locked Date: Thu, 14 Nov 2024 08:00:40 -0800 Message-ID: <20241114160131.48616-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::529; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x529.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org While this may at present be overly complicated for use by single page flushes, do so with the expectation that this will eventually allow simplification of large pages. Reviewed-by: Pierrick Bouvier Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 66 +++++++++++++++++++++++----------------------- 1 file changed, 33 insertions(+), 33 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index e37af24525..46fa0ae802 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -493,11 +493,6 @@ static bool tlb_flush_entry_mask_locked(CPUTLBEntry *tlb_entry, return false; } -static inline bool tlb_flush_entry_locked(CPUTLBEntry *tlb_entry, vaddr page) -{ - return tlb_flush_entry_mask_locked(tlb_entry, page, -1); -} - /* Called with tlb_c.lock held */ static void tlb_flush_vtlb_page_mask_locked(CPUState *cpu, int mmu_idx, vaddr page, @@ -520,10 +515,37 @@ static inline void tlb_flush_vtlb_page_locked(CPUState *cpu, int mmu_idx, tlb_flush_vtlb_page_mask_locked(cpu, mmu_idx, page, -1); } +static void tlbfast_flush_range_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast, + vaddr addr, vaddr len, vaddr mask) +{ + /* + * If @mask is smaller than the tlb size, there may be multiple entries + * within the TLB; for now, just flush the entire TLB. + * Otherwise all addresses that match under @mask hit the same TLB entry. + * + * If @len is larger than the tlb size, then it will take longer to + * test all of the entries in the TLB than it will to flush it all. + */ + if (mask < fast->mask || len > fast->mask) { + tlbfast_flush_locked(desc, fast); + return; + } + + for (vaddr i = 0; i < len; i += TARGET_PAGE_SIZE) { + vaddr page = addr + i; + CPUTLBEntry *entry = tlbfast_entry(fast, page); + + if (tlb_flush_entry_mask_locked(entry, page, mask)) { + desc->n_used_entries--; + } + } +} + static void tlb_flush_page_locked(CPUState *cpu, int midx, vaddr page) { - vaddr lp_addr = cpu->neg.tlb.d[midx].large_page_addr; - vaddr lp_mask = cpu->neg.tlb.d[midx].large_page_mask; + CPUTLBDesc *desc = &cpu->neg.tlb.d[midx]; + vaddr lp_addr = desc->large_page_addr; + vaddr lp_mask = desc->large_page_mask; /* Check if we need to flush due to large pages. */ if ((page & lp_mask) == lp_addr) { @@ -532,9 +554,8 @@ static void tlb_flush_page_locked(CPUState *cpu, int midx, vaddr page) midx, lp_addr, lp_mask); tlb_flush_one_mmuidx_locked(cpu, midx, get_clock_realtime()); } else { - if (tlb_flush_entry_locked(tlb_entry(cpu, midx, page), page)) { - tlb_n_used_entries_dec(cpu, midx); - } + tlbfast_flush_range_locked(desc, &cpu->neg.tlb.f[midx], + page, TARGET_PAGE_SIZE, -1); tlb_flush_vtlb_page_locked(cpu, midx, page); } } @@ -689,24 +710,6 @@ static void tlb_flush_range_locked(CPUState *cpu, int midx, CPUTLBDescFast *f = &cpu->neg.tlb.f[midx]; vaddr mask = MAKE_64BIT_MASK(0, bits); - /* - * If @bits is smaller than the tlb size, there may be multiple entries - * within the TLB; otherwise all addresses that match under @mask hit - * the same TLB entry. - * TODO: Perhaps allow bits to be a few bits less than the size. - * For now, just flush the entire TLB. - * - * If @len is larger than the tlb size, then it will take longer to - * test all of the entries in the TLB than it will to flush it all. - */ - if (mask < f->mask || len > f->mask) { - tlb_debug("forcing full flush midx %d (" - "%016" VADDR_PRIx "/%016" VADDR_PRIx "+%016" VADDR_PRIx ")\n", - midx, addr, mask, len); - tlb_flush_one_mmuidx_locked(cpu, midx, get_clock_realtime()); - return; - } - /* * Check if we need to flush due to large pages. * Because large_page_mask contains all 1's from the msb, @@ -720,13 +723,10 @@ static void tlb_flush_range_locked(CPUState *cpu, int midx, return; } + tlbfast_flush_range_locked(d, f, addr, len, mask); + for (vaddr i = 0; i < len; i += TARGET_PAGE_SIZE) { vaddr page = addr + i; - CPUTLBEntry *entry = tlb_entry(cpu, midx, page); - - if (tlb_flush_entry_mask_locked(entry, page, mask)) { - tlb_n_used_entries_dec(cpu, midx); - } tlb_flush_vtlb_page_mask_locked(cpu, midx, page, mask); } } From patchwork Thu Nov 14 16:00:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E15ACD68B33 for ; Thu, 14 Nov 2024 16:02:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHv-0004wK-1W; Thu, 14 Nov 2024 11:01:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHt-0004ue-0f for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHr-0002BM-3l for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:40 -0500 Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-7f45ab88e7fso619465a12.1 for ; Thu, 14 Nov 2024 08:01:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600096; x=1732204896; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=xBjb7ecHDPeo10d84Jo9N+bY5e+BwxfICVTefAKQV3o=; b=YKXEDKgYQAzfyWxXvedlBz+JpAG/hHBVVgcHoZH/I58mOtDblQFkV8YoER1OCQk+Ug lEcpPC0113eq8uPjzqv41ppeGpfA/SQhgOOvpBzbJth8mxY2l05PKEvDhdxEdKmBmZG9 EaFc6AqcWuVPXLt7s6o4x6yZgL8V4LCfDTONeY1nvE8uAZfXox+huPqp1qWAlHpnj+SL iJilmLyYOZtv0F4l4tMiNP15hI/ZjzRjcYYDcUvxkG3pbehjManxuzxQpSZw10F5Xkcq +Pk3xSv9PFzcibjKuZ/sJyP14ZcNHOLl4Ie9QCWzKi8EZF4J3IUoOiuhvRg1pM6Np4DU j6ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600096; x=1732204896; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xBjb7ecHDPeo10d84Jo9N+bY5e+BwxfICVTefAKQV3o=; b=OsaPvOVIMGRvLhYkR7peqkD8xWe5dZiX/qFWkBO3CqYDYNOsN3417CKveHd5XKuW4/ qwsGg11dTc2HV+qalkWz+hEG+Siu6gmcfGkTHTdY1QoPjZ6t1wnXXHVDtUVg2I+nv8HD SYFtOz2twtWAfjMQbAsw6wfspkdq+MAszt5KppKlkmOiaEU/xYFnosSurFxItYOuN8yZ 39kqzdOyj/VPuO5s/90ZdNWMVSBbL4g9qZf7VfiQzZBVpUYGZkczs0P7fxw1B2WHkSjZ j3KcbUdogIEwd6dqR8Cjmi1EYFWvEUjqf8TVc+08Mal3g/7GAwhYLXP5WUD3Gde0pmgd ozWw== X-Gm-Message-State: AOJu0YyAkftRFylzmZki62qFMcLoSIi66i218xXsQuDo1H+/9dd3ROt1 ZjQjD5NSgm6rvsG2sykacuEgVtpwXu5KXZR56ZdcojpEwNxe6VkD4Sgz3kFSZQyAqtTKI0GZkyx z X-Google-Smtp-Source: AGHT+IEnl0PgJGqLTR9i+8vRxjxCxE5d9vm4HYUQA/i/q9UpyST7qJtzTTkwC0BS/NulzX2uNFBPxw== X-Received: by 2002:a17:90b:3883:b0:2e2:d3e1:f863 with SMTP id 98e67ed59e1d1-2e9b171e360mr33852391a91.12.1731600096298; Thu, 14 Nov 2024 08:01:36 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:35 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 05/54] accel/tcg: Fix flags usage in mmu_lookup1, atomic_mmu_lookup Date: Thu, 14 Nov 2024 08:00:41 -0800 Message-ID: <20241114160131.48616-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The INVALID bit should only be auto-cleared when we have just called tlb_fill, not along the victim_tlb_hit path. In atomic_mmu_lookup, rename tlb_addr to flags, as that is what we're actually carrying around. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 46fa0ae802..77b972fd93 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1652,7 +1652,7 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, uint64_t tlb_addr = tlb_read_idx(entry, access_type); bool maybe_resized = false; CPUTLBEntryFull *full; - int flags; + int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; /* If the TLB entry is for a different page, reload and try again. */ if (!tlb_hit(tlb_addr, addr)) { @@ -1663,8 +1663,14 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, maybe_resized = true; index = tlb_index(cpu, mmu_idx, addr); entry = tlb_entry(cpu, mmu_idx, addr); + /* + * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, + * to force the next access through tlb_fill. We've just + * called tlb_fill, so we know that this entry *is* valid. + */ + flags &= ~TLB_INVALID_MASK; } - tlb_addr = tlb_read_idx(entry, access_type) & ~TLB_INVALID_MASK; + tlb_addr = tlb_read_idx(entry, access_type); } full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; @@ -1814,10 +1820,10 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, MemOp mop = get_memop(oi); uintptr_t index; CPUTLBEntry *tlbe; - vaddr tlb_addr; void *hostaddr; CPUTLBEntryFull *full; bool did_tlb_fill = false; + int flags; tcg_debug_assert(mmu_idx < NB_MMU_MODES); @@ -1828,8 +1834,8 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, tlbe = tlb_entry(cpu, mmu_idx, addr); /* Check TLB entry and enforce page permissions. */ - tlb_addr = tlb_addr_write(tlbe); - if (!tlb_hit(tlb_addr, addr)) { + flags = TLB_FLAGS_MASK; + if (!tlb_hit(tlb_addr_write(tlbe), addr)) { if (!victim_tlb_hit(cpu, mmu_idx, index, MMU_DATA_STORE, addr & TARGET_PAGE_MASK)) { tlb_fill_align(cpu, addr, MMU_DATA_STORE, mmu_idx, @@ -1837,8 +1843,13 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, did_tlb_fill = true; index = tlb_index(cpu, mmu_idx, addr); tlbe = tlb_entry(cpu, mmu_idx, addr); + /* + * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, + * to force the next access through tlb_fill. We've just + * called tlb_fill, so we know that this entry *is* valid. + */ + flags &= ~TLB_INVALID_MASK; } - tlb_addr = tlb_addr_write(tlbe) & ~TLB_INVALID_MASK; } /* @@ -1874,11 +1885,11 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, goto stop_the_world; } - /* Collect tlb flags for read. */ - tlb_addr |= tlbe->addr_read; + /* Collect tlb flags for read and write. */ + flags &= tlbe->addr_read | tlb_addr_write(tlbe); /* Notice an IO access or a needs-MMU-lookup access */ - if (unlikely(tlb_addr & (TLB_MMIO | TLB_DISCARD_WRITE))) { + if (unlikely(flags & (TLB_MMIO | TLB_DISCARD_WRITE))) { /* There's really nothing that can be done to support this apart from stop-the-world. */ goto stop_the_world; @@ -1887,11 +1898,11 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, hostaddr = (void *)((uintptr_t)addr + tlbe->addend); full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; - if (unlikely(tlb_addr & TLB_NOTDIRTY)) { + if (unlikely(flags & TLB_NOTDIRTY)) { notdirty_write(cpu, addr, size, full, retaddr); } - if (unlikely(tlb_addr & TLB_FORCE_SLOW)) { + if (unlikely(flags & TLB_FORCE_SLOW)) { int wp_flags = 0; if (full->slow_flags[MMU_DATA_STORE] & TLB_WATCHPOINT) { From patchwork Thu Nov 14 16:00:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08CCDD68B34 for ; Thu, 14 Nov 2024 16:02:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHw-0004x0-Td; Thu, 14 Nov 2024 11:01:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHs-0004ud-LY for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHr-0002Bb-37 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:40 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-7245a9d0e92so810085b3a.0 for ; Thu, 14 Nov 2024 08:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600097; x=1732204897; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=oFR54WJ7Z7wCqfLri0LCaQ3ITtfM85rp0V8bo+nOxWA=; b=QN+Tt6rYXifuXSBZjImQwSCH4hazW2pmsUppYijDkooQppGCwF8Zeom+Tt87NCCknA PKTIb2SuDSnKU/TUZygzryUM7Ed037XquoyspZ1lyxr8cbg7RK+joHBQlvmiMIzRd7Wc KWAoemzFolfSxMiXqLkn+qaWF/4UsoNCzbP2Rtr12EbjzOP0sn/EdcoC8Oao/EL13nft Reu91QNLXdkfYjWsDwek1zh4hFvjKuVt5U48psoljF1LljP7tZAPY9kd35Uh9I8EQ2ZY XCDocsGMQfXnFKPSr9aDHxYI2hjnh9nn72NJSV6LPtKXGyAIJExg5gqe7+uhKNYG90iC nBqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600097; x=1732204897; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oFR54WJ7Z7wCqfLri0LCaQ3ITtfM85rp0V8bo+nOxWA=; b=RrXr5pGpoDs/ykzhJGayu12xCeJ0eFgk89jnKVqrZOz8UOwVvw9AuHFxdlNQT5A9nB 2BmAA/3fDXFKhvuUK6OjA0kE/TT8jIMfAUvU3j8UbXxSYFtmiX1ypaujfOVrUQxqmX12 GkzGjWC2Fdo4qLT4FW7wutxst/IKrXS6PYTGbiqH66Qu7YBszFaI53/YyssgIv61swGJ ES34XsL/SJaCBj31Gp99IbYaVzeeBgf1Oo9RjrkoyOXQ4d/RQt81+Y5t0ryLGGFETWn4 W7Rydsx/eBDViJ/TZKUT+9QvcjZYlAs1MROuuGF2xSfIc3lH/KPiL9R/LV5usazAMyAu nNRw== X-Gm-Message-State: AOJu0YzTcNvOZjuEczxq+oRsRBtV+1vyqE9ymmQUqFlU0PinsKNpCxiM YGH+QJmRqYExx4Us/UP8oTwkQIL121izuzBhuZOWiLN2KjsMWm7m4xAaQHDnPduiSo71+nO/x0a r X-Google-Smtp-Source: AGHT+IEtgfnvE8wucOMCwWAfgCmZ8rB63917GWQJPZgDBTKLsxHXYSjWkH9si5U7nV6EibzRKcXRgA== X-Received: by 2002:a17:90b:3948:b0:2e2:d181:6808 with SMTP id 98e67ed59e1d1-2e9f2d5da6fmr8756010a91.30.1731600097110; Thu, 14 Nov 2024 08:01:37 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.36 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:36 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 06/54] accel/tcg: Assert non-zero length in tlb_flush_range_by_mmuidx* Date: Thu, 14 Nov 2024 08:00:42 -0800 Message-ID: <20241114160131.48616-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Next patches will assume non-zero length. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 77b972fd93..1346a26d90 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -791,6 +791,7 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr, TLBFlushRangeData d; assert_cpu_is_self(cpu); + assert(len != 0); /* * If all bits are significant, and len is small, @@ -830,6 +831,8 @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *src_cpu, TLBFlushRangeData d, *p; CPUState *dst_cpu; + assert(len != 0); + /* * If all bits are significant, and len is small, * this devolves to tlb_flush_page. From patchwork Thu Nov 14 16:00:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B952D68B35 for ; Thu, 14 Nov 2024 16:03:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHz-0004xb-8M; Thu, 14 Nov 2024 11:01:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHt-0004uf-17 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHr-0002Bo-Cd for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:40 -0500 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-720be27db27so664200b3a.2 for ; Thu, 14 Nov 2024 08:01:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600098; x=1732204898; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=V73iy9O4QBEVFd3/BQLHjhP35QEkGTMYrTHTqx+CnQ4=; b=uEz9pAdQIWtQJ/jtFaIUuz24sY/RuCsfLygWC3BZ9i3ClJtnz73XrdlP9to5sb0usy OK16vRHVfjx8YuFl5P/sj7iQtL6ZqnwskyMeb4WbYd52C4gKr7JCOLWk4zlon+9FWAmc HmhNbzON9JGuSIMEGdkTZdMl9Gjhxw+6z3GZNVD/fgF+SmxBTQzJclYOY++avxrCA03/ KzIoVKKIIhVJTGrAxNJuojWfjLepzvpHF3RMxn9I5HU77JpU1OPCK/VAZErmITyuOSBk DRSDYbnKxFqb7DI4ulnAUbAVb4td6KyDJXMz6eSPzgpxQqmPl7OdFYQyJP3oy4unZK+h lIlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600098; x=1732204898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V73iy9O4QBEVFd3/BQLHjhP35QEkGTMYrTHTqx+CnQ4=; b=Q66DAFcSZNOnkBw1LU9O4TxsTD8zoaKNT4gEWlZutnZnAfBA7L0deLnf+mcHePpcvK Td9sagCTfA01QosV5AZ6EnOR7GREzq0X6+w5qjcWcT+hZdxUDsN5F8xy8gw4YETZgy6q QrQSAVgUyVzNON+YfMqQdSBj5OyK+Dvkk8er/E7+piZPN/upXbyRIrlmT5zlVIB4LQTr bEatAnbTXjc3C9TPQ5VrYRYvjnz9eNzMb9fsnBE0AO7867qnuSRqxhfxEGHMelpMZphY aOoBBDY/Ycp/2+MCNI03gxFPXNSXVFdDg7TcG8PRuFfvvZhofqUILLh154M6wuxmDISL 3CnA== X-Gm-Message-State: AOJu0Yx4+45c6HxY3xWugbBKjepv5SqguF0IRnIUOgD3FZZu2lfyOEb/ 9dUOTj/U3f8tpGH2d1ZbDrKr4alYgelbTk5LMBEE02CTAiWj55FK2mkkrFQILgZcbhlBVhwHHid 6 X-Google-Smtp-Source: AGHT+IEYJK6UI4uKpmyyEECf0Os2bmLmCvDMldRXOxy4zA6Mj2Q3YhnggVlrohFXIGgZpzUqWYNX4Q== X-Received: by 2002:a17:90b:28c7:b0:2e2:8d33:f79d with SMTP id 98e67ed59e1d1-2e9e4bea639mr13502716a91.19.1731600097744; Thu, 14 Nov 2024 08:01:37 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:37 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 07/54] accel/tcg: Assert bits in range in tlb_flush_range_by_mmuidx* Date: Thu, 14 Nov 2024 08:00:43 -0800 Message-ID: <20241114160131.48616-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The only target that does not use TARGET_LONG_BITS is Arm, which only reduces bits based on TBI. There is no point in handling odd combinations of parameters. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 1346a26d90..5510f40333 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -792,20 +792,16 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr, assert_cpu_is_self(cpu); assert(len != 0); + assert(bits > TARGET_PAGE_BITS && bits <= TARGET_LONG_BITS); /* * If all bits are significant, and len is small, * this devolves to tlb_flush_page. */ - if (bits >= TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) { + if (bits == TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) { tlb_flush_page_by_mmuidx(cpu, addr, idxmap); return; } - /* If no page bits are significant, this devolves to tlb_flush. */ - if (bits < TARGET_PAGE_BITS) { - tlb_flush_by_mmuidx(cpu, idxmap); - return; - } /* This should already be page aligned */ d.addr = addr & TARGET_PAGE_MASK; @@ -832,20 +828,16 @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *src_cpu, CPUState *dst_cpu; assert(len != 0); + assert(bits > TARGET_PAGE_BITS && bits <= TARGET_LONG_BITS); /* * If all bits are significant, and len is small, * this devolves to tlb_flush_page. */ - if (bits >= TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) { + if (bits == TARGET_LONG_BITS && len <= TARGET_PAGE_SIZE) { tlb_flush_page_by_mmuidx_all_cpus_synced(src_cpu, addr, idxmap); return; } - /* If no page bits are significant, this devolves to tlb_flush. */ - if (bits < TARGET_PAGE_BITS) { - tlb_flush_by_mmuidx_all_cpus_synced(src_cpu, idxmap); - return; - } /* This should already be page aligned */ d.addr = addr & TARGET_PAGE_MASK; From patchwork Thu Nov 14 16:00:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35EE5D68B35 for ; Thu, 14 Nov 2024 16:03:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcHz-0004xc-8g; Thu, 14 Nov 2024 11:01:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHt-0004va-V9 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:42 -0500 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHs-0002C4-EJ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-7eab7622b61so600731a12.1 for ; Thu, 14 Nov 2024 08:01:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600099; x=1732204899; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=BsbKOlO+09+V06ilkiETtavIcLFq0cApuQcSVOlh0Qs=; b=AvxE0pzeZrEsfZBXtJlQo7H5U0BUzJZnBTiSj6KhQGv2DLs1wWNxrzAtnbHE9oixJS RMKm+8zrDXtqtbsZyzZEDnpO+xtEHU06nrEuTECzkUXTwqoJw3pjr9boCX6468v/X/Qu aelmKQ28uDU49QddtuQr80yiGuKAtZ/hL0ZuoMcOh5mr7W7Xcp17Vi08I7VsnjbkyC5/ erk5BANyqRvFrd7nDjmLy9466xG0Z04XKXxrAllT3hivQYtklFJ1Se76XtGov5QMhv5C Dkb28ogIE1TodYQs4Ijk6G0k8Cd5Cpd2/vkZiq0lPt80jbsnyDI5mxzxj7XGBxLSOcmc Ksjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600099; x=1732204899; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BsbKOlO+09+V06ilkiETtavIcLFq0cApuQcSVOlh0Qs=; b=jvad674pSrUdPxH6ZjIOqmFpyb3JXoTDSCOIGcrc+EvRDa9l9EH0/NGDpDabGgkC1G 1tAi+jgbP02jp5IEIIrSKO1UVqX9FnkyhIpV1FMWeFvb04zD2uDqGJsX7X9IEjtYzufA ik2rl+93t/We9gygL1S3KYNbPIAypiMbupddigLFn8+5lwsRDY7b08IaezGbPuG6UBiV pZqsrJvP7DrzzllYRPR8zR3dClmro/U/9McA9E7RrKkq/EN2yC4Eod0yjm2hzMvY7q+X tjzdCuGBPClxZw7ULT+9RNE2Y8ta4383yfm6YOJSEzyQQ2EUJNc/YOtVqwLUXUWwijNA LOxQ== X-Gm-Message-State: AOJu0Yx6j8gUkX/k3IeGAnyd9a6Vx8l5X0+p3w+cehQqGG1FzdBi7/NP MnkSyghpPHNbH+HGXcbj+AoZts+4KjfCq/RZIsURuwZztjqIA7ZGcbZ5B8kvbe4Bqx7r+JjGFzG w X-Google-Smtp-Source: AGHT+IGt3AC5gHbE95Yv2CV8baHn/zHIl3zbHtIEsS9n2l/6g/fpoWhk2keiF6Le3MGl5XN08m9MAQ== X-Received: by 2002:a17:90b:2248:b0:2cd:4593:2a8e with SMTP id 98e67ed59e1d1-2e9b171ff77mr31886980a91.15.1731600098555; Thu, 14 Nov 2024 08:01:38 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:38 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 08/54] accel/tcg: Flush entire tlb when a masked range wraps Date: Thu, 14 Nov 2024 08:00:44 -0800 Message-ID: <20241114160131.48616-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::533; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x533.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We expect masked address spaces to be quite large, e.g. 56 bits for AArch64 top-byte-ignore mode. We do not expect addr+len to wrap around, but it is possible with AArch64 guest flush range instructions. Convert this unlikely case to a full tlb flush. This can simplify the subroutines actually performing the range flush. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 5510f40333..31c45a6213 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -802,6 +802,11 @@ void tlb_flush_range_by_mmuidx(CPUState *cpu, vaddr addr, tlb_flush_page_by_mmuidx(cpu, addr, idxmap); return; } + /* If addr+len wraps in len bits, fall back to full flush. */ + if (bits < TARGET_LONG_BITS && ((addr ^ (addr + len - 1)) >> bits)) { + tlb_flush_by_mmuidx(cpu, idxmap); + return; + } /* This should already be page aligned */ d.addr = addr & TARGET_PAGE_MASK; @@ -838,6 +843,11 @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *src_cpu, tlb_flush_page_by_mmuidx_all_cpus_synced(src_cpu, addr, idxmap); return; } + /* If addr+len wraps in len bits, fall back to full flush. */ + if (bits < TARGET_LONG_BITS && ((addr ^ (addr + len - 1)) >> bits)) { + tlb_flush_by_mmuidx_all_cpus_synced(src_cpu, idxmap); + return; + } /* This should already be page aligned */ d.addr = addr & TARGET_PAGE_MASK; From patchwork Thu Nov 14 16:00:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C977D68B34 for ; Thu, 14 Nov 2024 16:09:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcI1-0004xi-GN; Thu, 14 Nov 2024 11:01:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcHu-0004vl-8l for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:42 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHs-0002CC-JE for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:41 -0500 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2ea0ad89e84so491935a91.3 for ; Thu, 14 Nov 2024 08:01:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600099; x=1732204899; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ZmBByG8TldvCZTNhOs8dLsatn8HWp1mSaaGpDvl40n8=; b=qB1GvrdCcsv6YqgL7klY/NLq/UD6jlFedudBCgt2IbHuteCNO/9B2PyZhI0shev6tD amsZhe4po3N6Q9mIqG1jgSUWnvhJzM9I8J9D77xAUsZRY56dr5qaN6eCQ7O8/4vJGLEV aOfYrzlooJ8b5q9W97kV0pPAIZK/n+vMc4UYhdmrKGPjeeevifpU28NWlib1Bdldpevx VP+GtjQsNie5ZgcFopzw17w6j9WyZ8Z0+r7+J0m8xIDRrqZT/XWni2lkExjU/yObT+OW QmevbqMGg7KDvOjpZcU2Cu4GXNejbbGRIieCDfUxRFExjd6i6A2pyiGDUsu8WzHoLq/i Ikiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600099; x=1732204899; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZmBByG8TldvCZTNhOs8dLsatn8HWp1mSaaGpDvl40n8=; b=KnAVqRoFsyrGXNpUsDyZIYv04QoRtMs9F4JE+4cfLIyNWrAD8kzrmtrUFzKTn1UW14 Tai87bWNncVeYb4CSA0PXKRCWYFGGoaMrZg+fK0l3QaKa1Ft9GThTTjitHRRcEnxr2N9 Cmo+FG3MRY3IPURVrR19CUKgKQlXf7vc+JwqtBGd3C5pI4fyk53K7C4kn6Js4kDQDqXw caFa7MCLyWrleZrWKQMsHFQ4N5GSGRRAK05rB672k6IpOPMOAV3qLVW+Jbwuskd0fSsH 6JGtIA2XfCnKy95jifevIDhouCwM57lv6LaTjMjMY1DKnjjl2AOM6hLr5/eST2odsG1l dtsg== X-Gm-Message-State: AOJu0Yx7adjL8HfLEQW+7LO7BAxNZfz+ujO7ZO5Gtf77U2n3ArzDHhaB fjyztEmrKV5XjvxnUDaQGt+rnc8832fov2kgDCcwyARPsJFg6dNmvj68iow8Sh3FraBkDzX6+1D M X-Google-Smtp-Source: AGHT+IEKv+BQbIXfODjLdJsnKc+8bAuwg6xhNZ5R6C+vHkigu3jSNFrpb3Z8cRRT5GY7aCzaFTIkHA== X-Received: by 2002:a17:90b:538f:b0:2e7:6e84:a854 with SMTP id 98e67ed59e1d1-2ea062dd600mr2936115a91.1.1731600099219; Thu, 14 Nov 2024 08:01:39 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:38 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 09/54] accel/tcg: Add IntervalTreeRoot to CPUTLBDesc Date: Thu, 14 Nov 2024 08:00:45 -0800 Message-ID: <20241114160131.48616-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add the data structures for tracking softmmu pages via a balanced interval tree. So far, only initialize and destroy the data structure. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/hw/core/cpu.h | 3 +++ accel/tcg/cputlb.c | 11 +++++++++++ 2 files changed, 14 insertions(+) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index db8a6fbc6e..1ebc999a73 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -35,6 +35,7 @@ #include "qemu/queue.h" #include "qemu/lockcnt.h" #include "qemu/thread.h" +#include "qemu/interval-tree.h" #include "qom/object.h" typedef int (*WriteCoreDumpFunction)(const void *buf, size_t size, @@ -290,6 +291,8 @@ typedef struct CPUTLBDesc { CPUTLBEntry vtable[CPU_VTLB_SIZE]; CPUTLBEntryFull vfulltlb[CPU_VTLB_SIZE]; CPUTLBEntryFull *fulltlb; + /* All active tlb entries for this address space. */ + IntervalTreeRoot iroot; } CPUTLBDesc; /* diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 31c45a6213..aa51fc1d26 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -89,6 +89,13 @@ QEMU_BUILD_BUG_ON(sizeof(vaddr) > sizeof(run_on_cpu_data)); QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16); #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1) +/* Extra data required to manage CPUTLBEntryFull within an interval tree. */ +typedef struct CPUTLBEntryTree { + IntervalTreeNode itree; + CPUTLBEntry copy; + CPUTLBEntryFull full; +} CPUTLBEntryTree; + static inline size_t tlb_n_entries(CPUTLBDescFast *fast) { return (fast->mask >> CPU_TLB_ENTRY_BITS) + 1; @@ -305,6 +312,7 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) desc->large_page_mask = -1; desc->vindex = 0; memset(desc->vtable, -1, sizeof(desc->vtable)); + interval_tree_free_nodes(&desc->iroot, offsetof(CPUTLBEntryTree, itree)); } static void tlb_flush_one_mmuidx_locked(CPUState *cpu, int mmu_idx, @@ -326,6 +334,7 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now) fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS; fast->table = g_new(CPUTLBEntry, n_entries); desc->fulltlb = g_new(CPUTLBEntryFull, n_entries); + memset(&desc->iroot, 0, sizeof(desc->iroot)); tlb_mmu_flush_locked(desc, fast); } @@ -365,6 +374,8 @@ void tlb_destroy(CPUState *cpu) g_free(fast->table); g_free(desc->fulltlb); + interval_tree_free_nodes(&cpu->neg.tlb.d[i].iroot, + offsetof(CPUTLBEntryTree, itree)); } } From patchwork Thu Nov 14 16:00:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A3C60D68B34 for ; Thu, 14 Nov 2024 16:04:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIH-0004zv-G5; Thu, 14 Nov 2024 11:02:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0004xj-Ns for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:49 -0500 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHt-0002CX-Q4 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:43 -0500 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-7ea8c4ce232so769670a12.0 for ; Thu, 14 Nov 2024 08:01:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600100; x=1732204900; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=XI4td4zXdpc3AvbTmQTiCq3YxLJsDwG9audmxtHxV6A=; b=IRfey1D/KtOfBn2QQfskqNzVyXMotSAmZrAeKvaqlXSbkua/H+wxbSAYa+4oYPMcKk ZtLIBb4AuM/OBG8FuDBlEa9JcYdRax4GxWqXgY5VNEayUEk8yaGAGDY3l0Yw4xHqqoBb EDlvBIWyBn4vZCtRRPizz/cS6qQc1cGyL54GYMJyHZIQOOfYgD2QgnQeyd4b7sKVRmWn RJdtG/fsex25es+Mx5nSBx1R/W/3xLyBNuweCUrz985qnt/ZbWpnncGEPoC+6newnEy8 i+tss3TmsbMrTrTZ/488ReNecFQgPR5j9bAuQE6zHxQR2UMrPlH+NGVLEtgMd9D1P1eB PKqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600100; x=1732204900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XI4td4zXdpc3AvbTmQTiCq3YxLJsDwG9audmxtHxV6A=; b=BThfWvBaL8YskgmnS3CG6WBBDi9g/wA5Y1sllw4NT6LycfIdeaD+O54aixIoQZMWBY wd/knMvmkKXq1wn0GWiawtUvY2b8NMtyz5rVPF6O7hVRfnKxAa/G6uo4XGN1FvgZ8qph p7/rVTwfle67a+B1qfk3wEMRhQ4qq4MeLNPaphCTEGjmENmwifmCjLIeug6apxjtx+Nw S/8byXRKglsh9nukRwghaF/4FK0uwOJBbUzlslro/6VBB/ZS528YT78lqr5+JDMvxTFV iu2d3JcT54/SxVRRJGip/y4/h544KnJWmZrGYFZTXXKxC0W0m7A4BNEeDTfbxLSp2J47 CrDg== X-Gm-Message-State: AOJu0YxSdz+/paZ6UC/oYjhYv2LFfeRxCXY6LRCh5kgaIfAHyj2gKjBB DfRSwk87Oz1yj1/RfnJbO08iVuVEcy3WAra8FrC7a034fRShNKYt3R9T7GWDr+WB97VN+ZLgKqU h X-Google-Smtp-Source: AGHT+IEuwH57RqruKUul5HK6LB7ubc7jYrLG/M06QUbRhTpEVdLaouUVjy8IM8+BBWnZ6hZ77SOcPg== X-Received: by 2002:a17:90a:d450:b0:2e0:894f:198e with SMTP id 98e67ed59e1d1-2ea0639fc36mr2943291a91.30.1731600100006; Thu, 14 Nov 2024 08:01:40 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:39 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 10/54] accel/tcg: Populate IntervalTree in tlb_set_page_full Date: Thu, 14 Nov 2024 08:00:46 -0800 Message-ID: <20241114160131.48616-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add or replace an entry in the IntervalTree for each page installed into softmmu. We do not yet use the tree for anything else. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 34 ++++++++++++++++++++++++++++------ 1 file changed, 28 insertions(+), 6 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index aa51fc1d26..ea6a5177de 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -305,6 +305,17 @@ static void tlbfast_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) memset(fast->table, -1, sizeof_tlb(fast)); } +static CPUTLBEntryTree *tlbtree_lookup_range(CPUTLBDesc *desc, vaddr s, vaddr l) +{ + IntervalTreeNode *i = interval_tree_iter_first(&desc->iroot, s, l); + return i ? container_of(i, CPUTLBEntryTree, itree) : NULL; +} + +static CPUTLBEntryTree *tlbtree_lookup_addr(CPUTLBDesc *desc, vaddr addr) +{ + return tlbtree_lookup_range(desc, addr, addr); +} + static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) { tlbfast_flush_locked(desc, fast); @@ -1072,7 +1083,8 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, MemoryRegionSection *section; unsigned int index, read_flags, write_flags; uintptr_t addend; - CPUTLBEntry *te, tn; + CPUTLBEntry *te; + CPUTLBEntryTree *node; hwaddr iotlb, xlat, sz, paddr_page; vaddr addr_page; int asidx, wp_flags, prot; @@ -1180,6 +1192,15 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, tlb_n_used_entries_dec(cpu, mmu_idx); } + /* Replace an old IntervalTree entry, or create a new one. */ + node = tlbtree_lookup_addr(desc, addr_page); + if (!node) { + node = g_new(CPUTLBEntryTree, 1); + node->itree.start = addr_page; + node->itree.last = addr_page + TARGET_PAGE_SIZE - 1; + interval_tree_insert(&node->itree, &desc->iroot); + } + /* refill the tlb */ /* * When memory region is ram, iotlb contains a TARGET_PAGE_BITS @@ -1201,15 +1222,15 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, full->phys_addr = paddr_page; /* Now calculate the new entry */ - tn.addend = addend - addr_page; + node->copy.addend = addend - addr_page; - tlb_set_compare(full, &tn, addr_page, read_flags, + tlb_set_compare(full, &node->copy, addr_page, read_flags, MMU_INST_FETCH, prot & PAGE_EXEC); if (wp_flags & BP_MEM_READ) { read_flags |= TLB_WATCHPOINT; } - tlb_set_compare(full, &tn, addr_page, read_flags, + tlb_set_compare(full, &node->copy, addr_page, read_flags, MMU_DATA_LOAD, prot & PAGE_READ); if (prot & PAGE_WRITE_INV) { @@ -1218,10 +1239,11 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, if (wp_flags & BP_MEM_WRITE) { write_flags |= TLB_WATCHPOINT; } - tlb_set_compare(full, &tn, addr_page, write_flags, + tlb_set_compare(full, &node->copy, addr_page, write_flags, MMU_DATA_STORE, prot & PAGE_WRITE); - copy_tlb_helper_locked(te, &tn); + node->full = *full; + copy_tlb_helper_locked(te, &node->copy); tlb_n_used_entries_inc(cpu, mmu_idx); qemu_spin_unlock(&tlb->c.lock); } From patchwork Thu Nov 14 16:00:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 208F0D68B35 for ; Thu, 14 Nov 2024 16:12:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcI4-0004zJ-5H; Thu, 14 Nov 2024 11:01:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0004xk-Nr for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:49 -0500 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHu-0002Cq-3h for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:43 -0500 Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-2e2ed2230d8so712292a91.0 for ; Thu, 14 Nov 2024 08:01:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600101; x=1732204901; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=hz/db0U//GrGJtv1TKp+9GNtVl8RODviEX3vH3DtlZo=; b=NLYWol+PQJdmkmF5vaOCTGF7mRWl+RO+T2Dab1p0wSSkr1fp67aHstLPHcIx7v69xH 0lOaShaXZlkuDs3h1nZETORfxvCAhqUAJgbb2a+4V2PMVjNCCZSTltys1+LB022Pr+mo YNQy3Q8h1BWEHQvtlb901uvrLX53DichaK4BaMqfetxA4Ppd8JXHJLQeePn8FAF8bTNz sdlsn71KrA+TAmMSoQWpKZ7B1F9TC2K4SSUy0LM997DR+XmJjuKvpJZ526ckGltBK2w7 Okg0S37BhQX0OK0j3bnK67JsgX/jd8bynazgK2j9IqNrNqS4V7FRpkwK1obmDwE6urTS hqaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600101; x=1732204901; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hz/db0U//GrGJtv1TKp+9GNtVl8RODviEX3vH3DtlZo=; b=Ga4/0EamY5VPjlxJ6CSNPYzW+umBLYpYrZbEZULKRYPNA4SwYxxu9YrxtiiYvqDn6x kNvbLhqMGIGQrYl7mlqTwdX2YnXEuGIwjCR48ws39vtZoYecGGG9DQ8XleT6laTGC+y7 E0KTbriLskLTaTAsTfBrKFQiv7pEe8dAFCT3aLV2Z2zE+ak0JIaGi1hUQAZFurxA6BN1 CWle/pJ9dS+iw12jOoCWs2GiqMk3ytjqSyzR+f0hIt26pRSTBjD44uBmM+Ck0WJ/lOsI rbWETlnY+otOT/o6TJkt9YKOQG0bEGCPKa/nLN+Ynp62K4nIP7FGg73N4uTd/hvVwjnc HhSA== X-Gm-Message-State: AOJu0YyUJ9X91t/7vSFTHuonTj5ZLuyvKwaHNY1PfwwCorDi7wAYAS46 gEDXqAVVboj2jbaLlBX3+82Z2CY4gDu27hE6REHesc307CA8aACOlhQ7w0GRJLZz6KD80Tn5TmX l X-Google-Smtp-Source: AGHT+IHcXVH3KaUHtQLFLDkMwOtcHGIrmarOlz54PW7Z4JpmceQuSWMN25JiJlrSZXxhUbG4TyrWBQ== X-Received: by 2002:a17:90b:4c03:b0:2c8:65cf:e820 with SMTP id 98e67ed59e1d1-2ea06346505mr2754826a91.2.1731600100692; Thu, 14 Nov 2024 08:01:40 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:40 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 11/54] accel/tcg: Remove IntervalTree entry in tlb_flush_page_locked Date: Thu, 14 Nov 2024 08:00:47 -0800 Message-ID: <20241114160131.48616-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Flush a page from the IntervalTree cache. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ea6a5177de..d532d69083 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -568,6 +568,7 @@ static void tlb_flush_page_locked(CPUState *cpu, int midx, vaddr page) CPUTLBDesc *desc = &cpu->neg.tlb.d[midx]; vaddr lp_addr = desc->large_page_addr; vaddr lp_mask = desc->large_page_mask; + CPUTLBEntryTree *node; /* Check if we need to flush due to large pages. */ if ((page & lp_mask) == lp_addr) { @@ -575,10 +576,17 @@ static void tlb_flush_page_locked(CPUState *cpu, int midx, vaddr page) VADDR_PRIx "/%016" VADDR_PRIx ")\n", midx, lp_addr, lp_mask); tlb_flush_one_mmuidx_locked(cpu, midx, get_clock_realtime()); - } else { - tlbfast_flush_range_locked(desc, &cpu->neg.tlb.f[midx], - page, TARGET_PAGE_SIZE, -1); - tlb_flush_vtlb_page_locked(cpu, midx, page); + return; + } + + tlbfast_flush_range_locked(desc, &cpu->neg.tlb.f[midx], + page, TARGET_PAGE_SIZE, -1); + tlb_flush_vtlb_page_locked(cpu, midx, page); + + node = tlbtree_lookup_addr(desc, page); + if (node) { + interval_tree_remove(&node->itree, &desc->iroot); + g_free(node); } } From patchwork Thu Nov 14 16:00:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3408BD68B33 for ; Thu, 14 Nov 2024 16:14:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004z4-EZ; Thu, 14 Nov 2024 11:01:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI1-0004xn-8e for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:49 -0500 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHu-0002D5-NU for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:49 -0500 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-2e9b4a5862fso689740a91.1 for ; Thu, 14 Nov 2024 08:01:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600101; x=1732204901; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=u1afL0RFM67W2rOuMJqQQROG23LY3jQbGC8krRZVFcY=; b=yVR9gF63XvgesbIfGba7n9fZZX3hgXVni7+13PJkbUps6RHYDnGg2P4+cSMPzpuy3y lpDuKyNvrh09B3elyqfRp1fd269Prs2+5XyXStCq7KYjkI9+5upKjiXByxVIz1U52fA0 2qsUCYtSpoPN4ZtTOlZCyj9Ur7ZW+hFuRKx0qvV0RBzrGt++jeNBUu1N/iSlbwyDqSw9 7ByqYW2iyGKJIUUmTa3kLkBaNKbo/F+/9a/fArGzHg5GlcZ2+NzsRIWrSAqSon1bqBnP aRYonFMxGQtFXLNCBkgek05k1QtI0zbf811nR4lBFvdWua2uv7qVYxS54IC81soNpPlY pLzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600101; x=1732204901; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u1afL0RFM67W2rOuMJqQQROG23LY3jQbGC8krRZVFcY=; b=IQ+wS5vaSLobUmx/pgrpuOoCwhaqrajh841WkJz3H3sevGmZ8blqtBJQ4BMq+1sa44 zNixf5/ptjJhZyJOsfeHHWn1Tfs7oiwPDIeARN/C2qeD5EWxCFR/APGwscT3xFQBmwrS kSxOQseKhVFFszkwFbNR8aUDfprQXxyXMItGcQq171uKsNKtnkl3/2ugSKqoW52GS8+Q fOyPSP0br85ZKhr3oBgAp2fQp35+g1So5ERhEGcRBLhGzJlRJCtyA+PvtQ/rRP2c42xa 0MnKZiZth1UuLDPf4kL/rMvvbxTGqX6gxoTQwrBuoc3hHG3gAem/1linqIMF3XrkXDZW 0KkA== X-Gm-Message-State: AOJu0YzAlebavtB2CGB6KY3wAMboH/7N2mhFUgAmHchQOYHyWiB1KpMH wni8yWuZRqiHGOZGjrq7Q7TFE7uEXa9JFQwWS+n/FxFfcd3a4/uqvxvn1YZWkof3yWrfW4zWe0U 2 X-Google-Smtp-Source: AGHT+IGd2wESbeUDZWZyULrJlD6ZhjqMg0umkV9Malj8l88C1hYJaC10rZTqQ9ZwkfZTBuGon/ZBDQ== X-Received: by 2002:a17:90b:4a90:b0:2d1:bf48:e767 with SMTP id 98e67ed59e1d1-2e9e4c73b9bmr14198227a91.29.1731600101380; Thu, 14 Nov 2024 08:01:41 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:40 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 12/54] accel/tcg: Remove IntervalTree entries in tlb_flush_range_locked Date: Thu, 14 Nov 2024 08:00:48 -0800 Message-ID: <20241114160131.48616-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Flush a masked range of pages from the IntervalTree cache. When the mask is not used there is a redundant comparison, but that is better than duplicating code at this point. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index d532d69083..e2c855f147 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -311,6 +311,13 @@ static CPUTLBEntryTree *tlbtree_lookup_range(CPUTLBDesc *desc, vaddr s, vaddr l) return i ? container_of(i, CPUTLBEntryTree, itree) : NULL; } +static CPUTLBEntryTree *tlbtree_lookup_range_next(CPUTLBEntryTree *prev, + vaddr s, vaddr l) +{ + IntervalTreeNode *i = interval_tree_iter_next(&prev->itree, s, l); + return i ? container_of(i, CPUTLBEntryTree, itree) : NULL; +} + static CPUTLBEntryTree *tlbtree_lookup_addr(CPUTLBDesc *desc, vaddr addr) { return tlbtree_lookup_range(desc, addr, addr); @@ -739,6 +746,8 @@ static void tlb_flush_range_locked(CPUState *cpu, int midx, CPUTLBDesc *d = &cpu->neg.tlb.d[midx]; CPUTLBDescFast *f = &cpu->neg.tlb.f[midx]; vaddr mask = MAKE_64BIT_MASK(0, bits); + CPUTLBEntryTree *node; + vaddr addr_mask, last_mask, last_imask; /* * Check if we need to flush due to large pages. @@ -759,6 +768,22 @@ static void tlb_flush_range_locked(CPUState *cpu, int midx, vaddr page = addr + i; tlb_flush_vtlb_page_mask_locked(cpu, midx, page, mask); } + + addr_mask = addr & mask; + last_mask = addr_mask + len - 1; + last_imask = last_mask | ~mask; + node = tlbtree_lookup_range(d, addr_mask, last_imask); + while (node) { + CPUTLBEntryTree *next = + tlbtree_lookup_range_next(node, addr_mask, last_imask); + vaddr page_mask = node->itree.start & mask; + + if (page_mask >= addr_mask && page_mask < last_mask) { + interval_tree_remove(&node->itree, &d->iroot); + g_free(node); + } + node = next; + } } typedef struct { From patchwork Thu Nov 14 16:00:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44346D68B34 for ; Thu, 14 Nov 2024 16:09:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIb-00059J-0P; Thu, 14 Nov 2024 11:02:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004z6-Kv for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHz-0002DP-OF for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:50 -0500 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-20c805a0753so8528885ad.0 for ; Thu, 14 Nov 2024 08:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600102; x=1732204902; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=8p+yJfYhOb0hWvZrgDQ+COQgpXoEivuXg56vfDbFxMU=; b=KiajS9YeEOeO4bZTjdzULh4VpNgvL5pqxep+40EDsK9N5NqxNL97d4xbSgDSTatEyq w6Hdnb0V0hssYu4RjogALWmFYqmSKxDc/LemuzMjSvZazqQEBm72pTNMwQYHftSidkIP njRuYqIe6ntvHlCWOc7Ib7so+bNhs08/lmPoENFWx//ueRmYO/krAPsUAWTDPJqM0aGx GxJrfTBQYbNjzivPTLQDtiUnhWuBDoonTNWBs9+keiKRIYzDfJ3M4tb2bL+O45818I1h 5Vsf358KuqiJUJjDGYNdgufKOoqUAoV/1baC7vDG1AHPgl1IlRn1bRj5+45UWjpFQhmw ofpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600102; x=1732204902; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8p+yJfYhOb0hWvZrgDQ+COQgpXoEivuXg56vfDbFxMU=; b=HOdDt71TwNh0uytU33D74xlbAVvRgjMBSY84LkSWjQm/iYOcCy5qTFW7zIV+7Ic1Op Pneu5VSyQ2pg7nTYDLv70Lyt8hT33ktn9+az1CxxpQcELPXsdzZgv19VQwzQ5785JdtI 6pCBwginatQRLcJ8LQFwBKkhYI0UCTR4Y5PXtKr1JmVNqSqiwN2edZx2HmIDINDYrqkl 60dcjPaDQIL/+KyvFIhBVBcsQDAijIFCcxma+KYfw8uIZDmvQme7xNeG0ghcyrcBLcso hKlsZxFvl4VkyPeA54r+ZXkl7Ri9SfokswdrMfQoAQQ9CG2R8R1uQBWzF5njN7pwMN1t O3PQ== X-Gm-Message-State: AOJu0YwsXl1TW+zddpnFlMhsmFCSx9TSSHamhL4sd3uHiqwTKUo1DNzK UPM1ini8fyZwtqqnpBdNLU3BBJtkl3hu1L1b4mNShyPLXUFAz5QT7mJAT1CXRwquk7W+1XVtw+G 7 X-Google-Smtp-Source: AGHT+IF+jprbCTqOgF4ottpXzA4kVHm5nA2aY5cLD3avsiOEsDtDYzt3Mjqc6TXOAMl/TajT0Xx8dA== X-Received: by 2002:a17:902:8f8c:b0:20c:bcd8:5ccb with SMTP id d9443c01a7336-21183d67984mr268991755ad.30.1731600101979; Thu, 14 Nov 2024 08:01:41 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:41 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 13/54] accel/tcg: Process IntervalTree entries in tlb_reset_dirty Date: Thu, 14 Nov 2024 08:00:49 -0800 Message-ID: <20241114160131.48616-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Update the addr_write copy within each interval tree node. Tidy the iteration within the other two loops as well. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index e2c855f147..0c9f834cbe 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1010,17 +1010,20 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) qemu_spin_lock(&cpu->neg.tlb.c.lock); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - unsigned int i; - unsigned int n = tlb_n_entries(&cpu->neg.tlb.f[mmu_idx]); + CPUTLBDesc *desc = &cpu->neg.tlb.d[mmu_idx]; + CPUTLBDescFast *fast = &cpu->neg.tlb.f[mmu_idx]; - for (i = 0; i < n; i++) { - tlb_reset_dirty_range_locked(&cpu->neg.tlb.f[mmu_idx].table[i], - start1, length); + for (size_t i = 0, n = tlb_n_entries(fast); i < n; i++) { + tlb_reset_dirty_range_locked(&fast->table[i], start1, length); } - for (i = 0; i < CPU_VTLB_SIZE; i++) { - tlb_reset_dirty_range_locked(&cpu->neg.tlb.d[mmu_idx].vtable[i], - start1, length); + for (size_t i = 0; i < CPU_VTLB_SIZE; i++) { + tlb_reset_dirty_range_locked(&desc->vtable[i], start1, length); + } + + for (CPUTLBEntryTree *t = tlbtree_lookup_range(desc, 0, -1); t; + t = tlbtree_lookup_range_next(t, 0, -1)) { + tlb_reset_dirty_range_locked(&t->copy, start1, length); } } qemu_spin_unlock(&cpu->neg.tlb.c.lock); From patchwork Thu Nov 14 16:00:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D48D7D68B33 for ; Thu, 14 Nov 2024 16:13:41 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIf-0005GS-Kb; Thu, 14 Nov 2024 11:02:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI5-0004zw-2m for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:56 -0500 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0002DZ-TS for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-72467c35ddeso751977b3a.0 for ; Thu, 14 Nov 2024 08:01:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600103; x=1732204903; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=E4pmRCXWy/4R+SRD3ehMnfQVgg8YaLPMO4JGcA4dSk8=; b=YU3qMoSb8fhIO3z6Bi1giY3GVMyh1JkLCYDKBz+q7RKVqMUph72B4i9dKVieXWCG3T X+h0p8QEFDPufOSNQLftQsKLVJUp2sP1Bk0xlQy+qMGy0ZZCrAJ7/YTNA+UmkgPllAMP iRMmZFb3PpgCEBw2IMizmVWEEoUbf3zbDuzpiK7Hy48v9RzfIWQCr0XAr9UYd2T+cbR4 kLgCkWnDMUIsExNDhio15v3PiQiavti8qVYivnf3sZsWcYTs8bqHHb5lqhcPVAiSY6Ru 91aQbE74zfF+8yVscdG4pOj00Hy3y0NqRdINa12EnmQWjoP+chG73ANwHHaZIQLCuhDV pSNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600103; x=1732204903; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E4pmRCXWy/4R+SRD3ehMnfQVgg8YaLPMO4JGcA4dSk8=; b=JOZZBgXFmrwuebcdg2ZtO80j5RD/BpZcAvkEUiihD4fJChU96rK7ukDyaB8mbNjw0T qxT8FJyfukYFGXB9PQI8nEN/9/zOKjQJZvTQh13eLpGtxW7me6DVuBA5Ovo1iVunarAw DFtbXGV6jBWmHoD8AR6mlikgU6MriYwhXp3HnYIKXHGxmNsKmnetSoksLAblgfGWHVtu UHD7Gnb9cqXCMDfj0k6Tvb9ActGNB49yblvw2rFWm+0QCjg6W/31ZYJ5j1L4BoOzReMI 1J/TD0QRywL8VLo/pOhGqHO0XJHA2EjdJHg+BdhFjZpfs5RrmVJUyDoVY+fJVPurlWfL 0KlQ== X-Gm-Message-State: AOJu0Yy8cdsVMb0mWohnPiKP+kBRx972qF5yqKbWJx0gv03yWYXCEpe6 ccB9VQX2Zl8DYjhj7lOpnk9vd87sQErzFKC7OtZ9gXGJXyTp2jf40PzAGMSiOl5h3U1Lmwy0Itx + X-Google-Smtp-Source: AGHT+IF7pTcW6gix9jbZ5CedeOcwDTDFruBmXkucISr2+sP4bwHmeWtl2vtFqCqf4Lx5c+cZETU5RQ== X-Received: by 2002:a17:90b:2801:b0:2e0:8780:ecb with SMTP id 98e67ed59e1d1-2e9fe649b2amr5717448a91.12.1731600103088; Thu, 14 Nov 2024 08:01:43 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:42 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 14/54] accel/tcg: Process IntervalTree entries in tlb_set_dirty Date: Thu, 14 Nov 2024 08:00:50 -0800 Message-ID: <20241114160131.48616-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Update the addr_write copy within an interval tree node. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 0c9f834cbe..eb85e96ee2 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1049,13 +1049,18 @@ static void tlb_set_dirty(CPUState *cpu, vaddr addr) addr &= TARGET_PAGE_MASK; qemu_spin_lock(&cpu->neg.tlb.c.lock); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - tlb_set_dirty1_locked(tlb_entry(cpu, mmu_idx, addr), addr); - } + CPUTLBDesc *desc = &cpu->neg.tlb.d[mmu_idx]; + CPUTLBEntryTree *node; - for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - int k; - for (k = 0; k < CPU_VTLB_SIZE; k++) { - tlb_set_dirty1_locked(&cpu->neg.tlb.d[mmu_idx].vtable[k], addr); + tlb_set_dirty1_locked(tlb_entry(cpu, mmu_idx, addr), addr); + + for (int k = 0; k < CPU_VTLB_SIZE; k++) { + tlb_set_dirty1_locked(&desc->vtable[k], addr); + } + + node = tlbtree_lookup_addr(desc, addr); + if (node) { + tlb_set_dirty1_locked(&node->copy, addr); } } qemu_spin_unlock(&cpu->neg.tlb.c.lock); From patchwork Thu Nov 14 16:00:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B101D68B33 for ; Thu, 14 Nov 2024 16:12:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIj-0005L9-9a; Thu, 14 Nov 2024 11:02:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004z5-Jz for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0002Db-GQ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:50 -0500 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-720aa3dbda5so600546b3a.1 for ; Thu, 14 Nov 2024 08:01:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600104; x=1732204904; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=CIfCBBS4NOF8aU4fkbSKUFc82IUgInZTkka8x32P+O8=; b=MxhnKw7Ycsd2Lz1o4fV3Te2gTmgA35JPkuYio81N7DSRFGZA8Nc6Xa9vzhqEg56Qw+ AM29PKMX+SwzPEwpUmahkZJxkmj2vhikmn+iQWVMm1T5s800EmwyziOhe2dV91hLZakc TNNxpOBDFmIp7UOOnakYkp6VJ40zXnfmDnCa2gdE8iXi/qR2nL0a3hfHG1Udw6gTlj1c VcpPvhzhQ2/wb+1cqUOV/uLfV1vt0+fRO8SAA2Qxpbh3cIXY5RSglEF8z04C5jAczGrr z/lV9BPwRNnZiYIDySJm72u23eeUBGIr1zChtRc7FNQgVXnk/XIPaiK+Ydx+26+yc59p FbKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600104; x=1732204904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CIfCBBS4NOF8aU4fkbSKUFc82IUgInZTkka8x32P+O8=; b=KK8kQ536REW9N1hJ/CLqPIPtJynknqaCk5b+yHEj/+uNO/Ucr2QWFmVWYaCgzH8WJv 5m+kXJx/3tzLm9uLYc8uoVXgyAn5bEUeCIp/edVsxiYUaZenWmogvMp7lPZ1Mmgwv4qI kII2MowWvKbxXZrY/pylHV/JdwJyzqjbbN2SFuMxFx5jS+P27FE2aKdpM8XUhcM2tzLf 7gYeewU2WC0JHDUBDXjyasJd/48QEFJpdkmwoCJscMzQNp0Ncuxb89h1aqV3xGErORHh f6dPf1aHaDlPMcgLIefzW7Ro0YhYcCrm0F0/Gi2Z+ntkEsGygE5740N9w0gRzPxEC7Og MWcw== X-Gm-Message-State: AOJu0YycVlgtXQIhBZLOJWo4PBOXdMzKI2RHdvcIEV9vOIjMQH7auRVH 2ZkOWwpysnmCgsqUznBgGlAa9ZWc8YSp1cu8z67TN5UJBG+mfyOnJtrIDhzxtaQ7yEIps/I40YQ + X-Google-Smtp-Source: AGHT+IFXgtuzAj+FLsRJNB5OYDKwFBpVXFWVzv6tNsTZOkil1C20oQhAsdnc9rvUvjaNpibRI0+eIg== X-Received: by 2002:a17:90b:2643:b0:2e9:20d8:4140 with SMTP id 98e67ed59e1d1-2e9e4c73ae4mr12373356a91.26.1731600103692; Thu, 14 Nov 2024 08:01:43 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:43 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 15/54] accel/tcg: Use tlb_hit_page in victim_tlb_hit Date: Thu, 14 Nov 2024 08:00:51 -0800 Message-ID: <20241114160131.48616-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is clearer than directly comparing the page address and the comparator. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index eb85e96ee2..7ecd327297 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1394,9 +1394,8 @@ static bool victim_tlb_hit(CPUState *cpu, size_t mmu_idx, size_t index, assert_cpu_is_self(cpu); for (vidx = 0; vidx < CPU_VTLB_SIZE; ++vidx) { CPUTLBEntry *vtlb = &cpu->neg.tlb.d[mmu_idx].vtable[vidx]; - uint64_t cmp = tlb_read_idx(vtlb, access_type); - if (cmp == page) { + if (tlb_hit_page(tlb_read_idx(vtlb, access_type), page)) { /* Found entry in victim tlb, swap tlb and iotlb. */ CPUTLBEntry tmptlb, *tlb = &cpu->neg.tlb.f[mmu_idx].table[index]; From patchwork Thu Nov 14 16:00:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EEBEAD68B34 for ; Thu, 14 Nov 2024 16:13:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ3-0005Ye-G1; Thu, 14 Nov 2024 11:02:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004z8-Mr for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcHz-0002Di-O9 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:50 -0500 Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-7f3da2c2cb5so599530a12.2 for ; Thu, 14 Nov 2024 08:01:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600104; x=1732204904; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=OAfTvQFTjKDkcAxt0Vto6R9Rpfvm8uCUUQSgIj7Y47Q=; b=IpdOpTmOclxCy1T6xFdK2s54qeIGfMEv+vgNHqFsC7Tp/x3jsHdno46cLLkxN8DWi2 xUl8tjWXteO7qBqw4kWX2dww/7TwbK9EFThNRC+mdApAIgWXmlkKInNXbj886hL9y7XI YSFGNkeCe1FEqaKdCRUn6PxbaAeoRMixo7a4uQ389hn75C4uKpfHoJFiqjtxW8tpKcgb Uj+7YOHyHoYfUNf+JdAgvtCxzsz9RBxt2PlrL9A+5G80yDmfYG6eLQlBGfvTfsAcrh9T qOanoCPcAQjpQ2hkRNMvMXWy+7zNtTZBID2gJsUCawbWlQczp3puoDncbEax8OvG28ij jAWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600104; x=1732204904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OAfTvQFTjKDkcAxt0Vto6R9Rpfvm8uCUUQSgIj7Y47Q=; b=bml73z9o+mvbU94PIUr4oXB7pnyZ2qWIKtMALgWRywTFqh3Hzw/xTCgSEF+GsWLeAV s0MpFRyC0AsR+W8hvvgBFAgGzPSeY+pWKgH2xSIjG+qqNV41Ilf//v/shNvmeNM8aDbq z74fiPFjZFV9CjFaR9NcTbGHE3OsCz17XxMGEI6YgGFJ0QAwcAjUFEFxfDoKZU/KmH/W kFn5uQvi4x46g+oFxxtdqVeuSVtaRGaK3YPjGgEOc+QcoEen27EbN+bLlYHvz3qSL+e7 Tlr6LDdVID9amtM1zpqyUITyZxOk5/9pMe9gekad9upk2Qq2kMq0jjID0fz5I4A+B4/a MUQw== X-Gm-Message-State: AOJu0YwjPggoKVoLTsmHrfNBNpdOF4HPB8oXNT4ft5MhHgV1VjL4trTA wC9oyPIfK+2Zu9sTzPmAEUpKhJMTvgApZAonjLU1x0Kt5jkNKFwzQsm/n4PmElVNPFwKEC9Z1nI f X-Google-Smtp-Source: AGHT+IEOa93NnKnA/1J0YX+utX824qtojXl7jMeiG9CZxZM2GsxmTSwtBWUaN/IGBsyoQKPQ09eaHw== X-Received: by 2002:a17:90b:4c06:b0:2e2:cd11:c9b with SMTP id 98e67ed59e1d1-2e9b16eb849mr30900735a91.3.1731600104353; Thu, 14 Nov 2024 08:01:44 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:44 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 16/54] accel/tcg: Pass full addr to victim_tlb_hit Date: Thu, 14 Nov 2024 08:00:52 -0800 Message-ID: <20241114160131.48616-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Do not mask the address to the page in these calls. It is easy enough to use a different helper instead. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 7ecd327297..3aab72ea82 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1387,7 +1387,7 @@ static void io_failed(CPUState *cpu, CPUTLBEntryFull *full, vaddr addr, /* Return true if ADDR is present in the victim tlb, and has been copied back to the main tlb. */ static bool victim_tlb_hit(CPUState *cpu, size_t mmu_idx, size_t index, - MMUAccessType access_type, vaddr page) + MMUAccessType access_type, vaddr addr) { size_t vidx; @@ -1395,7 +1395,7 @@ static bool victim_tlb_hit(CPUState *cpu, size_t mmu_idx, size_t index, for (vidx = 0; vidx < CPU_VTLB_SIZE; ++vidx) { CPUTLBEntry *vtlb = &cpu->neg.tlb.d[mmu_idx].vtable[vidx]; - if (tlb_hit_page(tlb_read_idx(vtlb, access_type), page)) { + if (tlb_hit(tlb_read_idx(vtlb, access_type), addr)) { /* Found entry in victim tlb, swap tlb and iotlb. */ CPUTLBEntry tmptlb, *tlb = &cpu->neg.tlb.f[mmu_idx].table[index]; @@ -1448,13 +1448,12 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, uintptr_t index = tlb_index(cpu, mmu_idx, addr); CPUTLBEntry *entry = tlb_entry(cpu, mmu_idx, addr); uint64_t tlb_addr = tlb_read_idx(entry, access_type); - vaddr page_addr = addr & TARGET_PAGE_MASK; int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; bool force_mmio = check_mem_cbs && cpu_plugin_mem_cbs_enabled(cpu); CPUTLBEntryFull *full; - if (!tlb_hit_page(tlb_addr, page_addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, page_addr)) { + if (!tlb_hit(tlb_addr, addr)) { + if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, addr)) { if (!tlb_fill_align(cpu, addr, access_type, mmu_idx, 0, fault_size, nonfault, retaddr)) { /* Non-faulting page table read failed. */ @@ -1734,8 +1733,7 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, /* If the TLB entry is for a different page, reload and try again. */ if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, - addr & TARGET_PAGE_MASK)) { + if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, addr)) { tlb_fill_align(cpu, addr, access_type, mmu_idx, memop, data->size, false, ra); maybe_resized = true; @@ -1914,8 +1912,7 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, /* Check TLB entry and enforce page permissions. */ flags = TLB_FLAGS_MASK; if (!tlb_hit(tlb_addr_write(tlbe), addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, MMU_DATA_STORE, - addr & TARGET_PAGE_MASK)) { + if (!victim_tlb_hit(cpu, mmu_idx, index, MMU_DATA_STORE, addr)) { tlb_fill_align(cpu, addr, MMU_DATA_STORE, mmu_idx, mop, size, false, retaddr); did_tlb_fill = true; From patchwork Thu Nov 14 16:00:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEEE8D68B33 for ; Thu, 14 Nov 2024 16:12:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJU-0006JX-0C; Thu, 14 Nov 2024 11:03:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004zA-NZ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0002Dp-OO for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-72410cc7be9so731796b3a.0 for ; Thu, 14 Nov 2024 08:01:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600105; x=1732204905; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=qWIXUHGQnTcdkm072yvGDVIoDbE1775WHviaFUbmmuU=; b=ovA8eWyTCJLl8iHhUBRmn93o83Eq2I4oEyHSNgLisnbg3WffO2OIvvhWs0N3LEZGcp +nntnn0adTEZ1UoPuh3EP9n+HgZ+scVffr+hGbkCuBijx4B6UMbyoxLvlRvGtzXIt0e5 YEHa/YR46Newt+2wdiI7pUslJSbH31ELw9qbZDE2H/oXnlOGswS9mXvI3Hu0EB6xxtLG OLXjMOngFlX/yDIJLgB+8DVY2ncqzmYI5qmS7ceS8u1rsUu8qrrG75GHC73JPrHx4lZU 5AuSl6T5+A5BQh+ZOPH84Vt6KzNl/N1ytmZ0hrhkVVWPf4CeN6FC4YaQaAYEsFRShiro 5QBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600105; x=1732204905; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qWIXUHGQnTcdkm072yvGDVIoDbE1775WHviaFUbmmuU=; b=TYFb/kSYtvymoFeF2WCQaUs+a4z7v/qKFlkI+rOEDAEGzJVUN16rfX72XXZLB1erIE DCyt01bcRUegcflaPuqd6xxmNpwsC2UQJvO15dgOzGVdrxAUaTjmiFmDk8E5TY6ucmZH aR9r6LGZHwdtXt60Zg+ktVyb1sVHhdqmHhiM2vQ79be62MyV49fAks70M+3RbM3AmOW6 T0tTZcMQEzTzxZo9MD8iHTSuSYUr7T8wd8wA9q+7i2OON9F39GnbLf3KXCepNo/Pb443 n5T85nP8mAlBPTpChubC2Hu+lLSIqYfKdv5zYTq5mjZesYCkMKbV7jQiVOohUEK0XyJB 8RfA== X-Gm-Message-State: AOJu0YwG5JBPcDRFwKfSqnnr22rgv5/rUMA4pGI1anBzS5QNvohEMCxQ NtpFJumlOczIo4z9GvD2XqMzpcPI1f7jQ6uNAS09faYIGlnVEMZpmurv4aRawKEW0Uhc1plEiwT / X-Google-Smtp-Source: AGHT+IHPvPz14MAvO19icCcoybrM8ZtKp3yBE5tx84N/3/o3EC5HVEijtTtZIi2335BwNZMR67Pj6A== X-Received: by 2002:a17:90b:3883:b0:2e2:d3e1:f863 with SMTP id 98e67ed59e1d1-2e9b171e360mr33853299a91.12.1731600105146; Thu, 14 Nov 2024 08:01:45 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.44 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:44 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 17/54] accel/tcg: Replace victim_tlb_hit with tlbtree_hit Date: Thu, 14 Nov 2024 08:00:53 -0800 Message-ID: <20241114160131.48616-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Change from a linear search on the victim tlb to a balanced binary tree search on the interval tree. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 59 ++++++++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 3aab72ea82..ea4b78866b 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1384,35 +1384,38 @@ static void io_failed(CPUState *cpu, CPUTLBEntryFull *full, vaddr addr, } } -/* Return true if ADDR is present in the victim tlb, and has been copied - back to the main tlb. */ -static bool victim_tlb_hit(CPUState *cpu, size_t mmu_idx, size_t index, - MMUAccessType access_type, vaddr addr) +/* + * Return true if ADDR is present in the interval tree, + * and has been copied back to the main tlb. + */ +static bool tlbtree_hit(CPUState *cpu, int mmu_idx, + MMUAccessType access_type, vaddr addr) { - size_t vidx; + CPUTLBDesc *desc = &cpu->neg.tlb.d[mmu_idx]; + CPUTLBDescFast *fast = &cpu->neg.tlb.f[mmu_idx]; + CPUTLBEntryTree *node; + size_t index; assert_cpu_is_self(cpu); - for (vidx = 0; vidx < CPU_VTLB_SIZE; ++vidx) { - CPUTLBEntry *vtlb = &cpu->neg.tlb.d[mmu_idx].vtable[vidx]; - - if (tlb_hit(tlb_read_idx(vtlb, access_type), addr)) { - /* Found entry in victim tlb, swap tlb and iotlb. */ - CPUTLBEntry tmptlb, *tlb = &cpu->neg.tlb.f[mmu_idx].table[index]; - - qemu_spin_lock(&cpu->neg.tlb.c.lock); - copy_tlb_helper_locked(&tmptlb, tlb); - copy_tlb_helper_locked(tlb, vtlb); - copy_tlb_helper_locked(vtlb, &tmptlb); - qemu_spin_unlock(&cpu->neg.tlb.c.lock); - - CPUTLBEntryFull *f1 = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; - CPUTLBEntryFull *f2 = &cpu->neg.tlb.d[mmu_idx].vfulltlb[vidx]; - CPUTLBEntryFull tmpf; - tmpf = *f1; *f1 = *f2; *f2 = tmpf; - return true; - } + node = tlbtree_lookup_addr(desc, addr); + if (!node) { + /* There is no cached mapping for this page. */ + return false; } - return false; + + if (!tlb_hit(tlb_read_idx(&node->copy, access_type), addr)) { + /* This access is not permitted. */ + return false; + } + + /* Install the cached entry. */ + index = tlbfast_index(fast, addr); + qemu_spin_lock(&cpu->neg.tlb.c.lock); + copy_tlb_helper_locked(&fast->table[index], &node->copy); + qemu_spin_unlock(&cpu->neg.tlb.c.lock); + + desc->fulltlb[index] = node->full; + return true; } static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size, @@ -1453,7 +1456,7 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, CPUTLBEntryFull *full; if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, addr)) { + if (!tlbtree_hit(cpu, mmu_idx, access_type, addr)) { if (!tlb_fill_align(cpu, addr, access_type, mmu_idx, 0, fault_size, nonfault, retaddr)) { /* Non-faulting page table read failed. */ @@ -1733,7 +1736,7 @@ static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, /* If the TLB entry is for a different page, reload and try again. */ if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, access_type, addr)) { + if (!tlbtree_hit(cpu, mmu_idx, access_type, addr)) { tlb_fill_align(cpu, addr, access_type, mmu_idx, memop, data->size, false, ra); maybe_resized = true; @@ -1912,7 +1915,7 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, /* Check TLB entry and enforce page permissions. */ flags = TLB_FLAGS_MASK; if (!tlb_hit(tlb_addr_write(tlbe), addr)) { - if (!victim_tlb_hit(cpu, mmu_idx, index, MMU_DATA_STORE, addr)) { + if (!tlbtree_hit(cpu, mmu_idx, MMU_DATA_STORE, addr)) { tlb_fill_align(cpu, addr, MMU_DATA_STORE, mmu_idx, mop, size, false, retaddr); did_tlb_fill = true; From patchwork Thu Nov 14 16:00:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13C71D68B33 for ; Thu, 14 Nov 2024 16:03:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJI-0005yK-V8; Thu, 14 Nov 2024 11:03:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI5-0004zx-GD for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:56 -0500 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0002E6-TE for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-71e5130832aso620755b3a.0 for ; Thu, 14 Nov 2024 08:01:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600106; x=1732204906; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=3Tsi6XDXpoiCgHSBmhIafrnzhnOLjjswRzo90Xc17TQ=; b=ntbMaoUrkaLnXBdMso4jRlo0CS6oDfDZuJRG97WW+cfp/6OCcUkGK7E0QkJ+u5sfVd Rcth8mpqkqLOb5W+gPof2u5+JfAyCKM+/Dq2mu1Ea9wHCLAi/qTMUnn3sR01IwvHSyQC csh6ErcCztrQCQaXOtBai4SmN7Mu+FxduTnseCMwNuU0jSUmWP1ySi7rIFxue0keVgPI 0oD+nTGFx0fq7WIx1qJfCZLMlOeaot9Ojgp4AooluyUSV5InXGxpeRIUNEQZI8oIGFr9 hqt/UjYLJ9tTKFsPp8p/GTiRSa3V+LXRGY+wp0PtpYATrJGM3mZBLG15wyk8hzLX5doP PiZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600106; x=1732204906; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Tsi6XDXpoiCgHSBmhIafrnzhnOLjjswRzo90Xc17TQ=; b=WeOQ3YAzjJ4pUxMzveXnCkveRGdkahiB6hTDQQv+rHvtDbNhFnuZAs3v9JU8QRvqNO GDF+i59xwHcrwoj/DWQzU1SOnxpcZQCYuZloha7dBmKXjKndvlKStCBzeDLpEC8nm4t/ QxL4nSaOxAi+CJfzjvMQ9WJwO6dxc1PgwiExc9IYPU5n3nGTpi37BMbp4A+k6YEpHmY/ p1PqsCzlDyRySXJ4NTTbCyERaHjUXsEWwKxV+kPuHWuH/MgY+Vz7+ETAOlvP0tJFhU6Z 0HsJdvsbl6z8OLjPddx9aLrYsicCaQUl0YNhbgk9xArHmQLGETCDKIlSGW0rNNY8zuPG +OFQ== X-Gm-Message-State: AOJu0YyanLo11wIu8a2BAg8VAy7BbSyYsCMF1vQKZfgJZyDhOOGrAjL/ tbdfSTw3KXTZuWwPKgLHDCk3xPwGqbmiP3GrRhifOYqE+647xAAhw4QgLDr9v9qWKxRK7jb4bDf n X-Google-Smtp-Source: AGHT+IFgqgc1AF2jPV/QhIO0BEvsoj+GnOyd4MyxiGFsZdNwzmSho2dBI3ACk8Z6JUPgWAA4Zj1XLg== X-Received: by 2002:a17:90b:4f47:b0:2e2:af52:a7b7 with SMTP id 98e67ed59e1d1-2ea06ab3892mr2496657a91.34.1731600105864; Thu, 14 Nov 2024 08:01:45 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:45 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 18/54] accel/tcg: Remove the victim tlb Date: Thu, 14 Nov 2024 08:00:54 -0800 Message-ID: <20241114160131.48616-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This has been functionally replaced by the IntervalTree. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/hw/core/cpu.h | 8 ----- accel/tcg/cputlb.c | 74 ------------------------------------------- 2 files changed, 82 deletions(-) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index 1ebc999a73..8eda0574b2 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -201,9 +201,6 @@ struct CPUClass { */ #define NB_MMU_MODES 16 -/* Use a fully associative victim tlb of 8 entries. */ -#define CPU_VTLB_SIZE 8 - /* * The full TLB entry, which is not accessed by generated TCG code, * so the layout is not as critical as that of CPUTLBEntry. This is @@ -285,11 +282,6 @@ typedef struct CPUTLBDesc { /* maximum number of entries observed in the window */ size_t window_max_entries; size_t n_used_entries; - /* The next index to use in the tlb victim table. */ - size_t vindex; - /* The tlb victim table, in two parts. */ - CPUTLBEntry vtable[CPU_VTLB_SIZE]; - CPUTLBEntryFull vfulltlb[CPU_VTLB_SIZE]; CPUTLBEntryFull *fulltlb; /* All active tlb entries for this address space. */ IntervalTreeRoot iroot; diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ea4b78866b..8caa8c0f1d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -328,8 +328,6 @@ static void tlb_mmu_flush_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast) tlbfast_flush_locked(desc, fast); desc->large_page_addr = -1; desc->large_page_mask = -1; - desc->vindex = 0; - memset(desc->vtable, -1, sizeof(desc->vtable)); interval_tree_free_nodes(&desc->iroot, offsetof(CPUTLBEntryTree, itree)); } @@ -361,11 +359,6 @@ static inline void tlb_n_used_entries_inc(CPUState *cpu, uintptr_t mmu_idx) cpu->neg.tlb.d[mmu_idx].n_used_entries++; } -static inline void tlb_n_used_entries_dec(CPUState *cpu, uintptr_t mmu_idx) -{ - cpu->neg.tlb.d[mmu_idx].n_used_entries--; -} - void tlb_init(CPUState *cpu) { int64_t now = get_clock_realtime(); @@ -496,20 +489,6 @@ static bool tlb_hit_page_mask_anyprot(CPUTLBEntry *tlb_entry, page == (tlb_entry->addr_code & mask)); } -static inline bool tlb_hit_page_anyprot(CPUTLBEntry *tlb_entry, vaddr page) -{ - return tlb_hit_page_mask_anyprot(tlb_entry, page, -1); -} - -/** - * tlb_entry_is_empty - return true if the entry is not in use - * @te: pointer to CPUTLBEntry - */ -static inline bool tlb_entry_is_empty(const CPUTLBEntry *te) -{ - return te->addr_read == -1 && te->addr_write == -1 && te->addr_code == -1; -} - /* Called with tlb_c.lock held */ static bool tlb_flush_entry_mask_locked(CPUTLBEntry *tlb_entry, vaddr page, @@ -522,28 +501,6 @@ static bool tlb_flush_entry_mask_locked(CPUTLBEntry *tlb_entry, return false; } -/* Called with tlb_c.lock held */ -static void tlb_flush_vtlb_page_mask_locked(CPUState *cpu, int mmu_idx, - vaddr page, - vaddr mask) -{ - CPUTLBDesc *d = &cpu->neg.tlb.d[mmu_idx]; - int k; - - assert_cpu_is_self(cpu); - for (k = 0; k < CPU_VTLB_SIZE; k++) { - if (tlb_flush_entry_mask_locked(&d->vtable[k], page, mask)) { - tlb_n_used_entries_dec(cpu, mmu_idx); - } - } -} - -static inline void tlb_flush_vtlb_page_locked(CPUState *cpu, int mmu_idx, - vaddr page) -{ - tlb_flush_vtlb_page_mask_locked(cpu, mmu_idx, page, -1); -} - static void tlbfast_flush_range_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast, vaddr addr, vaddr len, vaddr mask) { @@ -588,7 +545,6 @@ static void tlb_flush_page_locked(CPUState *cpu, int midx, vaddr page) tlbfast_flush_range_locked(desc, &cpu->neg.tlb.f[midx], page, TARGET_PAGE_SIZE, -1); - tlb_flush_vtlb_page_locked(cpu, midx, page); node = tlbtree_lookup_addr(desc, page); if (node) { @@ -764,11 +720,6 @@ static void tlb_flush_range_locked(CPUState *cpu, int midx, tlbfast_flush_range_locked(d, f, addr, len, mask); - for (vaddr i = 0; i < len; i += TARGET_PAGE_SIZE) { - vaddr page = addr + i; - tlb_flush_vtlb_page_mask_locked(cpu, midx, page, mask); - } - addr_mask = addr & mask; last_mask = addr_mask + len - 1; last_imask = last_mask | ~mask; @@ -1017,10 +968,6 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) tlb_reset_dirty_range_locked(&fast->table[i], start1, length); } - for (size_t i = 0; i < CPU_VTLB_SIZE; i++) { - tlb_reset_dirty_range_locked(&desc->vtable[i], start1, length); - } - for (CPUTLBEntryTree *t = tlbtree_lookup_range(desc, 0, -1); t; t = tlbtree_lookup_range_next(t, 0, -1)) { tlb_reset_dirty_range_locked(&t->copy, start1, length); @@ -1054,10 +1001,6 @@ static void tlb_set_dirty(CPUState *cpu, vaddr addr) tlb_set_dirty1_locked(tlb_entry(cpu, mmu_idx, addr), addr); - for (int k = 0; k < CPU_VTLB_SIZE; k++) { - tlb_set_dirty1_locked(&desc->vtable[k], addr); - } - node = tlbtree_lookup_addr(desc, addr); if (node) { tlb_set_dirty1_locked(&node->copy, addr); @@ -1216,23 +1159,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, /* Note that the tlb is no longer clean. */ tlb->c.dirty |= 1 << mmu_idx; - /* Make sure there's no cached translation for the new page. */ - tlb_flush_vtlb_page_locked(cpu, mmu_idx, addr_page); - - /* - * Only evict the old entry to the victim tlb if it's for a - * different page; otherwise just overwrite the stale data. - */ - if (!tlb_hit_page_anyprot(te, addr_page) && !tlb_entry_is_empty(te)) { - unsigned vidx = desc->vindex++ % CPU_VTLB_SIZE; - CPUTLBEntry *tv = &desc->vtable[vidx]; - - /* Evict the old entry into the victim tlb. */ - copy_tlb_helper_locked(tv, te); - desc->vfulltlb[vidx] = desc->fulltlb[index]; - tlb_n_used_entries_dec(cpu, mmu_idx); - } - /* Replace an old IntervalTree entry, or create a new one. */ node = tlbtree_lookup_addr(desc, addr_page); if (!node) { From patchwork Thu Nov 14 16:00:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A1A87D68B33 for ; Thu, 14 Nov 2024 16:02:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIR-00055D-Ex; Thu, 14 Nov 2024 11:02:16 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0004z7-MT for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:51 -0500 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI0-0002EC-OV for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:50 -0500 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2e91403950dso648078a91.3 for ; Thu, 14 Nov 2024 08:01:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600107; x=1732204907; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=aH/YSb9/f+Pg6UolkLA4sQEI53j6GiG7lP5Aqm9cSsA=; b=RT+Kl5faUclTlmPrp/awEHRtcu/yrF983B7fB5+W83ioqoDb0x3OvNFbBwlv1NoWXX 9ShF4/255jkvHPeDVYkEuKj0e9qUpMmhm9FDSuFeN5OEzTRI+QdDT+Ak5CxFbXw1/EEU d/qQz2yIFWDm+xW2z7b/F9LLLBkSROXsFFGdkmImo//GibhkqV7wz34Y1qWqnC5+ccM1 OYgW/NLI7HXXZek/++/5vBdW1WoH3V5CoDIxX1YR85jLcJaEiwNaGoiTZMswaI0uGvpi EQ9yuNOoSU2b1ZURL8k/CyNAY5PwKxX+5Pn+TgzTWEIrHqNZ33tfVnrjBjLnchFTAlpn m4LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600107; x=1732204907; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aH/YSb9/f+Pg6UolkLA4sQEI53j6GiG7lP5Aqm9cSsA=; b=cdOaYb/L1VRnmID6WixNuSPUPfSu++Fz+JIQhxbqmVaxDjnMNm3q9ORfVamUj3YJNG fUxU/bmGHycyAtAftYJgLIkXfipN8GqGmAf4qFAKXKXc5AK9abChiK6yAp6q6T7HUYh3 FpQDUtdjmPPZyeejtIB4eO2/Agyvm9JStnmgSwDiRiyLAxn0z0qgSxeNmEUE24Q1xPUs hJmsoptEgFHnHrQzC/xVIUjk5yAq8Er8ZMf6WNnPHxDNC+4IWisiSdMny7jNVBtQMBxC jm/L++eCRKr6E89/9OV0J9KxG96afLSC9ZAU8ynOHAhf4z1UbSSAtaw+lG+rLMyUKqQM 1nJA== X-Gm-Message-State: AOJu0YwCHPhRr/PO78ub7QJhsbKi7yAsT+mk/6JkKeblOYTNpAf2rgCg 5eXmbUUMT73Xrx4+7KH7n7RZstJ4dtEL1S0wo/lfUbEKqFG5NrpMeLCEFn1dqp+0Ghy45+3Hukl H X-Google-Smtp-Source: AGHT+IGO50T2+G3nh2qLqmoY9c/fFez6dpmtOhk4e1eGoW34d3YRYyRV916fAdqex1sAC1p3l4DIKA== X-Received: by 2002:a17:90b:4c06:b0:2e2:cd11:c9b with SMTP id 98e67ed59e1d1-2e9b16eb849mr30900909a91.3.1731600106566; Thu, 14 Nov 2024 08:01:46 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.45 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:46 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 19/54] accel/tcg: Remove tlb_n_used_entries_inc Date: Thu, 14 Nov 2024 08:00:55 -0800 Message-ID: <20241114160131.48616-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Expand the function into its only caller, using the existing CPUTLBDesc local pointer. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 8caa8c0f1d..3e24529f4f 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -354,11 +354,6 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now) tlb_mmu_flush_locked(desc, fast); } -static inline void tlb_n_used_entries_inc(CPUState *cpu, uintptr_t mmu_idx) -{ - cpu->neg.tlb.d[mmu_idx].n_used_entries++; -} - void tlb_init(CPUState *cpu) { int64_t now = get_clock_realtime(); @@ -1211,7 +1206,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, node->full = *full; copy_tlb_helper_locked(te, &node->copy); - tlb_n_used_entries_inc(cpu, mmu_idx); + desc->n_used_entries++; qemu_spin_unlock(&tlb->c.lock); } From patchwork Thu Nov 14 16:00:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0B75D68B33 for ; Thu, 14 Nov 2024 16:03:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ8-0005m6-Uc; Thu, 14 Nov 2024 11:02:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcI5-0004zy-IZ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:56 -0500 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI3-0002F4-AO for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:52 -0500 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2e3d523a24dso687271a91.0 for ; Thu, 14 Nov 2024 08:01:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600109; x=1732204909; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=q3hq8/WORX9tjGeTrB15u911iYQPv1dQgSnGBISFTlA=; b=Cmlz0NUQdtgnoOegqJSXWKxJwjpM6H8bkqcetUsacOznDI5tYiS5QAMjHnjulV/Boo schnG8YS1m/QVMe6fAOQvjOXgo2PSvHVkSdhbcWJAXbVk0znJB9jjFi3TuLxiH0uXTUK ykPlO9Iz3XQpF/hoTvXREIgkZS5YS2XbCCKDAE84yWLcdHgEtV1JHIVzyyDNNuuQxJ41 zjMotGN6ytYBqCgU//uQwVJbF+/emmQaE99minwO+E7Lp1hgoJpxSkw9gAYNjsyVMwk7 qvLVKWmHbvvNOykIwLPZHqUAZIXy5wWmfkZwgx54BdE4egUoIIPq6ShthM3Yc5mf0dNR ttSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600109; x=1732204909; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q3hq8/WORX9tjGeTrB15u911iYQPv1dQgSnGBISFTlA=; b=e4/lOS30re3zBXimCJlJ6vy0xJhehssd6v/tEv3wTrLcUI3nEGRYhxqTDJG2RsVajz eR71vc9I3h57xwcOI+g0ui/ULwvd2ycJm+rtlbarwv2SXtposDsQ8Dk/f10Org2i0o1A 5X/6VTVBPP8UkOSZCjCo5OPxkSKjTlb52OES5RZM31WsApei9RNl0P+tiJ+h1HcgrUX6 nwCVZK52RTpFi9gBPJ6F4YAfgLoOvr93vD4NhRtKAmAJf26JVCxUKUU3lLrwV0aE2NmF fCd4X+0QP/ReW2hCHiq9jmiaEhnlTmltLZUaykpzNDZvYb824L0sFxYgTUNUZaVDbdr/ Mxdg== X-Gm-Message-State: AOJu0YxG/fKtYUz8sQqyD4xHtpyf7k3vIfsAl76ml0bxkUsCWltEABo1 O3FxmluBeJiSL1OgejTawo76+S7S9W/5PeGSNkEhTav7oyAsp41e90ZNYhLMnlE0tjSEdbpm22M + X-Google-Smtp-Source: AGHT+IFtPIHgC6GpjZH9+1GsaXgTn2s1n/5wfBLUc1GdUUBRC2fxmiLGTPdJ65XmtPZb/7yBspHvrg== X-Received: by 2002:a17:90b:4acb:b0:2e2:d5fc:2847 with SMTP id 98e67ed59e1d1-2e9b177fe40mr32056117a91.30.1731600107271; Thu, 14 Nov 2024 08:01:47 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.46 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:46 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 20/54] include/exec/tlb-common: Move CPUTLBEntryFull from hw/core/cpu.h Date: Thu, 14 Nov 2024 08:00:56 -0800 Message-ID: <20241114160131.48616-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org CPUTLBEntryFull structures are no longer directly included within the CPUState structure. Move the structure definition out of cpu.h to reduce visibility. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/tlb-common.h | 63 +++++++++++++++++++++++++++++++++++++++ include/hw/core/cpu.h | 63 --------------------------------------- 2 files changed, 63 insertions(+), 63 deletions(-) diff --git a/include/exec/tlb-common.h b/include/exec/tlb-common.h index dc5a5faa0b..300f9fae67 100644 --- a/include/exec/tlb-common.h +++ b/include/exec/tlb-common.h @@ -53,4 +53,67 @@ typedef struct CPUTLBDescFast { CPUTLBEntry *table; } CPUTLBDescFast QEMU_ALIGNED(2 * sizeof(void *)); +/* + * The full TLB entry, which is not accessed by generated TCG code, + * so the layout is not as critical as that of CPUTLBEntry. This is + * also why we don't want to combine the two structs. + */ +struct CPUTLBEntryFull { + /* + * @xlat_section contains: + * - in the lower TARGET_PAGE_BITS, a physical section number + * - with the lower TARGET_PAGE_BITS masked off, an offset which + * must be added to the virtual address to obtain: + * + the ram_addr_t of the target RAM (if the physical section + * number is PHYS_SECTION_NOTDIRTY or PHYS_SECTION_ROM) + * + the offset within the target MemoryRegion (otherwise) + */ + hwaddr xlat_section; + + /* + * @phys_addr contains the physical address in the address space + * given by cpu_asidx_from_attrs(cpu, @attrs). + */ + hwaddr phys_addr; + + /* @attrs contains the memory transaction attributes for the page. */ + MemTxAttrs attrs; + + /* @prot contains the complete protections for the page. */ + uint8_t prot; + + /* @lg_page_size contains the log2 of the page size. */ + uint8_t lg_page_size; + + /* Additional tlb flags requested by tlb_fill. */ + uint8_t tlb_fill_flags; + + /* + * Additional tlb flags for use by the slow path. If non-zero, + * the corresponding CPUTLBEntry comparator must have TLB_FORCE_SLOW. + */ + uint8_t slow_flags[MMU_ACCESS_COUNT]; + + /* + * Allow target-specific additions to this structure. + * This may be used to cache items from the guest cpu + * page tables for later use by the implementation. + */ + union { + /* + * Cache the attrs and shareability fields from the page table entry. + * + * For ARMMMUIdx_Stage2*, pte_attrs is the S2 descriptor bits [5:2]. + * Otherwise, pte_attrs is the same as the MAIR_EL1 8-bit format. + * For shareability and guarded, as in the SH and GP fields respectively + * of the VMSAv8-64 PTEs. + */ + struct { + uint8_t pte_attrs; + uint8_t shareability; + bool guarded; + } arm; + } extra; +}; + #endif /* EXEC_TLB_COMMON_H */ diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index 8eda0574b2..4364ddb1db 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -201,69 +201,6 @@ struct CPUClass { */ #define NB_MMU_MODES 16 -/* - * The full TLB entry, which is not accessed by generated TCG code, - * so the layout is not as critical as that of CPUTLBEntry. This is - * also why we don't want to combine the two structs. - */ -struct CPUTLBEntryFull { - /* - * @xlat_section contains: - * - in the lower TARGET_PAGE_BITS, a physical section number - * - with the lower TARGET_PAGE_BITS masked off, an offset which - * must be added to the virtual address to obtain: - * + the ram_addr_t of the target RAM (if the physical section - * number is PHYS_SECTION_NOTDIRTY or PHYS_SECTION_ROM) - * + the offset within the target MemoryRegion (otherwise) - */ - hwaddr xlat_section; - - /* - * @phys_addr contains the physical address in the address space - * given by cpu_asidx_from_attrs(cpu, @attrs). - */ - hwaddr phys_addr; - - /* @attrs contains the memory transaction attributes for the page. */ - MemTxAttrs attrs; - - /* @prot contains the complete protections for the page. */ - uint8_t prot; - - /* @lg_page_size contains the log2 of the page size. */ - uint8_t lg_page_size; - - /* Additional tlb flags requested by tlb_fill. */ - uint8_t tlb_fill_flags; - - /* - * Additional tlb flags for use by the slow path. If non-zero, - * the corresponding CPUTLBEntry comparator must have TLB_FORCE_SLOW. - */ - uint8_t slow_flags[MMU_ACCESS_COUNT]; - - /* - * Allow target-specific additions to this structure. - * This may be used to cache items from the guest cpu - * page tables for later use by the implementation. - */ - union { - /* - * Cache the attrs and shareability fields from the page table entry. - * - * For ARMMMUIdx_Stage2*, pte_attrs is the S2 descriptor bits [5:2]. - * Otherwise, pte_attrs is the same as the MAIR_EL1 8-bit format. - * For shareability and guarded, as in the SH and GP fields respectively - * of the VMSAv8-64 PTEs. - */ - struct { - uint8_t pte_attrs; - uint8_t shareability; - bool guarded; - } arm; - } extra; -}; - /* * Data elements that are per MMU mode, minus the bits accessed by * the TCG fast path. From patchwork Thu Nov 14 16:00:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FC5DD68B34 for ; Thu, 14 Nov 2024 16:13:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIf-0005FD-C7; Thu, 14 Nov 2024 11:02:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcID-00050p-Rs for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:05 -0500 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI4-0002FH-Qk for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:54 -0500 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2e2fb304e7dso729060a91.1 for ; Thu, 14 Nov 2024 08:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600110; x=1732204910; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=lJHKnfJ9AOt1s0DhksJDYAiN+sOc9T9tuOMIGs7rfV8=; b=eHGfoqH87XZL3ZHbSDaWpDHHsUIg9j75KUYhfgnMO6a94XefjKiHjxxismqoUh30dI 7ypbwKhnJrwqcg3/slu982nntlvqpBebIE0EaJiMywfdETCO8/285ijCzDTD1SXfKdU7 AW9iGiPAYb5+GRuNCV0BSuJIcE9EsUBL6+l+4f0x9LXPd719ZLA8od3TGxo1JPI5wkTC 4UhRO8ChTWSORpjDQ2iigaVh97CHqvNiHuejG+pcuh6JMwBwBfuN2GsEAdnKxmvDJ93f 89pco8hHWafazJrcV/bO/mjR0cm0CNo/fQ7yciyCFuw9dlqmO/7XnRXb2RqWQE8UXSsX opyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600110; x=1732204910; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lJHKnfJ9AOt1s0DhksJDYAiN+sOc9T9tuOMIGs7rfV8=; b=b5hxAgIri/DRzhiY0knkYNVSNnn/GV+46Y0feyGY67JGCWsxrZwBerBOKA547zJ8QY 73cLKz0vc2h1fRfZFQ4q6A6gXWPD0xZKA2W74ZdJQag1gXSozib/QPDL11Ke7HNnh7Gk J0zZtPaWci56nOlERYRUHXZQ+J7mXiYI4GRSXmDmIf3hfyNF3fXnyyO5mp64PfdjoXkX rnW/PcmlFzRXYTnK4wemVZaxt8XB2Rl5RcXu2PgQasGa5zAS/ocS2lUbZdQSRMkQb/Ae w3j5Uu0mKgmaiNDmAToKWGymZhZinyhwkN2bS2Ypoat2aaC+bYBA8Cosnee17qEKw7uc 2Gow== X-Gm-Message-State: AOJu0Yz9fGpgvq9rB8lGIdV+m0gZgbskYF/m696zjhfHXb5RTfMahUlH G+Na3/zbBOTiVy0ZSx7MUsshXzI77jQOrkpZohvCCjPzZeVRDaLLZDetXF6ev/Gl3xGWTyzePGw 1 X-Google-Smtp-Source: AGHT+IGsHqXaMfO75Hr/xzzH8lYuJMrFbtExhGPHDG+/XpX/lNz4Zg95GZ2tSIZmtwzMf4aX6NejRg== X-Received: by 2002:a17:90b:4a87:b0:2e2:aef9:8f60 with SMTP id 98e67ed59e1d1-2e9b14ddb3bmr32406854a91.0.1731600110051; Thu, 14 Nov 2024 08:01:50 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.49 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:49 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 21/54] accel/tcg: Delay plugin adjustment in probe_access_internal Date: Thu, 14 Nov 2024 08:00:57 -0800 Message-ID: <20241114160131.48616-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Remove force_mmio and place the expression into the IF expression, behind the short-circuit logic expressions that might eliminate its computation. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 3e24529f4f..a4c69bcbf1 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1373,7 +1373,6 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, CPUTLBEntry *entry = tlb_entry(cpu, mmu_idx, addr); uint64_t tlb_addr = tlb_read_idx(entry, access_type); int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; - bool force_mmio = check_mem_cbs && cpu_plugin_mem_cbs_enabled(cpu); CPUTLBEntryFull *full; if (!tlb_hit(tlb_addr, addr)) { @@ -1404,9 +1403,14 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, *pfull = full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; flags |= full->slow_flags[access_type]; - /* Fold all "mmio-like" bits into TLB_MMIO. This is not RAM. */ - if (unlikely(flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY | TLB_CHECK_ALIGNED)) - || (access_type != MMU_INST_FETCH && force_mmio)) { + /* + * Fold all "mmio-like" bits, and required plugin callbacks, to TLB_MMIO. + * These cannot be treated as RAM. + */ + if ((flags & ~(TLB_WATCHPOINT | TLB_NOTDIRTY | TLB_CHECK_ALIGNED)) + || (access_type != MMU_INST_FETCH + && check_mem_cbs + && cpu_plugin_mem_cbs_enabled(cpu))) { *phost = NULL; return TLB_MMIO; } From patchwork Thu Nov 14 16:00:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C3FDD68B33 for ; Thu, 14 Nov 2024 16:12:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIj-0005Lq-8v; Thu, 14 Nov 2024 11:02:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcID-00050r-Sb for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:05 -0500 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI5-0002Fw-7B for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:54 -0500 Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2e5a0177531so675744a91.2 for ; Thu, 14 Nov 2024 08:01:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600111; x=1732204911; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ZzUqo5EQVmmILWK3/HrkYlBd4Dl5eFiN4ejkIAqPzrI=; b=arhER/1wqg1QpTgXVPn0NDZBKZSxaiE1pt8lAv5E7XFewOz2gWk0P6b7bEQAdFDxiM yCBjAEnGRjMSHd58arzyUHefPL0LZuFMRNOITl7t5OpYkJPk81ckp2+ChlXbEqC7+4D0 Gkjh3oGFwOYk/jIIJM8tmOLKatsn0/eOzlz0HJgNieuQjzJJKZcLfzVurpXmCtPi+jPj MCpw6xTw75ivUW7ezfF81R1nUwsjnIeBMvI3P4nMfdfOjC+/7EjnTB4n4m29C/ia/JId YaNvXJRCiscZzy5Y8FbeDvgQvN2U9cr3nRvn8o2ZHH/N1bp2Yv3FlP3omIvmvINaAumJ Eelg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600111; x=1732204911; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZzUqo5EQVmmILWK3/HrkYlBd4Dl5eFiN4ejkIAqPzrI=; b=a8UCiL/vjeFnHvDRidBZSB9ApfhtRq/Q82v2FeoL4lbWrFj8VsDxGH2JZKQvAeB6YF 94UWwfJDUlxivMVcOioc9DKbA+68yNmtiXi03POiv+7mIkGRbEFGJtA6z6+JmnuXGkcu 6s8Wj6gnZTYIbW8psX+57dxMKsChg0Iub0IsUiJ0iBYaRlXXV9fVw0/YSZetFJDzCLWD 1pIHqzNVV97XrIjP/vcC7LC4ePOoJWKZL3waHtk3W47FCj07MkqS9rzklaVSUWFnZPKJ /38s27DIZNIDUTk1pfaVS3NIg+gzmCzKLJ5qB78ONQBnb5X/x8G5AGBJMK7LZbtMDqLW 2Q6Q== X-Gm-Message-State: AOJu0Yzow/j0oLzGsY0bAConhPJw4X9UQdU4Q95MDhbDJ64kqXhzO8lx D5TmM8+a8Q3y3oA+G8Zs7ahwTaTV/1cCk8q5D6hW+b5h8l8gjGJpfNDDEoPGu5DcvSr7FUt2WWO O X-Google-Smtp-Source: AGHT+IEC8cVZjuJs8fTcgyRwOAB4vOr7dnWsxczMYwi3IO/oI4UAakxoV9jKUiEFGz7BQpnwwUHAUw== X-Received: by 2002:a17:90b:1a92:b0:2d8:82a2:b093 with SMTP id 98e67ed59e1d1-2e9b1720e9dmr31250421a91.13.1731600111017; Thu, 14 Nov 2024 08:01:51 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.50 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:50 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 22/54] accel/tcg: Call cpu_ld*_code_mmu from cpu_ld*_code Date: Thu, 14 Nov 2024 08:00:58 -0800 Message-ID: <20241114160131.48616-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Ensure a common entry point for all code lookups. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index a4c69bcbf1..c975dd2322 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2924,28 +2924,28 @@ uint32_t cpu_ldub_code(CPUArchState *env, abi_ptr addr) { CPUState *cs = env_cpu(env); MemOpIdx oi = make_memop_idx(MO_UB, cpu_mmu_index(cs, true)); - return do_ld1_mmu(cs, addr, oi, 0, MMU_INST_FETCH); + return cpu_ldb_code_mmu(env, addr, oi, 0); } uint32_t cpu_lduw_code(CPUArchState *env, abi_ptr addr) { CPUState *cs = env_cpu(env); MemOpIdx oi = make_memop_idx(MO_TEUW, cpu_mmu_index(cs, true)); - return do_ld2_mmu(cs, addr, oi, 0, MMU_INST_FETCH); + return cpu_ldw_code_mmu(env, addr, oi, 0); } uint32_t cpu_ldl_code(CPUArchState *env, abi_ptr addr) { CPUState *cs = env_cpu(env); MemOpIdx oi = make_memop_idx(MO_TEUL, cpu_mmu_index(cs, true)); - return do_ld4_mmu(cs, addr, oi, 0, MMU_INST_FETCH); + return cpu_ldl_code_mmu(env, addr, oi, 0); } uint64_t cpu_ldq_code(CPUArchState *env, abi_ptr addr) { CPUState *cs = env_cpu(env); MemOpIdx oi = make_memop_idx(MO_TEUQ, cpu_mmu_index(cs, true)); - return do_ld8_mmu(cs, addr, oi, 0, MMU_INST_FETCH); + return cpu_ldq_code_mmu(env, addr, oi, 0); } uint8_t cpu_ldb_code_mmu(CPUArchState *env, abi_ptr addr, From patchwork Thu Nov 14 16:00:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D6A0D68B34 for ; Thu, 14 Nov 2024 16:03:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIl-0005OL-Ku; Thu, 14 Nov 2024 11:02:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052L-K7 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcID-0002Gg-KD for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:06 -0500 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2ea0d64e78dso412481a91.1 for ; Thu, 14 Nov 2024 08:01:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600113; x=1732204913; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=X2yiS+ruW+gkzVWGmQM9TvYN4B8BJM2W1/RTtxTmteg=; b=SvUGTtTSIHBdbYvIX9y2bH/dIRfNWgZnAMq4nB5nQlWXF6D0cGHJLI1r5/6gUVqt8f SCheJco3l8P17/ipSMDikz+/0qFI0RuEKHfYrPjGOly1JS/PdBNBWk7s286tqhXMw1wi OwmpkjSTFB95LVykP0z6jbSE4aJp2D40LkeT4rSL+WGyFZhrZWgMEliYhRI2GT7GLfcd 3QTcyIhSTVZg22EeI5o68xsBfs159dqFIuQHoBYg+AeK5x7RWhoIcM+zcKkFhrGEFLWo psqVXQc/m1VzUoskGDbd2hHI71lOsnMhtWpFaWAIVCivh4P8ztFlWPy3bpHYUz0Ka3Gj BVzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600113; x=1732204913; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X2yiS+ruW+gkzVWGmQM9TvYN4B8BJM2W1/RTtxTmteg=; b=UBHEtzG4kYyq8jtXm4Lc9Uir/Qn8bXvxWYBqBLE8ck4U1nsCm3aXI/ztFkW45E1zlW so8jCciGg/yEJ3cbW9Nhh5vKhMTVtPexG4LKmO1WSU8Gz6bX6xRJfsd78i/UxX6OEFwm DMpIBYrr+PRDCGU++yxEwWbwi1DRahSaiFW5GfTJZGJKj5JBB31NIDH/jM5fq6l47Mmy EHp7eOhNm4lJ1rFk30IPakL4qVajQjbW+CehOD50Msm5PvWkuPl4Zg9FfAJcAF01uHzI iwUhdIh5MUe9plT7gRk3hYXBULhQ/L91GqWuWWil6c2V8DdTOJ23NcRw9nH1mL05smKj h3FA== X-Gm-Message-State: AOJu0YwTk9I1LYzgTQ8JvBH8zWo8X3sJsMAj2ct6lJjWI7bFQZcVQArx /l11p4sB3ciTpI1PfSky1CEYfomNIS8OBuZCVqV1f4DYS2qxiwHlHJ27Wcd7wGytV5BRmFo9nOB 5 X-Google-Smtp-Source: AGHT+IHuqzTniQ8yFTsrocheUor779ihuJrnuyUderscn0uCVN5kbFNNswxyGQ4yCSVvR+90QfRkvw== X-Received: by 2002:a17:90b:3b8b:b0:2e2:85b8:14e with SMTP id 98e67ed59e1d1-2e9fe6b8273mr5628147a91.15.1731600111789; Thu, 14 Nov 2024 08:01:51 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:51 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 23/54] accel/tcg: Check original prot bits for read in atomic_mmu_lookup Date: Thu, 14 Nov 2024 08:00:59 -0800 Message-ID: <20241114160131.48616-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In the mist before CPUTLBEntryFull existed, we had to be clever to detect write-only pages. Now we can directly test the saved prot bits, which is clearer. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index c975dd2322..ae3a99eb47 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1854,14 +1854,13 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, flags &= ~TLB_INVALID_MASK; } } + full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; /* * Let the guest notice RMW on a write-only page. * We have just verified that the page is writable. - * Subpage lookups may have left TLB_INVALID_MASK set, - * but addr_read will only be -1 if PAGE_READ was unset. */ - if (unlikely(tlbe->addr_read == -1)) { + if (unlikely(!(full->prot & PAGE_READ))) { tlb_fill_align(cpu, addr, MMU_DATA_LOAD, mmu_idx, 0, size, false, retaddr); /* @@ -1899,7 +1898,6 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, } hostaddr = (void *)((uintptr_t)addr + tlbe->addend); - full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; if (unlikely(flags & TLB_NOTDIRTY)) { notdirty_write(cpu, addr, size, full, retaddr); From patchwork Thu Nov 14 16:01:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB85FD68B34 for ; Thu, 14 Nov 2024 16:11:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIT-00055f-4N; Thu, 14 Nov 2024 11:02:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcID-00050q-SL for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:05 -0500 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcI6-0002GH-2A for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:01:55 -0500 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-7f43259d220so598426a12.3 for ; Thu, 14 Nov 2024 08:01:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600113; x=1732204913; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=AnWkT7tU0GDgeoHYcM5K7eCpXA9PnKUyZxtzeQFAarY=; b=xzsPK61FSnUindBobiONzn1KOBiD4SLunuIIgAkQ02i1kgRcs806SIhj8x58HS7oS+ mlX87IirUoVmD1dKtzDJz2P50ry6u/ElAHenTW6xktiDj1uRYf4lyReeCMcSVMYEHcGP eYWYxBN7ZS1f0lHx8IdL/yl1RgFPLsSbMBU8/COLW2C5GHkOfLPoDFHbL3rsGIbdfDl/ vJxaxk9mD8wll+fgp1PTqHfZDi1Z0tAZjZiQdhHA9lJBVo/lk3PEFn9qw6D3uOdnp0gg bjxyEF2MButakqYj5qyOKnk5Nz0ZHF3BObze5plbFPeEpzYGrD1ua2UHyk0Tcrejza05 8Z3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600113; x=1732204913; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AnWkT7tU0GDgeoHYcM5K7eCpXA9PnKUyZxtzeQFAarY=; b=Y4emIzIf/1CauYYk2xrkw9ACQBOV3BdyGBnDe3rm0SbElPvfSFFRaKrd+ll9aGiOmL 4APecnxYeZbRtUfaXqsAdinNUsLpHQPB5YtruZZwyolYYz4TKIcWQWSbJpt9QxPrQkBs JViDgvQCSCQ3lWrQEOtPujMDFu2lZPk9dXtcDBb0gS/iPen2A8W1M7R9XvxtzAmgBZkm P9gEFbigfyTMXJYTc40rCbTg+vVvtPNPnheB4nSbdIpykQ9ZN3mIysDH+CnOEzvvj+zs j83Y5IWJ1PuXN6hndssrvDHDYgyPSCw/j46gNylRboo+EStI2rxR7D4DguipPqmnRigV ACXA== X-Gm-Message-State: AOJu0Yy6r2DLPMJysgAn6wpbxI5prccH27Ye4zV7IOo3rnq6747Z4BTw /WJq7rMEou8N2qq5tyGHHHLA65JVINVyLurwup5ln7Y2JgW91/UfKz21mHi6qeEtnmtbcolsipH e X-Google-Smtp-Source: AGHT+IFvnIHz7a1MyEK30ABDABS/y04r0zAC5JEVsqHkdhBLA4A6WMpdB018cq83f65IAhiKpdf/Aw== X-Received: by 2002:a17:90b:2792:b0:2e2:effb:618b with SMTP id 98e67ed59e1d1-2ea0636fd3bmr2866679a91.13.1731600112677; Thu, 14 Nov 2024 08:01:52 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:52 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 24/54] accel/tcg: Preserve tlb flags in tlb_set_compare Date: Thu, 14 Nov 2024 08:01:00 -0800 Message-ID: <20241114160131.48616-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before, if !enable, we squashed the entire address comparator to -1. This works because TLB_INVALID_MASK is set. It seemed natural, because the tlb is cleared with memset of 0xff. With this patch, we retain all of the other TLB_* bits even when the page is not enabled. This works because TLB_INVALID_MASK is set. This will be used in a subsequent patch; the addr_read comparator contains the flags for pages that are executable but not readable. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ae3a99eb47..585f4171cc 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1032,15 +1032,13 @@ static inline void tlb_set_compare(CPUTLBEntryFull *full, CPUTLBEntry *ent, vaddr address, int flags, MMUAccessType access_type, bool enable) { - if (enable) { - address |= flags & TLB_FLAGS_MASK; - flags &= TLB_SLOW_FLAGS_MASK; - if (flags) { - address |= TLB_FORCE_SLOW; - } - } else { - address = -1; - flags = 0; + if (!enable) { + address = TLB_INVALID_MASK; + } + address |= flags & TLB_FLAGS_MASK; + flags &= TLB_SLOW_FLAGS_MASK; + if (flags) { + address |= TLB_FORCE_SLOW; } ent->addr_idx[access_type] = address; full->slow_flags[access_type] = flags; From patchwork Thu Nov 14 16:01:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89E12D68B33 for ; Thu, 14 Nov 2024 16:04:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIT-000562-Ix; Thu, 14 Nov 2024 11:02:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052V-MH for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIE-0002Gq-Gp for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:06 -0500 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-7240d93fffdso616485b3a.2 for ; Thu, 14 Nov 2024 08:01:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600114; x=1732204914; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=9Yssv0jEMu3znLJwX6X9mSGXOOpGLG/4PsoO3zzU/j8=; b=XgsI8Y7WRvgNkYvNbvQCseP/ZqUaN/ZNKMN+yGY+OsZ/MzUt+7bDRWnntY5ueT3fSl czls36G4ttsu70Tj5JjKxcqJrEvYlN5GMcYuo0nsRzM9Y01ng5K5byZRHN5cxYgWg3rq rONvu/YuteAC+Owmxbfq8MPrX/mpvLWFX1el/39quU+fRrW+p3EjM2C+5W42RwbdvUmk 8JcgwAi5OJ5pKZiVBuhMr3ZYnx7CrhyaIrCcIgU4W6MqzXHONDf29uJpqFqggJJNse5y gHo5y6DMlkdYNXT1s9ZZ2Fn4XvuyEDk1pujFKufhycvUuntZNiJZO1jR+h41fu6I3/91 GoCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600114; x=1732204914; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9Yssv0jEMu3znLJwX6X9mSGXOOpGLG/4PsoO3zzU/j8=; b=co25rNHQr1XaxeRXWBYd45Um+dOR64O/KXwyo49KbNYzfKoh42/gkzLm57su9ZXcPL H2njF7lSTfPJf3muY1cENcXtHoR2816TaCLINlejVEqm5HlavEibgU0fGxRVjoAed85v 8+z4oB4rbODzhMvVty8FjUho2VpVTqhG7sdyUdzDIU8FB8oJXTJ+QPFWNVB2FRAyZjeE pn3HWwDB/teg7ITlRjLxqV3uPNxnwluhRmLx1i/OhWaCfWlpiPx8C3WoErIWupc6+2A2 2vBdNAmuuMS6NKONMjTc/biOHCv4tVntejktEf3z+xMOkBmRd3AxCzawN7P78Qi7XPlK kppA== X-Gm-Message-State: AOJu0YzpjUI/KqBc4okCBBNXIu0JABvOyUH1+5ZCVk97+ie7YAEWfEKi qNAW+SngYKGQy2uicIiMRvbgYmIVlX48qDSSOoPKvfRnYbhw0cmSxeH+c9rYhvJyDHzA1xouwE8 b X-Google-Smtp-Source: AGHT+IFLdAAVJWloFL406nCZNpvgcVG5gYnNaGTp0OhtyT1eS3AqjXuJsSo+9xeXXm45DxgMRFh9vQ== X-Received: by 2002:a17:90b:51c4:b0:2e2:ffb0:89f6 with SMTP id 98e67ed59e1d1-2e9f2c78416mr9734707a91.15.1731600113640; Thu, 14 Nov 2024 08:01:53 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:53 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 25/54] accel/tcg: Return CPUTLBEntryFull not pointer in probe_access_full_mmu Date: Thu, 14 Nov 2024 08:01:01 -0800 Message-ID: <20241114160131.48616-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Return a copy of the structure, not a pointer. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/exec-all.h | 2 +- accel/tcg/cputlb.c | 13 ++++++++----- target/arm/ptw.c | 10 +++++----- target/i386/tcg/sysemu/excp_helper.c | 8 ++++---- 4 files changed, 18 insertions(+), 15 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 2e4c4cc4b4..df7d0b5ad0 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -393,7 +393,7 @@ int probe_access_full(CPUArchState *env, vaddr addr, int size, */ int probe_access_full_mmu(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, - void **phost, CPUTLBEntryFull **pfull); + void **phost, CPUTLBEntryFull *pfull); #endif /* !CONFIG_USER_ONLY */ #endif /* CONFIG_TCG */ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 585f4171cc..81135524eb 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1439,25 +1439,28 @@ int probe_access_full(CPUArchState *env, vaddr addr, int size, int probe_access_full_mmu(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, - void **phost, CPUTLBEntryFull **pfull) + void **phost, CPUTLBEntryFull *pfull) { void *discard_phost; - CPUTLBEntryFull *discard_tlb; + CPUTLBEntryFull *full; /* privately handle users that don't need full results */ phost = phost ? phost : &discard_phost; - pfull = pfull ? pfull : &discard_tlb; int flags = probe_access_internal(env_cpu(env), addr, size, access_type, - mmu_idx, true, phost, pfull, 0, false); + mmu_idx, true, phost, &full, 0, false); /* Handle clean RAM pages. */ if (unlikely(flags & TLB_NOTDIRTY)) { int dirtysize = size == 0 ? 1 : size; - notdirty_write(env_cpu(env), addr, dirtysize, *pfull, 0); + notdirty_write(env_cpu(env), addr, dirtysize, full, 0); flags &= ~TLB_NOTDIRTY; } + if (pfull) { + *pfull = *full; + } + return flags; } diff --git a/target/arm/ptw.c b/target/arm/ptw.c index 9849949508..3ae5f524de 100644 --- a/target/arm/ptw.c +++ b/target/arm/ptw.c @@ -592,7 +592,7 @@ static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw, ptw->out_space = s2.f.attrs.space; } else { #ifdef CONFIG_TCG - CPUTLBEntryFull *full; + CPUTLBEntryFull full; int flags; env->tlb_fi = fi; @@ -604,10 +604,10 @@ static bool S1_ptw_translate(CPUARMState *env, S1Translate *ptw, if (unlikely(flags & TLB_INVALID_MASK)) { goto fail; } - ptw->out_phys = full->phys_addr | (addr & ~TARGET_PAGE_MASK); - ptw->out_rw = full->prot & PAGE_WRITE; - pte_attrs = full->extra.arm.pte_attrs; - ptw->out_space = full->attrs.space; + ptw->out_phys = full.phys_addr | (addr & ~TARGET_PAGE_MASK); + ptw->out_rw = full.prot & PAGE_WRITE; + pte_attrs = full.extra.arm.pte_attrs; + ptw->out_space = full.attrs.space; #else g_assert_not_reached(); #endif diff --git a/target/i386/tcg/sysemu/excp_helper.c b/target/i386/tcg/sysemu/excp_helper.c index 02d3486421..168ff8e5f3 100644 --- a/target/i386/tcg/sysemu/excp_helper.c +++ b/target/i386/tcg/sysemu/excp_helper.c @@ -436,7 +436,7 @@ do_check_protect_pse36: * addresses) using the address with the A20 bit set. */ if (in->ptw_idx == MMU_NESTED_IDX) { - CPUTLBEntryFull *full; + CPUTLBEntryFull full; int flags, nested_page_size; flags = probe_access_full_mmu(env, paddr, 0, access_type, @@ -451,7 +451,7 @@ do_check_protect_pse36: } /* Merge stage1 & stage2 protection bits. */ - prot &= full->prot; + prot &= full.prot; /* Re-verify resulting protection. */ if ((prot & (1 << access_type)) == 0) { @@ -459,8 +459,8 @@ do_check_protect_pse36: } /* Merge stage1 & stage2 addresses to final physical address. */ - nested_page_size = 1 << full->lg_page_size; - paddr = (full->phys_addr & ~(nested_page_size - 1)) + nested_page_size = 1 << full.lg_page_size; + paddr = (full.phys_addr & ~(nested_page_size - 1)) | (paddr & (nested_page_size - 1)); /* From patchwork Thu Nov 14 16:01:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99DD0D68B33 for ; Thu, 14 Nov 2024 16:04:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ8-0005m8-Ui; Thu, 14 Nov 2024 11:02:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIO-00053N-II for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIJ-0002Gz-D8 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:12 -0500 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2ea0c38f0fdso444987a91.3 for ; Thu, 14 Nov 2024 08:01:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600115; x=1732204915; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=KiD2QRw11IfiGIBgEwLODrTbmi4491+hXDJh5Wff0xY=; b=hzZjhiJ7yxmab5kHBwn7TxHmvcC3MJ4lvsZzDr0MoD7RVgdTDxEwTqL9bsOSKUuXvl ZCv/91/sMpkvPSbZHDda2IvpmQOM5/gt1NZHwTMHko/ea0J/R4S0dV1FAg/q3n+GBQoz a1UUckLCduoSfQVSh568ARFqUoL1pC+jp0PxCKSOOF7K4kIj61N0stRpWWA/HNE8hD7m rHegTpfWZ7uy43ZzM5mQD2troZo8n1OZ2c8jy/y+dyYoYFKLBx/ZyRi9VMf/y2rUK+h0 rbxkUauOPWi1TAe385zaZrpcQG3mV28VsonbiJWTyRwh9S1G9RcGSWvf5s8jNaX5xgDR soHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600115; x=1732204915; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KiD2QRw11IfiGIBgEwLODrTbmi4491+hXDJh5Wff0xY=; b=dkFsENO8rHXvYLm4AyAvadq9gz1lYSOL8i7ZiJ5zls94AZHlItijDHMFC3XQearmh9 utnVgLvgZ9EBRm0ztMFZJ00XaRwrZYfJ5Sh3tH3IwaA0+UZZrNByzx5276IwuUcdqJmH 5JZZ+F3wBeL8RLhMnFMWcD+EMiKXESCbaP4j3bcb+KW44rKEqiMWFcl9w77vz9oVyKNt 5YCfa1sdUdIhgvwPKT5ph+A9alG45kTS9VFmV7DdHM449S/gYNiiozN5mKY36fPl5uuc TN72kKMHTDPSv6EbOhwU9omnVfisUkBG65RI0ZdbBZfhty5LtNtp6ctHc9+ZK0sMtHIB Jtiw== X-Gm-Message-State: AOJu0YyNtvtHamoCEgWjiJMNl0h0vNpKNSP/msx1uWgfxDXih6C+WQii lvOf96GxYqhehU5mw590zKqKLnVI89gghgJf3BhlrbGrCM0+jq5bWeoqFAkKsNeqNAZp91nMmu7 1 X-Google-Smtp-Source: AGHT+IEwAnamhG6Wsz49aAe8amyREuM4MSSY0/Enh1wOpOfXJdOfDELR22yWAUTA0birea61vPeX8A== X-Received: by 2002:a17:90a:d2cc:b0:2e9:c902:d53 with SMTP id 98e67ed59e1d1-2e9c9020e26mr22441031a91.14.1731600114497; Thu, 14 Nov 2024 08:01:54 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:53 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 26/54] accel/tcg: Return CPUTLBEntryFull not pointer in probe_access_full Date: Thu, 14 Nov 2024 08:01:02 -0800 Message-ID: <20241114160131.48616-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Return a copy of the structure, not a pointer. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/exec-all.h | 6 +----- accel/tcg/cputlb.c | 8 +++++--- target/arm/tcg/helper-a64.c | 4 ++-- target/arm/tcg/mte_helper.c | 15 ++++++--------- target/arm/tcg/sve_helper.c | 6 +++--- 5 files changed, 17 insertions(+), 22 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index df7d0b5ad0..69bdb77584 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -365,10 +365,6 @@ int probe_access_flags(CPUArchState *env, vaddr addr, int size, * probe_access_full: * Like probe_access_flags, except also return into @pfull. * - * The CPUTLBEntryFull structure returned via @pfull is transient - * and must be consumed or copied immediately, before any further - * access or changes to TLB @mmu_idx. - * * This function will not fault if @nonfault is set, but will * return TLB_INVALID_MASK if the page is not mapped, or is not * accessible with @access_type. @@ -379,7 +375,7 @@ int probe_access_flags(CPUArchState *env, vaddr addr, int size, int probe_access_full(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, bool nonfault, void **phost, - CPUTLBEntryFull **pfull, uintptr_t retaddr); + CPUTLBEntryFull *pfull, uintptr_t retaddr); /** * probe_access_full_mmu: diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 81135524eb..84e7e633e3 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1420,20 +1420,22 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, int probe_access_full(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, - bool nonfault, void **phost, CPUTLBEntryFull **pfull, + bool nonfault, void **phost, CPUTLBEntryFull *pfull, uintptr_t retaddr) { + CPUTLBEntryFull *full; int flags = probe_access_internal(env_cpu(env), addr, size, access_type, - mmu_idx, nonfault, phost, pfull, retaddr, + mmu_idx, nonfault, phost, &full, retaddr, true); /* Handle clean RAM pages. */ if (unlikely(flags & TLB_NOTDIRTY)) { int dirtysize = size == 0 ? 1 : size; - notdirty_write(env_cpu(env), addr, dirtysize, *pfull, retaddr); + notdirty_write(env_cpu(env), addr, dirtysize, full, retaddr); flags &= ~TLB_NOTDIRTY; } + *pfull = *full; return flags; } diff --git a/target/arm/tcg/helper-a64.c b/target/arm/tcg/helper-a64.c index 8f42a28d07..783864d6db 100644 --- a/target/arm/tcg/helper-a64.c +++ b/target/arm/tcg/helper-a64.c @@ -1883,14 +1883,14 @@ static bool is_guarded_page(CPUARMState *env, target_ulong addr, uintptr_t ra) #ifdef CONFIG_USER_ONLY return page_get_flags(addr) & PAGE_BTI; #else - CPUTLBEntryFull *full; + CPUTLBEntryFull full; void *host; int mmu_idx = cpu_mmu_index(env_cpu(env), true); int flags = probe_access_full(env, addr, 0, MMU_INST_FETCH, mmu_idx, false, &host, &full, ra); assert(!(flags & TLB_INVALID_MASK)); - return full->extra.arm.guarded; + return full.extra.arm.guarded; #endif } diff --git a/target/arm/tcg/mte_helper.c b/target/arm/tcg/mte_helper.c index 9d2ba287ee..870b2875af 100644 --- a/target/arm/tcg/mte_helper.c +++ b/target/arm/tcg/mte_helper.c @@ -83,8 +83,7 @@ uint8_t *allocation_tag_mem_probe(CPUARMState *env, int ptr_mmu_idx, TARGET_PAGE_BITS - LOG2_TAG_GRANULE - 1); return tags + index; #else - CPUTLBEntryFull *full; - MemTxAttrs attrs; + CPUTLBEntryFull full; int in_page, flags; hwaddr ptr_paddr, tag_paddr, xlat; MemoryRegion *mr; @@ -110,7 +109,7 @@ uint8_t *allocation_tag_mem_probe(CPUARMState *env, int ptr_mmu_idx, assert(!(flags & TLB_INVALID_MASK)); /* If the virtual page MemAttr != Tagged, access unchecked. */ - if (full->extra.arm.pte_attrs != 0xf0) { + if (full.extra.arm.pte_attrs != 0xf0) { return NULL; } @@ -129,9 +128,7 @@ uint8_t *allocation_tag_mem_probe(CPUARMState *env, int ptr_mmu_idx, * Remember these values across the second lookup below, * which may invalidate this pointer via tlb resize. */ - ptr_paddr = full->phys_addr | (ptr & ~TARGET_PAGE_MASK); - attrs = full->attrs; - full = NULL; + ptr_paddr = full.phys_addr | (ptr & ~TARGET_PAGE_MASK); /* * The Normal memory access can extend to the next page. E.g. a single @@ -150,17 +147,17 @@ uint8_t *allocation_tag_mem_probe(CPUARMState *env, int ptr_mmu_idx, if (!probe && unlikely(flags & TLB_WATCHPOINT)) { int wp = ptr_access == MMU_DATA_LOAD ? BP_MEM_READ : BP_MEM_WRITE; assert(ra != 0); - cpu_check_watchpoint(env_cpu(env), ptr, ptr_size, attrs, wp, ra); + cpu_check_watchpoint(env_cpu(env), ptr, ptr_size, full.attrs, wp, ra); } /* Convert to the physical address in tag space. */ tag_paddr = ptr_paddr >> (LOG2_TAG_GRANULE + 1); /* Look up the address in tag space. */ - tag_asi = attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS; + tag_asi = full.attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS; tag_as = cpu_get_address_space(env_cpu(env), tag_asi); mr = address_space_translate(tag_as, tag_paddr, &xlat, NULL, - tag_access == MMU_DATA_STORE, attrs); + tag_access == MMU_DATA_STORE, full.attrs); /* * Note that @mr will never be NULL. If there is nothing in the address diff --git a/target/arm/tcg/sve_helper.c b/target/arm/tcg/sve_helper.c index f1ee0e060f..dad0d5e518 100644 --- a/target/arm/tcg/sve_helper.c +++ b/target/arm/tcg/sve_helper.c @@ -5357,7 +5357,7 @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env, flags = probe_access_flags(env, addr, 0, access_type, mmu_idx, nofault, &info->host, retaddr); #else - CPUTLBEntryFull *full; + CPUTLBEntryFull full; flags = probe_access_full(env, addr, 0, access_type, mmu_idx, nofault, &info->host, &full, retaddr); #endif @@ -5373,8 +5373,8 @@ bool sve_probe_page(SVEHostPage *info, bool nofault, CPUARMState *env, /* Require both ANON and MTE; see allocation_tag_mem(). */ info->tagged = (flags & PAGE_ANON) && (flags & PAGE_MTE); #else - info->attrs = full->attrs; - info->tagged = full->extra.arm.pte_attrs == 0xf0; + info->attrs = full.attrs; + info->tagged = full.extra.arm.pte_attrs == 0xf0; #endif /* Ensure that info->host[] is relative to addr, not addr + mem_off. */ From patchwork Thu Nov 14 16:01:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C34C4D68B33 for ; Thu, 14 Nov 2024 16:11:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJR-0006D8-Ff; Thu, 14 Nov 2024 11:03:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052Q-LS for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIF-0002HP-JJ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:07 -0500 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2e59746062fso716694a91.2 for ; Thu, 14 Nov 2024 08:02:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600120; x=1732204920; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=nRyalkWLpxudzsmNEN2fR4VaSKzFhjYdvFTI7sg4gQs=; b=dT31tDmRlMxyUinjWEkJra+TYXFlZ3C6EieVlzwOZUEMG9WnM0xrWr2irVMbzv8lTH fmsFyStCAHPNrb63Ou7Qwi6Pk4P+i9ZXCD4t+udscjJJV8jTt9qZyqOPgypT1FdzYMTk KH8A7W+0ZPpy8F9srzIerC0Z4MSGLgmkeOCouPHBGXf7i0BUfrUUSVJC0Z6DZ+i/RuMn rWjxrWEuLLOaohh2UrHTvPWvhV7AP9t0s9NptpxNeQ0cQltVzbrYQ3RQVHLvUJYIzh8S vZ9kIR07tMC4mnuhomKXsPEuC9wXBEo3A7l4Buj+sRKxCtSTwKgiIYZlK2V3CCItGXqT uiJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600120; x=1732204920; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nRyalkWLpxudzsmNEN2fR4VaSKzFhjYdvFTI7sg4gQs=; b=XKIzZV3Aw2UZtZSdS94D3f9F+Nbsblyw8iIdfRrHYBZSX1pgpMxbBto+YuzuemjYeU gmZxld0+CE5FauL1VDHsTt0DY0nqI1QBRMHi7CFUo4mCwVCWczZ/EWm5xs2k+nRDZ488 lJ2cIB1v7mPW//DRZZZhf+mgo+1E7/UigclO7uV+1U01nPRgmtTX4aHyIkp9JVRfyZ2V HL54nUHs3+ZYKe1wg/hLqKTEK4hZN9FDPbIDp1jMKY2NOJHJxD6EVoG/yl9RxTi8vRcu 1CygqL/zRr+x9YCtVe6ppsIAvI4NUlOEg/AXOOJ+UwNNzZ+QPDwW0RSjr7A742sgNA10 SVrQ== X-Gm-Message-State: AOJu0YwvfXspHGkM7ImfT/phzwot+XYdN1ZppHO6LIbqriHem90Szrq8 ynILbulQAw/jLB0NDCiG++01x4IAVr13cjSB4C+1uUfb0HZlm7o5vVazUSzi/IpU/0cnHG9nN0k z X-Google-Smtp-Source: AGHT+IHvW2J974QSn4ulUGCLcikT5osBaZ6b2NsYmusO2gXxa02oGvOJq+SpsgcBtxDPhswQCGnmKA== X-Received: by 2002:a17:90b:3503:b0:2d3:d063:bdb6 with SMTP id 98e67ed59e1d1-2e9f2c4eee5mr8381820a91.4.1731600115459; Thu, 14 Nov 2024 08:01:55 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:54 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 27/54] accel/tcg: Return CPUTLBEntryFull not pointer in probe_access_internal Date: Thu, 14 Nov 2024 08:01:03 -0800 Message-ID: <20241114160131.48616-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Return a copy of the structure, not a pointer. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 40 ++++++++++++++++++---------------------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 84e7e633e3..41b2f76cc9 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1364,7 +1364,7 @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size, static int probe_access_internal(CPUState *cpu, vaddr addr, int fault_size, MMUAccessType access_type, int mmu_idx, bool nonfault, - void **phost, CPUTLBEntryFull **pfull, + void **phost, CPUTLBEntryFull *pfull, uintptr_t retaddr, bool check_mem_cbs) { uintptr_t index = tlb_index(cpu, mmu_idx, addr); @@ -1379,7 +1379,7 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, 0, fault_size, nonfault, retaddr)) { /* Non-faulting page table read failed. */ *phost = NULL; - *pfull = NULL; + memset(pfull, 0, sizeof(*pfull)); return TLB_INVALID_MASK; } @@ -1398,8 +1398,9 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, } flags &= tlb_addr; - *pfull = full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; + full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; flags |= full->slow_flags[access_type]; + *pfull = *full; /* * Fold all "mmio-like" bits, and required plugin callbacks, to TLB_MMIO. @@ -1423,19 +1424,17 @@ int probe_access_full(CPUArchState *env, vaddr addr, int size, bool nonfault, void **phost, CPUTLBEntryFull *pfull, uintptr_t retaddr) { - CPUTLBEntryFull *full; int flags = probe_access_internal(env_cpu(env), addr, size, access_type, - mmu_idx, nonfault, phost, &full, retaddr, + mmu_idx, nonfault, phost, pfull, retaddr, true); /* Handle clean RAM pages. */ if (unlikely(flags & TLB_NOTDIRTY)) { int dirtysize = size == 0 ? 1 : size; - notdirty_write(env_cpu(env), addr, dirtysize, full, retaddr); + notdirty_write(env_cpu(env), addr, dirtysize, pfull, retaddr); flags &= ~TLB_NOTDIRTY; } - *pfull = *full; return flags; } @@ -1444,25 +1443,22 @@ int probe_access_full_mmu(CPUArchState *env, vaddr addr, int size, void **phost, CPUTLBEntryFull *pfull) { void *discard_phost; - CPUTLBEntryFull *full; + CPUTLBEntryFull discard_full; /* privately handle users that don't need full results */ phost = phost ? phost : &discard_phost; + pfull = pfull ? pfull : &discard_full; int flags = probe_access_internal(env_cpu(env), addr, size, access_type, - mmu_idx, true, phost, &full, 0, false); + mmu_idx, true, phost, pfull, 0, false); /* Handle clean RAM pages. */ if (unlikely(flags & TLB_NOTDIRTY)) { int dirtysize = size == 0 ? 1 : size; - notdirty_write(env_cpu(env), addr, dirtysize, full, 0); + notdirty_write(env_cpu(env), addr, dirtysize, pfull, 0); flags &= ~TLB_NOTDIRTY; } - if (pfull) { - *pfull = *full; - } - return flags; } @@ -1470,7 +1466,7 @@ int probe_access_flags(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, bool nonfault, void **phost, uintptr_t retaddr) { - CPUTLBEntryFull *full; + CPUTLBEntryFull full; int flags; g_assert(-(addr | TARGET_PAGE_MASK) >= size); @@ -1482,7 +1478,7 @@ int probe_access_flags(CPUArchState *env, vaddr addr, int size, /* Handle clean RAM pages. */ if (unlikely(flags & TLB_NOTDIRTY)) { int dirtysize = size == 0 ? 1 : size; - notdirty_write(env_cpu(env), addr, dirtysize, full, retaddr); + notdirty_write(env_cpu(env), addr, dirtysize, &full, retaddr); flags &= ~TLB_NOTDIRTY; } @@ -1492,7 +1488,7 @@ int probe_access_flags(CPUArchState *env, vaddr addr, int size, void *probe_access(CPUArchState *env, vaddr addr, int size, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr) { - CPUTLBEntryFull *full; + CPUTLBEntryFull full; void *host; int flags; @@ -1513,12 +1509,12 @@ void *probe_access(CPUArchState *env, vaddr addr, int size, int wp_access = (access_type == MMU_DATA_STORE ? BP_MEM_WRITE : BP_MEM_READ); cpu_check_watchpoint(env_cpu(env), addr, size, - full->attrs, wp_access, retaddr); + full.attrs, wp_access, retaddr); } /* Handle clean RAM pages. */ if (flags & TLB_NOTDIRTY) { - notdirty_write(env_cpu(env), addr, size, full, retaddr); + notdirty_write(env_cpu(env), addr, size, &full, retaddr); } } @@ -1528,7 +1524,7 @@ void *probe_access(CPUArchState *env, vaddr addr, int size, void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, MMUAccessType access_type, int mmu_idx) { - CPUTLBEntryFull *full; + CPUTLBEntryFull full; void *host; int flags; @@ -1552,7 +1548,7 @@ void *tlb_vaddr_to_host(CPUArchState *env, abi_ptr addr, tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr, void **hostp) { - CPUTLBEntryFull *full; + CPUTLBEntryFull full; void *p; (void)probe_access_internal(env_cpu(env), addr, 1, MMU_INST_FETCH, @@ -1562,7 +1558,7 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr, return -1; } - if (full->lg_page_size < TARGET_PAGE_BITS) { + if (full.lg_page_size < TARGET_PAGE_BITS) { return -1; } From patchwork Thu Nov 14 16:01:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98F0FD68B34 for ; Thu, 14 Nov 2024 16:13:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJV-0006N4-Df; Thu, 14 Nov 2024 11:03:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIR-00055Q-Dc for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:15 -0500 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIL-0002H5-9V for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:14 -0500 Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2e2ed59a35eso736807a91.0 for ; Thu, 14 Nov 2024 08:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600117; x=1732204917; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=LsY9qG1BO160UZ3oT54r8qS8sJnVh1lfu8JqWoX9HKs=; b=Ff2wmvpUdD2EO1f7TlSEKgvFRCBGLSSVVfVZJTayf5x1Xodt5YQu9pXnksMJYLpu7z zOagINypcEtvZpNxrbklxytPXhWB1lM45ka2B2tf/b1YQ5M/IVKoo37NUtAI+q5BYlK8 ChfNGcy5EHbmgBCJ+4h5eS+1AkkOjlFcdkwyTFTVpo5giys/iQ3xkBZbWepXbLxJX+CD gSjGE+0T5KFGYrtkPrQ9lwP6jXk8mDRw/cAJFce9/J99BFjWblrU1z9LKjoXmkGI78gv ln6c49qJ5uLSKD6nqBLPHHTqzfxtONvK+jz5RqtpHPBmHJcfQ/q56CCFYDGFOD3q18mm Xevw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600117; x=1732204917; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LsY9qG1BO160UZ3oT54r8qS8sJnVh1lfu8JqWoX9HKs=; b=xMA/VbgRcDm89KX0zH2lP/of3BA4btFs8gkEYIv6IE5FrFngDWFk/UW6Vgxh/35PwX uEL8XUrsVfSOQSWnglMqvfvznRKpHug0mJNcnfPM0YnHX/8ocFXSgtkl21+8ARNBm5kR a7EfEeQpRfGCcUmAkg7hSxCG7L/ofsFJqZgPxRyqylucLVcP1vzWnjc9NyoJNSVFef+t mQOzSXtZfLVmLvafUqTA4wFbxrlswVMw4Oy9SUJf+u3dVPkzm15Ea4vuWzZKHixKYuaW nYbNueMCJE3uWyGvTr+KL/ZyJ7X/+RFCE1o/rhkGnOai0SNpXpAWXyykNeBkVz4n+mr0 d+Qw== X-Gm-Message-State: AOJu0YyNG14MKxbRvq02FFpGo/NR78neJaDrvXTyM3LqCJvgKy/oOZgE JC/4gKAZaPkvvV3MuGw+57xAaAo6tOdnnL7fj4eikG/mUTbU5n8FgrpI3ft+ccQZ01EfyHfjZxU 0 X-Google-Smtp-Source: AGHT+IHnNhDS49CGt44Gxs8BVTqUpIENQno7yxoNeG6lAU+lTa83Immbnso9WHnc97AumqC+86docw== X-Received: by 2002:a17:90a:dfc4:b0:2ea:1293:7db1 with SMTP id 98e67ed59e1d1-2ea12937f38mr645040a91.33.1731600116195; Thu, 14 Nov 2024 08:01:56 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:55 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 28/54] accel/tcg: Introduce tlb_lookup Date: Thu, 14 Nov 2024 08:01:04 -0800 Message-ID: <20241114160131.48616-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Unify 3 instances of tlb lookup, through tlb_hit, tlbtree_hit, and tlb_full_align. Use structures to avoid too many arguments. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 369 ++++++++++++++++++++++----------------------- 1 file changed, 178 insertions(+), 191 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 41b2f76cc9..a33bebf55a 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1271,6 +1271,118 @@ static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr, mmu_idx, retaddr); } +typedef struct TLBLookupInput { + vaddr addr; + uintptr_t ra; + int memop_probe : 16; + unsigned int size : 8; + MMUAccessType access_type : 4; + unsigned int mmu_idx : 4; +} TLBLookupInput; + +typedef struct TLBLookupOutput { + CPUTLBEntryFull full; + void *haddr; + int flags; + bool did_tlb_fill; +} TLBLookupOutput; + +static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, + const TLBLookupInput *i) +{ + CPUTLBDesc *desc = &cpu->neg.tlb.d[i->mmu_idx]; + CPUTLBDescFast *fast = &cpu->neg.tlb.f[i->mmu_idx]; + vaddr addr = i->addr; + MMUAccessType access_type = i->access_type; + CPUTLBEntryFull *full; + CPUTLBEntryTree *node; + CPUTLBEntry *entry; + uint64_t cmp; + bool probe = i->memop_probe < 0; + MemOp memop = probe ? 0 : i->memop_probe; + int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; + + assert_cpu_is_self(cpu); + o->did_tlb_fill = false; + + /* Primary lookup in the fast tlb. */ + entry = tlbfast_entry(fast, addr); + full = &desc->fulltlb[tlbfast_index(fast, addr)]; + cmp = tlb_read_idx(entry, access_type); + if (tlb_hit(cmp, addr)) { + goto found; + } + + /* Secondary lookup in the IntervalTree. */ + node = tlbtree_lookup_addr(desc, addr); + if (node) { + cmp = tlb_read_idx(&node->copy, access_type); + if (tlb_hit(cmp, addr)) { + /* Install the cached entry. */ + qemu_spin_lock(&cpu->neg.tlb.c.lock); + copy_tlb_helper_locked(entry, &node->copy); + qemu_spin_unlock(&cpu->neg.tlb.c.lock); + *full = node->full; + goto found; + } + } + + /* Finally, query the target hook. */ + if (!tlb_fill_align(cpu, addr, access_type, i->mmu_idx, + memop, i->size, probe, i->ra)) { + tcg_debug_assert(probe); + return false; + } + + o->did_tlb_fill = true; + + entry = tlbfast_entry(fast, addr); + full = &desc->fulltlb[tlbfast_index(fast, addr)]; + cmp = tlb_read_idx(entry, access_type); + /* + * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, + * to force the next access through tlb_fill_align. We've just + * called tlb_fill_align, so we know that this entry *is* valid. + */ + flags &= ~TLB_INVALID_MASK; + goto done; + + found: + /* Alignment has not been checked by tlb_fill_align. */ + { + int a_bits = memop_alignment_bits(memop); + + /* + * The TLB_CHECK_ALIGNED check differs from the normal alignment + * check, in that this is based on the atomicity of the operation. + * The intended use case is the ARM memory type field of each PTE, + * where access to pages with Device memory type require alignment. + */ + if (unlikely(flags & TLB_CHECK_ALIGNED)) { + int at_bits = memop_atomicity_bits(memop); + a_bits = MAX(a_bits, at_bits); + } + if (unlikely(addr & ((1 << a_bits) - 1))) { + cpu_unaligned_access(cpu, addr, access_type, i->mmu_idx, i->ra); + } + } + + done: + flags &= cmp; + flags |= full->slow_flags[access_type]; + o->flags = flags; + o->full = *full; + o->haddr = (void *)((uintptr_t)addr + entry->addend); + return true; +} + +static void tlb_lookup_nofail(CPUState *cpu, TLBLookupOutput *o, + const TLBLookupInput *i) +{ + bool ok = tlb_lookup(cpu, o, i); + tcg_debug_assert(ok); +} + static MemoryRegionSection * io_prepare(hwaddr *out_offset, CPUState *cpu, hwaddr xlat, MemTxAttrs attrs, vaddr addr, uintptr_t retaddr) @@ -1303,40 +1415,6 @@ static void io_failed(CPUState *cpu, CPUTLBEntryFull *full, vaddr addr, } } -/* - * Return true if ADDR is present in the interval tree, - * and has been copied back to the main tlb. - */ -static bool tlbtree_hit(CPUState *cpu, int mmu_idx, - MMUAccessType access_type, vaddr addr) -{ - CPUTLBDesc *desc = &cpu->neg.tlb.d[mmu_idx]; - CPUTLBDescFast *fast = &cpu->neg.tlb.f[mmu_idx]; - CPUTLBEntryTree *node; - size_t index; - - assert_cpu_is_self(cpu); - node = tlbtree_lookup_addr(desc, addr); - if (!node) { - /* There is no cached mapping for this page. */ - return false; - } - - if (!tlb_hit(tlb_read_idx(&node->copy, access_type), addr)) { - /* This access is not permitted. */ - return false; - } - - /* Install the cached entry. */ - index = tlbfast_index(fast, addr); - qemu_spin_lock(&cpu->neg.tlb.c.lock); - copy_tlb_helper_locked(&fast->table[index], &node->copy); - qemu_spin_unlock(&cpu->neg.tlb.c.lock); - - desc->fulltlb[index] = node->full; - return true; -} - static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size, CPUTLBEntryFull *full, uintptr_t retaddr) { @@ -1367,40 +1445,26 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, void **phost, CPUTLBEntryFull *pfull, uintptr_t retaddr, bool check_mem_cbs) { - uintptr_t index = tlb_index(cpu, mmu_idx, addr); - CPUTLBEntry *entry = tlb_entry(cpu, mmu_idx, addr); - uint64_t tlb_addr = tlb_read_idx(entry, access_type); - int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; - CPUTLBEntryFull *full; + TLBLookupInput i = { + .addr = addr, + .ra = retaddr, + .access_type = access_type, + .size = fault_size, + .memop_probe = nonfault ? -1 : 0, + .mmu_idx = mmu_idx, + }; + TLBLookupOutput o; + int flags; - if (!tlb_hit(tlb_addr, addr)) { - if (!tlbtree_hit(cpu, mmu_idx, access_type, addr)) { - if (!tlb_fill_align(cpu, addr, access_type, mmu_idx, - 0, fault_size, nonfault, retaddr)) { - /* Non-faulting page table read failed. */ - *phost = NULL; - memset(pfull, 0, sizeof(*pfull)); - return TLB_INVALID_MASK; - } - - /* TLB resize via tlb_fill_align may have moved the entry. */ - index = tlb_index(cpu, mmu_idx, addr); - entry = tlb_entry(cpu, mmu_idx, addr); - - /* - * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, - * to force the next access through tlb_fill_align. We've just - * called tlb_fill_align, so we know that this entry *is* valid. - */ - flags &= ~TLB_INVALID_MASK; - } - tlb_addr = tlb_read_idx(entry, access_type); + if (!tlb_lookup(cpu, &o, &i)) { + /* Non-faulting page table read failed. */ + *phost = NULL; + memset(pfull, 0, sizeof(*pfull)); + return TLB_INVALID_MASK; } - flags &= tlb_addr; - full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; - flags |= full->slow_flags[access_type]; - *pfull = *full; + *pfull = o.full; + flags = o.flags; /* * Fold all "mmio-like" bits, and required plugin callbacks, to TLB_MMIO. @@ -1415,7 +1479,7 @@ static int probe_access_internal(CPUState *cpu, vaddr addr, } /* Everything else is RAM. */ - *phost = (void *)((uintptr_t)addr + entry->addend); + *phost = o.haddr; return flags; } @@ -1625,6 +1689,7 @@ typedef struct MMULookupPageData { vaddr addr; int flags; int size; + TLBLookupOutput o; } MMULookupPageData; typedef struct MMULookupLocals { @@ -1644,67 +1709,25 @@ typedef struct MMULookupLocals { * * Resolve the translation for the one page at @data.addr, filling in * the rest of @data with the results. If the translation fails, - * tlb_fill_align will longjmp out. Return true if the softmmu tlb for - * @mmu_idx may have resized. + * tlb_fill_align will longjmp out. */ -static bool mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, +static void mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, int mmu_idx, MMUAccessType access_type, uintptr_t ra) { - vaddr addr = data->addr; - uintptr_t index = tlb_index(cpu, mmu_idx, addr); - CPUTLBEntry *entry = tlb_entry(cpu, mmu_idx, addr); - uint64_t tlb_addr = tlb_read_idx(entry, access_type); - bool maybe_resized = false; - CPUTLBEntryFull *full; - int flags = TLB_FLAGS_MASK & ~TLB_FORCE_SLOW; + TLBLookupInput i = { + .addr = data->addr, + .ra = ra, + .access_type = access_type, + .memop_probe = memop, + .size = data->size, + .mmu_idx = mmu_idx, + }; - /* If the TLB entry is for a different page, reload and try again. */ - if (!tlb_hit(tlb_addr, addr)) { - if (!tlbtree_hit(cpu, mmu_idx, access_type, addr)) { - tlb_fill_align(cpu, addr, access_type, mmu_idx, - memop, data->size, false, ra); - maybe_resized = true; - index = tlb_index(cpu, mmu_idx, addr); - entry = tlb_entry(cpu, mmu_idx, addr); - /* - * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, - * to force the next access through tlb_fill. We've just - * called tlb_fill, so we know that this entry *is* valid. - */ - flags &= ~TLB_INVALID_MASK; - } - tlb_addr = tlb_read_idx(entry, access_type); - } + tlb_lookup_nofail(cpu, &data->o, &i); - full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; - flags = tlb_addr & (TLB_FLAGS_MASK & ~TLB_FORCE_SLOW); - flags |= full->slow_flags[access_type]; - - if (likely(!maybe_resized)) { - /* Alignment has not been checked by tlb_fill_align. */ - int a_bits = memop_alignment_bits(memop); - - /* - * This alignment check differs from the one above, in that this is - * based on the atomicity of the operation. The intended use case is - * the ARM memory type field of each PTE, where access to pages with - * Device memory type require alignment. - */ - if (unlikely(flags & TLB_CHECK_ALIGNED)) { - int at_bits = memop_atomicity_bits(memop); - a_bits = MAX(a_bits, at_bits); - } - if (unlikely(addr & ((1 << a_bits) - 1))) { - cpu_unaligned_access(cpu, addr, access_type, mmu_idx, ra); - } - } - - data->full = full; - data->flags = flags; - /* Compute haddr speculatively; depending on flags it might be invalid. */ - data->haddr = (void *)((uintptr_t)addr + entry->addend); - - return maybe_resized; + data->full = &data->o.full; + data->flags = data->o.flags; + data->haddr = data->o.haddr; } /** @@ -1785,15 +1808,9 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, l->page[1].size = l->page[0].size - size0; l->page[0].size = size0; - /* - * Lookup both pages, recognizing exceptions from either. If the - * second lookup potentially resized, refresh first CPUTLBEntryFull. - */ + /* Lookup both pages, recognizing exceptions from either. */ mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra); - if (mmu_lookup1(cpu, &l->page[1], 0, l->mmu_idx, type, ra)) { - uintptr_t index = tlb_index(cpu, l->mmu_idx, addr); - l->page[0].full = &cpu->neg.tlb.d[l->mmu_idx].fulltlb[index]; - } + mmu_lookup1(cpu, &l->page[1], 0, l->mmu_idx, type, ra); flags = l->page[0].flags | l->page[1].flags; if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) { @@ -1819,49 +1836,26 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, int size, uintptr_t retaddr) { - uintptr_t mmu_idx = get_mmuidx(oi); - MemOp mop = get_memop(oi); - uintptr_t index; - CPUTLBEntry *tlbe; - void *hostaddr; - CPUTLBEntryFull *full; - bool did_tlb_fill = false; - int flags; + TLBLookupInput i = { + .addr = addr, + .ra = retaddr - GETPC_ADJ, + .access_type = MMU_DATA_STORE, + .memop_probe = get_memop(oi), + .mmu_idx = get_mmuidx(oi), + }; + TLBLookupOutput o; + int flags, wp_flags; - tcg_debug_assert(mmu_idx < NB_MMU_MODES); - - /* Adjust the given return address. */ - retaddr -= GETPC_ADJ; - - index = tlb_index(cpu, mmu_idx, addr); - tlbe = tlb_entry(cpu, mmu_idx, addr); - - /* Check TLB entry and enforce page permissions. */ - flags = TLB_FLAGS_MASK; - if (!tlb_hit(tlb_addr_write(tlbe), addr)) { - if (!tlbtree_hit(cpu, mmu_idx, MMU_DATA_STORE, addr)) { - tlb_fill_align(cpu, addr, MMU_DATA_STORE, mmu_idx, - mop, size, false, retaddr); - did_tlb_fill = true; - index = tlb_index(cpu, mmu_idx, addr); - tlbe = tlb_entry(cpu, mmu_idx, addr); - /* - * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, - * to force the next access through tlb_fill. We've just - * called tlb_fill, so we know that this entry *is* valid. - */ - flags &= ~TLB_INVALID_MASK; - } - } - full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; + i.size = memop_size(i.memop_probe); + tlb_lookup_nofail(cpu, &o, &i); /* * Let the guest notice RMW on a write-only page. * We have just verified that the page is writable. */ - if (unlikely(!(full->prot & PAGE_READ))) { - tlb_fill_align(cpu, addr, MMU_DATA_LOAD, mmu_idx, - 0, size, false, retaddr); + if (unlikely(!(o.full.prot & PAGE_READ))) { + tlb_fill_align(cpu, addr, MMU_DATA_LOAD, i.mmu_idx, + 0, i.size, false, i.ra); /* * Since we don't support reads and writes to different * addresses, and we do have the proper page loaded for @@ -1871,12 +1865,13 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, } /* Enforce guest required alignment, if not handled by tlb_fill_align. */ - if (!did_tlb_fill && (addr & ((1 << memop_alignment_bits(mop)) - 1))) { - cpu_unaligned_access(cpu, addr, MMU_DATA_STORE, mmu_idx, retaddr); + if (!o.did_tlb_fill + && (addr & ((1 << memop_alignment_bits(i.memop_probe)) - 1))) { + cpu_unaligned_access(cpu, addr, MMU_DATA_STORE, i.mmu_idx, i.ra); } /* Enforce qemu required alignment. */ - if (unlikely(addr & (size - 1))) { + if (unlikely(addr & (i.size - 1))) { /* * We get here if guest alignment was not requested, or was not * enforced by cpu_unaligned_access or tlb_fill_align above. @@ -1886,41 +1881,33 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, goto stop_the_world; } - /* Collect tlb flags for read and write. */ - flags &= tlbe->addr_read | tlb_addr_write(tlbe); - /* Notice an IO access or a needs-MMU-lookup access */ + flags = o.flags; if (unlikely(flags & (TLB_MMIO | TLB_DISCARD_WRITE))) { /* There's really nothing that can be done to support this apart from stop-the-world. */ goto stop_the_world; } - hostaddr = (void *)((uintptr_t)addr + tlbe->addend); - if (unlikely(flags & TLB_NOTDIRTY)) { - notdirty_write(cpu, addr, size, full, retaddr); + notdirty_write(cpu, addr, i.size, &o.full, i.ra); } - if (unlikely(flags & TLB_FORCE_SLOW)) { - int wp_flags = 0; - - if (full->slow_flags[MMU_DATA_STORE] & TLB_WATCHPOINT) { - wp_flags |= BP_MEM_WRITE; - } - if (full->slow_flags[MMU_DATA_LOAD] & TLB_WATCHPOINT) { - wp_flags |= BP_MEM_READ; - } - if (wp_flags) { - cpu_check_watchpoint(cpu, addr, size, - full->attrs, wp_flags, retaddr); - } + wp_flags = 0; + if (flags & TLB_WATCHPOINT) { + wp_flags |= BP_MEM_WRITE; + } + if (o.full.slow_flags[MMU_DATA_LOAD] & TLB_WATCHPOINT) { + wp_flags |= BP_MEM_READ; + } + if (unlikely(wp_flags)) { + cpu_check_watchpoint(cpu, addr, i.size, o.full.attrs, wp_flags, i.ra); } - return hostaddr; + return o.haddr; stop_the_world: - cpu_loop_exit_atomic(cpu, retaddr); + cpu_loop_exit_atomic(cpu, i.ra); } /* From patchwork Thu Nov 14 16:01:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B750D68B33 for ; Thu, 14 Nov 2024 16:11:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ1-0005Xs-Sj; Thu, 14 Nov 2024 11:02:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052R-LG for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcID-0002HB-KZ for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:07 -0500 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-2e2eba31d3aso649399a91.2 for ; Thu, 14 Nov 2024 08:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600118; x=1732204918; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=q0lcs2GzNK+WCVKsOkPx1Msyr0hhchWTgy7GKRRvyJs=; b=NwD7EbbbqdKYfkiyfIiwWXoQ+5zMdhGPGEPCtp4lhIZF5/xmydmqs8Q87c3cpEZtXJ eo6S5syHhFUiEAb3tss67e5uXjUg16xgSde+7jEkzlmt2aD4VY81nrmuyqZsqc2RlhVl lqPe8UZsQFrKLpNjdrhKX8R0uwWFAWXANSGv/gQtqr1IwOAhfwMIT08wMR+SWP0aqpXP JDbsnQJbHdzpPowaEo4BmANm8Hl+2x60UoHtn+m9AUq28IFXc24pptM2wzQ04WvgYIZ4 pDE7TodOJp3v6+BbJhl1nMXzlpV7zdSeElJ2RqF/tTJhADU17pD4fhp7gU58NE8Kjhrg LdFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600118; x=1732204918; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q0lcs2GzNK+WCVKsOkPx1Msyr0hhchWTgy7GKRRvyJs=; b=AJTRoK5MTib93qqIvpIiKcSieKC68ZHwBl5pHhBaw0cSDJwXMutv8mBS2fJxq1VPvY RtFBasbFyUz4vFbYoce24dsOud8/jYIC05sko1xO0TkG9QCcTofKjavI0XFpQVsiLZo+ JnYm0yLCR26npSG+FFzH7qWgzaFbxaHJZEfjMD/df1LHF3wLLv15DI3r1McuKSPIkAGg e57WeAQLd8IrhOO8iuP/a7VOg9JOmO1HoOWDlGkZ1gOlokbeqVa8+MHsnH47VvKlYmaM zsokKBK7gwz95g1k/dkICnlcn5oVNba0RjIp/uuKO7MTmyS92z6f8CM4EATaJSv0h5EW sn2g== X-Gm-Message-State: AOJu0Yxfc+AdYXnLZ5TkwCRyaMtQd4BCHpq8nmtSyurwix8zjhVPEEAg qJlnxYw/k3zb18FgthNr2SZkKVudXo1wEw+mPPmHbFKUAcVTo/C41ptIouay2GGeus1BzBJwb6n x X-Google-Smtp-Source: AGHT+IEtaxTu1otCcYY3xiXIG6nin4XuHZBue8sCsc6LMnjI3k75L1/JQ20UEVpUx5raZdSV9tgvVg== X-Received: by 2002:a17:90b:4a01:b0:2e9:2e69:ee10 with SMTP id 98e67ed59e1d1-2e9b173d95cmr32274780a91.19.1731600117798; Thu, 14 Nov 2024 08:01:57 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:57 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 29/54] accel/tcg: Partially unify MMULookupPageData and TLBLookupOutput Date: Thu, 14 Nov 2024 08:01:05 -0800 Message-ID: <20241114160131.48616-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 151 ++++++++++++++++++++++----------------------- 1 file changed, 74 insertions(+), 77 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index a33bebf55a..8f459be5a8 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1684,10 +1684,7 @@ bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, */ typedef struct MMULookupPageData { - CPUTLBEntryFull *full; - void *haddr; vaddr addr; - int flags; int size; TLBLookupOutput o; } MMULookupPageData; @@ -1724,10 +1721,6 @@ static void mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, }; tlb_lookup_nofail(cpu, &data->o, &i); - - data->full = &data->o.full; - data->flags = data->o.flags; - data->haddr = data->o.haddr; } /** @@ -1743,24 +1736,22 @@ static void mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, static void mmu_watch_or_dirty(CPUState *cpu, MMULookupPageData *data, MMUAccessType access_type, uintptr_t ra) { - CPUTLBEntryFull *full = data->full; - vaddr addr = data->addr; - int flags = data->flags; - int size = data->size; + int flags = data->o.flags; /* On watchpoint hit, this will longjmp out. */ if (flags & TLB_WATCHPOINT) { int wp = access_type == MMU_DATA_STORE ? BP_MEM_WRITE : BP_MEM_READ; - cpu_check_watchpoint(cpu, addr, size, full->attrs, wp, ra); + cpu_check_watchpoint(cpu, data->addr, data->size, + data->o.full.attrs, wp, ra); flags &= ~TLB_WATCHPOINT; } /* Note that notdirty is only set for writes. */ if (flags & TLB_NOTDIRTY) { - notdirty_write(cpu, addr, size, full, ra); + notdirty_write(cpu, data->addr, data->size, &data->o.full, ra); flags &= ~TLB_NOTDIRTY; } - data->flags = flags; + data->o.flags = flags; } /** @@ -1795,7 +1786,7 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, if (likely(!crosspage)) { mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra); - flags = l->page[0].flags; + flags = l->page[0].o.flags; if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) { mmu_watch_or_dirty(cpu, &l->page[0], type, ra); } @@ -1812,7 +1803,7 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra); mmu_lookup1(cpu, &l->page[1], 0, l->mmu_idx, type, ra); - flags = l->page[0].flags | l->page[1].flags; + flags = l->page[0].o.flags | l->page[1].o.flags; if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) { mmu_watch_or_dirty(cpu, &l->page[0], type, ra); mmu_watch_or_dirty(cpu, &l->page[1], type, ra); @@ -2029,7 +2020,7 @@ static Int128 do_ld16_mmio_beN(CPUState *cpu, CPUTLBEntryFull *full, */ static uint64_t do_ld_bytes_beN(MMULookupPageData *p, uint64_t ret_be) { - uint8_t *haddr = p->haddr; + uint8_t *haddr = p->o.haddr; int i, size = p->size; for (i = 0; i < size; i++) { @@ -2047,7 +2038,7 @@ static uint64_t do_ld_bytes_beN(MMULookupPageData *p, uint64_t ret_be) */ static uint64_t do_ld_parts_beN(MMULookupPageData *p, uint64_t ret_be) { - void *haddr = p->haddr; + void *haddr = p->o.haddr; int size = p->size; do { @@ -2097,7 +2088,7 @@ static uint64_t do_ld_parts_beN(MMULookupPageData *p, uint64_t ret_be) static uint64_t do_ld_whole_be4(MMULookupPageData *p, uint64_t ret_be) { int o = p->addr & 3; - uint32_t x = load_atomic4(p->haddr - o); + uint32_t x = load_atomic4(p->o.haddr - o); x = cpu_to_be32(x); x <<= o * 8; @@ -2117,7 +2108,7 @@ static uint64_t do_ld_whole_be8(CPUState *cpu, uintptr_t ra, MMULookupPageData *p, uint64_t ret_be) { int o = p->addr & 7; - uint64_t x = load_atomic8_or_exit(cpu, ra, p->haddr - o); + uint64_t x = load_atomic8_or_exit(cpu, ra, p->o.haddr - o); x = cpu_to_be64(x); x <<= o * 8; @@ -2137,7 +2128,7 @@ static Int128 do_ld_whole_be16(CPUState *cpu, uintptr_t ra, MMULookupPageData *p, uint64_t ret_be) { int o = p->addr & 15; - Int128 x, y = load_atomic16_or_exit(cpu, ra, p->haddr - o); + Int128 x, y = load_atomic16_or_exit(cpu, ra, p->o.haddr - o); int size = p->size; if (!HOST_BIG_ENDIAN) { @@ -2160,8 +2151,8 @@ static uint64_t do_ld_beN(CPUState *cpu, MMULookupPageData *p, MemOp atom; unsigned tmp, half_size; - if (unlikely(p->flags & TLB_MMIO)) { - return do_ld_mmio_beN(cpu, p->full, ret_be, p->addr, p->size, + if (unlikely(p->o.flags & TLB_MMIO)) { + return do_ld_mmio_beN(cpu, &p->o.full, ret_be, p->addr, p->size, mmu_idx, type, ra); } @@ -2210,8 +2201,9 @@ static Int128 do_ld16_beN(CPUState *cpu, MMULookupPageData *p, uint64_t b; MemOp atom; - if (unlikely(p->flags & TLB_MMIO)) { - return do_ld16_mmio_beN(cpu, p->full, a, p->addr, size, mmu_idx, ra); + if (unlikely(p->o.flags & TLB_MMIO)) { + return do_ld16_mmio_beN(cpu, &p->o.full, a, p->addr, + size, mmu_idx, ra); } /* @@ -2223,7 +2215,7 @@ static Int128 do_ld16_beN(CPUState *cpu, MMULookupPageData *p, case MO_ATOM_SUBALIGN: p->size = size - 8; a = do_ld_parts_beN(p, a); - p->haddr += size - 8; + p->o.haddr += size - 8; p->size = 8; b = do_ld_parts_beN(p, 0); break; @@ -2242,7 +2234,7 @@ static Int128 do_ld16_beN(CPUState *cpu, MMULookupPageData *p, case MO_ATOM_NONE: p->size = size - 8; a = do_ld_bytes_beN(p, a); - b = ldq_be_p(p->haddr + size - 8); + b = ldq_be_p(p->o.haddr + size - 8); break; default: @@ -2255,10 +2247,11 @@ static Int128 do_ld16_beN(CPUState *cpu, MMULookupPageData *p, static uint8_t do_ld_1(CPUState *cpu, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { - if (unlikely(p->flags & TLB_MMIO)) { - return do_ld_mmio_beN(cpu, p->full, 0, p->addr, 1, mmu_idx, type, ra); + if (unlikely(p->o.flags & TLB_MMIO)) { + return do_ld_mmio_beN(cpu, &p->o.full, 0, p->addr, 1, + mmu_idx, type, ra); } else { - return *(uint8_t *)p->haddr; + return *(uint8_t *)p->o.haddr; } } @@ -2267,14 +2260,15 @@ static uint16_t do_ld_2(CPUState *cpu, MMULookupPageData *p, int mmu_idx, { uint16_t ret; - if (unlikely(p->flags & TLB_MMIO)) { - ret = do_ld_mmio_beN(cpu, p->full, 0, p->addr, 2, mmu_idx, type, ra); + if (unlikely(p->o.flags & TLB_MMIO)) { + ret = do_ld_mmio_beN(cpu, &p->o.full, 0, p->addr, 2, + mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap16(ret); } } else { /* Perform the load host endian, then swap if necessary. */ - ret = load_atom_2(cpu, ra, p->haddr, memop); + ret = load_atom_2(cpu, ra, p->o.haddr, memop); if (memop & MO_BSWAP) { ret = bswap16(ret); } @@ -2287,14 +2281,15 @@ static uint32_t do_ld_4(CPUState *cpu, MMULookupPageData *p, int mmu_idx, { uint32_t ret; - if (unlikely(p->flags & TLB_MMIO)) { - ret = do_ld_mmio_beN(cpu, p->full, 0, p->addr, 4, mmu_idx, type, ra); + if (unlikely(p->o.flags & TLB_MMIO)) { + ret = do_ld_mmio_beN(cpu, &p->o.full, 0, p->addr, 4, + mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap32(ret); } } else { /* Perform the load host endian. */ - ret = load_atom_4(cpu, ra, p->haddr, memop); + ret = load_atom_4(cpu, ra, p->o.haddr, memop); if (memop & MO_BSWAP) { ret = bswap32(ret); } @@ -2307,14 +2302,15 @@ static uint64_t do_ld_8(CPUState *cpu, MMULookupPageData *p, int mmu_idx, { uint64_t ret; - if (unlikely(p->flags & TLB_MMIO)) { - ret = do_ld_mmio_beN(cpu, p->full, 0, p->addr, 8, mmu_idx, type, ra); + if (unlikely(p->o.flags & TLB_MMIO)) { + ret = do_ld_mmio_beN(cpu, &p->o.full, 0, p->addr, 8, + mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap64(ret); } } else { /* Perform the load host endian. */ - ret = load_atom_8(cpu, ra, p->haddr, memop); + ret = load_atom_8(cpu, ra, p->o.haddr, memop); if (memop & MO_BSWAP) { ret = bswap64(ret); } @@ -2414,15 +2410,15 @@ static Int128 do_ld16_mmu(CPUState *cpu, vaddr addr, cpu_req_mo(TCG_MO_LD_LD | TCG_MO_ST_LD); crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_LOAD, &l); if (likely(!crosspage)) { - if (unlikely(l.page[0].flags & TLB_MMIO)) { - ret = do_ld16_mmio_beN(cpu, l.page[0].full, 0, addr, 16, + if (unlikely(l.page[0].o.flags & TLB_MMIO)) { + ret = do_ld16_mmio_beN(cpu, &l.page[0].o.full, 0, addr, 16, l.mmu_idx, ra); if ((l.memop & MO_BSWAP) == MO_LE) { ret = bswap128(ret); } } else { /* Perform the load host endian. */ - ret = load_atom_16(cpu, ra, l.page[0].haddr, l.memop); + ret = load_atom_16(cpu, ra, l.page[0].o.haddr, l.memop); if (l.memop & MO_BSWAP) { ret = bswap128(ret); } @@ -2568,10 +2564,10 @@ static uint64_t do_st_leN(CPUState *cpu, MMULookupPageData *p, MemOp atom; unsigned tmp, half_size; - if (unlikely(p->flags & TLB_MMIO)) { - return do_st_mmio_leN(cpu, p->full, val_le, p->addr, + if (unlikely(p->o.flags & TLB_MMIO)) { + return do_st_mmio_leN(cpu, &p->o.full, val_le, p->addr, p->size, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { return val_le >> (p->size * 8); } @@ -2582,7 +2578,7 @@ static uint64_t do_st_leN(CPUState *cpu, MMULookupPageData *p, atom = mop & MO_ATOM_MASK; switch (atom) { case MO_ATOM_SUBALIGN: - return store_parts_leN(p->haddr, p->size, val_le); + return store_parts_leN(p->o.haddr, p->size, val_le); case MO_ATOM_IFALIGN_PAIR: case MO_ATOM_WITHIN16_PAIR: @@ -2593,9 +2589,9 @@ static uint64_t do_st_leN(CPUState *cpu, MMULookupPageData *p, ? p->size == half_size : p->size >= half_size) { if (!HAVE_al8_fast && p->size <= 4) { - return store_whole_le4(p->haddr, p->size, val_le); + return store_whole_le4(p->o.haddr, p->size, val_le); } else if (HAVE_al8) { - return store_whole_le8(p->haddr, p->size, val_le); + return store_whole_le8(p->o.haddr, p->size, val_le); } else { cpu_loop_exit_atomic(cpu, ra); } @@ -2605,7 +2601,7 @@ static uint64_t do_st_leN(CPUState *cpu, MMULookupPageData *p, case MO_ATOM_IFALIGN: case MO_ATOM_WITHIN16: case MO_ATOM_NONE: - return store_bytes_leN(p->haddr, p->size, val_le); + return store_bytes_leN(p->o.haddr, p->size, val_le); default: g_assert_not_reached(); @@ -2622,10 +2618,10 @@ static uint64_t do_st16_leN(CPUState *cpu, MMULookupPageData *p, int size = p->size; MemOp atom; - if (unlikely(p->flags & TLB_MMIO)) { - return do_st16_mmio_leN(cpu, p->full, val_le, p->addr, + if (unlikely(p->o.flags & TLB_MMIO)) { + return do_st16_mmio_leN(cpu, &p->o.full, val_le, p->addr, size, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { return int128_gethi(val_le) >> ((size - 8) * 8); } @@ -2636,8 +2632,8 @@ static uint64_t do_st16_leN(CPUState *cpu, MMULookupPageData *p, atom = mop & MO_ATOM_MASK; switch (atom) { case MO_ATOM_SUBALIGN: - store_parts_leN(p->haddr, 8, int128_getlo(val_le)); - return store_parts_leN(p->haddr + 8, p->size - 8, + store_parts_leN(p->o.haddr, 8, int128_getlo(val_le)); + return store_parts_leN(p->o.haddr + 8, p->size - 8, int128_gethi(val_le)); case MO_ATOM_WITHIN16_PAIR: @@ -2645,7 +2641,7 @@ static uint64_t do_st16_leN(CPUState *cpu, MMULookupPageData *p, if (!HAVE_CMPXCHG128) { cpu_loop_exit_atomic(cpu, ra); } - return store_whole_le16(p->haddr, p->size, val_le); + return store_whole_le16(p->o.haddr, p->size, val_le); case MO_ATOM_IFALIGN_PAIR: /* @@ -2655,8 +2651,8 @@ static uint64_t do_st16_leN(CPUState *cpu, MMULookupPageData *p, case MO_ATOM_IFALIGN: case MO_ATOM_WITHIN16: case MO_ATOM_NONE: - stq_le_p(p->haddr, int128_getlo(val_le)); - return store_bytes_leN(p->haddr + 8, p->size - 8, + stq_le_p(p->o.haddr, int128_getlo(val_le)); + return store_bytes_leN(p->o.haddr + 8, p->size - 8, int128_gethi(val_le)); default: @@ -2667,69 +2663,69 @@ static uint64_t do_st16_leN(CPUState *cpu, MMULookupPageData *p, static void do_st_1(CPUState *cpu, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { - if (unlikely(p->flags & TLB_MMIO)) { - do_st_mmio_leN(cpu, p->full, val, p->addr, 1, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + if (unlikely(p->o.flags & TLB_MMIO)) { + do_st_mmio_leN(cpu, &p->o.full, val, p->addr, 1, mmu_idx, ra); + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { - *(uint8_t *)p->haddr = val; + *(uint8_t *)p->o.haddr = val; } } static void do_st_2(CPUState *cpu, MMULookupPageData *p, uint16_t val, int mmu_idx, MemOp memop, uintptr_t ra) { - if (unlikely(p->flags & TLB_MMIO)) { + if (unlikely(p->o.flags & TLB_MMIO)) { if ((memop & MO_BSWAP) != MO_LE) { val = bswap16(val); } - do_st_mmio_leN(cpu, p->full, val, p->addr, 2, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + do_st_mmio_leN(cpu, &p->o.full, val, p->addr, 2, mmu_idx, ra); + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { /* Swap to host endian if necessary, then store. */ if (memop & MO_BSWAP) { val = bswap16(val); } - store_atom_2(cpu, ra, p->haddr, memop, val); + store_atom_2(cpu, ra, p->o.haddr, memop, val); } } static void do_st_4(CPUState *cpu, MMULookupPageData *p, uint32_t val, int mmu_idx, MemOp memop, uintptr_t ra) { - if (unlikely(p->flags & TLB_MMIO)) { + if (unlikely(p->o.flags & TLB_MMIO)) { if ((memop & MO_BSWAP) != MO_LE) { val = bswap32(val); } - do_st_mmio_leN(cpu, p->full, val, p->addr, 4, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + do_st_mmio_leN(cpu, &p->o.full, val, p->addr, 4, mmu_idx, ra); + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { /* Swap to host endian if necessary, then store. */ if (memop & MO_BSWAP) { val = bswap32(val); } - store_atom_4(cpu, ra, p->haddr, memop, val); + store_atom_4(cpu, ra, p->o.haddr, memop, val); } } static void do_st_8(CPUState *cpu, MMULookupPageData *p, uint64_t val, int mmu_idx, MemOp memop, uintptr_t ra) { - if (unlikely(p->flags & TLB_MMIO)) { + if (unlikely(p->o.flags & TLB_MMIO)) { if ((memop & MO_BSWAP) != MO_LE) { val = bswap64(val); } - do_st_mmio_leN(cpu, p->full, val, p->addr, 8, mmu_idx, ra); - } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { + do_st_mmio_leN(cpu, &p->o.full, val, p->addr, 8, mmu_idx, ra); + } else if (unlikely(p->o.flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { /* Swap to host endian if necessary, then store. */ if (memop & MO_BSWAP) { val = bswap64(val); } - store_atom_8(cpu, ra, p->haddr, memop, val); + store_atom_8(cpu, ra, p->o.haddr, memop, val); } } @@ -2822,19 +2818,20 @@ static void do_st16_mmu(CPUState *cpu, vaddr addr, Int128 val, cpu_req_mo(TCG_MO_LD_ST | TCG_MO_ST_ST); crosspage = mmu_lookup(cpu, addr, oi, ra, MMU_DATA_STORE, &l); if (likely(!crosspage)) { - if (unlikely(l.page[0].flags & TLB_MMIO)) { + if (unlikely(l.page[0].o.flags & TLB_MMIO)) { if ((l.memop & MO_BSWAP) != MO_LE) { val = bswap128(val); } - do_st16_mmio_leN(cpu, l.page[0].full, val, addr, 16, l.mmu_idx, ra); - } else if (unlikely(l.page[0].flags & TLB_DISCARD_WRITE)) { + do_st16_mmio_leN(cpu, &l.page[0].o.full, val, addr, + 16, l.mmu_idx, ra); + } else if (unlikely(l.page[0].o.flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { /* Swap to host endian if necessary, then store. */ if (l.memop & MO_BSWAP) { val = bswap128(val); } - store_atom_16(cpu, ra, l.page[0].haddr, l.memop, val); + store_atom_16(cpu, ra, l.page[0].o.haddr, l.memop, val); } return; } From patchwork Thu Nov 14 16:01:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56B8BD68B34 for ; Thu, 14 Nov 2024 16:08:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJl-000746-3j; Thu, 14 Nov 2024 11:03:38 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIH-00051a-Ly for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:07 -0500 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcID-0002HH-LR for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:04 -0500 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-7245a9d0e92so810658b3a.0 for ; Thu, 14 Nov 2024 08:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600119; x=1732204919; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=I5LJCTHDiHg5qibWbzvdQL13bnVE+48Q0Fcqf1mDluc=; b=dYKiJJX3bCzbH67y6W29tKuwasvWt4/6uRtdxwZHJotXkdBQoGnpcOHkERSPguVIt6 oOY23LsnSUFuDIj/rOFvurDemGuZ+vqrebSdUygJUM9BttS3ddrFYvCsAZQn9Rbm8p5P GiaWrk7oPdhSEwk14UHFJTOYQlVlUBDOCET4nMB9eVof0T70h7ebSAfaTnyTUzjXrpkh MCqGD+nt4PMhLG5Lbmz3GT1hs//ISWM7osLVdGcUpnGju+C6ClKzZwrhtwnhCvGcZ3mq yFw60LrizxqRQ2nCvxN8GadFN6GeiLL2t8tbnmPAiEQLD+q0/vB8+Xes+hcR96tBCkBJ BDag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600119; x=1732204919; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=I5LJCTHDiHg5qibWbzvdQL13bnVE+48Q0Fcqf1mDluc=; b=OlH9SYmA/PZ/uSI5Rx9K0QmrHHM8CmPXUR8QTL54V5ysOC86Qz05YLbj5CFFvCrPOX apwGEZQjjAQu0cu44N+W9Fsjv6lpv6Sw134D+UO8MOlRhnsb95LXaXQXBATM0fkLyg5M GSRZ2593mHCil4iqdSZF9A0wJ+XorFDzG0bl5HdRWvYNpsm5AJf5TpojTqT68SFy0aqg mkD2kL46vwVr6G+n++Wu6qPiZhpVtQ9YY2A9Vyu4w4vd5vIvqIrDDpKXa47/egHKQcYh 2u5g0sGryjTGdXSHwkZ7iPct5DhSmPATfpm5BEsgIXrpg+J1h+v7q0CGqO9IxrCawgwF fEng== X-Gm-Message-State: AOJu0Yztk7NiREMyFle/gW1Rc5ts0YFU1hwmlLZsV9C6hTCHJkEsVdXl 5ZaeouLlPhskVT029DL9900TUt5SKScmyT+7fuKWk58E4CkoZ/CcCKXcIyj2VA/6bMP+/Dw3c72 p X-Google-Smtp-Source: AGHT+IFtqNE4fDPhggwOVTT8HUG4faBR+wiHn7so7IBz8pIQxvUeMUYmt4T7rrgnrw3ulxBhRsPVgA== X-Received: by 2002:a17:90b:3c12:b0:2e2:b2ce:e41e with SMTP id 98e67ed59e1d1-2e9f2c78421mr8571311a91.13.1731600118769; Thu, 14 Nov 2024 08:01:58 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:58 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 30/54] accel/tcg: Merge mmu_lookup1 into mmu_lookup Date: Thu, 14 Nov 2024 08:01:06 -0800 Message-ID: <20241114160131.48616-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Reuse most of TLBLookupInput between calls to tlb_lookup. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 65 ++++++++++++++++++---------------------------- 1 file changed, 25 insertions(+), 40 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 8f459be5a8..981098a6f2 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1695,34 +1695,6 @@ typedef struct MMULookupLocals { int mmu_idx; } MMULookupLocals; -/** - * mmu_lookup1: translate one page - * @cpu: generic cpu state - * @data: lookup parameters - * @memop: memory operation for the access, or 0 - * @mmu_idx: virtual address context - * @access_type: load/store/code - * @ra: return address into tcg generated code, or 0 - * - * Resolve the translation for the one page at @data.addr, filling in - * the rest of @data with the results. If the translation fails, - * tlb_fill_align will longjmp out. - */ -static void mmu_lookup1(CPUState *cpu, MMULookupPageData *data, MemOp memop, - int mmu_idx, MMUAccessType access_type, uintptr_t ra) -{ - TLBLookupInput i = { - .addr = data->addr, - .ra = ra, - .access_type = access_type, - .memop_probe = memop, - .size = data->size, - .mmu_idx = mmu_idx, - }; - - tlb_lookup_nofail(cpu, &data->o, &i); -} - /** * mmu_watch_or_dirty * @cpu: generic cpu state @@ -1769,26 +1741,36 @@ static void mmu_watch_or_dirty(CPUState *cpu, MMULookupPageData *data, static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, uintptr_t ra, MMUAccessType type, MMULookupLocals *l) { + MemOp memop = get_memop(oi); + int mmu_idx = get_mmuidx(oi); + TLBLookupInput i = { + .addr = addr, + .ra = ra, + .access_type = type, + .memop_probe = memop, + .size = memop_size(memop), + .mmu_idx = mmu_idx, + }; bool crosspage; int flags; - l->memop = get_memop(oi); - l->mmu_idx = get_mmuidx(oi); + l->memop = memop; + l->mmu_idx = mmu_idx; - tcg_debug_assert(l->mmu_idx < NB_MMU_MODES); + tcg_debug_assert(mmu_idx < NB_MMU_MODES); l->page[0].addr = addr; - l->page[0].size = memop_size(l->memop); - l->page[1].addr = (addr + l->page[0].size - 1) & TARGET_PAGE_MASK; + l->page[0].size = i.size; + l->page[1].addr = (addr + i.size - 1) & TARGET_PAGE_MASK; l->page[1].size = 0; crosspage = (addr ^ l->page[1].addr) & TARGET_PAGE_MASK; if (likely(!crosspage)) { - mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra); + tlb_lookup_nofail(cpu, &l->page[0].o, &i); flags = l->page[0].o.flags; if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) { - mmu_watch_or_dirty(cpu, &l->page[0], type, ra); + mmu_watch_or_dirty(cpu, &l->page[0], i.access_type, i.ra); } if (unlikely(flags & TLB_BSWAP)) { l->memop ^= MO_BSWAP; @@ -1796,17 +1778,20 @@ static bool mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, } else { /* Finish compute of page crossing. */ int size0 = l->page[1].addr - addr; - l->page[1].size = l->page[0].size - size0; + l->page[1].size = i.size - size0; l->page[0].size = size0; /* Lookup both pages, recognizing exceptions from either. */ - mmu_lookup1(cpu, &l->page[0], l->memop, l->mmu_idx, type, ra); - mmu_lookup1(cpu, &l->page[1], 0, l->mmu_idx, type, ra); + i.size = size0; + tlb_lookup_nofail(cpu, &l->page[0].o, &i); + i.addr = l->page[1].addr; + i.size = l->page[1].size; + tlb_lookup_nofail(cpu, &l->page[1].o, &i); flags = l->page[0].o.flags | l->page[1].o.flags; if (unlikely(flags & (TLB_WATCHPOINT | TLB_NOTDIRTY))) { - mmu_watch_or_dirty(cpu, &l->page[0], type, ra); - mmu_watch_or_dirty(cpu, &l->page[1], type, ra); + mmu_watch_or_dirty(cpu, &l->page[0], i.access_type, i.ra); + mmu_watch_or_dirty(cpu, &l->page[1], i.access_type, i.ra); } /* From patchwork Thu Nov 14 16:01:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2BD7D68B34 for ; Thu, 14 Nov 2024 16:07:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJY-0006ik-TP; Thu, 14 Nov 2024 11:03:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIJ-000527-PF for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:09 -0500 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcID-0002HL-LI for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:05 -0500 Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-2e9ff7a778cso766594a91.1 for ; Thu, 14 Nov 2024 08:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600120; x=1732204920; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=gRqWI+rp5TQr50PN/dQhc9ynBvziyOUstyxN8uI5DJA=; b=zOXir8AT4W9R7vUhOk9GOfjuHugytrZOF2H3G0mB4JM15HGAk1KvkGilEzziqSNI/Y jZwrK8l7MBVNar1h8zwjMyzLQeaK1a8b7qlCeOU8H6oRpR26PliTEg2u+jTwxZ0h8NOC MalEWlBOOAAfx8fAcszj4lJjt9WentAUU5pEygvtbxejChXANTUCOD+Tm8sEepWVoLQj G/ikxU27HRtkHtADcKGW1rgGfxqNOeO80V2v0O0WXpLkjkWCM8RR7NT6t1ecxxoSLxEq 7S75rlm9JpYPNorQ18End7xqgsB0p54eBYFBiPZTvHI/FYiuWT3J+GY1DcTjXEcnJQan dcUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600120; x=1732204920; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gRqWI+rp5TQr50PN/dQhc9ynBvziyOUstyxN8uI5DJA=; b=ms+abNZT2Im/R2D2fV5fPNp+BnASk1mQHAmsJ0uOD7Fpw5dgLQohqEFhBGRxWMhkBw OZQ3VD8KGvKfbe4u44qNkETyD4hKX1BhstPGSfTi6zvD39x2hzfGlsFqYLHXy417rCr8 /nKR3aL0uOmQ3st0yySRsHp4N3NfkSXC7jSi0bBbfkGqjVvh+zuFJaWE6H1cCEAf91mG 1FY0GawSRpZmpN86G4fprF3vzXUhlLlmZffFHSGVp9CZj4A9kIT140yD+AtLwbl/36N2 w33uunjOG7ZWJiPtWikn2HXORjz65aGD1f8yUZemlpzvbmAuTN4Zk2wR6mVxZ2DDW6/A 7T3A== X-Gm-Message-State: AOJu0Yxmc7qQSod1KHH4Wf+ttsiAdDy/ZaHxUYb9q0WUGRzWNKi45Z+y GrEle56I0sBsyalhT7/eqG47NYTL6nBntxVgSQhErR64GbaVbqNwUNZNnNqOtDhw3HzizXVlKom / X-Google-Smtp-Source: AGHT+IFJ5BI//ltdBDttI/6INQQy1I3mSJ5gcLja933v+C+AcspAAagD/nkkOznE3BryN8lA/sIrqA== X-Received: by 2002:a17:90b:3503:b0:2e9:3056:71dd with SMTP id 98e67ed59e1d1-2e9fe62dd8cmr6507773a91.7.1731600119527; Thu, 14 Nov 2024 08:01:59 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:59 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 31/54] accel/tcg: Always use IntervalTree for code lookups Date: Thu, 14 Nov 2024 08:01:07 -0800 Message-ID: <20241114160131.48616-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Because translation is special, we don't need the speed of the direct-mapped softmmu tlb. We cache a lookups in DisasContextBase within the translator loop anyway. Drop the addr_code comparator from CPUTLBEntry. Go directly to the IntervalTree for MMU_INST_FETCH. Derive exec flags from read flags. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/cpu-all.h | 3 ++ include/exec/tlb-common.h | 5 ++- accel/tcg/cputlb.c | 76 ++++++++++++++++++++++++--------------- 3 files changed, 52 insertions(+), 32 deletions(-) diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index 45e6676938..ad160c328a 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -339,6 +339,9 @@ static inline int cpu_mmu_index(CPUState *cs, bool ifetch) (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO \ | TLB_FORCE_SLOW | TLB_DISCARD_WRITE) +/* Filter read flags to exec flags. */ +#define TLB_EXEC_FLAGS_MASK (TLB_MMIO) + /* * Flags stored in CPUTLBEntryFull.slow_flags[x]. * TLB_FORCE_SLOW must be set in CPUTLBEntry.addr_idx[x]. diff --git a/include/exec/tlb-common.h b/include/exec/tlb-common.h index 300f9fae67..feaa471299 100644 --- a/include/exec/tlb-common.h +++ b/include/exec/tlb-common.h @@ -26,7 +26,6 @@ typedef union CPUTLBEntry { struct { uint64_t addr_read; uint64_t addr_write; - uint64_t addr_code; /* * Addend to virtual address to get host address. IO accesses * use the corresponding iotlb value. @@ -35,7 +34,7 @@ typedef union CPUTLBEntry { }; /* * Padding to get a power of two size, as well as index - * access to addr_{read,write,code}. + * access to addr_{read,write}. */ uint64_t addr_idx[(1 << CPU_TLB_ENTRY_BITS) / sizeof(uint64_t)]; } CPUTLBEntry; @@ -92,7 +91,7 @@ struct CPUTLBEntryFull { * Additional tlb flags for use by the slow path. If non-zero, * the corresponding CPUTLBEntry comparator must have TLB_FORCE_SLOW. */ - uint8_t slow_flags[MMU_ACCESS_COUNT]; + uint8_t slow_flags[2]; /* * Allow target-specific additions to this structure. diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 981098a6f2..be2ea1bc70 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -114,8 +114,9 @@ static inline uint64_t tlb_read_idx(const CPUTLBEntry *entry, MMU_DATA_LOAD * sizeof(uint64_t)); QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_write) != MMU_DATA_STORE * sizeof(uint64_t)); - QEMU_BUILD_BUG_ON(offsetof(CPUTLBEntry, addr_code) != - MMU_INST_FETCH * sizeof(uint64_t)); + + tcg_debug_assert(access_type == MMU_DATA_LOAD || + access_type == MMU_DATA_STORE); #if TARGET_LONG_BITS == 32 /* Use qatomic_read, in case of addr_write; only care about low bits. */ @@ -480,8 +481,7 @@ static bool tlb_hit_page_mask_anyprot(CPUTLBEntry *tlb_entry, mask &= TARGET_PAGE_MASK | TLB_INVALID_MASK; return (page == (tlb_entry->addr_read & mask) || - page == (tlb_addr_write(tlb_entry) & mask) || - page == (tlb_entry->addr_code & mask)); + page == (tlb_addr_write(tlb_entry) & mask)); } /* Called with tlb_c.lock held */ @@ -1184,9 +1184,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, /* Now calculate the new entry */ node->copy.addend = addend - addr_page; - tlb_set_compare(full, &node->copy, addr_page, read_flags, - MMU_INST_FETCH, prot & PAGE_EXEC); - if (wp_flags & BP_MEM_READ) { read_flags |= TLB_WATCHPOINT; } @@ -1308,22 +1305,30 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, /* Primary lookup in the fast tlb. */ entry = tlbfast_entry(fast, addr); full = &desc->fulltlb[tlbfast_index(fast, addr)]; - cmp = tlb_read_idx(entry, access_type); - if (tlb_hit(cmp, addr)) { - goto found; + if (access_type != MMU_INST_FETCH) { + cmp = tlb_read_idx(entry, access_type); + if (tlb_hit(cmp, addr)) { + goto found_data; + } } /* Secondary lookup in the IntervalTree. */ node = tlbtree_lookup_addr(desc, addr); if (node) { - cmp = tlb_read_idx(&node->copy, access_type); - if (tlb_hit(cmp, addr)) { - /* Install the cached entry. */ - qemu_spin_lock(&cpu->neg.tlb.c.lock); - copy_tlb_helper_locked(entry, &node->copy); - qemu_spin_unlock(&cpu->neg.tlb.c.lock); - *full = node->full; - goto found; + if (access_type == MMU_INST_FETCH) { + if (node->full.prot & PAGE_EXEC) { + goto found_code; + } + } else { + cmp = tlb_read_idx(&node->copy, access_type); + if (tlb_hit(cmp, addr)) { + /* Install the cached entry. */ + qemu_spin_lock(&cpu->neg.tlb.c.lock); + copy_tlb_helper_locked(entry, &node->copy); + qemu_spin_unlock(&cpu->neg.tlb.c.lock); + *full = node->full; + goto found_data; + } } } @@ -1333,9 +1338,14 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, tcg_debug_assert(probe); return false; } - o->did_tlb_fill = true; + if (access_type == MMU_INST_FETCH) { + node = tlbtree_lookup_addr(desc, addr); + tcg_debug_assert(node); + goto found_code; + } + entry = tlbfast_entry(fast, addr); full = &desc->fulltlb[tlbfast_index(fast, addr)]; cmp = tlb_read_idx(entry, access_type); @@ -1345,14 +1355,29 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, * called tlb_fill_align, so we know that this entry *is* valid. */ flags &= ~TLB_INVALID_MASK; + goto found_data; + + found_data: + flags &= cmp; + flags |= full->slow_flags[access_type]; + o->flags = flags; + o->full = *full; + o->haddr = (void *)((uintptr_t)addr + entry->addend); goto done; - found: - /* Alignment has not been checked by tlb_fill_align. */ - { + found_code: + o->flags = node->copy.addr_read & TLB_EXEC_FLAGS_MASK; + o->full = node->full; + o->haddr = (void *)((uintptr_t)addr + node->copy.addend); + goto done; + + done: + if (!o->did_tlb_fill) { int a_bits = memop_alignment_bits(memop); /* + * Alignment has not been checked by tlb_fill_align. + * * The TLB_CHECK_ALIGNED check differs from the normal alignment * check, in that this is based on the atomicity of the operation. * The intended use case is the ARM memory type field of each PTE, @@ -1366,13 +1391,6 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, cpu_unaligned_access(cpu, addr, access_type, i->mmu_idx, i->ra); } } - - done: - flags &= cmp; - flags |= full->slow_flags[access_type]; - o->flags = flags; - o->full = *full; - o->haddr = (void *)((uintptr_t)addr + entry->addend); return true; } From patchwork Thu Nov 14 16:01:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A0C4D68B34 for ; Thu, 14 Nov 2024 16:11:02 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ5-0005cE-4L; Thu, 14 Nov 2024 11:02:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052J-Jh for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIF-0002HU-Ij for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:07 -0500 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-7ed9c16f687so615276a12.0 for ; Thu, 14 Nov 2024 08:02:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600121; x=1732204921; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=C92gX+NWcvvyINoBcMpyemaXSV00oH2ZBkWs2vS0nJE=; b=WcY5B3wR5JMvxhfpO2pxX925ODmxlMKnsvSEdi4s6uhJhBvD3V60MqViUXf1tKzxKQ 7qQqrJTfolPTgB/90aj2lEiWqSj3K40nUrF5k/0k8fbfhMpd//nOWuMO6W62EuGEN5Hq kyzHX82LHmucwXRhzNtJvm3xapdchg7l0e7xpKk9MW+9Ca+7k0nqxSwzZq6JlipCiRAc yQKYW80uhuH1pLGEQrWwJV4JLKyVhRWvgeciZYLapevBONGT0EVyHAMLkFmnbN4qzXuf 4RJ4clkueykR+dOD9pQFJhfaf4KNDhVIHLB+3n88Xc3pnhKjVATq39pt+CJtXmGg/1Ro wCeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600121; x=1732204921; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C92gX+NWcvvyINoBcMpyemaXSV00oH2ZBkWs2vS0nJE=; b=r7bcBgRxkYpdGb/QpOxgGg00rZzxp5cP8wSG4SKEYLqx4Gxdf3ywpvKppY7zhS2asG ujSoVfHOf8AF3T8o0N7+8d9303UJvLfoShLjOQ8EqI3gLMfge+dkjfWi8p0ncmdbxa3z TrYfo7H0Yw1Qxoz54yW0BZdCW1DTdT/Mw7+ny/8EbsY7D6Dx75Mue4sIkZSoOjOkA6ok lyjbmEFmOtEdG0xJHUeBs/ZN6VIQlkol0k84lUAZg8LmD8BKJFU3uv8Vl4l8WO3zNVkb gEQorTt/8TO2OAmENY5cwXJEnPLGRFW8IKwqYPosJxCANdJqu+Z1xzFrJzSjXP25x9wT p1XA== X-Gm-Message-State: AOJu0Yyxjg7d8heqz39NtD/MUermjWeP1CSea5spb9w/pIyP6rf7qlCy JSA2MAGsUautGIuHlprCY9uNl5pxh7wKDDSugUG/DDMFDKpWjM235VkVArX8+KxEULPF9fSqRai P X-Google-Smtp-Source: AGHT+IH1bmGFUpP5442RkaAfKsgvmPq8vkXQThC5va/ntSAMd2rmyRSBzkZHXnVp8ir6wCg57S9F4g== X-Received: by 2002:a17:90b:4b0f:b0:2e2:a013:859a with SMTP id 98e67ed59e1d1-2e9f2c4ef3bmr9095877a91.7.1731600120301; Thu, 14 Nov 2024 08:02:00 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.01.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:01:59 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 32/54] accel/tcg: Link CPUTLBEntry to CPUTLBEntryTree Date: Thu, 14 Nov 2024 08:01:08 -0800 Message-ID: <20241114160131.48616-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x536.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Link from the fast tlb entry to the interval tree node. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/tlb-common.h | 2 ++ accel/tcg/cputlb.c | 26 +++++++++++++------------- 2 files changed, 15 insertions(+), 13 deletions(-) diff --git a/include/exec/tlb-common.h b/include/exec/tlb-common.h index feaa471299..3b57d61112 100644 --- a/include/exec/tlb-common.h +++ b/include/exec/tlb-common.h @@ -31,6 +31,8 @@ typedef union CPUTLBEntry { * use the corresponding iotlb value. */ uintptr_t addend; + /* The defining IntervalTree entry. */ + struct CPUTLBEntryTree *tree; }; /* * Padding to get a power of two size, as well as index diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index be2ea1bc70..3282436752 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -490,7 +490,10 @@ static bool tlb_flush_entry_mask_locked(CPUTLBEntry *tlb_entry, vaddr mask) { if (tlb_hit_page_mask_anyprot(tlb_entry, page, mask)) { - memset(tlb_entry, -1, sizeof(*tlb_entry)); + tlb_entry->addr_read = -1; + tlb_entry->addr_write = -1; + tlb_entry->addend = 0; + tlb_entry->tree = NULL; return true; } return false; @@ -1183,6 +1186,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, /* Now calculate the new entry */ node->copy.addend = addend - addr_page; + node->copy.tree = node; if (wp_flags & BP_MEM_READ) { read_flags |= TLB_WATCHPOINT; @@ -1291,7 +1295,6 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, CPUTLBDescFast *fast = &cpu->neg.tlb.f[i->mmu_idx]; vaddr addr = i->addr; MMUAccessType access_type = i->access_type; - CPUTLBEntryFull *full; CPUTLBEntryTree *node; CPUTLBEntry *entry; uint64_t cmp; @@ -1304,9 +1307,9 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, /* Primary lookup in the fast tlb. */ entry = tlbfast_entry(fast, addr); - full = &desc->fulltlb[tlbfast_index(fast, addr)]; if (access_type != MMU_INST_FETCH) { cmp = tlb_read_idx(entry, access_type); + node = entry->tree; if (tlb_hit(cmp, addr)) { goto found_data; } @@ -1326,7 +1329,6 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, qemu_spin_lock(&cpu->neg.tlb.c.lock); copy_tlb_helper_locked(entry, &node->copy); qemu_spin_unlock(&cpu->neg.tlb.c.lock); - *full = node->full; goto found_data; } } @@ -1347,8 +1349,8 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, } entry = tlbfast_entry(fast, addr); - full = &desc->fulltlb[tlbfast_index(fast, addr)]; cmp = tlb_read_idx(entry, access_type); + node = entry->tree; /* * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, * to force the next access through tlb_fill_align. We've just @@ -1359,19 +1361,18 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, found_data: flags &= cmp; - flags |= full->slow_flags[access_type]; + flags |= node->full.slow_flags[access_type]; o->flags = flags; - o->full = *full; - o->haddr = (void *)((uintptr_t)addr + entry->addend); - goto done; + goto found_common; found_code: o->flags = node->copy.addr_read & TLB_EXEC_FLAGS_MASK; + goto found_common; + + found_common: o->full = node->full; o->haddr = (void *)((uintptr_t)addr + node->copy.addend); - goto done; - done: if (!o->did_tlb_fill) { int a_bits = memop_alignment_bits(memop); @@ -1669,7 +1670,6 @@ bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, bool is_store, struct qemu_plugin_hwaddr *data) { CPUTLBEntry *tlbe = tlb_entry(cpu, mmu_idx, addr); - uintptr_t index = tlb_index(cpu, mmu_idx, addr); MMUAccessType access_type = is_store ? MMU_DATA_STORE : MMU_DATA_LOAD; uint64_t tlb_addr = tlb_read_idx(tlbe, access_type); CPUTLBEntryFull *full; @@ -1678,7 +1678,7 @@ bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, return false; } - full = &cpu->neg.tlb.d[mmu_idx].fulltlb[index]; + full = &tlbe->tree->full; data->phys_addr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); /* We must have an iotlb entry for MMIO */ From patchwork Thu Nov 14 16:01:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 350E7D68B35 for ; Thu, 14 Nov 2024 16:11:11 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJR-0006DX-GS; Thu, 14 Nov 2024 11:03:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIL-00052K-Kt for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIF-0002I2-JR for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:08 -0500 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2ea09a033e2so583641a91.3 for ; Thu, 14 Nov 2024 08:02:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600121; x=1732204921; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=IURXvWR9Voo5SpXlji5M2ea/nd6IIOVKEQdxMQkSin0=; b=A2IqJvA2xpVtZJn3lw34e1h8A9xoVU1I3HH2NV6TTnOxUHQ8Y1ytbpVCAiJBNeN8j4 AbGRZ1Q8vCgI2nZprhf2ytEP8dFj2Dj28ar4Ij/bZB6eofZng1NixcmA5xbzOA2taAvJ Wt53e0/RxGq8srQWiYw9pF/SZgeI18gKfwDbNx9U5DzzleI9yAf2Vt2cnSOvsNgysm6c 022s1r7h/XHZZRVlZPqd+fjRbtTNmjlsK/QKSFtBtME/IqGPi5qmUWAHrKRaspPjyqkn oefxhBaF598EEHHTRmWozKdiVn5AYAO26e+WnYZbVUqYIUbCowqj5SXQg8YES6au69vf 8Klw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600121; x=1732204921; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IURXvWR9Voo5SpXlji5M2ea/nd6IIOVKEQdxMQkSin0=; b=PMuWRtDpY+IxBDj37s5Vguoz92OsLdA+KHY2lm4WJcJzaFR20Urp4q433q4C0HdWqb /sytLzkLOSDAT4IQRUAKRWpdG8swFaXa2uYvaAcSzX18jE9ox12riGfq5zE4ZBA0L3VE RM8V+OW1FmJpWYm66VJYttMsBFrToxYD0+xTV/Z053PCUzrEHeKh+IQ7HyS2QL2atDGt ibzQ+zFfiEQxzYsMM3uRmIMFGp5/VSp012wobsO975SFGyk7r9g4WFaC9Hcv3QSrckoh CxE2ShaFROFP2hGNugDdwR1T2XX86HTkpb6BY+KSkKREE5nnFqDbglod7WE982Ru1eWf W6Dw== X-Gm-Message-State: AOJu0YyfD0IeK1LxgdE2vyfx1qKlKzMK2VTLW+A0l41HdlVn/cyUSrql XeViVpvMGO+Uc8bLIWKWj6XOuvUxVGSiofS95+ggv6S0PxjyxPg3EiDDDWHlBUwwO0QTOVfBgRa a X-Google-Smtp-Source: AGHT+IEjX+Ulkz2mMI/955PVO4PcTvmrJ75S2hcYlmsqS/eC4gXSvI3ufVaBEIIueRuHWvdVSwHrMg== X-Received: by 2002:a17:90a:e7c3:b0:2e2:cd79:ec06 with SMTP id 98e67ed59e1d1-2e9f2c77384mr9130730a91.10.1731600121352; Thu, 14 Nov 2024 08:02:01 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:00 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 33/54] accel/tcg: Remove CPUTLBDesc.fulltlb Date: Thu, 14 Nov 2024 08:01:09 -0800 Message-ID: <20241114160131.48616-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This array is now write-only, and may be removed. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/hw/core/cpu.h | 1 - accel/tcg/cputlb.c | 34 +++++++--------------------------- 2 files changed, 7 insertions(+), 28 deletions(-) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index 4364ddb1db..5c069f2a00 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -219,7 +219,6 @@ typedef struct CPUTLBDesc { /* maximum number of entries observed in the window */ size_t window_max_entries; size_t n_used_entries; - CPUTLBEntryFull *fulltlb; /* All active tlb entries for this address space. */ IntervalTreeRoot iroot; } CPUTLBDesc; diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 3282436752..7f63dc3fd8 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -149,13 +149,6 @@ static inline CPUTLBEntry *tlbfast_entry(CPUTLBDescFast *fast, vaddr addr) return fast->table + tlbfast_index(fast, addr); } -/* Find the TLB index corresponding to the mmu_idx + address pair. */ -static inline uintptr_t tlb_index(CPUState *cpu, uintptr_t mmu_idx, - vaddr addr) -{ - return tlbfast_index(&cpu->neg.tlb.f[mmu_idx], addr); -} - /* Find the TLB entry corresponding to the mmu_idx + address pair. */ static inline CPUTLBEntry *tlb_entry(CPUState *cpu, uintptr_t mmu_idx, vaddr addr) @@ -270,22 +263,20 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast, } g_free(fast->table); - g_free(desc->fulltlb); tlb_window_reset(desc, now, 0); /* desc->n_used_entries is cleared by the caller */ fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS; fast->table = g_try_new(CPUTLBEntry, new_size); - desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size); /* - * If the allocations fail, try smaller sizes. We just freed some + * If the allocation fails, try smaller sizes. We just freed some * memory, so going back to half of new_size has a good chance of working. * Increased memory pressure elsewhere in the system might cause the * allocations to fail though, so we progressively reduce the allocation * size, aborting if we cannot even allocate the smallest TLB we support. */ - while (fast->table == NULL || desc->fulltlb == NULL) { + while (fast->table == NULL) { if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) { error_report("%s: %s", __func__, strerror(errno)); abort(); @@ -294,9 +285,7 @@ static void tlb_mmu_resize_locked(CPUTLBDesc *desc, CPUTLBDescFast *fast, fast->mask = (new_size - 1) << CPU_TLB_ENTRY_BITS; g_free(fast->table); - g_free(desc->fulltlb); fast->table = g_try_new(CPUTLBEntry, new_size); - desc->fulltlb = g_try_new(CPUTLBEntryFull, new_size); } } @@ -350,7 +339,6 @@ static void tlb_mmu_init(CPUTLBDesc *desc, CPUTLBDescFast *fast, int64_t now) desc->n_used_entries = 0; fast->mask = (n_entries - 1) << CPU_TLB_ENTRY_BITS; fast->table = g_new(CPUTLBEntry, n_entries); - desc->fulltlb = g_new(CPUTLBEntryFull, n_entries); memset(&desc->iroot, 0, sizeof(desc->iroot)); tlb_mmu_flush_locked(desc, fast); } @@ -372,15 +360,9 @@ void tlb_init(CPUState *cpu) void tlb_destroy(CPUState *cpu) { - int i; - qemu_spin_destroy(&cpu->neg.tlb.c.lock); - for (i = 0; i < NB_MMU_MODES; i++) { - CPUTLBDesc *desc = &cpu->neg.tlb.d[i]; - CPUTLBDescFast *fast = &cpu->neg.tlb.f[i]; - - g_free(fast->table); - g_free(desc->fulltlb); + for (int i = 0; i < NB_MMU_MODES; i++) { + g_free(cpu->neg.tlb.f[i].table); interval_tree_free_nodes(&cpu->neg.tlb.d[i].iroot, offsetof(CPUTLBEntryTree, itree)); } @@ -1061,7 +1043,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, CPUTLB *tlb = &cpu->neg.tlb; CPUTLBDesc *desc = &tlb->d[mmu_idx]; MemoryRegionSection *section; - unsigned int index, read_flags, write_flags; + unsigned int read_flags, write_flags; uintptr_t addend; CPUTLBEntry *te; CPUTLBEntryTree *node; @@ -1140,7 +1122,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, wp_flags = cpu_watchpoint_address_matches(cpu, addr_page, TARGET_PAGE_SIZE); - index = tlb_index(cpu, mmu_idx, addr_page); te = tlb_entry(cpu, mmu_idx, addr_page); /* @@ -1179,8 +1160,8 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, * subtract here is that of the page base, and not the same as the * vaddr we add back in io_prepare()/get_page_addr_code(). */ - desc->fulltlb[index] = *full; - full = &desc->fulltlb[index]; + node->full = *full; + full = &node->full; full->xlat_section = iotlb - addr_page; full->phys_addr = paddr_page; @@ -1203,7 +1184,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, tlb_set_compare(full, &node->copy, addr_page, write_flags, MMU_DATA_STORE, prot & PAGE_WRITE); - node->full = *full; copy_tlb_helper_locked(te, &node->copy); desc->n_used_entries++; qemu_spin_unlock(&tlb->c.lock); From patchwork Thu Nov 14 16:01:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875372 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56183D68B35 for ; Thu, 14 Nov 2024 16:10:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJR-0006DW-Gs; Thu, 14 Nov 2024 11:03:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIO-00053M-E9 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIH-0002IL-EO for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:12 -0500 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-2ea0c38f0fdso445138a91.3 for ; Thu, 14 Nov 2024 08:02:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600122; x=1732204922; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=EV/r07ktWnLxfLmkCAX5tos07YFLZkZRGQVALb+yUNI=; b=H0VUJMztw/msRM7o+u2NlBt+hA+iTUI4lbaDdutJIC2HbR5JM5hU1Tg2AcA2mdQke1 kljYLq8X0ik/0Jc4AegTU6E28/4yr86P1vQOrGPVuGcHvTBGWEgAoEofQoVH3Gf/cAN4 18lChzKs/5qzrl3zWRiZPFqbGMAmdeUHCnSV9vr/zEhvFWahLc9vVQq/fKSqFWEKHDq4 YYAX8FkCh9TpT8dWjqs/z9f3LaXaSgy/QiA3G5OWaDO7iz50O19jKXy4Dlty03lGQsYu mKr7889mAdNO6i/xAcLLgj7MigAaMW3GBOG/kMkaknEFVod1c4FEar/peXFtOOcZVRzm x10A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600122; x=1732204922; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EV/r07ktWnLxfLmkCAX5tos07YFLZkZRGQVALb+yUNI=; b=wIVJEUlNX2TYsGyzSKqu/aECHUxCxvqQc4DRqADNcsRRAsjv8bLCwxKxa8pRwvC9Kr nEADhyhY5MQwFHv7o4xNJmJ8NiIz4hMe5uhg78Z6kDoVY23uEKCgPKl2QICAsKKjf1Tn 3blpQeGx9ZXwvi6PJzVWzvdNJ7XWrjGFkrjnIfc4iJ4oHE6kWGf0ZS6pRzM5djg1dzxa 4zoGT0pPNJIC2gYLfyflUQunoSekdvNuT56D4xU0u7PAgQQjydDajuOM9I8NbgIwNM/V wSI6zSNnci6wCLSPRHjurTDPIPY3eqHB1HyW0NBTBO/pga+OCrpkTFQNt+oCIIOjLBLJ SwkQ== X-Gm-Message-State: AOJu0YzJvkkUeZKkqRiaMmfoEeqQBUlHSc7oakmLRPh7hlA5VqrOFeMb tyoaQq3sH7JsJS/lB2TxvHnExfaMaiNmj+chIqItyB3ALgnvaWzR+hLuw9DdfDB8AEI9/s46H64 Y X-Google-Smtp-Source: AGHT+IGO2bsWqDyD+rL/vjKX4YAILOdj+fMZ7o0pC47qOMByxfmN28ckAczbfFCVv3HsxgM3Ws+Xpw== X-Received: by 2002:a17:90b:1c8d:b0:2e2:de27:db09 with SMTP id 98e67ed59e1d1-2e9b17415demr31437722a91.19.1731600122206; Thu, 14 Nov 2024 08:02:02 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:01 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 34/54] target/alpha: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:10 -0800 Message-ID: <20241114160131.48616-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/alpha/cpu.h | 6 +++--- target/alpha/cpu.c | 2 +- target/alpha/helper.c | 23 +++++++++++++++++------ 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/target/alpha/cpu.h b/target/alpha/cpu.h index 3556d3227f..70331c0b83 100644 --- a/target/alpha/cpu.h +++ b/target/alpha/cpu.h @@ -449,9 +449,9 @@ void alpha_cpu_record_sigsegv(CPUState *cs, vaddr address, void alpha_cpu_record_sigbus(CPUState *cs, vaddr address, MMUAccessType access_type, uintptr_t retaddr); #else -bool alpha_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool alpha_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); G_NORETURN void alpha_cpu_do_unaligned_access(CPUState *cpu, vaddr addr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr); diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c index 5d75c941f7..7bcc48420d 100644 --- a/target/alpha/cpu.c +++ b/target/alpha/cpu.c @@ -228,7 +228,7 @@ static const TCGCPUOps alpha_tcg_ops = { .record_sigsegv = alpha_cpu_record_sigsegv, .record_sigbus = alpha_cpu_record_sigbus, #else - .tlb_fill = alpha_cpu_tlb_fill, + .tlb_fill_align = alpha_cpu_tlb_fill_align, .cpu_exec_interrupt = alpha_cpu_exec_interrupt, .cpu_exec_halt = alpha_cpu_has_work, .do_interrupt = alpha_cpu_do_interrupt, diff --git a/target/alpha/helper.c b/target/alpha/helper.c index 2f1000c99f..26eadfe3ca 100644 --- a/target/alpha/helper.c +++ b/target/alpha/helper.c @@ -294,14 +294,21 @@ hwaddr alpha_cpu_get_phys_page_debug(CPUState *cs, vaddr addr) return (fail >= 0 ? -1 : phys); } -bool alpha_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool alpha_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra) { CPUAlphaState *env = cpu_env(cs); target_ulong phys; int prot, fail; + if (addr & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + alpha_cpu_do_unaligned_access(cs, addr, access_type, mmu_idx, ra); + } + fail = get_physical_address(env, addr, 1 << access_type, mmu_idx, &phys, &prot); if (unlikely(fail >= 0)) { @@ -314,11 +321,15 @@ bool alpha_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, env->trap_arg2 = (access_type == MMU_DATA_LOAD ? 0ull : access_type == MMU_DATA_STORE ? 1ull : /* access_type == MMU_INST_FETCH */ -1ull); - cpu_loop_exit_restore(cs, retaddr); + cpu_loop_exit_restore(cs, ra); } - tlb_set_page(cs, addr & TARGET_PAGE_MASK, phys & TARGET_PAGE_MASK, - prot, mmu_idx, TARGET_PAGE_SIZE); + memset(out, 0, sizeof(*out)); + out->phys_addr = phys; + out->prot = prot; + out->attrs = MEMTXATTRS_UNSPECIFIED; + out->lg_page_size = TARGET_PAGE_BITS; + return true; } From patchwork Thu Nov 14 16:01:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 199A8D68B34 for ; Thu, 14 Nov 2024 16:10:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIh-0005JV-52; Thu, 14 Nov 2024 11:02:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIN-00052s-1G for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:11 -0500 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIH-0002IS-EI for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:10 -0500 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-7ea7e250c54so586571a12.0 for ; Thu, 14 Nov 2024 08:02:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600123; x=1732204923; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=tAszhXmbZO1yYFyoLknvdTRmxQFGefE5ni84P6772pk=; b=ydf2ymps/IarHO5o/GpyAgt3AsjHQd14WE2T7hKQKMT4PM/kaZY6oMKLj4XkMaH8l3 /lYbpAlYAKdUY2P0HxO6btQPd9WERHnzN929ACbcb5jD+Q3tNpTRkh8u+qXPHDIW+lY/ lMlnv5g8/HoexYPneOLb14i4AyxSqE3CLfVIvS5tUFTeqfOXWH2pAl5Ip+gk93T3BLMv WUDu+DoQWjXc/QDfe7hlWkSFwBbW6RePz59ltv1Nm2nRTg/xiXNTvy5UrPm0oe3QooG8 YCaw87m2YFo0xOfYYdC/PNJHHg2W7qXwtjMX8WWc8+9BuF7+nBGyetkvOciEX85ypLe3 DZ9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600123; x=1732204923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tAszhXmbZO1yYFyoLknvdTRmxQFGefE5ni84P6772pk=; b=i047K5z0+dH4iunKH45pucYlksl5z1EDJfuVJPSTHrZbpG7hSN8xXE8rSxmkp5Wr52 kOjYDAXWLhXUEet1/5SxzdyQqaeZ7d3QUE/QoeYFvo1VnByat7Eg6g2VSXW4tHcYant+ KrtPlkd22PrN/z7JFKDWb7YoZqd87xWt3uRUi0ysrF5Jsy2jzdQoYOaoQT3VeuGaKIhE pdjkxj1SKtSFE2fnWnwzNs1D2rM7kYYnOvTMjE3jwm0d3fwCQpJchT08NgQpdIfy3y6y PXdKauqPNReHA6aQMh0OLQhQfkYCu5szaVn2XTphsv4mEVy5dSq2WZTFtckjF3yJS3f5 IrBQ== X-Gm-Message-State: AOJu0YwWLGyDfX9nSwlZfA3XlrYpWJzkU8qZp/4ntz/EWBjlVbDzuy6I Q4HkskJvlE9GAOOdsCCLIvK3sTDdHPIevSw6+vc3cEj+EPf1KanE0gIzBSBiGwhHt3/QuiGU+KE 9 X-Google-Smtp-Source: AGHT+IEyV/vCAiEEJ8K8TBwPUbaYg4B2DmSZnWhdKc8ort+PrLfvfAeh3rjDBqzYjVZcpVu2Ohs8iQ== X-Received: by 2002:a17:90b:2e8e:b0:2e2:f044:caaa with SMTP id 98e67ed59e1d1-2ea06a63e77mr3122535a91.37.1731600122925; Thu, 14 Nov 2024 08:02:02 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:02 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 35/54] target/avr: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:11 -0800 Message-ID: <20241114160131.48616-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::529; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x529.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/avr/cpu.h | 7 ++++--- target/avr/cpu.c | 2 +- target/avr/helper.c | 19 ++++++++++++------- 3 files changed, 17 insertions(+), 11 deletions(-) diff --git a/target/avr/cpu.h b/target/avr/cpu.h index 4725535102..cdd3bcd418 100644 --- a/target/avr/cpu.h +++ b/target/avr/cpu.h @@ -23,6 +23,7 @@ #include "cpu-qom.h" #include "exec/cpu-defs.h" +#include "exec/memop.h" #ifdef CONFIG_USER_ONLY #error "AVR 8-bit does not support user mode" @@ -238,9 +239,9 @@ static inline void cpu_set_sreg(CPUAVRState *env, uint8_t sreg) env->sregI = (sreg >> 7) & 0x01; } -bool avr_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool avr_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); #include "exec/cpu-all.h" diff --git a/target/avr/cpu.c b/target/avr/cpu.c index 3132842d56..a7fe869396 100644 --- a/target/avr/cpu.c +++ b/target/avr/cpu.c @@ -211,7 +211,7 @@ static const TCGCPUOps avr_tcg_ops = { .restore_state_to_opc = avr_restore_state_to_opc, .cpu_exec_interrupt = avr_cpu_exec_interrupt, .cpu_exec_halt = avr_cpu_has_work, - .tlb_fill = avr_cpu_tlb_fill, + .tlb_fill_align = avr_cpu_tlb_fill_align, .do_interrupt = avr_cpu_do_interrupt, }; diff --git a/target/avr/helper.c b/target/avr/helper.c index 345708a1b3..a18f11aa9f 100644 --- a/target/avr/helper.c +++ b/target/avr/helper.c @@ -104,11 +104,11 @@ hwaddr avr_cpu_get_phys_page_debug(CPUState *cs, vaddr addr) return addr; /* I assume 1:1 address correspondence */ } -bool avr_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool avr_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr address, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra) { - int prot, page_size = TARGET_PAGE_SIZE; + int prot, lg_page_size = TARGET_PAGE_BITS; uint32_t paddr; address &= TARGET_PAGE_MASK; @@ -141,15 +141,20 @@ bool avr_cpu_tlb_fill(CPUState *cs, vaddr address, int size, * to force tlb_fill to be called for the next access. */ if (probe) { - page_size = 1; + lg_page_size = 0; } else { cpu_env(cs)->fullacc = 1; - cpu_loop_exit_restore(cs, retaddr); + cpu_loop_exit_restore(cs, ra); } } } - tlb_set_page(cs, address, paddr, prot, mmu_idx, page_size); + memset(out, 0, sizeof(*out)); + out->phys_addr = paddr; + out->prot = prot; + out->attrs = MEMTXATTRS_UNSPECIFIED; + out->lg_page_size = lg_page_size; + return true; } From patchwork Thu Nov 14 16:01:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875326 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 52BC0D68B33 for ; Thu, 14 Nov 2024 16:03:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJ8-0005mB-UJ; Thu, 14 Nov 2024 11:02:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIO-00053P-O5 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIL-0002Ic-AU for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:12 -0500 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2e9b55b83d2so651264a91.3 for ; Thu, 14 Nov 2024 08:02:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600124; x=1732204924; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=QF7xEZ+McMkBHbikVqFxssmRyGZDCQmLdpfkAwqKmS0=; b=akE++NJmTcdv3IZB59T9YFINNNU5WjK+2XnnuVSRAg2E5qf0OsmBuaAhQHC4HuhzeB vbciDnrkhNJPQUzpXWL3osYaMm572kJE1Iadkx0K5DOmFnYaP6LFXtefx5v7mdqZRbkI wC0qEDzlFXRNou30JDcPOCB+AN+w0Nsf+rbmS+zEshbLKueCKR/HH5BYW5phfzHEykC7 tnyFvOgePg3S+KzvK50fyjHSjrvohxIRjihTkamtFcBTwXaTyn7jGhjm0qhuCRG8fVQQ Bkm83tL4ezwYYv9S+VFEQnWKZ3UomAclmR79pRmGmSV/DK9Cjyj68Oh4fptltITof029 IwLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600124; x=1732204924; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QF7xEZ+McMkBHbikVqFxssmRyGZDCQmLdpfkAwqKmS0=; b=PiBDb+ulxvJYSFqliUCOdG7Ms9UdoocKnZF7ZuDN0aqz7fUbux191E1DfydmYVIkgo WrmG8XJrqmpD7yDBhE7dDhtwBcwXm3GTQCVkQhK1iENMdCtnw3lARRIdmH3NFjDWO4hA LJgOd6jJfcEb6vk4f/tEWe0yu/3pjoGrQhpsJ7rX96/C0oIBR4Yz9Lqy9sTaUdyMaqLb BFPhU+0U6+qYQ9RiEj5cNUQAHeDlX87csYb0TPPxeCTBjR7/OXBRVmdLNRiqQcsgcBn6 sjgg41kHvF6vRAQqA6Oe5FcrqHDcr9+yqd6rKsdYKGFN02PR2ZCJvqqAkdhS2leBT2B9 CByQ== X-Gm-Message-State: AOJu0YzLA0gGlGpvkbWC2ylU8tSFCa8bdRRQnYV2CuTgPg4vaUKz6y4z 4Q3C7kus7aD67+vHoGz3HJQ0qRUDKpmNgZ3MPv+ltsUxdG72hcyv1i8RD/GVAKbyL251gsvLHiM t X-Google-Smtp-Source: AGHT+IH0C7SLi7d1YnIk4aDAQmiGhzzuP6i5coFmCgfbh1Y/MtJ+bvLwoXl0pz1rmNshi3q9u326SQ== X-Received: by 2002:a17:90b:5444:b0:2e2:e597:6cdc with SMTP id 98e67ed59e1d1-2e9f2c906c0mr7877746a91.22.1731600123715; Thu, 14 Nov 2024 08:02:03 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.03 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:03 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 36/54] target/i386: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:12 -0800 Message-ID: <20241114160131.48616-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/i386/tcg/helper-tcg.h | 6 +++--- target/i386/tcg/sysemu/excp_helper.c | 28 ++++++++++++++++------------ target/i386/tcg/tcg-cpu.c | 2 +- 3 files changed, 20 insertions(+), 16 deletions(-) diff --git a/target/i386/tcg/helper-tcg.h b/target/i386/tcg/helper-tcg.h index 696d6ef016..b2164f41e6 100644 --- a/target/i386/tcg/helper-tcg.h +++ b/target/i386/tcg/helper-tcg.h @@ -79,9 +79,9 @@ void x86_cpu_record_sigsegv(CPUState *cs, vaddr addr, void x86_cpu_record_sigbus(CPUState *cs, vaddr addr, MMUAccessType access_type, uintptr_t ra); #else -bool x86_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool x86_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); G_NORETURN void x86_cpu_do_unaligned_access(CPUState *cs, vaddr vaddr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr); diff --git a/target/i386/tcg/sysemu/excp_helper.c b/target/i386/tcg/sysemu/excp_helper.c index 168ff8e5f3..d23d28fef5 100644 --- a/target/i386/tcg/sysemu/excp_helper.c +++ b/target/i386/tcg/sysemu/excp_helper.c @@ -601,25 +601,29 @@ static bool get_physical_address(CPUX86State *env, vaddr addr, return true; } -bool x86_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool x86_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *full, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, + uintptr_t retaddr) { CPUX86State *env = cpu_env(cs); TranslateResult out; TranslateFault err; + if (addr & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + x86_cpu_do_unaligned_access(cs, addr, access_type, mmu_idx, retaddr); + } + if (get_physical_address(env, addr, access_type, mmu_idx, &out, &err, retaddr)) { - /* - * Even if 4MB pages, we map only one 4KB page in the cache to - * avoid filling it too fast. - */ - assert(out.prot & (1 << access_type)); - tlb_set_page_with_attrs(cs, addr & TARGET_PAGE_MASK, - out.paddr & TARGET_PAGE_MASK, - cpu_get_mem_attrs(env), - out.prot, mmu_idx, out.page_size); + memset(full, 0, sizeof(*full)); + full->phys_addr = out.paddr; + full->prot = out.prot; + full->lg_page_size = ctz32(out.page_size); + full->attrs = cpu_get_mem_attrs(env); return true; } diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c index cca19cd40e..6fce6227c7 100644 --- a/target/i386/tcg/tcg-cpu.c +++ b/target/i386/tcg/tcg-cpu.c @@ -117,7 +117,7 @@ static const TCGCPUOps x86_tcg_ops = { .record_sigsegv = x86_cpu_record_sigsegv, .record_sigbus = x86_cpu_record_sigbus, #else - .tlb_fill = x86_cpu_tlb_fill, + .tlb_fill_align = x86_cpu_tlb_fill_align, .do_interrupt = x86_cpu_do_interrupt, .cpu_exec_halt = x86_cpu_exec_halt, .cpu_exec_interrupt = x86_cpu_exec_interrupt, From patchwork Thu Nov 14 16:01:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5BDF3D68B34 for ; Thu, 14 Nov 2024 16:03:26 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcJI-0005yJ-RT; Thu, 14 Nov 2024 11:03:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIR-00055R-Ef for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:16 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIL-0002Is-Ac for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-7ee020ec76dso610587a12.3 for ; Thu, 14 Nov 2024 08:02:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600125; x=1732204925; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=SAnTQwSzdXGj8FtiG366nzj4chRAQvpTBuCvSfGJDoU=; b=f2cqVK+DI9DALgZuOtT/cFVU4OxXZ0XIe0VoWGVw+clCRil6WifIuy6mJXoWELClLw oRAXH30gtoxx3moWoiwlmL4MkZHVd/bWeKbGkcbC8Da1VAbmxc6udw5W2r1cdY4usINA dYepNbMiGYsFEDgm4SBQDD0TutyhPFsnwkPn+EPSNAvdkQ/KtTstTTEUQ2CEbiquIdLq QmpkLHj1qUoqDlu8Y3apntpIfSIppUk0D//c1OXIEd9IQukcx9E/2P7TibBS7z6WmwVg rBg6Xxu0Wm7PrExKmncgWDHV18FggRAjPgqwocSnac+vTblVV647fmsy4rhDPuGt5xna Rx2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600125; x=1732204925; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SAnTQwSzdXGj8FtiG366nzj4chRAQvpTBuCvSfGJDoU=; b=q/ytjOIvuHz9Cd9nGg7A6YGcvpJcC23VEVeLUMGRmYR1cxsaLLY27TXNPsfNG70+E8 sLshzWsEJ0n68Tq/NdFw/ZignkVYGawzgnoQ38alfp78I42ricOYdzGhM7ZbBZGudKWJ yawdJwvxK0pZQUwnbUYSzvNtLri8p8tXRAGM2VGTKkAL9agEnKWHhwqzI7M/TgAbHE5U coqNILimxKkbp0EDXF1xgD5VzGEl1DYaMj4QHeYfnQkalWrvsGIUOImwKORDm7JWS9ao jLJOeMQrkwPxXNCTY+B5Ej/KI/xnCc1gQY6p3R6QYP/sGIWU1ogBX0TeKp3RlTUuxX8R //Wg== X-Gm-Message-State: AOJu0YzmvcFZkqoDoa/UWlkTB85GMzCPkBECNjT4Jpki9QIg28z+V5cB lJxZKSpDKEI9H+kjZCBjrGwMiojXmfl0soZWNZRXX1nA+oUargOA6oHYvxgtsMdCzFVq7LQV6qV 2 X-Google-Smtp-Source: AGHT+IEF5h1yU/MTIq/579zafoCKA9obeHgknNJhdyCc70CyiIPZtnW4cbtX0pAochofmI5hUNHq7w== X-Received: by 2002:a17:90b:3ec5:b0:2e2:8472:c350 with SMTP id 98e67ed59e1d1-2e9b173dd4cmr34193452a91.17.1731600125199; Thu, 14 Nov 2024 08:02:05 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:04 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 37/54] target/loongarch: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:13 -0800 Message-ID: <20241114160131.48616-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/loongarch/internals.h | 7 ++++--- target/loongarch/cpu.c | 2 +- target/loongarch/tcg/tlb_helper.c | 17 +++++++++++------ 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/target/loongarch/internals.h b/target/loongarch/internals.h index 1a02427627..a9f73f27b2 100644 --- a/target/loongarch/internals.h +++ b/target/loongarch/internals.h @@ -60,9 +60,10 @@ int get_physical_address(CPULoongArchState *env, hwaddr *physical, hwaddr loongarch_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr); #ifdef CONFIG_TCG -bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool loongarch_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t ra); #endif #endif /* !CONFIG_USER_ONLY */ diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c index 57cc4f314b..47d69f1788 100644 --- a/target/loongarch/cpu.c +++ b/target/loongarch/cpu.c @@ -798,7 +798,7 @@ static const TCGCPUOps loongarch_tcg_ops = { .restore_state_to_opc = loongarch_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = loongarch_cpu_tlb_fill, + .tlb_fill_align = loongarch_cpu_tlb_fill_align, .cpu_exec_interrupt = loongarch_cpu_exec_interrupt, .cpu_exec_halt = loongarch_cpu_has_work, .do_interrupt = loongarch_cpu_do_interrupt, diff --git a/target/loongarch/tcg/tlb_helper.c b/target/loongarch/tcg/tlb_helper.c index 97f38fc391..94d5df08a4 100644 --- a/target/loongarch/tcg/tlb_helper.c +++ b/target/loongarch/tcg/tlb_helper.c @@ -474,9 +474,10 @@ void helper_invtlb_page_asid_or_g(CPULoongArchState *env, tlb_flush(env_cpu(env)); } -bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool loongarch_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPULoongArchState *env = cpu_env(cs); hwaddr physical; @@ -488,12 +489,16 @@ bool loongarch_cpu_tlb_fill(CPUState *cs, vaddr address, int size, access_type, mmu_idx); if (ret == TLBRET_MATCH) { - tlb_set_page(cs, address & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, prot, - mmu_idx, TARGET_PAGE_SIZE); qemu_log_mask(CPU_LOG_MMU, "%s address=%" VADDR_PRIx " physical " HWADDR_FMT_plx " prot %d\n", __func__, address, physical, prot); + + memset(out, 0, sizeof(*out)); + out->phys_addr = physical; + out->prot = prot; + out->attrs = MEMTXATTRS_UNSPECIFIED; + out->lg_page_size = TARGET_PAGE_BITS; + return true; } else { qemu_log_mask(CPU_LOG_MMU, From patchwork Thu Nov 14 16:01:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08866D68B33 for ; Thu, 14 Nov 2024 16:09:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIc-00059v-1m; Thu, 14 Nov 2024 11:02:28 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIO-00053O-Ix for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIL-0002J0-AD for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:12 -0500 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-7f46d5d1ad5so601308a12.3 for ; Thu, 14 Nov 2024 08:02:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600126; x=1732204926; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=mq+U5IXUnwEDeWR+OTODGDDIW/H0+4jqgpkcDbxZSzQ=; b=ch3zCWn3m6vVm0/JhzFsbYEJVXMIqlFLVVNLbO4EZDhBLwzpKGC+15GnmMVEZiBSaQ sE3yCWwU/G9YQgZRgZ4BUI6a2hgf9YdJAoLyvaapwxzcM7x7E2GAnz8NTzI+1mxwAIw4 qol8Oz2unj7GIrtYuJuAIKxBa1377IA8CTLb3Xc599KFm1+JEynCvtiA3krfgbOrlmxo Owbhn6XUwGhOiAkXnc3cxeD1ZKYrhALDSHunOmez+EYrgDuvSon8bPes/dtkLm0Qx7BC cPPCBiw9nuD7DLemX/dK5Q6+zCa+Z57rQK+fge+bOePsU+dDOB55+lZ2EBzMnFkkgRWe esXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600126; x=1732204926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mq+U5IXUnwEDeWR+OTODGDDIW/H0+4jqgpkcDbxZSzQ=; b=MhkR77j0SdHXSWa1Y1mLxEBTliB77pPE05AT8KQAUZJ82BP4EE3Zz+OWBkL9VVzFDT EiaUPvSfgCGuD7NXjJ8ZwTHpBQQgZzUtXs900+WE4vXEsCYKiVHhP7y3TztCPkI7HWGD pUGASGQD9EZVvzMwWr4G1ibUrpKfA/5SJhdUuJ1mYB4u1ZYSf87uXeNeHH0f/MdZ3+eQ v67Gh1RlATsefDAw3ZopeEBDqrANhOp9YNeh5pb4kr6FErl+DWb5yL49nltj8oS1GypW CB0lh4NpmttUMGMPW1k9rGyCo3XAvJTOh/XMY5FauTIkY6FXS3v1eiMIOG/qtBAXFvDV 6d2w== X-Gm-Message-State: AOJu0YxyEYNA4Qe5PK576Z3DqioWZL2vnUmA8bcYh8zzcCM0nPLwUvh5 BD7qmduFEopVdrwuA1/TmswX9l7ny06pDNdG9BDj/9i6fk9JAPewippImr0L9ki9x8dngOJmVxs 4 X-Google-Smtp-Source: AGHT+IGaIzzKrdFQmlHStSAhN9967MNqQOTFltJwD57bsotl36k3RDoYmyM7ot6mo6bkgjcnNC28uA== X-Received: by 2002:a17:90b:1d81:b0:2e2:b41b:854e with SMTP id 98e67ed59e1d1-2ea0631e509mr3132742a91.20.1731600126372; Thu, 14 Nov 2024 08:02:06 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.05 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:05 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 38/54] target/m68k: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:14 -0800 Message-ID: <20241114160131.48616-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/m68k/cpu.h | 7 ++++--- target/m68k/cpu.c | 2 +- target/m68k/helper.c | 22 +++++++++++++--------- 3 files changed, 18 insertions(+), 13 deletions(-) diff --git a/target/m68k/cpu.h b/target/m68k/cpu.h index b5bbeedb7a..4401426a0b 100644 --- a/target/m68k/cpu.h +++ b/target/m68k/cpu.h @@ -22,6 +22,7 @@ #define M68K_CPU_H #include "exec/cpu-defs.h" +#include "exec/memop.h" #include "qemu/cpu-float.h" #include "cpu-qom.h" @@ -582,10 +583,10 @@ enum { #define MMU_KERNEL_IDX 0 #define MMU_USER_IDX 1 -bool m68k_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); #ifndef CONFIG_USER_ONLY +bool m68k_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr addr, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); void m68k_cpu_transaction_failed(CPUState *cs, hwaddr physaddr, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, MemTxAttrs attrs, diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c index 5fe335558a..5316cf8922 100644 --- a/target/m68k/cpu.c +++ b/target/m68k/cpu.c @@ -550,7 +550,7 @@ static const TCGCPUOps m68k_tcg_ops = { .restore_state_to_opc = m68k_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = m68k_cpu_tlb_fill, + .tlb_fill_align = m68k_cpu_tlb_fill_align, .cpu_exec_interrupt = m68k_cpu_exec_interrupt, .cpu_exec_halt = m68k_cpu_has_work, .do_interrupt = m68k_cpu_do_interrupt, diff --git a/target/m68k/helper.c b/target/m68k/helper.c index 9bfc6ae97c..1decb6f39c 100644 --- a/target/m68k/helper.c +++ b/target/m68k/helper.c @@ -950,9 +950,10 @@ void m68k_set_irq_level(M68kCPU *cpu, int level, uint8_t vector) } } -bool m68k_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType qemu_access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool m68k_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType qemu_access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUM68KState *env = cpu_env(cs); hwaddr physical; @@ -961,12 +962,14 @@ bool m68k_cpu_tlb_fill(CPUState *cs, vaddr address, int size, int ret; target_ulong page_size; + memset(out, 0, sizeof(*out)); + out->attrs = MEMTXATTRS_UNSPECIFIED; + if ((env->mmu.tcr & M68K_TCR_ENABLED) == 0) { /* MMU disabled */ - tlb_set_page(cs, address & TARGET_PAGE_MASK, - address & TARGET_PAGE_MASK, - PAGE_READ | PAGE_WRITE | PAGE_EXEC, - mmu_idx, TARGET_PAGE_SIZE); + out->phys_addr = address; + out->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; + out->lg_page_size = TARGET_PAGE_BITS; return true; } @@ -985,8 +988,9 @@ bool m68k_cpu_tlb_fill(CPUState *cs, vaddr address, int size, ret = get_physical_address(env, &physical, &prot, address, access_type, &page_size); if (likely(ret == 0)) { - tlb_set_page(cs, address & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, prot, mmu_idx, page_size); + out->phys_addr = physical; + out->prot = prot; + out->lg_page_size = ctz32(page_size); return true; } From patchwork Thu Nov 14 16:01:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75F34D68B36 for ; Thu, 14 Nov 2024 16:11:55 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcIb-00059P-Lo; Thu, 14 Nov 2024 11:02:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcIO-00053L-9s for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:13 -0500 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcIL-0002JE-9r for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:02:12 -0500 Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-2e31af47681so652330a91.2 for ; Thu, 14 Nov 2024 08:02:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600128; x=1732204928; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=l6VeKTlRuD0Xy07QiXe4r1yW5BzO376HoZWUmX4DLoU=; b=ga9lUu8Lch3Ve1RO2rV7glzYz/sSC/u2UKPn+xqVKlfxvl79ff9+VRNPqppKL6Olbn L0edK1GuLHkXpHwJuw5320905ElB+EvdTiH0Ay51X9R99IMCoZF+5xkmiooDfBJRMb5N uAP3gjyJn/oIWOx/nHhP6xKFaaFfW4ZfhME+dVokU7Z2JlZM8f5oQB2TXUE9+srIC1ym j5A+lIk+NMHfc2matmw5k0sIr55YqQMlcX2go5cPfdnPzwcjfUAW47jxd1N6AhxEGsy+ 9JSYCBg0yFon88/1MOZtfuVbxDyKh40xdtDzV3tPKFeZRX1pMP6voVMK2XbDD2lT+Svj 0bvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600128; x=1732204928; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l6VeKTlRuD0Xy07QiXe4r1yW5BzO376HoZWUmX4DLoU=; b=p16Cwty/6Kv2Tt7+gKWxGVC7Tfz/OCsyTa9A31Aq8rrzskeaNBPU+sWH8oM8/1U3u7 kz9tWj3EQ/AMxGsRyx2a0hf/H9LByNiYQfe/rqjTDnIGn+gk5y/ahtIoASspniphmoFM XggQGYk/tO3Aqi1zIdEVrmB0bqSKijdQnBtYYXAHEnvKlWAKT358YgdoQfS+2zeIJ7CP JS54ewyWIPTMstQq5ZUxsP+qC+ZNgT2TTahARXquL36F/9uQDQN1baGsYWeRfbqpNERT x0uwZ5joa213HodejhzKPk2GWkyUn9fweOyuGpxMs7k/PCUfxxCpHvAfzGbiXxieDrtE eizA== X-Gm-Message-State: AOJu0Yz7vHibg7w5VHb+AgfVEzBXECUyOmDFW0g3als0JzQkDeBBVxz1 iE7RNMXL33FID/+KggTQ8rSR4zDBWv9oORS3GNABns6IpIHCIQjrNPk6+vMdcUxmGusbp8P+c1q Z X-Google-Smtp-Source: AGHT+IHF5hmTJojDMsBQyX7XdkOmm91aDExazdJ38la63nTUZIw4KmnyvC76QsCNcvcA+3I7Ip1syA== X-Received: by 2002:a17:90b:2548:b0:2e2:e092:5323 with SMTP id 98e67ed59e1d1-2e9f2d5cc36mr9029570a91.29.1731600127730; Thu, 14 Nov 2024 08:02:07 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ea024ec723sm1484438a91.46.2024.11.14.08.02.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:02:06 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 39/54] target/m68k: Do not call tlb_set_page in helper_ptest Date: Thu, 14 Nov 2024 08:01:15 -0800 Message-ID: <20241114160131.48616-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The entire operation of ptest is performed within get_physical_address as part of ACCESS_PTEST. There is no need to install the page into softmmu. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/m68k/helper.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/target/m68k/helper.c b/target/m68k/helper.c index 1decb6f39c..0a54eca9bb 100644 --- a/target/m68k/helper.c +++ b/target/m68k/helper.c @@ -1460,7 +1460,6 @@ void HELPER(ptest)(CPUM68KState *env, uint32_t addr, uint32_t is_read) hwaddr physical; int access_type; int prot; - int ret; target_ulong page_size; access_type = ACCESS_PTEST; @@ -1476,14 +1475,7 @@ void HELPER(ptest)(CPUM68KState *env, uint32_t addr, uint32_t is_read) env->mmu.mmusr = 0; env->mmu.ssw = 0; - ret = get_physical_address(env, &physical, &prot, addr, - access_type, &page_size); - if (ret == 0) { - tlb_set_page(env_cpu(env), addr & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, - prot, access_type & ACCESS_SUPER ? - MMU_KERNEL_IDX : MMU_USER_IDX, page_size); - } + get_physical_address(env, &physical, &prot, addr, access_type, &page_size); } void HELPER(pflush)(CPUM68KState *env, uint32_t addr, uint32_t opmode) From patchwork Thu Nov 14 16:01:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A076D68B34 for ; Thu, 14 Nov 2024 16:12:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcM0-00020K-Nk; Thu, 14 Nov 2024 11:06:02 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLJ-0001dy-Rx for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:16 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLH-000314-QN for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:13 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-7240d93fffdso620182b3a.2 for ; Thu, 14 Nov 2024 08:05:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600309; x=1732205109; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=4jkf2KaCd4Q7nWGpqdRhWv638MMOGvY55gPInd4iokE=; b=b4bQzPKk66Q3c61TVcrRkiOxgH0vHPzwVvItgZSVJ5kLwsN2Vh53vzXtY+WtqFb1M9 q2DWZc1DQOxM8iECoyo+iU373/1VTKNJP990HvSDM25cPY/nVfBnHdtCEmEF1TBrbpQH n+KjXHzk2C/xjXBxEeBZbtcF98Cge4axwnZzZp155yznejwqqermD9bf6gu2cBG25Fk4 Q6Do57vHmMj5E/ivI1y2Q8O2fISTnGPLRc0uo9VEiR/5bgeu83q5Gno9cJOJVFu3cxL6 /WEO3X/KKVGEksgwviwHglw5pYxGmqZkk6f9WOrLuRrvZBW3thscqNpC/Xuz00WFcJN9 uQKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600309; x=1732205109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4jkf2KaCd4Q7nWGpqdRhWv638MMOGvY55gPInd4iokE=; b=blBHoq02kY05FRxdf3zJtsZDCwhZHQaS2g0SjtZ9vCQaxPGR216ejSTm7lTIBthpAX 2bYbSGHxas4j4pkHqE9ElTqdTFRHcForx22OtHxRgosVWJQkNeDwt1FWUu2OJI0U1yu2 zZ4pV4y9rtkYzQtvIW8R1tvrTMNE/cBp6AO/0wuH2FiAFPoyHUNL/JYDTI7YGrNW1VVd DoWzisS8SXSvFd7vKPBbCR6ywDZop//TyMWeZk63Vx51h4kbwh+XkTBy/24DPpZnav/j QI8FhHjBMI4dmEFK/z5KeEDgNqovN/kXjY61h4oljpWfFMLjwGV6leazaPPzPjgtXo3I i7Kg== X-Gm-Message-State: AOJu0YyJvxVwC0WozJPlhqRsfmi3pkOWkago2ejffcweWf60Jjv8RDWo XNSKV1i/8GNy/o2QZ4mYTkWQbg7c74eY5DUknaOY7fsi1I9vwNTI1VbPiA6sGlmNHfNIldPps+8 1 X-Google-Smtp-Source: AGHT+IFFeIzm6hWAZPY2v/wwoyv/wazNDLdLgnmOTVwADsqlppY7G8NekegJo5pZrItxcmG+MXrgXg== X-Received: by 2002:a05:6a00:21ca:b0:71e:6a57:7290 with SMTP id d2e1a72fcca58-72457828d9amr9003406b3a.0.1731600309345; Thu, 14 Nov 2024 08:05:09 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:08 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 40/54] target/microblaze: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:16 -0800 Message-ID: <20241114160131.48616-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/microblaze/cpu.h | 7 +++---- target/microblaze/cpu.c | 2 +- target/microblaze/helper.c | 33 ++++++++++++++++++++------------- 3 files changed, 24 insertions(+), 18 deletions(-) diff --git a/target/microblaze/cpu.h b/target/microblaze/cpu.h index 3e5a3e5c60..b0eadfd9b1 100644 --- a/target/microblaze/cpu.h +++ b/target/microblaze/cpu.h @@ -421,10 +421,9 @@ static inline void cpu_get_tb_cpu_state(CPUMBState *env, vaddr *pc, } #if !defined(CONFIG_USER_ONLY) -bool mb_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); - +bool mb_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr address, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); void mb_cpu_transaction_failed(CPUState *cs, hwaddr physaddr, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, MemTxAttrs attrs, diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c index 710eb1146c..212cad2143 100644 --- a/target/microblaze/cpu.c +++ b/target/microblaze/cpu.c @@ -425,7 +425,7 @@ static const TCGCPUOps mb_tcg_ops = { .restore_state_to_opc = mb_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = mb_cpu_tlb_fill, + .tlb_fill_align = mb_cpu_tlb_fill_align, .cpu_exec_interrupt = mb_cpu_exec_interrupt, .cpu_exec_halt = mb_cpu_has_work, .do_interrupt = mb_cpu_do_interrupt, diff --git a/target/microblaze/helper.c b/target/microblaze/helper.c index 5d3259ce31..b6375564b4 100644 --- a/target/microblaze/helper.c +++ b/target/microblaze/helper.c @@ -36,37 +36,44 @@ static bool mb_cpu_access_is_secure(MicroBlazeCPU *cpu, } } -bool mb_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool mb_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr address, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, + bool probe, uintptr_t retaddr) { MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs); CPUMBState *env = &cpu->env; MicroBlazeMMULookup lu; unsigned int hit; - int prot; - MemTxAttrs attrs = {}; - attrs.secure = mb_cpu_access_is_secure(cpu, access_type); + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + mb_cpu_do_unaligned_access(cs, address, access_type, mmu_idx, retaddr); + } + + memset(out, 0, sizeof(*out)); + out->attrs.secure = mb_cpu_access_is_secure(cpu, access_type); + out->lg_page_size = TARGET_PAGE_BITS; if (mmu_idx == MMU_NOMMU_IDX) { /* MMU disabled or not available. */ - address &= TARGET_PAGE_MASK; - prot = PAGE_RWX; - tlb_set_page_with_attrs(cs, address, address, attrs, prot, mmu_idx, - TARGET_PAGE_SIZE); + out->phys_addr = address; + out->prot = PAGE_RWX; return true; } hit = mmu_translate(cpu, &lu, address, access_type, mmu_idx); if (likely(hit)) { - uint32_t vaddr = address & TARGET_PAGE_MASK; + uint32_t vaddr = address; uint32_t paddr = lu.paddr + vaddr - lu.vaddr; qemu_log_mask(CPU_LOG_MMU, "MMU map mmu=%d v=%x p=%x prot=%x\n", mmu_idx, vaddr, paddr, lu.prot); - tlb_set_page_with_attrs(cs, vaddr, paddr, attrs, lu.prot, mmu_idx, - TARGET_PAGE_SIZE); + + out->phys_addr = paddr; + out->prot = lu.prot; return true; } From patchwork Thu Nov 14 16:01:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B24FD68B35 for ; Thu, 14 Nov 2024 16:12:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcMn-0002YX-3y; Thu, 14 Nov 2024 11:06:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLO-0001gU-7Y for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:20 -0500 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLH-00031B-Qk for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:14 -0500 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-20e576dbc42so9330005ad.0 for ; Thu, 14 Nov 2024 08:05:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600310; x=1732205110; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=o/Mf+gksylMlQN4tQ9eK2BqWgWCztBtTJCqnsjP2hVE=; b=btLcgMsxgDdMq1Gxjxc3ntlYPAhD0fOhAshWxejL12B/H5iuAtR+l4gH6qCNO/WMBE aQsQeLvZiZmUXZe/WlNwbzToxzMcQujtdpdIVLEyF0HcEal81Ah4bHJ+9O8lbArePm6R NAUxTZDRjEpLXLkv6iIjDKhrAiC/n04weAdd8E4Lt3OWAhBzFMo7GxUBjTNFpY0h0EX1 Z5Q/8Bgk3nR2U47iWI9/3cccEs6czXCBaIFJ9e4OZA8gSDETmx1wl5L6UaqOK2In4oJj UdC00EVQ6bWuF7NSNz8TOA6pRvQ8VzhaJDBxZnLi68WjvIvlrYW5z+gNU+T3uoQ3K3P5 PrNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600310; x=1732205110; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o/Mf+gksylMlQN4tQ9eK2BqWgWCztBtTJCqnsjP2hVE=; b=Ur8J5NbBcPrSTn9qzuIG0KMENtFxJh+S2stN7LgENBjcaOeXFgjgVmRH+SMhVEdhyh laG5EOxsrE0A4rzeFTwcUotuR2QA9Sz0Vw1FKW3b8wYpem/hg687fURoewedKF+tqxbF IsbhiYmbpYAoakBl7eLSlBJGghBs8YfYO2pP4vCBzJQ+m90vsdI8cvs1RffS27GAglCa Hc76+vJAV16fGg6jcD0/6dyIR4s2Rw+ayo+ZW/vLu+yVChqxBB+NazIsT409LwPOvtuz 3hWugHUGgrLQ5cgyw3Zy9Wez3bSxX9rrwx2pqc7g+GQehTGS0lZ7+2R3mF6DWHla8jZ6 Zwkg== X-Gm-Message-State: AOJu0YzoAvMHfmYE709j0GBC1P8ycWy5BuKfS30xkOxLb/IV0lnJekPQ tdBCPZrwNnA7uLv7vUIXyRv6SixmpzITYK31GhreA5qiRVud0MYlRnJC1ELCn1LcU8X6whVP2mb R X-Google-Smtp-Source: AGHT+IHQN+uOU3UVH1D1p7YIV0yK+4+t6thqFAG0EHZgQpoAv0yDRjFLg3x2Q8ssA/CLjeSxHUzB7g== X-Received: by 2002:a17:903:185:b0:211:3275:3fe with SMTP id d9443c01a7336-211b661af32mr80266145ad.17.1731600310179; Thu, 14 Nov 2024 08:05:10 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:09 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 41/54] target/mips: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:17 -0800 Message-ID: <20241114160131.48616-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/mips/tcg/tcg-internal.h | 6 +++--- target/mips/cpu.c | 2 +- target/mips/tcg/sysemu/tlb_helper.c | 29 ++++++++++++++++++++--------- 3 files changed, 24 insertions(+), 13 deletions(-) diff --git a/target/mips/tcg/tcg-internal.h b/target/mips/tcg/tcg-internal.h index aef032c48d..f4b00354af 100644 --- a/target/mips/tcg/tcg-internal.h +++ b/target/mips/tcg/tcg-internal.h @@ -61,9 +61,9 @@ void mips_cpu_do_transaction_failed(CPUState *cs, hwaddr physaddr, MemTxResult response, uintptr_t retaddr); void cpu_mips_tlb_flush(CPUMIPSState *env); -bool mips_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool mips_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr address, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, bool probe, uintptr_t ra); void mips_semihosting(CPUMIPSState *env); diff --git a/target/mips/cpu.c b/target/mips/cpu.c index d0a43b6d5c..3a453c9285 100644 --- a/target/mips/cpu.c +++ b/target/mips/cpu.c @@ -556,7 +556,7 @@ static const TCGCPUOps mips_tcg_ops = { .restore_state_to_opc = mips_restore_state_to_opc, #if !defined(CONFIG_USER_ONLY) - .tlb_fill = mips_cpu_tlb_fill, + .tlb_fill_align = mips_cpu_tlb_fill_align, .cpu_exec_interrupt = mips_cpu_exec_interrupt, .cpu_exec_halt = mips_cpu_has_work, .do_interrupt = mips_cpu_do_interrupt, diff --git a/target/mips/tcg/sysemu/tlb_helper.c b/target/mips/tcg/sysemu/tlb_helper.c index e98bb95951..ac76396525 100644 --- a/target/mips/tcg/sysemu/tlb_helper.c +++ b/target/mips/tcg/sysemu/tlb_helper.c @@ -904,15 +904,28 @@ refill: } #endif -bool mips_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool mips_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, vaddr address, + MMUAccessType access_type, int mmu_idx, + MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUMIPSState *env = cpu_env(cs); hwaddr physical; int prot; int ret = TLBRET_BADADDR; + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + mips_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + + memset(out, 0, sizeof(*out)); + out->attrs = MEMTXATTRS_UNSPECIFIED; + out->lg_page_size = TARGET_PAGE_BITS; + /* data access */ /* XXX: put correct access by using cpu_restore_state() correctly */ ret = get_physical_address(env, &physical, &prot, address, @@ -930,9 +943,8 @@ bool mips_cpu_tlb_fill(CPUState *cs, vaddr address, int size, break; } if (ret == TLBRET_MATCH) { - tlb_set_page(cs, address & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, prot, - mmu_idx, TARGET_PAGE_SIZE); + out->phys_addr = physical; + out->prot = prot; return true; } #if !defined(TARGET_MIPS64) @@ -948,9 +960,8 @@ bool mips_cpu_tlb_fill(CPUState *cs, vaddr address, int size, ret = get_physical_address(env, &physical, &prot, address, access_type, mmu_idx); if (ret == TLBRET_MATCH) { - tlb_set_page(cs, address & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, prot, - mmu_idx, TARGET_PAGE_SIZE); + out->phys_addr = physical; + out->prot = prot; return true; } } From patchwork Thu Nov 14 16:01:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EE6DBD68B34 for ; Thu, 14 Nov 2024 16:08:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcM6-00022X-Ch; Thu, 14 Nov 2024 11:06:04 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLO-0001gT-7Y for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:20 -0500 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLI-00031L-KC for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:14 -0500 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-720c286bcd6so729630b3a.3 for ; Thu, 14 Nov 2024 08:05:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600311; x=1732205111; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=lXqCqubeCT0QNAMQLrmscFNdjuU77OyZLOi8s3DYmiM=; b=j4QrfB6GyOucNYUDaNyshWMzV2KpKNYbGeSCs1yusJ2bEKVqr8fHWUk1IADVgIIwOC kmt419hEjy7jaPmZzL0PEcLDpCJr2leGlxBvk0ko+ZPdJzoMDl1NzBwSfZuPp5v9leiQ rv/hozMTfUnqHbBRG1pG+KfMWjT1Xret9/bVNdzc82oImS7OEjYhyD6pSGZVAfDILO4o 3kmj+S0wKwt7oyh4/roFbhPUS0rXff4d78cCQK3SKCMSe0rMd4OOrzuwOlJWaRQ9FsUj l6Yt+ZSzO0qhW2hVJtqGEWIrtmZzdCtn6fNpd/UprRgvp4JiiQvGOf+7ptAvBWln1fgz ysng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600311; x=1732205111; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lXqCqubeCT0QNAMQLrmscFNdjuU77OyZLOi8s3DYmiM=; b=t4f7CDRvzoz7OMk4njDgRRjKMBnGKGG6iWRDcnzv/nvCEAbX0fbSQYPieKZSiUIhCh 6GEQCR8/9oZN3DHTQH0BErc4H8lZ4iEt0Ed/F7b4ybq4py0tLmR2hl6pXq+K57ffFrPB kcKfDHh9Hw3xOLjTgZ7LSrwlhtk6eHKVSRDrJjD0IY68Wh626LrFK75uo6YqbdUcgemm YpCqg1uP//nqpFQaB1yhiKUtmMO2gVf6z+dVWNfRWUNwvYaje2x/miI31DkntoNPrgHM IQLobeFHfvsnEJfTEvRrd09GJJ1GjrQvU44gMAm7XOyYBFBTsSwpn2xtjfq+X7vYATRc aIpg== X-Gm-Message-State: AOJu0YxYdfZpbbC3tHPsN9cieeo+fcBLy8e0roeaEPjRrlJzGvB3BhDA 9nF3D7C6slE4DTPsFToIgPeaOOzkNVr6CcrEkeOUDog6n2mQXrLo+tOtPr72T0nr0/9ojvdBN3A s X-Google-Smtp-Source: AGHT+IE7g5LfgO4J+I6h04FvuntiNTMaS3e0coLJeLtKjCUdDv7iMXtK+Oc4rDakvNBP+F9Ef07cJQ== X-Received: by 2002:a05:6a00:124c:b0:71e:722b:ae1d with SMTP id d2e1a72fcca58-724133b687emr30389114b3a.25.1731600311041; Thu, 14 Nov 2024 08:05:11 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:10 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 42/54] target/openrisc: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:18 -0800 Message-ID: <20241114160131.48616-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/openrisc/cpu.h | 8 +++++--- target/openrisc/cpu.c | 2 +- target/openrisc/mmu.c | 39 +++++++++++++++++++++------------------ 3 files changed, 27 insertions(+), 22 deletions(-) diff --git a/target/openrisc/cpu.h b/target/openrisc/cpu.h index c9fe9ae12d..e177ad8b84 100644 --- a/target/openrisc/cpu.h +++ b/target/openrisc/cpu.h @@ -22,6 +22,7 @@ #include "cpu-qom.h" #include "exec/cpu-defs.h" +#include "exec/memop.h" #include "fpu/softfloat-types.h" /** @@ -306,9 +307,10 @@ int print_insn_or1k(bfd_vma addr, disassemble_info *info); #ifndef CONFIG_USER_ONLY hwaddr openrisc_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr); -bool openrisc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool openrisc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t ra); extern const VMStateDescription vmstate_openrisc_cpu; diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c index b96561d1f2..6aa04ff7d3 100644 --- a/target/openrisc/cpu.c +++ b/target/openrisc/cpu.c @@ -237,7 +237,7 @@ static const TCGCPUOps openrisc_tcg_ops = { .restore_state_to_opc = openrisc_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = openrisc_cpu_tlb_fill, + .tlb_fill_align = openrisc_cpu_tlb_fill_align, .cpu_exec_interrupt = openrisc_cpu_exec_interrupt, .cpu_exec_halt = openrisc_cpu_has_work, .do_interrupt = openrisc_cpu_do_interrupt, diff --git a/target/openrisc/mmu.c b/target/openrisc/mmu.c index c632d5230b..eafab356a6 100644 --- a/target/openrisc/mmu.c +++ b/target/openrisc/mmu.c @@ -104,39 +104,42 @@ static void raise_mmu_exception(OpenRISCCPU *cpu, target_ulong address, cpu->env.lock_addr = -1; } -bool openrisc_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool openrisc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { OpenRISCCPU *cpu = OPENRISC_CPU(cs); - int excp = EXCP_DPF; int prot; hwaddr phys_addr; + /* TODO: alignment faults not currently handled. */ + if (mmu_idx == MMU_NOMMU_IDX) { /* The mmu is disabled; lookups never fail. */ get_phys_nommu(&phys_addr, &prot, addr); - excp = 0; } else { bool super = mmu_idx == MMU_SUPERVISOR_IDX; int need = (access_type == MMU_INST_FETCH ? PAGE_EXEC : access_type == MMU_DATA_STORE ? PAGE_WRITE : PAGE_READ); - excp = get_phys_mmu(cpu, &phys_addr, &prot, addr, need, super); + int excp = get_phys_mmu(cpu, &phys_addr, &prot, addr, need, super); + + if (unlikely(excp)) { + if (probe) { + return false; + } + raise_mmu_exception(cpu, addr, excp); + cpu_loop_exit_restore(cs, retaddr); + } } - if (likely(excp == 0)) { - tlb_set_page(cs, addr & TARGET_PAGE_MASK, - phys_addr & TARGET_PAGE_MASK, prot, - mmu_idx, TARGET_PAGE_SIZE); - return true; - } - if (probe) { - return false; - } - - raise_mmu_exception(cpu, addr, excp); - cpu_loop_exit_restore(cs, retaddr); + memset(out, 0, sizeof(*out)); + out->phys_addr = phys_addr; + out->prot = prot; + out->lg_page_size = TARGET_PAGE_BITS; + out->attrs = MEMTXATTRS_UNSPECIFIED; + return true; } hwaddr openrisc_cpu_get_phys_page_debug(CPUState *cs, vaddr addr) From patchwork Thu Nov 14 16:01:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 220F9D68B34 for ; Thu, 14 Nov 2024 16:13:19 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcMo-0002Zv-8A; Thu, 14 Nov 2024 11:06:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLO-0001gV-85 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:20 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLJ-00033L-KB for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:15 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-7245a9d0e92so815347b3a.0 for ; Thu, 14 Nov 2024 08:05:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600312; x=1732205112; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=6ZNcg828iubR7iZKSP5M7BkBGmFtklgKRp8DrOlWIfY=; b=B8dtA3+4K5ZJXvkhImcNeLWJvQnqYnQe/1rnzdw/1QBAUm00pKkKo9NptafVBPtgT7 zb0mhc34mFOUMxyla/tSe8Xnic0b9TTew87fWxpCedqtB1moi4x/ZHoSccXv93uByd5f okT5cC5wPiAKi2Y31q78bhzRZ/AN34T5E3znCaw3afZJKhgLh0Zrq2OfGS4W7heCtyC/ F/UV/xgJzrsVcOpqsvfZV1k3aInkNkeJpAmcO9AyRHRfvvVwhdBYggJ2tlRZm6yxClf3 lR4ODPmhop6gYPJF1Z/WStO/UpgN0gLbycLRLbEYlxnC15Cf6RD60fKskew29CkYA1v+ bc7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600312; x=1732205112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6ZNcg828iubR7iZKSP5M7BkBGmFtklgKRp8DrOlWIfY=; b=Piunb46B5hEswoV/hYsr4rUZMg5oNB996Ga+ZBW+pXLS5hKpuoVqhXbc47aVj/PtuR f7EubWW9wGrE/2Yke9roOT2BNId5QPklz2VH2Pixf21HS2eg56gR3tfYFSL9DS4qnLM+ D8v02wRO/CKP6K1gXcll1U+A8+l4ImeGge+lnt94xRSaUXywwhfn2NFuwRSr1tNeLPZ6 rN4nyURgKoRLd1KtMvUGzn/mlOPUtpIXvVcDMIL0329Q2cvdvfyjhPYRK+zBJVVZwQdJ UNViBk4QLatY1KQqzQr5gxbNgYxP92wPs2QUzYWRCZnnGt2yCIIsIVhwHtQ6iEyL2SnC L7PQ== X-Gm-Message-State: AOJu0Yy9TTqLcC/LGn5g/2xSuTFXfcEgr9n+8aFD8pLHh+rReOXEPuyg IDCbF0qDT/M1OC1SNy1krEfgQrJ188+V/DmZ+queoQDg1Qg3MbrYRz6VZZUUx5gNEK4tPNC5nUJ C X-Google-Smtp-Source: AGHT+IG8r4WbMldXia9Btm6/A9nnQQIH7XWeXWgD4MMho9TEz/vY24mIEL+xQy0vGk4oqgrCs/tz/A== X-Received: by 2002:a05:6a00:3a06:b0:71e:4c34:e294 with SMTP id d2e1a72fcca58-7245799d47bmr8657997b3a.7.1731600311849; Thu, 14 Nov 2024 08:05:11 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:11 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 43/54] target/ppc: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:19 -0800 Message-ID: <20241114160131.48616-44-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/ppc/internal.h | 7 ++++--- target/ppc/cpu_init.c | 2 +- target/ppc/mmu_helper.c | 21 ++++++++++++++++----- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/target/ppc/internal.h b/target/ppc/internal.h index 20fb2ec593..9d132d35a1 100644 --- a/target/ppc/internal.h +++ b/target/ppc/internal.h @@ -273,9 +273,10 @@ void ppc_cpu_record_sigsegv(CPUState *cs, vaddr addr, MMUAccessType access_type, bool maperr, uintptr_t ra); #else -bool ppc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool ppc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t ra); G_NORETURN void ppc_cpu_do_unaligned_access(CPUState *cs, vaddr addr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr); diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c index efcb80d1c2..387c7ff2da 100644 --- a/target/ppc/cpu_init.c +++ b/target/ppc/cpu_init.c @@ -7422,7 +7422,7 @@ static const TCGCPUOps ppc_tcg_ops = { #ifdef CONFIG_USER_ONLY .record_sigsegv = ppc_cpu_record_sigsegv, #else - .tlb_fill = ppc_cpu_tlb_fill, + .tlb_fill_align = ppc_cpu_tlb_fill_align, .cpu_exec_interrupt = ppc_cpu_exec_interrupt, .cpu_exec_halt = ppc_cpu_has_work, .do_interrupt = ppc_cpu_do_interrupt, diff --git a/target/ppc/mmu_helper.c b/target/ppc/mmu_helper.c index b167b37e0a..bf98e0efb0 100644 --- a/target/ppc/mmu_helper.c +++ b/target/ppc/mmu_helper.c @@ -1357,18 +1357,29 @@ void helper_check_tlb_flush_global(CPUPPCState *env) } -bool ppc_cpu_tlb_fill(CPUState *cs, vaddr eaddr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool ppc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr eaddr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { PowerPCCPU *cpu = POWERPC_CPU(cs); hwaddr raddr; int page_size, prot; + if (eaddr & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + ppc_cpu_do_unaligned_access(cs, eaddr, access_type, mmu_idx, retaddr); + } + if (ppc_xlate(cpu, eaddr, access_type, &raddr, &page_size, &prot, mmu_idx, !probe)) { - tlb_set_page(cs, eaddr & TARGET_PAGE_MASK, raddr & TARGET_PAGE_MASK, - prot, mmu_idx, 1UL << page_size); + memset(out, 0, sizeof(*out)); + out->phys_addr = raddr; + out->prot = prot; + out->lg_page_size = page_size; + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } if (probe) { From patchwork Thu Nov 14 16:01:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6EE7D68B34 for ; Thu, 14 Nov 2024 16:09:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcNP-00047j-Jd; Thu, 14 Nov 2024 11:07:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLQ-0001mC-MS for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLL-00033h-Qo for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:19 -0500 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-7e9e38dd5f1so674386a12.0 for ; Thu, 14 Nov 2024 08:05:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600312; x=1732205112; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=PvazYVJMP5GcUwBS2MCJUYaWeFV3enL9YDBstfMYOoA=; b=eezgv9/Ws6bo1OcLC9OrFR8jGQBTmgBUKeB1CdpdcD6wpPen0NQBzpaPkS3eK5VtXe QfCyac/31cFN/uV4BuTU1szEnIDZ4uZ3LGEQ7C6I3SQPnKOU5k1h/8sio0aFWxHT1ntK YWZbe3oHzrWUkD0z8hhvEZYjOPULlC00TcaoaCu/Nuk2YX+O4La2rZuGM7/ZoR6dwcLg 4iafvvi3aFRz9PoH/Td3L8lVaqwCyvWU1rUqdBxLHb1diMZK4+W8WmRJnXgyUTSdTDF9 3QaO34r3/ywfMAw3wxQ38u6oJYlD2mvmTSbaeErm7Foeucak66P0FhrED+Q93ZJvzCQw uUrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600312; x=1732205112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PvazYVJMP5GcUwBS2MCJUYaWeFV3enL9YDBstfMYOoA=; b=B6L6JpLSsd3zNzIgwj6KUBbVsHGnP+5hQSUHf3qExwO+ZgoH9J0M/xwfczqGTZ0uP+ w5L2baH9Z2LqJXki7PDiIeO/9UasDkiDPgUDCj07lX58+MgbFv0CRXSbV1xsgACouZBJ UCJ08T6Q5mGw14hr5YG2/6ScByWUzna5gT2XDArml3RuBKzGcId25Aige00MAd7/5apz rcbnSQF99E6lcz29TGtvkLhC9o1F+Pfyq1WAG0TFOqJ0r2SPtOiML+qDrRrLpRXsNK47 fk6v6XsnXmAfvt4EpPJzzQZGhVHOqm5xD9PWUnYK/Ul8LzHYRRbfUTSE7GbJAbLlbOZQ AWSg== X-Gm-Message-State: AOJu0Yz96jk2V6KVhlEeJjMlCNIacCTDNA84eYMbC+RE1uXon3deDTF3 nkXMwU4N4aghNb6KdVqy7FcpDfvaCCSBRRvBxmC0Vl0zeGD+S6NtZBKhPjg6kWmD+3NifhLqDgU 8 X-Google-Smtp-Source: AGHT+IHRRECedHgC8UA8c0cg98zU4ej6leJ/NcHYkkXVAzng1uypKrz9H9pIVnN1g0TWsoz6F1d0KQ== X-Received: by 2002:a05:6a20:4325:b0:1d6:fb1b:d07a with SMTP id adf61e73a8af0-1dc22b57f1dmr30731144637.31.1731600312512; Thu, 14 Nov 2024 08:05:12 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.11 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:12 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 44/54] target/riscv: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:20 -0800 Message-ID: <20241114160131.48616-45-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::536; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x536.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/riscv/cpu.h | 8 +++++--- target/riscv/cpu_helper.c | 22 +++++++++++++++++----- target/riscv/tcg/tcg-cpu.c | 2 +- 3 files changed, 23 insertions(+), 9 deletions(-) diff --git a/target/riscv/cpu.h b/target/riscv/cpu.h index 284b112821..f97c4f3410 100644 --- a/target/riscv/cpu.h +++ b/target/riscv/cpu.h @@ -25,6 +25,7 @@ #include "hw/qdev-properties.h" #include "exec/cpu-defs.h" #include "exec/gdbstub.h" +#include "exec/memop.h" #include "qemu/cpu-float.h" #include "qom/object.h" #include "qemu/int128.h" @@ -563,9 +564,10 @@ bool cpu_get_bcfien(CPURISCVState *env); G_NORETURN void riscv_cpu_do_unaligned_access(CPUState *cs, vaddr addr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr); -bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool riscv_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t ra); char *riscv_isa_string(RISCVCPU *cpu); int riscv_cpu_max_xlen(RISCVCPUClass *mcc); bool riscv_cpu_option_set(const char *optname); diff --git a/target/riscv/cpu_helper.c b/target/riscv/cpu_helper.c index 0a3ead69ea..edb2edfc55 100644 --- a/target/riscv/cpu_helper.c +++ b/target/riscv/cpu_helper.c @@ -1429,9 +1429,10 @@ static void pmu_tlb_fill_incr_ctr(RISCVCPU *cpu, MMUAccessType access_type) riscv_pmu_incr_ctr(cpu, pmu_event_type); } -bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool riscv_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { RISCVCPU *cpu = RISCV_CPU(cs); CPURISCVState *env = &cpu->env; @@ -1452,6 +1453,14 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, qemu_log_mask(CPU_LOG_MMU, "%s ad %" VADDR_PRIx " rw %d mmu_idx %d\n", __func__, address, access_type, mmu_idx); + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + riscv_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + pmu_tlb_fill_incr_ctr(cpu, access_type); if (two_stage_lookup) { /* Two stage lookup */ @@ -1544,8 +1553,11 @@ bool riscv_cpu_tlb_fill(CPUState *cs, vaddr address, int size, } if (ret == TRANSLATE_SUCCESS) { - tlb_set_page(cs, address & ~(tlb_size - 1), pa & ~(tlb_size - 1), - prot, mmu_idx, tlb_size); + memset(out, 0, sizeof(*out)); + out->phys_addr = pa; + out->prot = prot; + out->lg_page_size = ctz64(tlb_size); + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } else if (probe) { return false; diff --git a/target/riscv/tcg/tcg-cpu.c b/target/riscv/tcg/tcg-cpu.c index c62c221696..f3b436bb86 100644 --- a/target/riscv/tcg/tcg-cpu.c +++ b/target/riscv/tcg/tcg-cpu.c @@ -138,7 +138,7 @@ static const TCGCPUOps riscv_tcg_ops = { .restore_state_to_opc = riscv_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = riscv_cpu_tlb_fill, + .tlb_fill_align = riscv_cpu_tlb_fill_align, .cpu_exec_interrupt = riscv_cpu_exec_interrupt, .cpu_exec_halt = riscv_cpu_has_work, .do_interrupt = riscv_cpu_do_interrupt, From patchwork Thu Nov 14 16:01:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 498A4D68B34 for ; Thu, 14 Nov 2024 16:10:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcNP-00047h-UP; Thu, 14 Nov 2024 11:07:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLQ-0001mA-LU for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00034C-0S for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:19 -0500 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-723db2798caso751484b3a.0 for ; Thu, 14 Nov 2024 08:05:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600313; x=1732205113; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=nc2uEpCTHuP4WgyDpbnyFTPw+frcLZQWSz4dicwWNic=; b=o0XFkiQmvEzGYalgTWKCceMWsbKFRP8vYi/6a4G+l1Fa/79+7kamd2qdo/kMwuBKMQ aPRJSc0V5/6A8FhUFxHw6twjn+f/AtnIWf4l5Mf39ifOTtSlRmPRuk9n498y61nJQzCV FsQ14Wrh+GajGr+oYl2ewhfsd2O+1A59nHZEloAzSi5rKrDMGwpL+kz3EkXDGPHjqtmt 6cLwiZKBiMFe5BuUsfiYHrrF99hDpxUxwddutaAVEpgRw747n6gwWU4ATYvzM7vMtDU2 BwFx0NydBuZOtyKzVnORgqLKtcDCz0+1gm0vmWAXLy97ZIiSummH1n4FV+3B7MUucMas Sodw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600313; x=1732205113; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nc2uEpCTHuP4WgyDpbnyFTPw+frcLZQWSz4dicwWNic=; b=TY20KlZZGONqyzvMpMMVEfMM0Yso8CsQi/8gYm1Xk0J926iWf/aDOonrf1BxOD/1vg Zv45AZlmil5Q796WYs8sEiCkaD8wITrbgOpWGZ8OTh/G4an2dy2MyQPxZGEqF05K8TwP w8GNpyDjV8q9owYMEjV+PsWmy5y9R3Rb4rdgX/ObtRO3dTFIUQwkgMO89/RkBDRqhsHI PXWWdNA7Jxc9jwM8QC/0X9Hg7p30Cn5XN3HbYHcohzEihEJ6kYmNX55gTnA/cl1K/1BS krKLP+K5QaQwXYnwdQZgdkxP3G6E50ZSY6hy82osrFjxXKp8s8NuWa4/ZpWXjcAyd9W7 TY8A== X-Gm-Message-State: AOJu0Yz9YbSynId93uwhZ+AcWobXsUZZ77CAHcRB77eFC9nHeXJXiB79 Xu4GVGI5LFU4/G0bAxyMso0J8arH2aRrVUO1Ge4a9rdc8bE9t2rnGPIW3+lb6oqRhTcEjbh+RVp i X-Google-Smtp-Source: AGHT+IHQRU9f+dWRMqldDHp7M+xVk37euYiBWfVuo+7EeuLplWzUbxFNAjcJ2Wvq4BK4cu6aKsrS3w== X-Received: by 2002:a05:6a00:3d01:b0:71e:8046:2728 with SMTP id d2e1a72fcca58-7244a59f534mr15160202b3a.17.1731600313250; Thu, 14 Nov 2024 08:05:13 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.12 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:12 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 45/54] target/rx: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:21 -0800 Message-ID: <20241114160131.48616-46-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/rx/cpu.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/target/rx/cpu.c b/target/rx/cpu.c index 65a74ce720..c83a582141 100644 --- a/target/rx/cpu.c +++ b/target/rx/cpu.c @@ -161,16 +161,19 @@ static void rx_cpu_disas_set_info(CPUState *cpu, disassemble_info *info) info->print_insn = print_insn_rx; } -static bool rx_cpu_tlb_fill(CPUState *cs, vaddr addr, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +static bool rx_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { - uint32_t address, physical, prot; + /* TODO: alignment faults not currently handled. */ /* Linear mapping */ - address = physical = addr & TARGET_PAGE_MASK; - prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; - tlb_set_page(cs, address, physical, prot, mmu_idx, TARGET_PAGE_SIZE); + memset(out, 0, sizeof(*out)); + out->phys_addr = addr; + out->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; + out->lg_page_size = TARGET_PAGE_BITS; + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } @@ -195,7 +198,7 @@ static const TCGCPUOps rx_tcg_ops = { .initialize = rx_translate_init, .synchronize_from_tb = rx_cpu_synchronize_from_tb, .restore_state_to_opc = rx_restore_state_to_opc, - .tlb_fill = rx_cpu_tlb_fill, + .tlb_fill_align = rx_cpu_tlb_fill_align, #ifndef CONFIG_USER_ONLY .cpu_exec_interrupt = rx_cpu_exec_interrupt, From patchwork Thu Nov 14 16:01:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB179D68B33 for ; Thu, 14 Nov 2024 16:11:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcMx-0002fK-2Z; Thu, 14 Nov 2024 11:06:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLS-0001mU-Me for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:28 -0500 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00036B-9Y for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:21 -0500 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-72061bfec2dso590718b3a.2 for ; Thu, 14 Nov 2024 08:05:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600314; x=1732205114; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=XzIjtrWoIZsOWLPYm1QVw15canDC/nSQkt85xis2OP4=; b=IfBm3AwKTdf/B9PpEaSEqxqequ1nXEtbz0Bz7xXDanCdHDcILBM6zgF94jArq/Tly7 uZENvWz8XGJmD0e5uOEU+sfbGU31X48kgUn1URah8slQ+mpJAvVb7062RErxghd+ul4X 0oHlgqdjF7ZT9Np3VOLRqgsNbTdxrDLvzKf9w/ORi55cy9/NykOKOIrDeNuN2kX9OJG/ pVgAae314ZPZxSE1qRVlLSrYhQ5j/KAp78C8VpdY8X8NOJzfBrnvRSRhHPOmwWLintY+ 5ODFNRVz75vlGvQXlFKIfKSdu0Yu3fzpaneKhhiqIibMMvxwM/I39aT5lIqCFNYsx5pA UhsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600314; x=1732205114; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XzIjtrWoIZsOWLPYm1QVw15canDC/nSQkt85xis2OP4=; b=JIxwONkILEcx/nGIJ81kZKRTmIPs4Y9sLzWsYeeuD5Mx6ZBMcgNTwGlfm0KWwJPXNO pV5Hgd1vJzGizQDzJTL8JA4+kSYm4zRHd9Aw4F2ySCl31f3ELRwWsJVkQUuHTUwn/aLl W1F/b3ifxqbqt6L/aBkRTDTF9Rjs5qeBDBtc33E1pBKbeuBbZwLOalNR43pEzVZzC1gE KuZiPrBpFPjk5bE+XDfEBJzeHB3namQGiV49hcJXrX+UQzbxaYn05pKLVbIYOvshXOsp pVu9C7fypXhKpXRG7+4Ya0uk+dX6F32Rvzb1ywmKPsvza3andFj1aVHKQzqm1R9NOXWO nVTQ== X-Gm-Message-State: AOJu0YyAQ3vJ0ZnjCuKqWkSgafiCnA7TfOyx5zGch/EJGOYwPgt++Gmp Am+wGnk0xxArmIwCFfCodTkmxxamtjgfy8YZZTPlipsepi8julu8p1tT1m4SSkVfaiOPnsB6MSz P X-Google-Smtp-Source: AGHT+IHyZb/kd+QONYmo2tnnYuVkEviprnW6EUVFPAsQsweUgcLfJtHzNUAcamblJ5QgNVkAwU9drQ== X-Received: by 2002:a05:6a00:1394:b0:724:59e0:5d22 with SMTP id d2e1a72fcca58-72469df0a7fmr3581313b3a.20.1731600313922; Thu, 14 Nov 2024 08:05:13 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.13 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:13 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 46/54] target/s390x: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:22 -0800 Message-ID: <20241114160131.48616-47-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/s390x/s390x-internal.h | 7 ++++--- target/s390x/cpu.c | 4 ++-- target/s390x/tcg/excp_helper.c | 23 ++++++++++++++++++----- 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/target/s390x/s390x-internal.h b/target/s390x/s390x-internal.h index 825252d728..eb6fe24c9a 100644 --- a/target/s390x/s390x-internal.h +++ b/target/s390x/s390x-internal.h @@ -278,9 +278,10 @@ void s390_cpu_record_sigsegv(CPUState *cs, vaddr address, void s390_cpu_record_sigbus(CPUState *cs, vaddr address, MMUAccessType access_type, uintptr_t retaddr); #else -bool s390_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool s390x_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr); G_NORETURN void s390x_cpu_do_unaligned_access(CPUState *cs, vaddr addr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr); diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c index 514c70f301..4d0eb129e3 100644 --- a/target/s390x/cpu.c +++ b/target/s390x/cpu.c @@ -330,7 +330,7 @@ void cpu_get_tb_cpu_state(CPUS390XState *env, vaddr *pc, * Instructions must be at even addresses. * This needs to be checked before address translation. */ - env->int_pgm_ilen = 2; /* see s390_cpu_tlb_fill() */ + env->int_pgm_ilen = 2; /* see s390x_cpu_tlb_fill_align() */ tcg_s390_program_interrupt(env, PGM_SPECIFICATION, 0); } @@ -364,7 +364,7 @@ static const TCGCPUOps s390_tcg_ops = { .record_sigsegv = s390_cpu_record_sigsegv, .record_sigbus = s390_cpu_record_sigbus, #else - .tlb_fill = s390_cpu_tlb_fill, + .tlb_fill_align = s390x_cpu_tlb_fill_align, .cpu_exec_interrupt = s390_cpu_exec_interrupt, .cpu_exec_halt = s390_cpu_has_work, .do_interrupt = s390_cpu_do_interrupt, diff --git a/target/s390x/tcg/excp_helper.c b/target/s390x/tcg/excp_helper.c index 4c0b692c9e..6d61032a4a 100644 --- a/target/s390x/tcg/excp_helper.c +++ b/target/s390x/tcg/excp_helper.c @@ -139,9 +139,10 @@ static inline uint64_t cpu_mmu_idx_to_asc(int mmu_idx) } } -bool s390_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool s390x_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUS390XState *env = cpu_env(cs); target_ulong vaddr, raddr; @@ -151,6 +152,14 @@ bool s390_cpu_tlb_fill(CPUState *cs, vaddr address, int size, qemu_log_mask(CPU_LOG_MMU, "%s: addr 0x%" VADDR_PRIx " rw %d mmu_idx %d\n", __func__, address, access_type, mmu_idx); + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + s390x_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + vaddr = address; if (mmu_idx < MMU_REAL_IDX) { @@ -177,8 +186,12 @@ bool s390_cpu_tlb_fill(CPUState *cs, vaddr address, int size, qemu_log_mask(CPU_LOG_MMU, "%s: set tlb %" PRIx64 " -> %" PRIx64 " (%x)\n", __func__, (uint64_t)vaddr, (uint64_t)raddr, prot); - tlb_set_page(cs, address & TARGET_PAGE_MASK, raddr, prot, - mmu_idx, TARGET_PAGE_SIZE); + + memset(out, 0, sizeof(*out)); + out->phys_addr = raddr; + out->prot = prot; + out->lg_page_size = TARGET_PAGE_BITS; + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } if (probe) { From patchwork Thu Nov 14 16:01:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC833D68B34 for ; Thu, 14 Nov 2024 16:10:52 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcNJ-0003lS-Et; Thu, 14 Nov 2024 11:07:17 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLQ-0001mE-UB for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00036L-9s for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:20 -0500 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-71e681bc315so597778b3a.0 for ; Thu, 14 Nov 2024 08:05:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600315; x=1732205115; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=/iKuhhsqlR0EuYEh+MzJkcjXdpnBDTTasC9nFU82FI0=; b=GHW9UMAK3ncz9iOPZ/c7Cn+L/0cTqVzgbQ3pJHQ25HdofRMzr79EY69TN84A+J+0vm XTOHFd+bkF6XzQ5sbLvIq9xzPjXRoNvyzct59Jlxlyu+BACLUaXFBpy06+63b3IevLh/ ox8v6XPCW1UXd2r0teyFhE6S1lYkWvCVNG/gSeVqDRpuEb9P3DySJYYnT4OBOsIdalsT xUMuLf/EoZBW6XseLaKyFpc3fZAu6oWVOJNGXkJMoMf1O+4zk7LHWPlsOXTnp4ffHwkV pV2f8Svr/lX/UBQ8DBXREmV/I0kxnIfiBIPakuJZKWvkcEZOoimGWuMqZXXeTBx8razC ce+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600315; x=1732205115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/iKuhhsqlR0EuYEh+MzJkcjXdpnBDTTasC9nFU82FI0=; b=bAOk9aE6BA6YvgJqaD/W5+rKDjVkSUUHSnuKPRkupd79cKQEIDVNbSyQ3Aam2Oqle/ +QaPcmSBWDncmj2tryoOxdWyjJzyIGoonU5hnZCE3vR/0Jass7uVZi6Zit7OG/NV65g1 HP4yYX9Ve43wGt0Mxl3FZ6/rA1HRQ5DBRtqNY/zvZDUOEvMp16EWQxLMNGlgMASBEIwB zJ99u4MJkzUQksajcHuDUL9JWfrMqi6UgDUYPAjgtvOa321ZsfZKPX1M7O5XA+/14AJq dRY2EzLnHsWZQ8247pbA/oRXh3tF06VYXAVSs1WRCMd3XfJExAUOtTGIPhzue5DP8CGo JoAA== X-Gm-Message-State: AOJu0YwD5DGnycTU5wlD5np3aNOc21q6Kpe17ODNj8Kfa+Dji/Yo36Wg l0bzErJ7FNLSxVmg7hwhKT8QykdzprA0T4KlnYNoOIcrJlaeatmPqZl1jsb4GXGh49F2qZ3NyAp m X-Google-Smtp-Source: AGHT+IGVRLIEpFo5IYTccYGmcv3ovGhtJ2PB1h97+XbdHJh4OBbjXz/1MeXXWgdkVOzRDwCGvzyMKg== X-Received: by 2002:a05:6a00:3bf4:b0:724:6804:d663 with SMTP id d2e1a72fcca58-7246804d6ddmr4556490b3a.1.1731600314704; Thu, 14 Nov 2024 08:05:14 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 47/54] target/sh4: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:23 -0800 Message-ID: <20241114160131.48616-48-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/sh4/cpu.h | 8 +++++--- target/sh4/cpu.c | 2 +- target/sh4/helper.c | 24 +++++++++++++++++------- 3 files changed, 23 insertions(+), 11 deletions(-) diff --git a/target/sh4/cpu.h b/target/sh4/cpu.h index d928bcf006..161efdefcf 100644 --- a/target/sh4/cpu.h +++ b/target/sh4/cpu.h @@ -22,6 +22,7 @@ #include "cpu-qom.h" #include "exec/cpu-defs.h" +#include "exec/memop.h" #include "qemu/cpu-float.h" /* CPU Subtypes */ @@ -251,9 +252,10 @@ void sh4_translate_init(void); #if !defined(CONFIG_USER_ONLY) hwaddr superh_cpu_get_phys_page_debug(CPUState *cpu, vaddr addr); -bool superh_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool superh_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr); void superh_cpu_do_interrupt(CPUState *cpu); bool superh_cpu_exec_interrupt(CPUState *cpu, int int_req); void cpu_sh4_invalidate_tlb(CPUSH4State *s); diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c index 8f07261dcf..8ca8b90e3c 100644 --- a/target/sh4/cpu.c +++ b/target/sh4/cpu.c @@ -252,7 +252,7 @@ static const TCGCPUOps superh_tcg_ops = { .restore_state_to_opc = superh_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = superh_cpu_tlb_fill, + .tlb_fill_align = superh_cpu_tlb_fill_align, .cpu_exec_interrupt = superh_cpu_exec_interrupt, .cpu_exec_halt = superh_cpu_has_work, .do_interrupt = superh_cpu_do_interrupt, diff --git a/target/sh4/helper.c b/target/sh4/helper.c index 9659c69550..543ac1b843 100644 --- a/target/sh4/helper.c +++ b/target/sh4/helper.c @@ -792,22 +792,32 @@ bool superh_cpu_exec_interrupt(CPUState *cs, int interrupt_request) return false; } -bool superh_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool superh_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUSH4State *env = cpu_env(cs); int ret; - target_ulong physical; int prot; + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + superh_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + ret = get_physical_address(env, &physical, &prot, address, access_type); if (ret == MMU_OK) { - address &= TARGET_PAGE_MASK; - physical &= TARGET_PAGE_MASK; - tlb_set_page(cs, address, physical, prot, mmu_idx, TARGET_PAGE_SIZE); + memset(out, 0, sizeof(*out)); + out->phys_addr = physical; + out->prot = prot; + out->lg_page_size = TARGET_PAGE_BITS; + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } if (probe) { From patchwork Thu Nov 14 16:01:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9AD4D68B34 for ; Thu, 14 Nov 2024 16:12:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcNc-0004WK-LD; Thu, 14 Nov 2024 11:07:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLS-0001mT-MG for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00036Q-9y for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:21 -0500 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-20e576dbc42so9331385ad.0 for ; Thu, 14 Nov 2024 08:05:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600315; x=1732205115; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=72w17lGLoodIUSQZn/s5b+crRwhLJRO07+3KfBN9KKk=; b=PHOtPFfOYxe6eiMf3ClgLfSaoMxF3PM5++CNFvrYZ4YUVq3XHRt34XfYpzqeYclqgJ Rawn8t1NYUVKdThvGT3Z6PedYJ5pLrYVc68k6QKZ7bOVgFt8Hf1kvojSnkuTMDEAmDNs s7RzDem3W8GtWIJdHUIe+Q3kVa/dXbC6yDd1IkjTAPFB7eENbfDmta2zV6sqm9QMlseX oldchgxhxak9SksD4/nh+p/5kkncshKhdFWVsyfGw9C7lnvNBMzaFXivgV8xPrmZG9yq q86SrxjQL/vtSgynvrCCDxkZWl7fozUF4Jj5m1tUu3d8APBpW5/I7D1s8jCVsqtjvQ9a Vdxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600315; x=1732205115; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=72w17lGLoodIUSQZn/s5b+crRwhLJRO07+3KfBN9KKk=; b=q8TSV4Z+Y+zW6SHeKoTIpETf31aCR+5TZDjDj1O5/1LVpjitj6ea4n6mhWm9PsrVhA L3T0Lo/cFceodqnGvfm+lTClIK+HkCsC+70VDaeHvSg5k6DKzA9/e6z/kBVV8LO4NC5S LPYq4yueO1v81YkiTZo7hCWkq71vmtcaPuJjJDnx1NKWrR1s89NGvay60uPidcyIG7gG QhiA3DKg/le32Zi39sMlBNtIKmWOzpVyiQwfsdLyYVBqOzoJ5JuwMG4SvDqRI+0LYhO5 jKzWAWt9HbYxRBOG4EtdHB2ufetviuIA6kti2JlEnhg7wFsrXtr3XcsxRr/jxdLgAlfn 1VMw== X-Gm-Message-State: AOJu0Yy7l8dd7cKG1WrHW6Efnkq0VttqTnKloCv6ClHoR17J7wjO4jf/ 9uMjSRUkONhZGj0QHcANRvTywzd2EyqHW0lgkc2p1oi3VTp2xMLKb2P2IJydwDbApRNvWMTviG0 g X-Google-Smtp-Source: AGHT+IE0QyvZdI3vLQn4HufGwcv0pfYvUNiM+X3EG/cq1gFpGvkpib2IeHYpzweL4FriZ25FFpIMew== X-Received: by 2002:a17:903:2ac6:b0:20c:e169:eb75 with SMTP id d9443c01a7336-211b65d93c4mr87296725ad.2.1731600315493; Thu, 14 Nov 2024 08:05:15 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 48/54] target/sparc: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:24 -0800 Message-ID: <20241114160131.48616-49-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/sparc/cpu.h | 8 ++++--- target/sparc/cpu.c | 2 +- target/sparc/mmu_helper.c | 44 +++++++++++++++++++++++++-------------- 3 files changed, 34 insertions(+), 20 deletions(-) diff --git a/target/sparc/cpu.h b/target/sparc/cpu.h index f517e5a383..4c8927e9fa 100644 --- a/target/sparc/cpu.h +++ b/target/sparc/cpu.h @@ -4,6 +4,7 @@ #include "qemu/bswap.h" #include "cpu-qom.h" #include "exec/cpu-defs.h" +#include "exec/memop.h" #include "qemu/cpu-float.h" #if !defined(TARGET_SPARC64) @@ -596,9 +597,10 @@ G_NORETURN void cpu_raise_exception_ra(CPUSPARCState *, int, uintptr_t); void cpu_sparc_set_id(CPUSPARCState *env, unsigned int cpu); void sparc_cpu_list(void); /* mmu_helper.c */ -bool sparc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool sparc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr); target_ulong mmu_probe(CPUSPARCState *env, target_ulong address, int mmulev); void dump_mmu(CPUSPARCState *env); diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c index dd7af86de7..57ae53bd71 100644 --- a/target/sparc/cpu.c +++ b/target/sparc/cpu.c @@ -932,7 +932,7 @@ static const TCGCPUOps sparc_tcg_ops = { .restore_state_to_opc = sparc_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = sparc_cpu_tlb_fill, + .tlb_fill_align = sparc_cpu_tlb_fill_align, .cpu_exec_interrupt = sparc_cpu_exec_interrupt, .cpu_exec_halt = sparc_cpu_has_work, .do_interrupt = sparc_cpu_do_interrupt, diff --git a/target/sparc/mmu_helper.c b/target/sparc/mmu_helper.c index 9ff06026b8..32766a37d6 100644 --- a/target/sparc/mmu_helper.c +++ b/target/sparc/mmu_helper.c @@ -203,12 +203,12 @@ static int get_physical_address(CPUSPARCState *env, CPUTLBEntryFull *full, } /* Perform address translation */ -bool sparc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool sparc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUSPARCState *env = cpu_env(cs); - CPUTLBEntryFull full = {}; target_ulong vaddr; int error_code = 0, access_index; @@ -220,16 +220,21 @@ bool sparc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, */ assert(!probe); + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + sparc_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + + memset(out, 0, sizeof(*out)); address &= TARGET_PAGE_MASK; - error_code = get_physical_address(env, &full, &access_index, + error_code = get_physical_address(env, out, &access_index, address, access_type, mmu_idx); vaddr = address; if (likely(error_code == 0)) { qemu_log_mask(CPU_LOG_MMU, "Translate at %" VADDR_PRIx " -> " HWADDR_FMT_plx ", vaddr " TARGET_FMT_lx "\n", - address, full.phys_addr, vaddr); - tlb_set_page_full(cs, mmu_idx, vaddr, &full); + address, out->phys_addr, vaddr); return true; } @@ -244,8 +249,7 @@ bool sparc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, permissions. If no mapping is available, redirect accesses to neverland. Fake/overridden mappings will be flushed when switching to normal mode. */ - full.prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; - tlb_set_page_full(cs, mmu_idx, vaddr, &full); + out->prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; return true; } else { if (access_type == MMU_INST_FETCH) { @@ -754,22 +758,30 @@ static int get_physical_address(CPUSPARCState *env, CPUTLBEntryFull *full, } /* Perform address translation */ -bool sparc_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool sparc_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUSPARCState *env = cpu_env(cs); - CPUTLBEntryFull full = {}; int error_code = 0, access_index; + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + sparc_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + + memset(out, 0, sizeof(*out)); address &= TARGET_PAGE_MASK; - error_code = get_physical_address(env, &full, &access_index, + error_code = get_physical_address(env, out, &access_index, address, access_type, mmu_idx); if (likely(error_code == 0)) { - trace_mmu_helper_mmu_fault(address, full.phys_addr, mmu_idx, env->tl, + trace_mmu_helper_mmu_fault(address, out->phys_addr, mmu_idx, env->tl, env->dmmu.mmu_primary_context, env->dmmu.mmu_secondary_context); - tlb_set_page_full(cs, mmu_idx, address, &full); return true; } if (probe) { From patchwork Thu Nov 14 16:01:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5FD90D68B34 for ; Thu, 14 Nov 2024 16:14:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcOQ-0005Jt-2q; Thu, 14 Nov 2024 11:08:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLQ-0001mB-MG for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00036X-9p for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:19 -0500 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-71e4e481692so636044b3a.1 for ; Thu, 14 Nov 2024 08:05:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600316; x=1732205116; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=9nz66+lLu2GQe/uhCiK8Lu0UqaZ0it8xcxl9KTGeICs=; b=hrhomyc7oDrtl/T+yrlHVuMTxj2igSBr9po7/BPODnR5tcCrnENEn237BYsdgdYo9Y ha1+xKlopP/X/fVvHyNDLK3ixSS2WiPeQwnMCTA95fTD/RLRLJVcHyjhsnsjSz9PCHi9 0HGT8foPKd9NGJTLz+itgKrlK60ISSIhDJGWVCcrdozbFw9Dk6VpczFvRpHrCf9Ib7wh 1PNernK2LwD0v261CsenbYzJObTAdrYsUp+1U4h4lgmdb+33NQoSvy3Ko3QK5DQxb5SD ZFxMYu1rjarSK7HACJCoKiakRhpYOoShylOvDLSAOmpoLVQffUKEDtnyv5YI4W8KqX9M B3Eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600316; x=1732205116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9nz66+lLu2GQe/uhCiK8Lu0UqaZ0it8xcxl9KTGeICs=; b=fze+oRAmb+b7gyI9CbPuX/9BT5NglE9Vc18VfezpllVwhfwCoC2NGQbhBNTi4zb7dB iUf4vqahfndBttfekcEU0o+NavXtCVe75zjCW4ma+REwQaeAvIcUlfr8MFG0jvNgp+gP LVxO4eJKcrOCbl46gY5BTXQ5DkfFDga+/VJM/k051oKV0yaH1/4z8wc9M/XqKpL1bp9e 8isBHKiqMqJqSMikoynK2+PL16AL+O1VY1ZKIazM1gwRj870r9QvTyYc1klDJUoUdxgA A5P/NPCzu2IW6zxgfCZTgFSnjqqSnIMAR4LmRWplWXfM7UYQtTu8G+ICf3N5dNarZuE/ Qtyw== X-Gm-Message-State: AOJu0YzcBP6eZGfhI2qunHunPNxKrrQX/MY7W0vswDBNOztJyTYrk7zm oCNO8dFomjmuI80Rd/3GWuEo0vDdR8Pgb5jemVbp2HGnJsDDJNc7xAsgpQs1SMBXM3eqpyy7tTw 9 X-Google-Smtp-Source: AGHT+IF8SLdAKjSr5fcT9yE2AZtutkW+OgOe31Z/tOjbkTvktdkTDUwzv3LOGjNXVzdwBlfCyc5j2g== X-Received: by 2002:aa7:88c8:0:b0:71e:f83:5c00 with SMTP id d2e1a72fcca58-72469c485bfmr3740544b3a.2.1731600316282; Thu, 14 Nov 2024 08:05:16 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:15 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 49/54] target/tricore: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:25 -0800 Message-ID: <20241114160131.48616-50-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/tricore/cpu.h | 7 ++++--- target/tricore/cpu.c | 2 +- target/tricore/helper.c | 19 ++++++++++++------- 3 files changed, 17 insertions(+), 11 deletions(-) diff --git a/target/tricore/cpu.h b/target/tricore/cpu.h index 220af69fc2..5f141ce8f3 100644 --- a/target/tricore/cpu.h +++ b/target/tricore/cpu.h @@ -268,8 +268,9 @@ static inline void cpu_get_tb_cpu_state(CPUTriCoreState *env, vaddr *pc, #define CPU_RESOLVING_TYPE TYPE_TRICORE_CPU /* helpers.c */ -bool tricore_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool tricore_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr); #endif /* TRICORE_CPU_H */ diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c index 1a26171590..29e0b5d129 100644 --- a/target/tricore/cpu.c +++ b/target/tricore/cpu.c @@ -173,7 +173,7 @@ static const TCGCPUOps tricore_tcg_ops = { .initialize = tricore_tcg_init, .synchronize_from_tb = tricore_cpu_synchronize_from_tb, .restore_state_to_opc = tricore_restore_state_to_opc, - .tlb_fill = tricore_cpu_tlb_fill, + .tlb_fill_align = tricore_cpu_tlb_fill_align, .cpu_exec_interrupt = tricore_cpu_exec_interrupt, .cpu_exec_halt = tricore_cpu_has_work, }; diff --git a/target/tricore/helper.c b/target/tricore/helper.c index 7014255f77..8c6bf63298 100644 --- a/target/tricore/helper.c +++ b/target/tricore/helper.c @@ -64,16 +64,19 @@ static void raise_mmu_exception(CPUTriCoreState *env, target_ulong address, { } -bool tricore_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType rw, int mmu_idx, - bool probe, uintptr_t retaddr) +bool tricore_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUTriCoreState *env = cpu_env(cs); hwaddr physical; int prot; int ret = 0; + int rw = access_type & 1; + + /* TODO: alignment faults not currently handled. */ - rw &= 1; ret = get_physical_address(env, &physical, &prot, address, rw, mmu_idx); @@ -82,9 +85,11 @@ bool tricore_cpu_tlb_fill(CPUState *cs, vaddr address, int size, __func__, address, ret, physical, prot); if (ret == TLBRET_MATCH) { - tlb_set_page(cs, address & TARGET_PAGE_MASK, - physical & TARGET_PAGE_MASK, prot | PAGE_EXEC, - mmu_idx, TARGET_PAGE_SIZE); + memset(out, 0, sizeof(*out)); + out->phys_addr = physical; + out->prot = prot | PAGE_EXEC; + out->lg_page_size = TARGET_PAGE_BITS; + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } else { assert(ret < 0); From patchwork Thu Nov 14 16:01:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7780ED68B33 for ; Thu, 14 Nov 2024 16:13:47 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcO7-0004tS-BL; Thu, 14 Nov 2024 11:08:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLS-0001mS-M9 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLO-00036c-Rd for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:21 -0500 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-724455f40a0so576984b3a.0 for ; Thu, 14 Nov 2024 08:05:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600317; x=1732205117; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Bj/Re7YaCahV1M4/AVlcGxlPUjwCTTqNcJS3jvHiVnM=; b=sm79gGcY8d7R9U9YKWSOalv1jMZv7UuBvgPq7cdJ6da+g9NKr/UziNuCvqYSNH47hQ Srumo0PlGl/nla/gilPMRIM9t8Hw6wjsiaasmqa56T5oP/fWWrXxaDgw8jRzRsdEhpYz chkQwvqcaOdeXA1ZHsQLlbHd1b4Rs8Y9QRulNIFardDgZ1giQmgWH5TeEdREBlCFoWoW 5RTXRSDjWbJwN/A9HKPFvLr6MjpR+e2fu65zGAZQh9OYTT4HtIvLnM11+0lMTHIBtSka JddtHbtV0kVwKYbmQ970agVARjrNylphT6XkzRbTMKvEgQQ8VVCpfbI+4DJYCa/uUasj lcYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600317; x=1732205117; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bj/Re7YaCahV1M4/AVlcGxlPUjwCTTqNcJS3jvHiVnM=; b=ZquKvEDMMsg5ky/ObvffKmMpE4Y6MSYA1gQ4yiLTwr2djc2X2798/9WqZkUi39MTZA G4h46Bl6eJsiOZcSVCWVRPQgP3ewoJSPVm8EeBKaGIKZd3uaKV1YS0BKESBMkNhLd61q beECbKUQM4XmkiMOF6rFeYJ6RwTZZZXq6YOydlJRt11Z9OtX+YJLWsev75RMSNyzQs+x rU9lvabAgLkrnCUOcMz+IjH6nGcuefSjbmP/HggxvA5u9N8n6bGLO6TY/9z3G2Qas/lk etyZ5D64ZdeeC1MS+djDk9COnKmEsc3tEKhmryqNZIzVagSSPVVPIaqs9JcRyoILhU8K Z1BA== X-Gm-Message-State: AOJu0YwsbDmsJL2N09nu7UqGN31CfvQUEBdBzeRAepFPxziNiHWNscSR unQcw+OW8BVT/RE6WWn5RLArVaHGR1/L+7YNcA9uO9NRx1eJjTeJOKmIXNMnZsS5MaHRaqkdwaD 0 X-Google-Smtp-Source: AGHT+IHHzrlH5cH6Mf/qYJuZTB1/8Ct1LLnp88Scf1vNvjfLmK/5f02XQ1P2dX3PBcxmhrvvqlLEOg== X-Received: by 2002:aa7:88c9:0:b0:71e:592d:6fa9 with SMTP id d2e1a72fcca58-7241327b6a4mr30110491b3a.1.1731600317101; Thu, 14 Nov 2024 08:05:17 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.16 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:16 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 50/54] target/xtensa: Convert to TCGCPUOps.tlb_fill_align Date: Thu, 14 Nov 2024 08:01:26 -0800 Message-ID: <20241114160131.48616-51-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- target/xtensa/cpu.h | 8 +++++--- target/xtensa/cpu.c | 2 +- target/xtensa/helper.c | 28 ++++++++++++++++++++-------- 3 files changed, 26 insertions(+), 12 deletions(-) diff --git a/target/xtensa/cpu.h b/target/xtensa/cpu.h index 77e48eef19..68c3d90d41 100644 --- a/target/xtensa/cpu.h +++ b/target/xtensa/cpu.h @@ -31,6 +31,7 @@ #include "cpu-qom.h" #include "qemu/cpu-float.h" #include "exec/cpu-defs.h" +#include "exec/memop.h" #include "hw/clock.h" #include "xtensa-isa.h" @@ -580,9 +581,10 @@ struct XtensaCPUClass { }; #ifndef CONFIG_USER_ONLY -bool xtensa_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); +bool xtensa_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr addr, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr); void xtensa_cpu_do_interrupt(CPUState *cpu); bool xtensa_cpu_exec_interrupt(CPUState *cpu, int interrupt_request); void xtensa_cpu_do_transaction_failed(CPUState *cs, hwaddr physaddr, vaddr addr, diff --git a/target/xtensa/cpu.c b/target/xtensa/cpu.c index 6f9039abae..3e4ec97e0e 100644 --- a/target/xtensa/cpu.c +++ b/target/xtensa/cpu.c @@ -232,7 +232,7 @@ static const TCGCPUOps xtensa_tcg_ops = { .restore_state_to_opc = xtensa_restore_state_to_opc, #ifndef CONFIG_USER_ONLY - .tlb_fill = xtensa_cpu_tlb_fill, + .tlb_fill_align = xtensa_cpu_tlb_fill_align, .cpu_exec_interrupt = xtensa_cpu_exec_interrupt, .cpu_exec_halt = xtensa_cpu_has_work, .do_interrupt = xtensa_cpu_do_interrupt, diff --git a/target/xtensa/helper.c b/target/xtensa/helper.c index ca214b948a..69b0e661c8 100644 --- a/target/xtensa/helper.c +++ b/target/xtensa/helper.c @@ -261,15 +261,26 @@ void xtensa_cpu_do_unaligned_access(CPUState *cs, addr); } -bool xtensa_cpu_tlb_fill(CPUState *cs, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr) +bool xtensa_cpu_tlb_fill_align(CPUState *cs, CPUTLBEntryFull *out, + vaddr address, MMUAccessType access_type, + int mmu_idx, MemOp memop, int size, + bool probe, uintptr_t retaddr) { CPUXtensaState *env = cpu_env(cs); uint32_t paddr; uint32_t page_size; unsigned access; - int ret = xtensa_get_physical_addr(env, true, address, access_type, + int ret; + + if (address & ((1 << memop_alignment_bits(memop)) - 1)) { + if (probe) { + return false; + } + xtensa_cpu_do_unaligned_access(cs, address, access_type, + mmu_idx, retaddr); + } + + ret = xtensa_get_physical_addr(env, true, address, access_type, mmu_idx, &paddr, &page_size, &access); qemu_log_mask(CPU_LOG_MMU, "%s(%08" VADDR_PRIx @@ -277,10 +288,11 @@ bool xtensa_cpu_tlb_fill(CPUState *cs, vaddr address, int size, __func__, address, access_type, mmu_idx, paddr, ret); if (ret == 0) { - tlb_set_page(cs, - address & TARGET_PAGE_MASK, - paddr & TARGET_PAGE_MASK, - access, mmu_idx, page_size); + memset(out, 0, sizeof(*out)); + out->phys_addr = paddr; + out->prot = access; + out->lg_page_size = ctz32(page_size); + out->attrs = MEMTXATTRS_UNSPECIFIED; return true; } else if (probe) { return false; From patchwork Thu Nov 14 16:01:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CBCFCD68B34 for ; Thu, 14 Nov 2024 16:11:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcMx-0002fI-1p; Thu, 14 Nov 2024 11:06:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLS-0001mR-M3 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:27 -0500 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLQ-000372-E2 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:22 -0500 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-723f37dd76cso758026b3a.0 for ; Thu, 14 Nov 2024 08:05:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600318; x=1732205118; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=hjYrYa5EsHh3SE2oZ1zQ8eTrBOhtgNkNlLaG4FiIf5M=; b=iXIqKF3UcTJ2ELfpuHHTsvloBCJ/VdGkHM58d54rtNF0yPlMwgHAwUOou+jBxyVqhJ SQX290WHeKHa1+mCnt6GTvEPiXn/4ycs+mNWr1dDAU3s+Jvc6EfuYMsh8c84kP+nvYHK lHgQcnJGh8ugv61XTgdFGnatANCQsNCEorJmob3qckucanMJGmBVBOMdmRxRLO87nLui FudC8LOsE5xymPywi2v/Iz+20vrm9E1j2aSGyiU7KcJnTEeh0tAnbkmgiuz9Ts8PAj7D Mp7iXPtxr7I1POxIE0cXRMy3mb7/S5ju5gfukUzhS2jpB903d31vMNXtTOCAIDl+KZ28 bA9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600318; x=1732205118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hjYrYa5EsHh3SE2oZ1zQ8eTrBOhtgNkNlLaG4FiIf5M=; b=t6HDKuMUeeNjqfjwaRQgoLpIAoN75CJzcWQWE0hcUJj6/XjaBeg5V6Rg47J8FKuC4M A/qGrh627c166P7IdEjscVU3nAb3DcoitDvxehs+5Om1zniy/SeXPk6W7MCyAP93up+t kzxBSE23Zt4hscPsO2tdjhj+YJ75qOAw96QiJXUCeHg14zDOiVLH3mXimmUErCOxtenZ 7mC8MzyZEYgeorGAKeyRUBAZHpItddYYbC8n3mQVe5hmbVidyo4TrGU08VSfbiO/m0sO qni+73XpCXvg+3HwNY+SYbEn0RdAL30bqoTq/Wpg8tIdIittRo0Xk+poRMx9RVygV5rj iCaw== X-Gm-Message-State: AOJu0YzpVafqnm88nUftXCidAXtYZkFJ7ufmvBVMh47cg0RZ54XALamf Ik3j8rBkGvXBRM30hC/ZJGPeLfLjn6lCd6bFogMVZXS/kckYWoZqgwZwsIacJN+jGQHGaZq88vZ f X-Google-Smtp-Source: AGHT+IGfHX5pYHxsHA7K4uM46iOMwPpNfvOwCwQV10VKTCN6yRoVDj0v4MXJ81BIqIbxRSbsPDjEqA== X-Received: by 2002:a05:6a00:234b:b0:71e:7d52:fa8c with SMTP id d2e1a72fcca58-72457a7563bmr9083242b3a.22.1731600318394; Thu, 14 Nov 2024 08:05:18 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:17 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 51/54] accel/tcg: Drop TCGCPUOps.tlb_fill Date: Thu, 14 Nov 2024 08:01:27 -0800 Message-ID: <20241114160131.48616-52-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Now that all targets have been converted to tlb_fill_align, remove the tlb_fill hook. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/hw/core/tcg-cpu-ops.h | 10 ---------- accel/tcg/cputlb.c | 19 ++++--------------- 2 files changed, 4 insertions(+), 25 deletions(-) diff --git a/include/hw/core/tcg-cpu-ops.h b/include/hw/core/tcg-cpu-ops.h index 663efb9133..70cafcc6cd 100644 --- a/include/hw/core/tcg-cpu-ops.h +++ b/include/hw/core/tcg-cpu-ops.h @@ -157,16 +157,6 @@ struct TCGCPUOps { bool (*tlb_fill_align)(CPUState *cpu, CPUTLBEntryFull *out, vaddr addr, MMUAccessType access_type, int mmu_idx, MemOp memop, int size, bool probe, uintptr_t ra); - /** - * @tlb_fill: Handle a softmmu tlb miss - * - * If the access is valid, call tlb_set_page and return true; - * if the access is invalid and probe is true, return false; - * otherwise raise an exception and do not return. - */ - bool (*tlb_fill)(CPUState *cpu, vaddr address, int size, - MMUAccessType access_type, int mmu_idx, - bool probe, uintptr_t retaddr); /** * @do_transaction_failed: Callback for handling failed memory transactions * (ie bus faults or external aborts; not MMU faults) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 7f63dc3fd8..ec597ed6f5 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1222,23 +1222,12 @@ static bool tlb_fill_align(CPUState *cpu, vaddr addr, MMUAccessType type, int mmu_idx, MemOp memop, int size, bool probe, uintptr_t ra) { - const TCGCPUOps *ops = cpu->cc->tcg_ops; CPUTLBEntryFull full; - if (ops->tlb_fill_align) { - if (ops->tlb_fill_align(cpu, &full, addr, type, mmu_idx, - memop, size, probe, ra)) { - tlb_set_page_full(cpu, mmu_idx, addr, &full); - return true; - } - } else { - /* Legacy behaviour is alignment before paging. */ - if (addr & ((1u << memop_alignment_bits(memop)) - 1)) { - ops->do_unaligned_access(cpu, addr, type, mmu_idx, ra); - } - if (ops->tlb_fill(cpu, addr, size, type, mmu_idx, probe, ra)) { - return true; - } + if (cpu->cc->tcg_ops->tlb_fill_align(cpu, &full, addr, type, mmu_idx, + memop, size, probe, ra)) { + tlb_set_page_full(cpu, mmu_idx, addr, &full); + return true; } assert(probe); return false; From patchwork Thu Nov 14 16:01:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B93E8D68B34 for ; Thu, 14 Nov 2024 16:13:43 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcON-0005IR-IN; Thu, 14 Nov 2024 11:08:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLi-0001v0-AM for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:41 -0500 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLd-00039O-1w for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:34 -0500 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-7205b6f51f3so553364b3a.1 for ; Thu, 14 Nov 2024 08:05:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600330; x=1732205130; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=KflgX9SpwT1nWAco5X5rdlvjIGuNkPShcCz1Q+BgM4A=; b=XfSyYBwDHMCAvNQd6rQiqbC0jfTSBt2vDp3WcM56kZlEg7a8n1swaR9mCdc/bQ31gL 51DEGkXv1U1zWRCjmXW3S6MVQAqZALS0jZA/TmcjX73PrwBeSh9vaxLTPSCLFaBFhCVE iZXCTJy0xiWpYTbrWlIi97eXgCaWIGXm94liw8b9WxdJJExvG2EwzjAguUuCpjN5u85v uRuUMVLwG+XJgWEBYHP4gc7Ssyry/+3MwwIEb1Acqdfo2+jRWertP3N4JXasLSr/GZb9 gSSyMSINQfu3Hl1nAzc73i1nXyB8g9rglzvYl0I7h2TOKDS/+eJ81a10Y/74ulej80zL Bp5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600330; x=1732205130; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KflgX9SpwT1nWAco5X5rdlvjIGuNkPShcCz1Q+BgM4A=; b=MPh1D4YAe196kHPRu2uNpd6Z9dT/yrm1sEWsYyQ5F87IuTOpDm0PivyN3mZHTCrm5C q9FOLwgvnwxhdZroJe4c6n5irDr7K2FjAVMYNdtYvLJTx0JsRnpTBDZTD0cauaheSuBt 6PMIave0QvIX9N+SllElaurQdqFRxIauvCjDRczmbozAzC6WhH1gxp9iU7SSH+otZ6RZ n+f0B6/bF3y+Ya4De9yLvsW3K/fRxsf9p+RvnVJxnYfv6DVrM4d5Q+Z6GzF2WLYzogmg EuBr586378eePmpF527QWI/NqYf6lPxKk1vryxd8DlCoIHdukw77xzmWBjV/CT5/pcXS 18ww== X-Gm-Message-State: AOJu0YxBHSMU3bIgg7MScdPILfwT5eQpW90/rken14JTegtma8CI8D1P CASkcMEslaB/2ZnnVR3KKlNG9mr/9tGMvGiMLmyPep0pIlxpkb3kAz68LbEVDQfHD+cVNiQmqiY q X-Google-Smtp-Source: AGHT+IGO3jezWbdtn0uPJqPEsS05jrM4krkq5yq0KebbNJckuiUrDWpVdCdNdkuMMrJmBoVvUDX4KQ== X-Received: by 2002:a05:6a00:3d49:b0:724:680d:d12c with SMTP id d2e1a72fcca58-72469cd7843mr2774768b3a.12.1731600319259; Thu, 14 Nov 2024 08:05:19 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.18 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:18 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 52/54] accel/tcg: Unexport tlb_set_page* Date: Thu, 14 Nov 2024 08:01:28 -0800 Message-ID: <20241114160131.48616-53-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The new tlb_fill_align hook returns page data via structure rather than by function call, so we can make tlb_set_page_full be local to cputlb.c. There are no users of tlb_set_page or tlb_set_page_with_attrs, so those can be eliminated. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- include/exec/exec-all.h | 57 ----------------------------------------- accel/tcg/cputlb.c | 27 ++----------------- 2 files changed, 2 insertions(+), 82 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 69bdb77584..b65fc547bd 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -184,63 +184,6 @@ void tlb_flush_range_by_mmuidx_all_cpus_synced(CPUState *cpu, vaddr len, uint16_t idxmap, unsigned bits); - -/** - * tlb_set_page_full: - * @cpu: CPU context - * @mmu_idx: mmu index of the tlb to modify - * @addr: virtual address of the entry to add - * @full: the details of the tlb entry - * - * Add an entry to @cpu tlb index @mmu_idx. All of the fields of - * @full must be filled, except for xlat_section, and constitute - * the complete description of the translated page. - * - * This is generally called by the target tlb_fill function after - * having performed a successful page table walk to find the physical - * address and attributes for the translation. - * - * At most one entry for a given virtual address is permitted. Only a - * single TARGET_PAGE_SIZE region is mapped; @full->lg_page_size is only - * used by tlb_flush_page. - */ -void tlb_set_page_full(CPUState *cpu, int mmu_idx, vaddr addr, - CPUTLBEntryFull *full); - -/** - * tlb_set_page_with_attrs: - * @cpu: CPU to add this TLB entry for - * @addr: virtual address of page to add entry for - * @paddr: physical address of the page - * @attrs: memory transaction attributes - * @prot: access permissions (PAGE_READ/PAGE_WRITE/PAGE_EXEC bits) - * @mmu_idx: MMU index to insert TLB entry for - * @size: size of the page in bytes - * - * Add an entry to this CPU's TLB (a mapping from virtual address - * @addr to physical address @paddr) with the specified memory - * transaction attributes. This is generally called by the target CPU - * specific code after it has been called through the tlb_fill() - * entry point and performed a successful page table walk to find - * the physical address and attributes for the virtual address - * which provoked the TLB miss. - * - * At most one entry for a given virtual address is permitted. Only a - * single TARGET_PAGE_SIZE region is mapped; the supplied @size is only - * used by tlb_flush_page. - */ -void tlb_set_page_with_attrs(CPUState *cpu, vaddr addr, - hwaddr paddr, MemTxAttrs attrs, - int prot, int mmu_idx, vaddr size); -/* tlb_set_page: - * - * This function is equivalent to calling tlb_set_page_with_attrs() - * with an @attrs argument of MEMTXATTRS_UNSPECIFIED. It's provided - * as a convenience for CPUs which don't use memory transaction attributes. - */ -void tlb_set_page(CPUState *cpu, vaddr addr, - hwaddr paddr, int prot, - int mmu_idx, vaddr size); #else static inline void tlb_init(CPUState *cpu) { diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ec597ed6f5..3d731b8f3d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1037,8 +1037,8 @@ static inline void tlb_set_compare(CPUTLBEntryFull *full, CPUTLBEntry *ent, * Called from TCG-generated code, which is under an RCU read-side * critical section. */ -void tlb_set_page_full(CPUState *cpu, int mmu_idx, - vaddr addr, CPUTLBEntryFull *full) +static void tlb_set_page_full(CPUState *cpu, int mmu_idx, + vaddr addr, CPUTLBEntryFull *full) { CPUTLB *tlb = &cpu->neg.tlb; CPUTLBDesc *desc = &tlb->d[mmu_idx]; @@ -1189,29 +1189,6 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, qemu_spin_unlock(&tlb->c.lock); } -void tlb_set_page_with_attrs(CPUState *cpu, vaddr addr, - hwaddr paddr, MemTxAttrs attrs, int prot, - int mmu_idx, uint64_t size) -{ - CPUTLBEntryFull full = { - .phys_addr = paddr, - .attrs = attrs, - .prot = prot, - .lg_page_size = ctz64(size) - }; - - assert(is_power_of_2(size)); - tlb_set_page_full(cpu, mmu_idx, addr, &full); -} - -void tlb_set_page(CPUState *cpu, vaddr addr, - hwaddr paddr, int prot, - int mmu_idx, uint64_t size) -{ - tlb_set_page_with_attrs(cpu, addr, paddr, MEMTXATTRS_UNSPECIFIED, - prot, mmu_idx, size); -} - /* * Note: tlb_fill_align() can trigger a resize of the TLB. * This means that all of the caller's prior references to the TLB table From patchwork Thu Nov 14 16:01:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6A1E8D68B35 for ; Thu, 14 Nov 2024 16:13:42 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcNN-0003zo-G5; Thu, 14 Nov 2024 11:07:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLj-0001wv-Ew for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:43 -0500 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLd-00039n-ND for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:39 -0500 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-20cdb889222so8296375ad.3 for ; Thu, 14 Nov 2024 08:05:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600331; x=1732205131; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=kszkBxHzd0i3pHfwTrRD6LMnpSdmBjH+8x+WNocCo4c=; b=JaYBhne0qGJs/FNxFWMqbHW0IPxJpAkOqHv0C0W4DlHd9nO2lzjhq/wsRypJFhvemb swIth7Wn8+MtjZ1v1yt/vRMV4abzkO7CHZxf6J2XuD6mDgiZF43ojEwIQoiisKy/IV2z 03EefWumcoJSsJ5PmMimbl+PFw59yW6KOp6mDh2SBf8y1+2yT9bI4xiAQi0Co/ObrePK UCxzTuBoANmKjl10U5r7Q61fKcFCECHiX/XMZwoBS+S4LNvjFRN1SWIRuuWO5MBzsKhz VtwYHb9w5oP1pwdrVqrm8UGH/mPWsJZcr++t4dIX1g5rZNhiR31fYLxuI9NB/INOCZqe EUZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600331; x=1732205131; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kszkBxHzd0i3pHfwTrRD6LMnpSdmBjH+8x+WNocCo4c=; b=ExDG3kBgesUxCeNgiOXiINGZHyDefJ6FgeuV6ITG6qxujwGhuj9zLCMUW222lLyce9 lzNnW3ZnPUmY7tPw/BO84muF2C1kgJlMZlclgaUF87h9cqZU2d8HzX3TSoxz8ZD+vE6n KrmExvhbxD2aNqz8TvqkNenCesfB94Fx15Ny9JPRoKsE4/4jDMEVRduxgEF1hMImOOGd 5RGSsbGbDXS7AhyPMqb1/icxtXoGuGNH2tDx5JwsIGMaZOxsMP2LR2yoarqq6Fd/DFvL TJab+CZv8zmd7kMHdYt+1uxP5OZulkk1DRyE1M1i42iz1+O3JCOX3n5Wlo1dKeY03FGT PhQQ== X-Gm-Message-State: AOJu0YzQBHDBUH+CBOvUYsX4S3t0BveOsWSTf907SYd+IrxHbSsqYF8b 0+/9RNA5Ys9FenLtLwdyT7lCncRr0c/oLL30+rb+soelHWSEON7m9fy5udBynJo9Go2j87h/+rc w X-Google-Smtp-Source: AGHT+IHfV2PmkvW1JQisvwXqKrg1ceGhBQyVkQZzMtmDKkNczdk132iVVqRFoO6F0xVqEilcj9oFCA== X-Received: by 2002:a17:902:d4d1:b0:210:fce4:11ec with SMTP id d9443c01a7336-211b5bcc223mr100430365ad.1.1731600330998; Thu, 14 Nov 2024 08:05:30 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:30 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 53/54] accel/tcg: Merge tlb_fill_align into callers Date: Thu, 14 Nov 2024 08:01:29 -0800 Message-ID: <20241114160131.48616-54-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In tlb_lookup, we still call tlb_set_page_full. In atomic_mmu_lookup, we're expecting noreturn. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 31 ++++++------------------------- 1 file changed, 6 insertions(+), 25 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 3d731b8f3d..20af48c6c5 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1189,27 +1189,6 @@ static void tlb_set_page_full(CPUState *cpu, int mmu_idx, qemu_spin_unlock(&tlb->c.lock); } -/* - * Note: tlb_fill_align() can trigger a resize of the TLB. - * This means that all of the caller's prior references to the TLB table - * (e.g. CPUTLBEntry pointers) must be discarded and looked up again - * (e.g. via tlb_entry()). - */ -static bool tlb_fill_align(CPUState *cpu, vaddr addr, MMUAccessType type, - int mmu_idx, MemOp memop, int size, - bool probe, uintptr_t ra) -{ - CPUTLBEntryFull full; - - if (cpu->cc->tcg_ops->tlb_fill_align(cpu, &full, addr, type, mmu_idx, - memop, size, probe, ra)) { - tlb_set_page_full(cpu, mmu_idx, addr, &full); - return true; - } - assert(probe); - return false; -} - static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr, MMUAccessType access_type, int mmu_idx, uintptr_t retaddr) @@ -1281,11 +1260,13 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, } /* Finally, query the target hook. */ - if (!tlb_fill_align(cpu, addr, access_type, i->mmu_idx, - memop, i->size, probe, i->ra)) { + if (!cpu->cc->tcg_ops->tlb_fill_align(cpu, &o->full, addr, access_type, + i->mmu_idx, memop, i->size, + probe, i->ra)) { tcg_debug_assert(probe); return false; } + tlb_set_page_full(cpu, i->mmu_idx, addr, &o->full); o->did_tlb_fill = true; if (access_type == MMU_INST_FETCH) { @@ -1794,8 +1775,8 @@ static void *atomic_mmu_lookup(CPUState *cpu, vaddr addr, MemOpIdx oi, * We have just verified that the page is writable. */ if (unlikely(!(o.full.prot & PAGE_READ))) { - tlb_fill_align(cpu, addr, MMU_DATA_LOAD, i.mmu_idx, - 0, i.size, false, i.ra); + cpu->cc->tcg_ops->tlb_fill_align(cpu, &o.full, addr, MMU_DATA_LOAD, + i.mmu_idx, 0, i.size, false, i.ra); /* * Since we don't support reads and writes to different * addresses, and we do have the proper page loaded for From patchwork Thu Nov 14 16:01:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 13875383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1E47D68B33 for ; Thu, 14 Nov 2024 16:11:54 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tBcO4-0004kx-4V; Thu, 14 Nov 2024 11:08:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tBcLj-0001ww-H8 for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:43 -0500 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tBcLg-00039v-Rz for qemu-devel@nongnu.org; Thu, 14 Nov 2024 11:05:39 -0500 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-20cdb889222so8296595ad.3 for ; Thu, 14 Nov 2024 08:05:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1731600332; x=1732205132; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=TEPg1Hj+WyfSzdzbNBp7DHrrLsvR0rYuhsge67okkJI=; b=Z3M0RlbEIVjs2QMJ8IuimdYZ+paVJ59QyL0/DRbXWo+SE5nFvKY5Q4HoCNZ5T1rliA JrbcwMDzcZlf2ED+iLIzUHLvg6cVskMbK1L7uq9Vn5gMO39jzNS1t4/G1ATOrk2vlZt9 /cElN6ZYFTr4PoNppiidukIk/YrAuWs9h5eIIJztQ2XtG34LN0vMAawsDTXK0BbmFkC+ BDi2u7KoWyNhIfECvO2Ht1oFebW6W7AkkJqiRYpNdgQxOzNIrUqTMIPWKTnBP4gyaFp6 w0B7faLsqK34bCA7B0ul09EtBobiHBzstZBR9DKU+mNJx38sSdFQqYKqn6sSk1I41QRA nQrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731600332; x=1732205132; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TEPg1Hj+WyfSzdzbNBp7DHrrLsvR0rYuhsge67okkJI=; b=CHktdoFu2acmjo8By3qAmyyMPP+DOlttgZK4CLoQgihfFP3uL0IVgMzeuoj2hRv3Er yQhr/6ZyE3jmDoFzWHYyLthdJ9uX5zNJegYOu7vvmDlNos/bOCPqT2k06Xribv9wlApX xxDcThDcyW2UWZy465EwtC6DFp6hKYtZJG9ya/6d4UMljgVOIzxuWKWzQAR5ihl7MGYY cUa8z03LUslT8W7gASaMllHo1wv5+7BBD1OpANj4rHX2kQS9f/8HCz6YXTQNRsjeBu64 3+WXrXNpjheFGMxqkxEws2bA1IDtYqv2lUB9Hm2kPIJ3LDD1j6X3bu3M+cpCCsijk6Lu 1Kjg== X-Gm-Message-State: AOJu0YxpJNHQvHqoe1wxMXofspraiGk11UZZMpRjVBjgWrX4MOU0cFo/ W6/sK1pszBAHWUdp6DeZHxway21OfUXnFf+wRLuRRgX2wM10yu8flzt+It/ictdleGT4CfvRswG 1 X-Google-Smtp-Source: AGHT+IHwD2ZH3nElpu72z9wlCwe5s6d4kK4b2hjowEVGska+Pc3jcr4cK/alUydGLC/WMG1nCq9u7w== X-Received: by 2002:a17:902:da87:b0:20b:7be8:8eb9 with SMTP id d9443c01a7336-211b5d5d4c9mr103164115ad.54.1731600331965; Thu, 14 Nov 2024 08:05:31 -0800 (PST) Received: from stoup.. ([71.212.136.242]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7246a9bab70sm1417926b3a.152.2024.11.14.08.05.31 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Nov 2024 08:05:31 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 54/54] accel/tcg: Return CPUTLBEntryTree from tlb_set_page_full Date: Thu, 14 Nov 2024 08:01:30 -0800 Message-ID: <20241114160131.48616-55-richard.henderson@linaro.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241114160131.48616-1-richard.henderson@linaro.org> References: <20241114160131.48616-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Avoid a lookup to find the node that we have just inserted. Signed-off-by: Richard Henderson Reviewed-by: Pierrick Bouvier --- accel/tcg/cputlb.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 20af48c6c5..6d316e8767 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1037,8 +1037,8 @@ static inline void tlb_set_compare(CPUTLBEntryFull *full, CPUTLBEntry *ent, * Called from TCG-generated code, which is under an RCU read-side * critical section. */ -static void tlb_set_page_full(CPUState *cpu, int mmu_idx, - vaddr addr, CPUTLBEntryFull *full) +static CPUTLBEntryTree *tlb_set_page_full(CPUState *cpu, int mmu_idx, + vaddr addr, CPUTLBEntryFull *full) { CPUTLB *tlb = &cpu->neg.tlb; CPUTLBDesc *desc = &tlb->d[mmu_idx]; @@ -1187,6 +1187,8 @@ static void tlb_set_page_full(CPUState *cpu, int mmu_idx, copy_tlb_helper_locked(te, &node->copy); desc->n_used_entries++; qemu_spin_unlock(&tlb->c.lock); + + return node; } static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr, @@ -1266,18 +1268,14 @@ static bool tlb_lookup(CPUState *cpu, TLBLookupOutput *o, tcg_debug_assert(probe); return false; } - tlb_set_page_full(cpu, i->mmu_idx, addr, &o->full); + node = tlb_set_page_full(cpu, i->mmu_idx, addr, &o->full); o->did_tlb_fill = true; if (access_type == MMU_INST_FETCH) { - node = tlbtree_lookup_addr(desc, addr); - tcg_debug_assert(node); goto found_code; } - entry = tlbfast_entry(fast, addr); - cmp = tlb_read_idx(entry, access_type); - node = entry->tree; + cmp = tlb_read_idx(&node->copy, access_type); /* * With PAGE_WRITE_INV, we set TLB_INVALID_MASK immediately, * to force the next access through tlb_fill_align. We've just