From patchwork Fri Jun 3 20:40:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 9154055 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CC74960751 for ; Fri, 3 Jun 2016 21:03:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B134D282E8 for ; Fri, 3 Jun 2016 21:03:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A24BE28337; Fri, 3 Jun 2016 21:03:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 14A66282E8 for ; Fri, 3 Jun 2016 21:03:08 +0000 (UTC) Received: from localhost ([::1]:57750 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b8wFH-0004Sv-5D for patchwork-qemu-devel@patchwork.kernel.org; Fri, 03 Jun 2016 17:03:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39937) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b8vzf-0005zK-T1 for qemu-devel@nongnu.org; Fri, 03 Jun 2016 16:47:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b8vzb-0004gA-MX for qemu-devel@nongnu.org; Fri, 03 Jun 2016 16:46:58 -0400 Received: from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:38588) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b8vzb-0004fx-C8 for qemu-devel@nongnu.org; Fri, 03 Jun 2016 16:46:55 -0400 Received: by mail-wm0-x235.google.com with SMTP id m124so4258842wme.1 for ; Fri, 03 Jun 2016 13:46:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7qAahBz0WKGXhj587/U1i1c4f1tS81001sMmOmgHEkM=; b=RcKCQDWFkECGPARVnanPlX22hPudoWrE1F1FFkytXQEF0ZTOMyPMByn4/EbddAAYqV +B7Qy7n0FWTllSud7oi8cNllLZTCdGkAYCjT3t9N7LBjevvCuN0l8iiXeWupy7VgEq94 7kveZ5u3f1jVOqsHj9GUWAPMeRwyS2Phrjix4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7qAahBz0WKGXhj587/U1i1c4f1tS81001sMmOmgHEkM=; b=PvlGtAFOMFuCYqhfvWDhaNK5+UxOlXB2sv3tquKgpiuDIDEQH86nYyuzYAMBtfCAJv Re5ETebLNQ/2CCCxi2BNQkiw+U4uoIi1EfxFrKNG1dJQ+SAyVmk610xg+rmSiMD+hCnf 50OqkYEHikfh6/BLIRKCWW8HE6wRcRazc3zMSLUPrJ0kK6ZBIrOmQxdFWerDKY5IM8dq vY2VRAlV/wMAxHeitRkdeyANXVxlhXzj+iXuclHdAa5aqrNRJli9X7YG/CBtIYZDidyI 8AYnGQPE9ebBcP75RGBiMvINlIORmFum2V1oc/MAYCTFah2cVq5mxSK8nRXV4b7oQ2ZF 6z0w== X-Gm-Message-State: ALyK8tIxKwffbhD8cCI02Unk5E0V383oCDvx/LT8N1hjIi+TcMz57+hr1cXy1HIH69f6GUZ2 X-Received: by 10.28.7.208 with SMTP id 199mr1130293wmh.74.1464986814531; Fri, 03 Jun 2016 13:46:54 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id k3sm6200911wjh.7.2016.06.03.13.46.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 03 Jun 2016 13:46:53 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id B19FE3E3244; Fri, 3 Jun 2016 21:40:40 +0100 (BST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: mttcg@listserver.greensocs.com, qemu-devel@nongnu.org, fred.konrad@greensocs.com, a.rigo@virtualopensystems.com, serge.fdrv@gmail.com, cota@braap.org, bobby.prani@gmail.com Date: Fri, 3 Jun 2016 21:40:28 +0100 Message-Id: <1464986428-6739-20-git-send-email-alex.bennee@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1464986428-6739-1-git-send-email-alex.bennee@linaro.org> References: <1464986428-6739-1-git-send-email-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::235 Subject: [Qemu-devel] [RFC v3 19/19] cpu-exec: remove tb_lock from the hot-path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, Peter Crosthwaite , claudio.fontana@huawei.com, mark.burton@greensocs.com, jan.kiszka@siemens.com, pbonzini@redhat.com, =?UTF-8?q?Alex=20Benn=C3=A9e?= , rth@twiddle.net Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Lock contention in the hot path of moving between existing patched TranslationBlocks is the main drag on MTTCG performance. This patch pushes the tb_lock() usage down to the two places that really need it: - code generation (tb_gen_code) - jump patching (tb_add_jump) The rest of the code doesn't really need to hold a lock as it is either using per-CPU structures or designed to be used in concurrent read situations (qht_lookup). Signed-off-by: Alex Bennée --- v3 - fix merge conflicts with Sergey's patch --- cpu-exec.c | 59 ++++++++++++++++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 29 deletions(-) diff --git a/cpu-exec.c b/cpu-exec.c index b017643..4af0b52 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -298,41 +298,38 @@ static TranslationBlock *tb_find_slow(CPUState *cpu, * Pairs with smp_wmb() in tb_phys_invalidate(). */ smp_rmb(); tb = tb_find_physical(cpu, pc, cs_base, flags); - if (tb) { - goto found; - } + if (!tb) { - /* mmap_lock is needed by tb_gen_code, and mmap_lock must be - * taken outside tb_lock. Since we're momentarily dropping - * tb_lock, there's a chance that our desired tb has been - * translated. - */ - tb_unlock(); - mmap_lock(); - tb_lock(); - tb = tb_find_physical(cpu, pc, cs_base, flags); - if (tb) { - mmap_unlock(); - goto found; - } + /* mmap_lock is needed by tb_gen_code, and mmap_lock must be + * taken outside tb_lock. + */ + mmap_lock(); + tb_lock(); - /* if no translated code available, then translate it now */ - tb = tb_gen_code(cpu, pc, cs_base, flags, 0); + /* There's a chance that our desired tb has been translated while + * taking the locks so we check again inside the lock. + */ + tb = tb_find_physical(cpu, pc, cs_base, flags); + if (!tb) { + /* if no translated code available, then translate it now */ + tb = tb_gen_code(cpu, pc, cs_base, flags, 0); + } - mmap_unlock(); + tb_unlock(); + mmap_unlock(); + } -found: - /* we add the TB in the virtual pc hash table */ + /* We add the TB in the virtual pc hash table for the fast lookup */ cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)] = tb; return tb; } static inline TranslationBlock *tb_find_fast(CPUState *cpu, - TranslationBlock **last_tb, + TranslationBlock **ltbp, int tb_exit) { CPUArchState *env = (CPUArchState *)cpu->env_ptr; - TranslationBlock *tb; + TranslationBlock *tb, *last_tb; target_ulong cs_base, pc; uint32_t flags; @@ -340,7 +337,6 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu, always be the same before a given translated block is executed. */ cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags); - tb_lock(); tb = atomic_read(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)]); if (unlikely(!tb || tb->pc != pc || tb->cs_base != cs_base || tb->flags != flags)) { @@ -350,7 +346,7 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu, /* Ensure that no TB jump will be modified as the * translation buffer has been flushed. */ - *last_tb = NULL; + *ltbp = NULL; cpu->tb_flushed = false; } #ifndef CONFIG_USER_ONLY @@ -359,14 +355,19 @@ static inline TranslationBlock *tb_find_fast(CPUState *cpu, * spanning two pages because the mapping for the second page can change. */ if (tb->page_addr[1] != -1) { - *last_tb = NULL; + *ltbp = NULL; } #endif + /* See if we can patch the calling TB. */ - if (*last_tb && !qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) { - tb_add_jump(*last_tb, tb_exit, tb); + last_tb = *ltbp; + if (!qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN) && + last_tb && + !last_tb->jmp_list_next[tb_exit]) { + tb_lock(); + tb_add_jump(last_tb, tb_exit, tb); + tb_unlock(); } - tb_unlock(); return tb; }