From patchwork Thu Jun 25 18:45:11 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 6677201 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 510239F3A0 for ; Thu, 25 Jun 2015 18:45:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6FDBE20734 for ; Thu, 25 Jun 2015 18:45:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 69A162073C for ; Thu, 25 Jun 2015 18:45:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752312AbbFYSpZ (ORCPT ); Thu, 25 Jun 2015 14:45:25 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43127 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752029AbbFYSpU (ORCPT ); Thu, 25 Jun 2015 14:45:20 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 382431BE365; Thu, 25 Jun 2015 18:45:20 +0000 (UTC) Received: from hawk.localdomain.com (dhcp-1-158.brq.redhat.com [10.34.1.158]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t5PIjDXb002467; Thu, 25 Jun 2015 14:45:19 -0400 From: Andrew Jones To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: christoffer.dall@linaro.org, pbonzini@redhat.com Subject: [PATCH 3/3] arm/arm64: speed up spinlocks and atomic ops Date: Thu, 25 Jun 2015 20:45:11 +0200 Message-Id: <1435257911-9715-4-git-send-email-drjones@redhat.com> In-Reply-To: <1435257911-9715-1-git-send-email-drjones@redhat.com> References: <1435257911-9715-1-git-send-email-drjones@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-8.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP spinlock torture tests made it clear that checking mmu_enabled() every time we call spin_lock is a bad idea. As most tests will want the MMU enabled the entire time, then we can inline a light weight "nobody disabled the mmu" check, and bail out early. Adding a light weight inlined check vs. a compile-time flag suggested by Paolo. Signed-off-by: Andrew Jones --- lib/arm/asm/mmu-api.h | 7 ++++++- lib/arm/mmu.c | 9 +++++++-- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/lib/arm/asm/mmu-api.h b/lib/arm/asm/mmu-api.h index 12fdc57918c27..b2fc900a30add 100644 --- a/lib/arm/asm/mmu-api.h +++ b/lib/arm/asm/mmu-api.h @@ -1,7 +1,12 @@ #ifndef __ASMARM_MMU_API_H_ #define __ASMARM_MMU_API_H_ extern pgd_t *mmu_idmap; -extern bool mmu_enabled(void); +extern unsigned int mmu_disabled_cpu_count; +extern bool __mmu_enabled(void); +static inline bool mmu_enabled(void) +{ + return mmu_disabled_cpu_count == 0 || __mmu_enabled(); +} extern void mmu_enable(pgd_t *pgtable); extern void mmu_disable(void); extern void mmu_enable_idmap(void); diff --git a/lib/arm/mmu.c b/lib/arm/mmu.c index ad558e177dd1c..e661d4f26e4b7 100644 --- a/lib/arm/mmu.c +++ b/lib/arm/mmu.c @@ -15,8 +15,9 @@ extern unsigned long etext; pgd_t *mmu_idmap; static cpumask_t mmu_disabled_cpumask; +unsigned int mmu_disabled_cpu_count; -bool mmu_enabled(void) +bool __mmu_enabled(void) { int cpu = current_thread_info()->cpu; @@ -30,7 +31,9 @@ void mmu_enable(pgd_t *pgtable) asm_mmu_enable(__pa(pgtable)); flush_tlb_all(); - cpumask_clear_cpu(cpu, &mmu_disabled_cpumask); + + if (cpumask_test_and_clear_cpu(cpu, &mmu_disabled_cpumask)) + --mmu_disabled_cpu_count; } extern void asm_mmu_disable(void); @@ -39,6 +42,8 @@ void mmu_disable(void) int cpu = current_thread_info()->cpu; cpumask_set_cpu(cpu, &mmu_disabled_cpumask); + ++mmu_disabled_cpu_count; + asm_mmu_disable(); }