From patchwork Fri Mar 9 09:07:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 10270387 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 220B96016D for ; Fri, 9 Mar 2018 10:23:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0731829D71 for ; Fri, 9 Mar 2018 10:23:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F025229D75; Fri, 9 Mar 2018 10:23:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4838529D71 for ; Fri, 9 Mar 2018 10:23:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hlPt/EG5P/R4EaN9W+VsOSeLG66zBcGzuHGBI6uQa+E=; b=BjK/NSkAGkNwB3 S1EdgOptYxRIeDvi7mUPv8rJdSg3YwEBIkCUMaFQMeXS1Aa4NVFbKNHq/FSyi4EGUPc09pFziHw9R gR5aX2esgQN1y86ei3Lj2jHN8PTpJOY48aykPosQ9/Z1j11vKNfrirGVDum26pDxZLTFMyZYtOZIu HGVF0IPuAMIvrrsMcOx9c73yX74gxoYNBJ7RwqRTBsAf3eBNpmhQOsAjE1499SDbrBu9sWdzcGWXP b5oQHkqgpPqT+OaY8MlWLpSTUQs+GL8ZZqNg82pFUQMIJPsYeeJ4BO9uYKZt0iyoz2/FzdW2rqC2D gd4HvGSpCCJ8HIqFzhcw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1euFBW-0006NE-5M; Fri, 09 Mar 2018 10:23:34 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1euF7g-0002qN-CZ for linux-arm-kernel@bombadil.infradead.org; Fri, 09 Mar 2018 10:19:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:To:From:Sender:Reply-To:Cc:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+PcdKf8ujd1cp52R8I4Cxe9V9TfApAiDGhcs/vamm1A=; b=oDiq/hRJr8FkRAYbisYUWWMtd XdVH2jRXJJwx4VP7Rw6rYMGM9pC4nrzH+ZfQbIoaWnLTg8+SaiJlu/9QZJTlF7FT/geoOrOY8CGBb M/NCGiLvdWP1TH353v4L89MPJd6hr7lmZh0wCyun4hx2W/QmeviPYAPa1pFXi3KKxP1EWevZFK3/v VWqZiDR7vRRJBcWJor41cHsmPofy1e0x3oCDOxnaoDF7QV2PTgeKrBel5QOaxVfg942GXPOKbs3N+ v6bNRNoCGAWXk26S7QKcbXCmtSyilMvU3TRRrFiiogUVfUh1IqTiWykE4NmtT/VZUdyXMQ0pO2bip kl4ivo1/w==; Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]) by merlin.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1euE2l-0002fG-9E for linux-arm-kernel@lists.infradead.org; Fri, 09 Mar 2018 09:10:28 +0000 Received: by mail-pg0-x242.google.com with SMTP id i14so3337863pgv.3 for ; Fri, 09 Mar 2018 01:10:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=+PcdKf8ujd1cp52R8I4Cxe9V9TfApAiDGhcs/vamm1A=; b=VhToIwZRev+jM6flw/KtfEhNK8lDJdq6lFk+7p/luxrvfkC6pKN94mu24r6TCifoVg qraYQnAfLQj3UwcGP+l+CYXsjTPcCCT+iF27uYR4ieu1/VxHrKXfzmc9giF4lmBdLTfx e1c8IyBFm4f9M2rDjGcwfUVfmBVMAiUBT2VpI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+PcdKf8ujd1cp52R8I4Cxe9V9TfApAiDGhcs/vamm1A=; b=VzympQqrSjaObrOA/IEl5sWEbtghZwYzy1WzRCItM91hUCW/5QTCYuDupSrq7ZLpgy VeJ4YeNdYZkIxikZJEocWZVw/dBE7IyeXTSdQozeo2uoOeaZQ77IofJ0ltM3p/wAhnQz I7+fw2qlOOxHz+kntqAzz1EbK8IgjRC0HvoQvzBOdNCaJgshPuuQb/PuMoCDvJImmi7z Pq7B3RpYGcLQI8LJhAhKdJKKTrcQTuUBVnfNA6LfxMvMT4jnPeiNAFfEV7yj4JbjACOB n3wKtuowBOmzer+Mg4DloUI4gS+ont2Kg7p68pEn+u+CROo44v27i16fFBLhYDaLOalc XVoA== X-Gm-Message-State: APf1xPBlP/FOGXicUmnWrZdj/OcAs53MVauG2yVGa0zlHfgKv462W1YJ 4BcYqwsigGVyJk+R1WRYW6tu1g== X-Google-Smtp-Source: AG47ELtJ9+KFbxfi2mNNr2CKXyFUcH2mdabqEtIaOWKisf/pRsEstMKjSbUZF+18sq8imd/DKLXQPA== X-Received: by 10.101.93.82 with SMTP id e18mr23441870pgt.371.1520586613130; Fri, 09 Mar 2018 01:10:13 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id c7sm1961752pfg.36.2018.03.09.01.10.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 09 Mar 2018 01:10:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , broonie@linaro.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 19/39] arm64: prefetch: add alternative pattern for CPUs without a prefetcher Date: Fri, 9 Mar 2018 17:07:02 +0800 Message-Id: <20180309090722.26279-20-alex.shi@linaro.org> X-Mailer: git-send-email 2.16.2.440.gc6284da In-Reply-To: <20180309090722.26279-1-alex.shi@linaro.org> References: <20180309090722.26279-1-alex.shi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180309_041027_438764_4853A6C3 X-CRM114-Status: GOOD ( 16.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Will Deacon Most CPUs have a hardware prefetcher which generally performs better without explicit prefetch instructions issued by software, however some CPUs (e.g. Cavium ThunderX) rely solely on explicit prefetch instructions. This patch adds an alternative pattern (ARM64_HAS_NO_HW_PREFETCH) to allow our library code to make use of explicit prefetch instructions during things like copy routines only when the CPU does not have the capability to perform the prefetching itself. Signed-off-by: Will Deacon Tested-by: Andrew Pinski Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/cpufeature.h | 3 ++- arch/arm64/include/asm/cputype.h | 17 ++++++++++++++++- arch/arm64/kernel/cpu_errata.c | 18 +++--------------- arch/arm64/kernel/cpufeature.c | 17 +++++++++++++++++ 4 files changed, 38 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index ede73d2c09a4..77dd88f0d1fc 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -32,8 +32,9 @@ #define ARM64_WORKAROUND_834220 7 #define ARM64_WORKAROUND_CAVIUM_27456 8 #define ARM64_HARDEN_BRANCH_PREDICTOR 9 +#define ARM64_HAS_NO_HW_PREFETCH 10 -#define ARM64_NCAPS 10 +#define ARM64_NCAPS 11 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 1a5949364ed0..7540284a17fe 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -57,11 +57,22 @@ #define MIDR_IMPLEMENTOR(midr) \ (((midr) & MIDR_IMPLEMENTOR_MASK) >> MIDR_IMPLEMENTOR_SHIFT) -#define MIDR_CPU_PART(imp, partnum) \ +#define MIDR_CPU_MODEL(imp, partnum) \ (((imp) << MIDR_IMPLEMENTOR_SHIFT) | \ (0xf << MIDR_ARCHITECTURE_SHIFT) | \ ((partnum) << MIDR_PARTNUM_SHIFT)) +#define MIDR_CPU_MODEL_MASK (MIDR_IMPLEMENTOR_MASK | MIDR_PARTNUM_MASK | \ + MIDR_ARCHITECTURE_MASK) + +#define MIDR_IS_CPU_MODEL_RANGE(midr, model, rv_min, rv_max) \ +({ \ + u32 _model = (midr) & MIDR_CPU_MODEL_MASK; \ + u32 rv = (midr) & (MIDR_REVISION_MASK | MIDR_VARIANT_MASK); \ + \ + _model == (model) && rv >= (rv_min) && rv <= (rv_max); \ + }) + #define ARM_CPU_IMP_ARM 0x41 #define ARM_CPU_IMP_APM 0x50 #define ARM_CPU_IMP_CAVIUM 0x43 @@ -75,6 +86,10 @@ #define CAVIUM_CPU_PART_THUNDERX 0x0A1 +#define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) +#define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) +#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 0d3ea874133d..b84a93e8c8bd 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -21,24 +21,12 @@ #include #include -#define MIDR_CORTEX_A53 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) -#define MIDR_CORTEX_A57 MIDR_CPU_PART(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) -#define MIDR_THUNDERX MIDR_CPU_PART(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) - -#define CPU_MODEL_MASK (MIDR_IMPLEMENTOR_MASK | MIDR_PARTNUM_MASK | \ - MIDR_ARCHITECTURE_MASK) - static bool __maybe_unused is_affected_midr_range(const struct arm64_cpu_capabilities *entry) { - u32 midr = read_cpuid_id(); - - if ((midr & CPU_MODEL_MASK) != entry->midr_model) - return false; - - midr &= MIDR_REVISION_MASK | MIDR_VARIANT_MASK; - - return (midr >= entry->midr_range_min && midr <= entry->midr_range_max); + return MIDR_IS_CPU_MODEL_RANGE(read_cpuid_id(), entry->midr_model, + entry->midr_range_min, + entry->midr_range_max); } #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 8488a829e24e..2cf2f799455a 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -624,6 +624,18 @@ static bool has_useable_gicv3_cpuif(const struct arm64_cpu_capabilities *entry) return has_sre; } +static bool has_no_hw_prefetch(const struct arm64_cpu_capabilities *entry) +{ + u32 midr = read_cpuid_id(); + u32 rv_min, rv_max; + + /* Cavium ThunderX pass 1.x and 2.x */ + rv_min = 0; + rv_max = (1 << MIDR_VARIANT_SHIFT) | MIDR_REVISION_MASK; + + return MIDR_IS_CPU_MODEL_RANGE(midr, MIDR_THUNDERX, rv_min, rv_max); +} + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -654,6 +666,11 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .min_field_value = 2, }, #endif /* CONFIG_AS_LSE && CONFIG_ARM64_LSE_ATOMICS */ + { + .desc = "Software prefetching using PRFM", + .capability = ARM64_HAS_NO_HW_PREFETCH, + .matches = has_no_hw_prefetch, + }, {}, };