From patchwork Tue Aug 28 15:51:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8B4014BD for ; Tue, 28 Aug 2018 15:53:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D57AF2A3CB for ; Tue, 28 Aug 2018 15:53:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C974F2A405; Tue, 28 Aug 2018 15:53:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 44AFA2A3CB for ; Tue, 28 Aug 2018 15:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PjXBD0wuPW2fyEUxgTwhDw0ny/oOJyDS23GYnFmMQoQ=; b=X1Ua4aX9Qi44ZMQ7E36dKL+rrw rPpjpECGvhTFE3nxlv9gV3kaGqhoqP/tr9LFk6YQKsJwCWSq0iM/haykcDoTNlc4LuwJYt9BTdkqe fHt5G4ER8K95ar6tkd2Q2pkNjIgBaUm8e9+Bsxhbt6Tyf6cTqD7Rt3zrYxC88ShdNd9mzX6ujgaaq KEttN1FPitSUkA1gt95AnZ3+1vfuirupRc8J0icHgYSQmJT6m8mZ9qX3pp2Xhy4W3f+smCKeLqdad najOTjWDW11SQrDKUt/iIhnwoqX49vyMSWwzPMwQ0Gji30aH5QjDdULY0A1uvIIW8tfXCvs0L3CCP c7XwUy7w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIS-0005RN-Bt; Tue, 28 Aug 2018 15:52:48 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHf-00056d-HR for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 03309ED1; Tue, 28 Aug 2018 08:51:51 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DC8F43F557; Tue, 28 Aug 2018 08:51:48 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 01/27] arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature Date: Tue, 28 Aug 2018 16:51:11 +0100 Message-Id: <1535471497-38854-2-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085159_621053_372F90FD X-CRM114-Status: GOOD ( 12.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Suzuki K Poulose , will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Catalin Marinas Cc: Will Deacon Cc: Suzuki K Poulose Cc: Marc Zyngier --- arch/arm64/kernel/cpufeature.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index e238b79..1e433ac 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1039,7 +1039,7 @@ static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused) { .desc = "GIC system register CPU interface", .capability = ARM64_HAS_SYSREG_GIC_CPUIF, - .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, .matches = has_useable_gicv3_cpuif, .sys_reg = SYS_ID_AA64PFR0_EL1, .field_pos = ID_AA64PFR0_GIC_SHIFT, From patchwork Tue Aug 28 15:51:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BAF314BD for ; Tue, 28 Aug 2018 15:55:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29CE02A5FD for ; Tue, 28 Aug 2018 15:55:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D7AD2A610; Tue, 28 Aug 2018 15:55:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9E3F12A5FD for ; Tue, 28 Aug 2018 15:55:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=uVh60k+lzMJutMdwfY2SLjBP4jMywIW4pbxb6iFkM1k=; b=D4TWEs1m9fr7O6R/l38X4dtbUy W30RJQmgxtCtd85CehFhuKUtvf4KvdbWLGbASWW5B/rhtYVOAwSf//t+Lps6hhg4EGo8Z8ODylPS9 mipMTHgL3NRgnRTSzf3TIYcSZIOCgTFWSzWMn8Qt7Hzj3o1O722eGOMzit5Y2h2Y/uD8Ha+khDZOt a8BZtywRcnouj3bT685pwTKZKe7puvrnzWiG9MVpmWa4vJuMm91CTVbWLjCcj2zE9bEtA/3FM0M+a zC+Si1gI951j/IBNf1iIK1P/9cyS4J8SQ5g/xSflWq3IN6Q0U749CXCFf6qXAbR1lzJqFhqjNQMd6 XY8+0Wpg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugKz-00075K-5k; Tue, 28 Aug 2018 15:55:25 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHf-00056m-HT for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A444A15BF; Tue, 28 Aug 2018 08:51:53 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 430903F557; Tue, 28 Aug 2018 08:51:51 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 02/27] arm64: cpufeature: Use alternatives for VHE cpu_enable Date: Tue, 28 Aug 2018 16:51:12 +0100 Message-Id: <1535471497-38854-3-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085159_640310_A85B2ED9 X-CRM114-Status: GOOD ( 12.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , Marc Zyngier , catalin.marinas@arm.com, Suzuki K Poulose , will.deacon@arm.com, linux-kernel@vger.kernel.org, Christoffer Dall , james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The cpu_enable callback for VHE feature requires all alternatives to have been applied. This prevents applying VHE alternative separately from the rest. Use an alternative depending on VHE feature to know whether VHE alternatives have already been applied. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Suzuki K Poulose Cc: Marc Zyngier Cc: Christoffer Dall --- arch/arm64/kernel/cpufeature.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 1e433ac..3bc1c8b 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1014,6 +1014,8 @@ static bool runs_at_el2(const struct arm64_cpu_capabilities *entry, int __unused static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused) { + u64 tmp = 0; + /* * Copy register values that aren't redirected by hardware. * @@ -1022,8 +1024,15 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused) * that, freshly-onlined CPUs will set tpidr_el2, so we don't need to * do anything here. */ - if (!alternatives_applied) - write_sysreg(read_sysreg(tpidr_el1), tpidr_el2); + asm volatile(ALTERNATIVE( + "mrs %0, tpidr_el1\n" + "msr tpidr_el2, %0", + "nop\n" + "nop", + ARM64_HAS_VIRT_HOST_EXTN) + : "+r" (tmp) + : + : "memory"); } #endif From patchwork Tue Aug 28 15:51:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C97AD15A7 for ; Tue, 28 Aug 2018 15:54:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B49B52A495 for ; Tue, 28 Aug 2018 15:54:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A682F2A613; Tue, 28 Aug 2018 15:54:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 019162A495 for ; Tue, 28 Aug 2018 15:54:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ddpOprpGZvohi7qNIB554CHl6gmuZDeO8/xt+zkASxg=; b=ZFKzJ45DsYyhCcI955uYqy2INs /59Ud7bEbjzUf0KZzqlO9dLWJDas876izSFDW+gk8khk4wg+cEpXuddPpv1/IrKj9BLGBw8xvEZan TuGJ5udL/ZO/rcSwbL8aqKSGKjLVhFWyNaBkhzSeV8sOzg58i+BmFpnx1qh69VEAor9LTOadQrtCP QSeibNY0PLmzGDreyP5Hq/se6AFWlIQfdVXt6GQMd8b78nTy1I7HqHCK+qsstWzl5gsFK5mL+gqn1 hMBur+g9DhVHSYAoli9mOT1MzfONH0qIcYUn/uNirO98OLoRl5vm3UextwXRfaPREm/UtyZROqv3z UcskFldg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugKA-00066S-Hk; Tue, 28 Aug 2018 15:54:34 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHf-000575-HS for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 015C11682; Tue, 28 Aug 2018 08:51:56 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E42403F557; Tue, 28 Aug 2018 08:51:53 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 03/27] arm64: alternative: Apply alternatives early in boot process Date: Tue, 28 Aug 2018 16:51:13 +0100 Message-Id: <1535471497-38854-4-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085159_629572_68D51EE6 X-CRM114-Status: GOOD ( 18.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Suzuki K Poulose , will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Daniel Thompson Currently alternatives are applied very late in the boot process (and a long time after we enable scheduling). Some alternative sequences, such as those that alter the way CPU context is stored, must be applied much earlier in the boot sequence. Introduce apply_boot_alternatives() to allow some alternatives to be applied immediately after we detect the CPU features of the boot CPU. Signed-off-by: Daniel Thompson [julien.thierry@arm.com: rename to fit new cpufeature framework better, apply BOOT_SCOPE feature early in boot] Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Christoffer Dall Cc: Suzuki K Poulose --- arch/arm64/include/asm/alternative.h | 3 +-- arch/arm64/include/asm/cpufeature.h | 2 ++ arch/arm64/kernel/alternative.c | 28 +++++++++++++++++++++++++--- arch/arm64/kernel/cpufeature.c | 5 +++++ arch/arm64/kernel/smp.c | 7 +++++++ 5 files changed, 40 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/alternative.h b/arch/arm64/include/asm/alternative.h index 4b650ec..17f4554 100644 --- a/arch/arm64/include/asm/alternative.h +++ b/arch/arm64/include/asm/alternative.h @@ -14,8 +14,6 @@ #include #include -extern int alternatives_applied; - struct alt_instr { s32 orig_offset; /* offset to original instruction */ s32 alt_offset; /* offset to replacement instruction */ @@ -27,6 +25,7 @@ struct alt_instr { typedef void (*alternative_cb_t)(struct alt_instr *alt, __le32 *origptr, __le32 *updptr, int nr_inst); +void __init apply_boot_alternatives(void); void __init apply_alternatives_all(void); #ifdef CONFIG_MODULES diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 1717ba1..e6c030a 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -357,6 +357,8 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap) extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS]; extern struct static_key_false arm64_const_caps_ready; +extern unsigned long boot_capabilities; + bool this_cpu_has_cap(unsigned int cap); static inline bool cpu_have_feature(unsigned int num) diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index b5d6039..70c2604 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -145,7 +145,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end) } while (cur += d_size, cur < end); } -static void __apply_alternatives(void *alt_region, bool is_module) +static void __apply_alternatives(void *alt_region, bool is_module, + unsigned long feature_mask) { struct alt_instr *alt; struct alt_region *region = alt_region; @@ -155,6 +156,9 @@ static void __apply_alternatives(void *alt_region, bool is_module) for (alt = region->begin; alt < region->end; alt++) { int nr_inst; + if ((BIT(alt->cpufeature) & feature_mask) == 0) + continue; + /* Use ARM64_CB_PATCH as an unconditional patch */ if (alt->cpufeature < ARM64_CB_PATCH && !cpus_have_cap(alt->cpufeature)) @@ -213,7 +217,7 @@ static int __apply_alternatives_multi_stop(void *unused) isb(); } else { BUG_ON(alternatives_applied); - __apply_alternatives(®ion, false); + __apply_alternatives(®ion, false, ~boot_capabilities); /* Barriers provided by the cache flushing */ WRITE_ONCE(alternatives_applied, 1); } @@ -227,6 +231,24 @@ void __init apply_alternatives_all(void) stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask); } +/* + * This is called very early in the boot process (directly after we run + * a feature detect on the boot CPU). No need to worry about other CPUs + * here. + */ +void __init apply_boot_alternatives(void) +{ + struct alt_region region = { + .begin = (struct alt_instr *)__alt_instructions, + .end = (struct alt_instr *)__alt_instructions_end, + }; + + /* If called on non-boot cpu things could go wrong */ + WARN_ON(smp_processor_id() != 0); + + __apply_alternatives(®ion, false, boot_capabilities); +} + #ifdef CONFIG_MODULES void apply_alternatives_module(void *start, size_t length) { @@ -235,6 +257,6 @@ void apply_alternatives_module(void *start, size_t length) .end = start + length, }; - __apply_alternatives(®ion, true); + __apply_alternatives(®ion, true, -1); } #endif diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3bc1c8b..0d1e41e 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -52,6 +52,8 @@ DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS); EXPORT_SYMBOL(cpu_hwcaps); +unsigned long boot_capabilities; + /* * Flag to indicate if we have computed the system wide * capabilities based on the boot time active CPUs. This @@ -1375,6 +1377,9 @@ static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps, if (!cpus_have_cap(caps->capability) && caps->desc) pr_info("%s %s\n", info, caps->desc); cpus_set_cap(caps->capability); + + if (caps->type & SCOPE_BOOT_CPU) + __set_bit(caps->capability, &boot_capabilities); } } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 25fcd22..22c9a0a 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -414,6 +414,13 @@ void __init smp_prepare_boot_cpu(void) */ jump_label_init(); cpuinfo_store_boot_cpu(); + + /* + * We now know enough about the boot CPU to apply the + * alternatives that cannot wait until interrupt handling + * and/or scheduling is enabled. + */ + apply_boot_alternatives(); } static u64 __init of_get_cpu_mpidr(struct device_node *dn) From patchwork Tue Aug 28 15:51:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578747 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2AB514BD for ; Tue, 28 Aug 2018 15:52:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99B4A2A5A2 for ; Tue, 28 Aug 2018 15:52:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 89EAC2A607; Tue, 28 Aug 2018 15:52:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 30A382A5A2 for ; Tue, 28 Aug 2018 15:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=phbg3k6BDCb/Vo8KL1EWEBobx05flucJ0XFTwZBo+Uo=; b=ajeAMoVhLcGj64zx/HLbJy7q69 9tWDQh+TUHzu1rXF4m8AHuIhI7imHO0YquTVXiUR61879YY3JRY3/Q9JLhlunbzUBRXP65SbYrLiN R+s1hRDhnT/71wN8Em7m/uyYGtEvWDe5sBRn3ImVz6sEMBolblzDvgSOnmV1ewAT0nhRRAAUn1ur4 fj4jie9D+Ey2U3d9nQx9gDp9eL7x31dgSujml6n6SA67RFm2sSzs6ylqdMaYG7uaNWskbzWr4MffI BjeZYXeci9lyU6748C/+Xxo87kUMdRvgSdQ3/cNMqUWAwfzBUoF1jQCo73RQvYwzXMUOVgTkB3fiV xR2QFEMA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHj-00058c-I0; Tue, 28 Aug 2018 15:52:03 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHf-00057K-HU for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38AE41684; Tue, 28 Aug 2018 08:51:58 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 40CB03F557; Tue, 28 Aug 2018 08:51:56 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 04/27] arm64: daifflags: Use irqflags functions for daifflags Date: Tue, 28 Aug 2018 16:51:14 +0100 Message-Id: <1535471497-38854-5-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085159_628560_1BEDCD80 X-CRM114-Status: GOOD ( 12.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Some of the work done in daifflags save/restore is already provided by irqflags functions. Daifflags should always be a superset of irqflags (it handles irq status + status of other flags). Modifying behaviour of irqflags should alter the behaviour of daifflags. Use irqflags_save/restore functions for the corresponding daifflags operation. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: James Morse Reviewed-by: James Morse --- arch/arm64/include/asm/daifflags.h | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 22e4c83..8d91f22 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -36,11 +36,8 @@ static inline unsigned long local_daif_save(void) { unsigned long flags; - asm volatile( - "mrs %0, daif // local_daif_save\n" - : "=r" (flags) - : - : "memory"); + flags = arch_local_save_flags(); + local_daif_mask(); return flags; @@ -60,11 +57,9 @@ static inline void local_daif_restore(unsigned long flags) { if (!arch_irqs_disabled_flags(flags)) trace_hardirqs_on(); - asm volatile( - "msr daif, %0 // local_daif_restore" - : - : "r" (flags) - : "memory"); + + arch_local_irq_restore(flags); + if (arch_irqs_disabled_flags(flags)) trace_hardirqs_off(); } From patchwork Tue Aug 28 15:51:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578759 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12AA114BD for ; Tue, 28 Aug 2018 15:57:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F3E892A60C for ; Tue, 28 Aug 2018 15:57:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E87232A613; Tue, 28 Aug 2018 15:57:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 516092A60C for ; Tue, 28 Aug 2018 15:57:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=rK9I6ERaxlNisT4mrrPYaCLa19GEHXDNW6cLznXZcGg=; b=iVQSKJ4skq3HPDHpbG4G6a27jC AWqZ1FHu9VVfIwgKpsYyaRxMobltx0/T2iXMEk02aK3K/j22mOJGVNfeZnYQMaZ22ZQtKT78ArmkH kxJeaft1if+vSQ1bz7iFaS3r631j5G6HULvRb4vOEDiNheS4O4Y0wR2cl5+PFOCqoqYy6+5j0wsdI HmjUrhdc7oKV8EU9ykRKlLTjNcirXrGqYtBZTgdgNFboPZqkeGMA+S60ndI+VHoRCdURQlQzKyjun GuokQP3+GGOP1qxNp1yNOhNK3GXGycBcIDAEbpomjrY7gtMspwowBlP+Mh2jPAxUHXTjYU5fDnPvS ZTGZceWA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugMV-0008Qu-5K; Tue, 28 Aug 2018 15:56:59 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHr-00058P-5o for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 70542168F; Tue, 28 Aug 2018 08:52:00 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 78C983F557; Tue, 28 Aug 2018 08:51:58 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 05/27] arm64: Use daifflag_restore after bp_hardening Date: Tue, 28 Aug 2018 16:51:15 +0100 Message-Id: <1535471497-38854-6-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085211_273023_DA73A69B X-CRM114-Status: GOOD ( 12.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP For EL0 entries requiring bp_hardening, daif status is kept at DAIF_PROCCTX_NOIRQ until after hardening has been done. Then interrupts are enabled through local_irq_enable(). Before using local_irq_* functions, daifflags should be properly restored to a state where IRQs are enabled. Enable IRQs by restoring DAIF_PROCCTX state after bp hardening. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: James Morse Acked-by: James Morse --- arch/arm64/mm/fault.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 50b30ff..dfa2c43 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -37,6 +37,7 @@ #include #include #include +#include #include #include #include @@ -771,7 +772,7 @@ asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, if (addr > TASK_SIZE) arm64_apply_bp_hardening(); - local_irq_enable(); + local_daif_restore(DAIF_PROCCTX); do_mem_abort(addr, esr, regs); } @@ -785,7 +786,7 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, if (user_mode(regs)) { if (instruction_pointer(regs) > TASK_SIZE) arm64_apply_bp_hardening(); - local_irq_enable(); + local_daif_restore(DAIF_PROCCTX); } clear_siginfo(&info); From patchwork Tue Aug 28 15:51:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578765 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19B0E15A7 for ; Tue, 28 Aug 2018 15:59:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0824E2A61C for ; Tue, 28 Aug 2018 15:59:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EFB7D2A620; Tue, 28 Aug 2018 15:59:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 684B92A61C for ; Tue, 28 Aug 2018 15:59:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=s1gF+0ha2852SgWFH5hk3EgrCBDSQvvxJS2oeJ2WiyE=; b=J0//kiVOErTcFUrQEYRyaR1GP9 JqIF3uAhmia5oq2g0SynpJWvYtk9mTznMUjljH2Tx2uFYlrIQ6cTeqw5kRihXs0W+LvfvCeqg0rDF 2gAErCl4jaVjAx9pghrRGF4xAbNLCtihzXW56U0Z//cXgvyyA+Ga5n/LGS4flWMk6StYh7oOyfAvT sV0e8jvBrM3g9NXoJX32/3nx7YqfWrJMgYv7ZecYtp58Y8r1NnXWCvoDylWvksuRxGmhMYgpcchc0 0DKYNrQnchfnjxIHT4wMycU5Sr4kWtC+dgLMZa1Q4RL9OAxtIfvHoLNbvLPyMztgWm4qJlMZOArrS YrBzBLvw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugOU-0000pw-AZ; Tue, 28 Aug 2018 15:59:02 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHr-00058b-5n for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A6FDCED1; Tue, 28 Aug 2018 08:52:02 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B01533F557; Tue, 28 Aug 2018 08:52:00 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 06/27] arm64: Delay daif masking for user return Date: Tue, 28 Aug 2018 16:51:16 +0100 Message-Id: <1535471497-38854-7-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085211_297126_E2E09F4E X-CRM114-Status: GOOD ( 11.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Masking daif flags is done very early before returning to EL0. Only toggle the interrupt masking while in the vector entry and mask daif once in kernel_exit. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: James Morse --- arch/arm64/kernel/entry.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 09dbea22..85ce06ac 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -259,9 +259,9 @@ alternative_else_nop_endif .endm .macro kernel_exit, el - .if \el != 0 disable_daif + .if \el != 0 /* Restore the task's original addr_limit. */ ldr x20, [sp, #S_ORIG_ADDR_LIMIT] str x20, [tsk, #TSK_TI_ADDR_LIMIT] @@ -896,7 +896,7 @@ work_pending: * "slow" syscall return path. */ ret_to_user: - disable_daif + disable_irq // disable interrupts ldr x1, [tsk, #TSK_TI_FLAGS] and x2, x1, #_TIF_WORK_MASK cbnz x2, work_pending From patchwork Tue Aug 28 15:51:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A423514BD for ; Tue, 28 Aug 2018 15:56:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90CFE2A60C for ; Tue, 28 Aug 2018 15:56:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84A422A613; Tue, 28 Aug 2018 15:56:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0C4892A60C for ; Tue, 28 Aug 2018 15:56:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=XA0FyLlt9jrR89QnrZCz9q+YSy2gEQrwG/2XM4obDsc=; b=RPjhpLUJDfbY+Iyqajgv/UzAQL G0o03K8O5GNYi6HuEddVOE/poCmQ+NsQKSx4UpEHtXTd7japPzmTeXiUO/St/rkdEYoEDxdXu1Kbb yj/4YsLK79mfj/kFk8rZ4HUdSa4zSNzI0zXwn2yG4u4IQsBIEqTIdzYADkU7itsN6XYDaXsakKC+Z tRawj8BmMZFlTnk5/rRqth8SPnj0J9pKVC3RW3VsMo7GFj3oMcG+XWRFj0YJrloFa81g41f7LWlF4 MAIzf05V8o/4N2op+1b38olMFZkFW64du6cwM6veEAzjPdkr/NG9VE5WUp8CHMxakViEc/w4PDkAZ 0nFOZrPw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugLr-00088o-8F; Tue, 28 Aug 2018 15:56:19 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHr-000595-5o for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:12 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0DC7980D; Tue, 28 Aug 2018 08:52:05 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E6CD73F557; Tue, 28 Aug 2018 08:52:02 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 07/27] arm64: xen: Use existing helper to check interrupt status Date: Tue, 28 Aug 2018 16:51:17 +0100 Message-Id: <1535471497-38854-8-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085211_281473_0AFD2109 X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Stefano Stabellini , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The status of interrupts might depend on more than just pstate. Use interrupts_disabled() instead of raw_irqs_disabled_flags() to take the full context into account. Signed-off-by: Julien Thierry Cc: Stefano Stabellini Cc: Catalin Marinas Cc: Will Deacon Acked-by: Stefano Stabellini --- arch/arm64/include/asm/xen/events.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/xen/events.h b/arch/arm64/include/asm/xen/events.h index 4e22b7a..2788e95 100644 --- a/arch/arm64/include/asm/xen/events.h +++ b/arch/arm64/include/asm/xen/events.h @@ -14,7 +14,7 @@ enum ipi_vector { static inline int xen_irqs_disabled(struct pt_regs *regs) { - return raw_irqs_disabled_flags((unsigned long) regs->pstate); + return !interrupts_enabled(regs); } #define xchg_xen_ulong(ptr, val) xchg((ptr), (val)) From patchwork Tue Aug 28 15:51:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE0CC15A7 for ; Tue, 28 Aug 2018 15:57:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA6682A610 for ; Tue, 28 Aug 2018 15:57:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDCD42A614; Tue, 28 Aug 2018 15:57:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 46D8C2A610 for ; Tue, 28 Aug 2018 15:57:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=X/PIVBP0XIfRX5ARTUR6fesFkBBlpACeatdlLWs6bbQ=; b=ez5/fsA7kfB3czlnUWyp48Ne7B cOkw2Huz5VUH0Smik/T/7DRF4McA9Cw9+BzJkx4UUCuAa+dTCeaUZTdDo6vVRVnoYZWszJyfX519g QkZWWBrmw09vhlggqUlAJ9ppRvgN5KGuXj97oIcShS8F/eecDSWX3HLA/d6wGzQuAs9juXpPRv0Gr irHert3Ut0mwfxvw1SUrF9BCww2KeuFe0f3l2/Lai6ZPdAHRKDzdgH41PCXUt1yHfvepuna3mk/te OUXLJSbheZz3Xmk7dUmSllDNaKF8EszqxL5cnB8gRbG94hEJx+Hj1UoOnvFUeLM/XwyVn9Sjk7ie8 MijKqC+A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugNB-0000GW-9P; Tue, 28 Aug 2018 15:57:41 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHr-00059k-5i for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F40015BF; Tue, 28 Aug 2018 08:52:07 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4D5E33F557; Tue, 28 Aug 2018 08:52:05 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 08/27] irqchip/gic: Unify GIC priority definitions Date: Tue, 28 Aug 2018 16:51:18 +0100 Message-Id: <1535471497-38854-9-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085211_294969_6535374B X-CRM114-Status: GOOD ( 10.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP LPIs use the same priority value as other GIC interrupts. Make the GIC default priority definition visible to ITS implementation and use this same definition for LPI priorities. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-v3-its.c | 2 +- include/linux/irqchip/arm-gic-common.h | 6 ++++++ include/linux/irqchip/arm-gic.h | 5 ----- 3 files changed, 7 insertions(+), 6 deletions(-) -- 1.9.1 diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index 316a575..f5391f2 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -64,7 +64,7 @@ #define LPI_PROPBASE_SZ ALIGN(BIT(LPI_NRBITS), SZ_64K) #define LPI_PENDBASE_SZ ALIGN(BIT(LPI_NRBITS) / 8, SZ_64K) -#define LPI_PROP_DEFAULT_PRIO 0xa0 +#define LPI_PROP_DEFAULT_PRIO GICD_INT_DEF_PRI /* * Collection structure - just an ID, and a redistributor address to diff --git a/include/linux/irqchip/arm-gic-common.h b/include/linux/irqchip/arm-gic-common.h index 0a83b43..9a1a479 100644 --- a/include/linux/irqchip/arm-gic-common.h +++ b/include/linux/irqchip/arm-gic-common.h @@ -13,6 +13,12 @@ #include #include +#define GICD_INT_DEF_PRI 0xa0 +#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ + (GICD_INT_DEF_PRI << 16) |\ + (GICD_INT_DEF_PRI << 8) |\ + GICD_INT_DEF_PRI) + enum gic_type { GIC_V2, GIC_V3, diff --git a/include/linux/irqchip/arm-gic.h b/include/linux/irqchip/arm-gic.h index 6c4aaf0..6261790 100644 --- a/include/linux/irqchip/arm-gic.h +++ b/include/linux/irqchip/arm-gic.h @@ -65,11 +65,6 @@ #define GICD_INT_EN_CLR_X32 0xffffffff #define GICD_INT_EN_SET_SGI 0x0000ffff #define GICD_INT_EN_CLR_PPI 0xffff0000 -#define GICD_INT_DEF_PRI 0xa0 -#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ - (GICD_INT_DEF_PRI << 16) |\ - (GICD_INT_DEF_PRI << 8) |\ - GICD_INT_DEF_PRI) #define GICD_IIDR_IMPLEMENTER_SHIFT 0 #define GICD_IIDR_IMPLEMENTER_MASK (0xfff << GICD_IIDR_IMPLEMENTER_SHIFT) From patchwork Tue Aug 28 15:51:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578763 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2094714BD for ; Tue, 28 Aug 2018 15:58:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D33E2A5E1 for ; Tue, 28 Aug 2018 15:58:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 00C322A61E; Tue, 28 Aug 2018 15:58:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 90CD72A5E1 for ; Tue, 28 Aug 2018 15:58:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=yIdjYPvZ7V+ml5W0oO5gNbd+rPuZRkVuKnDGEcA9OQM=; b=ruAhcsw5YhAtRCGnEnxmjDWVE3 yWMwLALGsQyt3z2g84CH4uurmhBeEwZSjfo7EASG8MQWWhFnfjPLIsq5U+rbWAxxsjtTZ+zByd4zm OwsRbi5BTYwcxdJxd5fEcwRSHb8Kxw1JNhWP8xUCb6V8DtWX+v30bIjApi/TLfVUMRzWtGv+Jz5uE zXlSXNhEkQKltwxZOqPvlAr0DsIKdgqJIAp6HiWp73Sacx00VGvsa3+ciShUMialCi5goNEwlK8R1 52djtkoTRVGpb8urqySbqhmfGD96nh2qbcck5g/AZDqRMF5/fkKAD6pCY5oMcY25A3aEEEQLcPq79 A89y/03w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugNp-0000Xr-3s; Tue, 28 Aug 2018 15:58:21 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHr-0005AQ-5h for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:13 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 956821682; Tue, 28 Aug 2018 08:52:09 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9E6DE3F557; Tue, 28 Aug 2018 08:52:07 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 09/27] irqchip/gic: Lower priority of GIC interrupts Date: Tue, 28 Aug 2018 16:51:19 +0100 Message-Id: <1535471497-38854-10-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085211_286160_FC83E9AD X-CRM114-Status: GOOD ( 12.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The current value used for IRQ priorities is high among the non-secure interrupt priority values. Lower the default priority of interrupts so there is more flexibility to define higher priority interrupts. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Marc Zyngier --- include/linux/irqchip/arm-gic-common.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 1.9.1 diff --git a/include/linux/irqchip/arm-gic-common.h b/include/linux/irqchip/arm-gic-common.h index 9a1a479..2c9a4b3 100644 --- a/include/linux/irqchip/arm-gic-common.h +++ b/include/linux/irqchip/arm-gic-common.h @@ -13,7 +13,7 @@ #include #include -#define GICD_INT_DEF_PRI 0xa0 +#define GICD_INT_DEF_PRI 0xc0 #define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ (GICD_INT_DEF_PRI << 16) |\ (GICD_INT_DEF_PRI << 8) |\ From patchwork Tue Aug 28 15:51:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578777 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CEB4920 for ; Tue, 28 Aug 2018 16:04:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 89F0F2A63A for ; Tue, 28 Aug 2018 16:04:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D6792A651; Tue, 28 Aug 2018 16:04:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4DB082A63A for ; Tue, 28 Aug 2018 16:04:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nmIH3JtjJJ+aVsFjnIkLgI8krS/WgK1CKwYiUcCugIs=; b=HAdyFa0hMxn3/YFSnexFHwmKz1 KJbISXlVH9zvh++sHx/zDUatMyTVuqhbzmuupq+pCgc5orj6xnPjYeakDJIwbYd/bZ45vBsdrfnOQ hNLUkjYtCdA/AWKdWEGL+gQVw5PAbXhCx/NGDDh8TS1sj1+929SCUxEHOMIy7YwR7Ocn56r9Ne6sf J3hTFewQOHZZRCA4z31z6g2VGVFnvag5Z/nXRJ7d3DDiiWkMMdO39g67pqvCPdCrOqYulhUeSZunt U+FrkKAV0OCyeInWg5kQRaN+InB0Kb2oG1b+6Z92aWkf9LgC6j+n37u8vvj+tbI3L14/4se/mOcM+ 05Ch34pw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugT2-0003wQ-Sa; Tue, 28 Aug 2018 16:03:45 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugI2-0005CD-Dh for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:24 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E6C861684; Tue, 28 Aug 2018 08:52:11 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D56483F557; Tue, 28 Aug 2018 08:52:09 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 10/27] arm64: cpufeature: Add cpufeature for IRQ priority masking Date: Tue, 28 Aug 2018 16:51:20 +0100 Message-Id: <1535471497-38854-11-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085222_520816_C7A9A366 X-CRM114-Status: GOOD ( 14.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Suzuki K Poulose , will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add a cpufeature indicating whether a cpu supports masking interrupts by priority. Add command line option to disable that feature at runtime. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Suzuki K Poulose --- Documentation/admin-guide/kernel-parameters.txt | 3 +++ arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpufeature.c | 31 +++++++++++++++++++++++++ 3 files changed, 36 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 9871e64..d3e1170 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2764,6 +2764,9 @@ noexec=on: enable non-executable mappings (default) noexec=off: disable non-executable mappings + nogicprios [ARM64] + Disable usage of GIC priorities to toggle interrupt status. + nosmap [X86] Disable SMAP (Supervisor Mode Access Prevention) even if it is supported by processor. diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index ae1f704..8cc2ae5 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -51,7 +51,8 @@ #define ARM64_SSBD 30 #define ARM64_MISMATCHED_CACHE_TYPE 31 #define ARM64_HAS_STAGE2_FWB 32 +#define ARM64_HAS_IRQ_PRIO_MASKING 33 -#define ARM64_NCAPS 33 +#define ARM64_NCAPS 34 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 0d1e41e..2f2c557 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1046,6 +1046,22 @@ static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused) WARN_ON(val & (7 << 27 | 7 << 21)); } +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +static bool nogicprios = false; + +static int __init early_nogicprios(char *p) +{ + nogicprios = true; + return 0; +} +early_param("nogicprios", early_nogicprios); + +static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry, int scope) +{ + return !nogicprios && has_useable_gicv3_cpuif(entry, scope); +} +#endif + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -1233,6 +1249,21 @@ static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused) .cpu_enable = cpu_enable_hw_dbm, }, #endif +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + { + /* + * Depends on having GICv3 + */ + .desc = "IRQ priority masking", + .capability = ARM64_HAS_IRQ_PRIO_MASKING, + .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE, + .matches = can_use_gic_priorities, + .sys_reg = SYS_ID_AA64PFR0_EL1, + .field_pos = ID_AA64PFR0_GIC_SHIFT, + .sign = FTR_UNSIGNED, + .min_field_value = 1, + }, +#endif {}, }; From patchwork Tue Aug 28 15:51:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56CA214BD for ; Tue, 28 Aug 2018 15:59:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 445D72A61C for ; Tue, 28 Aug 2018 15:59:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 377952A620; Tue, 28 Aug 2018 15:59:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 865B92A61C for ; Tue, 28 Aug 2018 15:59:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=yJM38lYnH5GGMxRdLwZKr1zEk9E9eu1bJEXpUnFDJ8Y=; b=hUFc3QXUeSx809GObv8dR/Avv4 s9SX7eBdrr9yyDhNUdJrIusEp4G5Ka4MkUPFbO5KFMx1IxRXxxLoFIUD1Zg7eTF15jyPWy8jH2CPM aDX7g9t407jydAdiU5Pl8MpRXt2HZhJkO9UV+FqQDoLVvyfig/7pACxazwepgFVJ+Etyn1VTFAnlN apO6ijrlJK+B1Ph0q3b5GQerGboVlMIwzh+Lpt9q0FSW11MGm9R7brtGOG1kPF50uYBou0fQEUQbo MMfW1o161KP/ceoUOjA+/iP5Y/YYeXgVYFihKki9ET977NUwI+76Tw4+gg96VODDRM3HEHWJCRi+W byjRxnTA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugP7-00017N-Oi; Tue, 28 Aug 2018 15:59:41 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHu-0005AQ-Hl for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:16 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 705EE80D; Tue, 28 Aug 2018 08:52:14 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 322193F557; Tue, 28 Aug 2018 08:52:12 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 11/27] arm64: Make PMR part of task context Date: Tue, 28 Aug 2018 16:51:21 +0100 Message-Id: <1535471497-38854-12-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085214_667417_9EA56B1B X-CRM114-Status: GOOD ( 15.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, Oleg Nesterov , joel@joelfernandes.org, Dave Martin MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP If ICC_PMR_EL1 is used to mask interrupts, its value should be saved/restored whenever a task is context switched out/in or gets an exception. Add PMR to the registers to save in the pt_regs struct upon kernel entry, and restore it before ERET. Also, initialize it to a sane value when creating new tasks. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Oleg Nesterov Cc: Dave Martin --- arch/arm64/include/asm/processor.h | 1 + arch/arm64/include/asm/ptrace.h | 5 ++++- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 16 ++++++++++++++++ arch/arm64/kernel/process.c | 2 ++ 5 files changed, 24 insertions(+), 1 deletion(-) -- 1.9.1 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 79657ad..45a2e08 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -167,6 +167,7 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc) memset(regs, 0, sizeof(*regs)); forget_syscall(regs); regs->pc = pc; + regs->pmr_save = ICC_PMR_EL1_UNMASKED; } static inline void start_thread(struct pt_regs *regs, unsigned long pc, diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 177b851..29ec217 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -25,6 +25,9 @@ #define CurrentEL_EL1 (1 << 2) #define CurrentEL_EL2 (2 << 2) +/* PMR value use to unmask interrupts */ +#define ICC_PMR_EL1_UNMASKED 0xf0 + /* AArch32-specific ptrace requests */ #define COMPAT_PTRACE_GETREGS 12 #define COMPAT_PTRACE_SETREGS 13 @@ -163,7 +166,7 @@ struct pt_regs { #endif u64 orig_addr_limit; - u64 unused; // maintain 16 byte alignment + u64 pmr_save; u64 stackframe[2]; }; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 323aeb5..bab4122 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -78,6 +78,7 @@ int main(void) DEFINE(S_ORIG_X0, offsetof(struct pt_regs, orig_x0)); DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_ORIG_ADDR_LIMIT, offsetof(struct pt_regs, orig_addr_limit)); + DEFINE(S_PMR_SAVE, offsetof(struct pt_regs, pmr_save)); DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs)); BLANK(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 85ce06ac..79b06af 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -249,6 +249,15 @@ alternative_else_nop_endif msr sp_el0, tsk .endif + /* Save pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs_s x20, SYS_ICC_PMR_EL1 +alternative_else + /* Keep a sane value in the task context */ + mov x20, ICC_PMR_EL1_UNMASKED +alternative_endif + str x20, [sp, #S_PMR_SAVE] + /* * Registers that may be useful after this macro is invoked: * @@ -269,6 +278,13 @@ alternative_else_nop_endif /* No need to restore UAO, it will be restored from SPSR_EL1 */ .endif + /* Restore pmr */ +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + ldr x20, [sp, #S_PMR_SAVE] + msr_s SYS_ICC_PMR_EL1, x20 + dsb sy +alternative_else_nop_endif + ldp x21, x22, [sp, #S_PC] // load ELR, SPSR .if \el == 0 ct_user_enter diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 7f1628e..1f6a4d5 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -230,6 +230,7 @@ void __show_regs(struct pt_regs *regs) } printk("sp : %016llx\n", sp); + printk("pmr_save: %08llx\n", regs->pmr_save); i = top_reg; @@ -355,6 +356,7 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, } else { memset(childregs, 0, sizeof(struct pt_regs)); childregs->pstate = PSR_MODE_EL1h; + childregs->pmr_save = ICC_PMR_EL1_UNMASKED; if (IS_ENABLED(CONFIG_ARM64_UAO) && cpus_have_const_cap(ARM64_HAS_UAO)) childregs->pstate |= PSR_UAO_BIT; From patchwork Tue Aug 28 15:51:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578769 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C2BB14BD for ; Tue, 28 Aug 2018 16:00:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 391252A638 for ; Tue, 28 Aug 2018 16:00:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 368A42A678; Tue, 28 Aug 2018 16:00:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8CF9A2A638 for ; Tue, 28 Aug 2018 16:00:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=cG9xlcgeBwniMeGQDrKi8GMNNJpoJIzdnl2Hz8C6qbM=; b=K+KdsTMX/nx5fvIkcj8BzFXekQ Y8HjMhVbs6uzHDUyAN/HZ4oWuUmY2Z0yWR+KITiHFFo23Bp9myk2nJYSkkHVpjbicJS6EFMG7gbBw 7OhhDqJgedBESfGwLGB6z43XNCf4Fy165+Rjxr5zerbiUwdt2ZNfypJC6/kesVLJ5eCLR3oZ7BhqZ HThGBNtowc4rM/tQtuL3rFOhcNkOS64xuuBMcwPgChVIlL0rnBCKsStn8InWxmMDb5KLkVs7HnTAi KvyUpeCPJrDfF5hSLdMMoDAFe4HQniV+XEEQqKSO9C5UM7nByIylwvyeZx/oR93mEaJ3WX5JOZPXe JEOLm8mw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugPt-0002Cx-8t; Tue, 28 Aug 2018 16:00:29 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHw-00058P-Ln for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:18 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9DC1BED1; Tue, 28 Aug 2018 08:52:16 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AF97B3F557; Tue, 28 Aug 2018 08:52:14 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 12/27] arm64: Unmask PMR before going idle Date: Tue, 28 Aug 2018 16:51:22 +0100 Message-Id: <1535471497-38854-13-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085216_785642_5CB1900E X-CRM114-Status: GOOD ( 12.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP CPU does not received signals for interrupts with a priority masked by ICC_PMR_EL1. This means the CPU might not come back from a WFI instruction. Make sure ICC_PMR_EL1 does not mask interrupts when doing a WFI. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Catalin Marinas Cc: Will Deacon --- arch/arm64/mm/proc.S | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) -- 1.9.1 diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 03646e6..cc1caf7 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -53,10 +54,27 @@ * cpu_do_idle() * * Idle the processor (wait for interrupt). + * + * If the CPU supports priority masking we must do additional work to + * ensure that interrupts are not masked at the PMR (because the core will + * not wake up if we block the wake up signal in the interrupt controller). */ ENTRY(cpu_do_idle) +alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING + dsb sy // WFI may enter a low-power mode + wfi + ret +alternative_else + mrs x0, daif // save I bit + msr daifset, #2 // set I bit + mrs_s x1, SYS_ICC_PMR_EL1 // save PMR +alternative_endif + mov x2, #ICC_PMR_EL1_UNMASKED + msr_s SYS_ICC_PMR_EL1, x2 // unmask at PMR dsb sy // WFI may enter a low-power mode wfi + msr_s SYS_ICC_PMR_EL1, x1 // restore PMR + msr daif, x0 // restore I bit ret ENDPROC(cpu_do_idle) From patchwork Tue Aug 28 15:51:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5022920 for ; Tue, 28 Aug 2018 16:01:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D05A42A6C5 for ; Tue, 28 Aug 2018 16:01:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDECC2A6CF; Tue, 28 Aug 2018 16:01:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C17E82A6C5 for ; Tue, 28 Aug 2018 16:01:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=LUD2yJCY5gxmzyx0gpJnXPJqXN8DgNq7rK84VICrK/k=; b=d8NCtQ8xHbCine4koEfspdnw7F NoaSe6vkk1pfGyFs8VitdKt2h49Bl7i2/070JeX5FhoWHIUjNrzN7lWzHmzD5seAH31D8w/kztFGD Y176glvwSpYT54IX6d4M37ic/meG2u9r7ZdE60mmyqW81WyTfSZFnRFi1LwW9yq+fSDkyQC5sdR2f APea0nHgQyFrbBq7EmhRiAkqsV7vdlpZiJTdbYDW+RPVeDNrkAqztzHm99ARDHyFg9zrnLzMC4G34 o0AOn7NQqIaKTzfVyRtj06yKYqT1g+DF9RveoCtdgyEz3FMA1/4uqyIMIYSr5/TaPErDlantJqi61 aeb1JTlw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugQj-0002yA-NF; Tue, 28 Aug 2018 16:01:21 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugHz-00058P-0T for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:21 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EFF1E80D; Tue, 28 Aug 2018 08:52:18 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DDC7A3F557; Tue, 28 Aug 2018 08:52:16 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 13/27] arm/arm64: gic-v3: Add helper functions to manage IRQ priorities Date: Tue, 28 Aug 2018 16:51:23 +0100 Message-Id: <1535471497-38854-14-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085219_194164_32023704 X-CRM114-Status: GOOD ( 11.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Russell King MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add a function to check if priority masking is supported and accessors for PMR/RPR. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier --- arch/arm/include/asm/arch_gicv3.h | 21 +++++++++++++++++++++ arch/arm64/include/asm/arch_gicv3.h | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+) -- 1.9.1 diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h index 0bd5307..58d5d3e 100644 --- a/arch/arm/include/asm/arch_gicv3.h +++ b/arch/arm/include/asm/arch_gicv3.h @@ -34,6 +34,7 @@ #define ICC_SRE __ACCESS_CP15(c12, 0, c12, 5) #define ICC_IGRPEN1 __ACCESS_CP15(c12, 0, c12, 7) #define ICC_BPR1 __ACCESS_CP15(c12, 0, c12, 3) +#define ICC_RPR __ACCESS_CP15(c12, 0, c11, 3) #define __ICC_AP0Rx(x) __ACCESS_CP15(c12, 0, c8, 4 | x) #define ICC_AP0R0 __ICC_AP0Rx(0) @@ -245,6 +246,21 @@ static inline void gic_write_bpr1(u32 val) write_sysreg(val, ICC_BPR1); } +static inline u32 gic_read_pmr(void) +{ + return read_sysreg(ICC_PMR); +} + +static inline void gic_write_pmr(u32 val) +{ + write_sysreg(val, ICC_PMR); +} + +static inline u32 gic_read_rpr(void) +{ + return read_sysreg(ICC_RPR); +} + /* * Even in 32bit systems that use LPAE, there is no guarantee that the I/O * interface provides true 64bit atomic accesses, so using strd/ldrd doesn't @@ -347,5 +363,10 @@ static inline void gits_write_vpendbaser(u64 val, void * __iomem addr) #define gits_read_vpendbaser(c) __gic_readq_nonatomic(c) +static inline bool gic_prio_masking_enabled(void) +{ + return false; +} + #endif /* !__ASSEMBLY__ */ #endif /* !__ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index e278f94..19a5b1f 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -114,6 +114,21 @@ static inline void gic_write_bpr1(u32 val) write_sysreg_s(val, SYS_ICC_BPR1_EL1); } +static inline u32 gic_read_pmr(void) +{ + return read_sysreg_s(SYS_ICC_PMR_EL1); +} + +static inline void gic_write_pmr(u32 val) +{ + write_sysreg_s(val, SYS_ICC_PMR_EL1); +} + +static inline u32 gic_read_rpr(void) +{ + return read_sysreg_s(SYS_ICC_RPR_EL1); +} + #define gic_read_typer(c) readq_relaxed(c) #define gic_write_irouter(v, c) writeq_relaxed(v, c) #define gic_read_lpir(c) readq_relaxed(c) @@ -140,5 +155,11 @@ static inline void gic_write_bpr1(u32 val) #define gits_write_vpendbaser(v, c) writeq_relaxed(v, c) #define gits_read_vpendbaser(c) readq_relaxed(c) +static inline bool gic_prio_masking_enabled(void) +{ + return IS_ENABLED(CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS) + && cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING); +} + #endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */ From patchwork Tue Aug 28 15:51:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C405139B for ; Tue, 28 Aug 2018 16:02:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7936E2A628 for ; Tue, 28 Aug 2018 16:02:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B0D32A639; Tue, 28 Aug 2018 16:02:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 71DA32A628 for ; Tue, 28 Aug 2018 16:02:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=H3DdNYeusJqibBfkxFEaswhchseq+c59ASFErSnKDVA=; b=bzvsbX8FGI6Dsdn1r0HeQNlUqm x4tPMq76pS0KFXT2Uef2VoNicE/3aiuySXj7vLxCEgjiYPMKeVMGHflvgfEHv8kXTRQg3ZDh5HhYq r3IEvgTtO5XykeilU8lUn/HOXnakR1/WYnpJTd/Ym5XGJzRbbfrEi/nw5tJEmNiMhLMbTQXDZygZ/ S8lmYlZB21mgPY3IdB84Grrk27wMZjrHhXF/KLhgZLyvcQmfkMAh+npbAnx4l6oGDk3PQYLZdKLr/ HYmrANZehpQa8f9Z8Ub0SmuVyR5SJnAQ32SkdnmKIMISjOzD3UEXEPyf40HU8R0DZCfiC63pkV6JE tWej2y1A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugRN-0003Fm-Km; Tue, 28 Aug 2018 16:02:01 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugI1-00058P-BV for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:23 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4D35AED1; Tue, 28 Aug 2018 08:52:21 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3BB273F557; Tue, 28 Aug 2018 08:52:19 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 14/27] arm64: kvm: Unmask PMR before entering guest Date: Tue, 28 Aug 2018 16:51:24 +0100 Message-Id: <1535471497-38854-15-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085221_490841_3BBC4103 X-CRM114-Status: GOOD ( 14.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, kvmarm@lists.cs.columbia.edu MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This means that hypervisor will not receive masked interrupts while running a guest. Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Christoffer Dall Cc: Marc Zyngier Cc: Catalin Marinas Cc: Will Deacon Cc: kvmarm@lists.cs.columbia.edu --- arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++ arch/arm64/kvm/hyp/switch.c | 17 +++++++++++++++++ 2 files changed, 29 insertions(+) -- 1.9.1 diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f26055f..9689592 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -24,6 +24,7 @@ #include #include +#include #include #include #include @@ -466,6 +467,17 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu) static inline void kvm_arm_vhe_guest_enter(void) { local_daif_mask(); + + /* + * Having IRQs masked via PMR when entering the guest means the GIC + * will not signal the CPU of interrupts of lower priority, and the + * only way to get out will be via guest exceptions. + * Naturally, we want to avoid this. + */ + if (gic_prio_masking_enabled()) { + gic_write_pmr(ICC_PMR_EL1_UNMASKED); + dsb(sy); + } } static inline void kvm_arm_vhe_guest_exit(void) diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index d496ef5..074ab21 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -22,6 +22,7 @@ #include +#include #include #include #include @@ -533,6 +534,19 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *host_ctxt; struct kvm_cpu_context *guest_ctxt; u64 exit_code; + u32 host_pmr = ICC_PMR_EL1_UNMASKED; + + /* + * Having IRQs masked via PMR when entering the guest means the GIC + * will not signal the CPU of interrupts of lower priority, and the + * only way to get out will be via guest exceptions. + * Naturally, we want to avoid this. + */ + if (gic_prio_masking_enabled()) { + host_pmr = gic_read_pmr(); + gic_write_pmr(ICC_PMR_EL1_UNMASKED); + dsb(sy); + } vcpu = kern_hyp_va(vcpu); @@ -586,6 +600,9 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) */ __debug_switch_to_host(vcpu); + if (gic_prio_masking_enabled()) + gic_write_pmr(host_pmr); + return exit_code; } From patchwork Tue Aug 28 15:51:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578775 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2676139B for ; Tue, 28 Aug 2018 16:03:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F5252A63A for ; Tue, 28 Aug 2018 16:03:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 82B1D2A651; Tue, 28 Aug 2018 16:03:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B42522A63A for ; Tue, 28 Aug 2018 16:03:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=NFe0M4kSneqiRhfFMApyj9fZ+Zafq8B5vpLpf3rmc5E=; b=hQXRxTE4zVpGUXeKsZvmZo/ngN g4BoY2wNRGCEF7l5KC1Va9j4+TmgWPy1eSQd/I/9vTfsF6dTkscMv0FNyD9uZjrR209ZVr4mmrpnZ 1Xod9VK5Vl+6/xUpjANJXDiFh8dtSpLpXANWANH+E62Nk6wF/i56fdM6V/SKSyzsZoigITHE4SD7R E0AOrv02nkVMq8pBkgcYVb410xBs56XdWC4o3DiVilLhDjnJiHxEYy0IvF6dP+HHM0uhfJiFRjZz+ m2EdyvDfK3LvRXeW892chYf3AKYOevEPG1+5qESnnStDaBDKsyeg/mCp2DU/l34rxQantNGlsuhIr Z7nL4SMQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugS2-0003WG-L4; Tue, 28 Aug 2018 16:02:42 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugI3-00058P-MV for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:25 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9EEB8ED1; Tue, 28 Aug 2018 08:52:23 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8D26A3F557; Tue, 28 Aug 2018 08:52:21 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 15/27] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking Date: Tue, 28 Aug 2018 16:51:25 +0100 Message-Id: <1535471497-38854-16-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085223_809548_39046912 X-CRM114-Status: GOOD ( 17.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Ard Biesheuvel , will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, Oleg Nesterov , joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Instead disabling interrupts by setting the PSR.I bit, use a priority higher than the one used for interrupts to mask them via PMR. The value chosen for PMR to enable/disable interrupts encodes the status of interrupts on a single bit. This information is stored in the irqflags values used when saving/restoring IRQ status. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Catalin Marinas Cc: Will Deacon Cc: Ard Biesheuvel Cc: Oleg Nesterov --- arch/arm64/include/asm/assembler.h | 17 ++++++- arch/arm64/include/asm/efi.h | 3 +- arch/arm64/include/asm/irqflags.h | 97 ++++++++++++++++++++++++++++++-------- arch/arm64/include/asm/ptrace.h | 10 ++-- arch/arm64/kernel/entry.S | 2 +- 5 files changed, 102 insertions(+), 27 deletions(-) -- 1.9.1 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0bcc98d..0b2dcfd 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -23,6 +23,7 @@ #ifndef __ASM_ASSEMBLER_H #define __ASM_ASSEMBLER_H +#include #include #include #include @@ -62,12 +63,24 @@ /* * Enable and disable interrupts. */ - .macro disable_irq + .macro disable_irq, tmp + mov \tmp, #ICC_PMR_EL1_MASKED +alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING msr daifset, #2 +alternative_else + msr_s SYS_ICC_PMR_EL1, \tmp +alternative_endif .endm - .macro enable_irq + .macro enable_irq, tmp + mov \tmp, #ICC_PMR_EL1_UNMASKED +alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING msr daifclr, #2 + nop +alternative_else + msr_s SYS_ICC_PMR_EL1, \tmp + dsb sy +alternative_endif .endm .macro save_and_disable_irq, flags diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 7ed3208..3e06891 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -42,7 +42,8 @@ efi_status_t __efi_rt_asm_wrapper(void *, const char *, ...); -#define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) +#define ARCH_EFI_IRQ_FLAGS_MASK \ + (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | ARCH_FLAG_PMR_EN) /* arch specific definitions used by the stub code */ diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index 24692ed..193cfd0 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -18,7 +18,27 @@ #ifdef __KERNEL__ +#include +#include #include +#include + + +/* + * When ICC_PMR_EL1 is used for interrupt masking, only the bit indicating + * whether the normal interrupts are masked is kept along with the daif + * flags. + */ +#define ARCH_FLAG_PMR_EN 0x1 + +#define MAKE_ARCH_FLAGS(daif, pmr) \ + ((daif) | (((pmr) >> ICC_PMR_EL1_EN_SHIFT) & ARCH_FLAG_PMR_EN)) + +#define ARCH_FLAGS_GET_PMR(flags) \ + ((((flags) & ARCH_FLAG_PMR_EN) << ICC_PMR_EL1_EN_SHIFT) \ + | ICC_PMR_EL1_MASKED) + +#define ARCH_FLAGS_GET_DAIF(flags) ((flags) & ~ARCH_FLAG_PMR_EN) /* * Aarch64 has flags for masking: Debug, Asynchronous (serror), Interrupts and @@ -38,31 +58,50 @@ */ static inline unsigned long arch_local_irq_save(void) { - unsigned long flags; - asm volatile( + unsigned long flags, masked = ICC_PMR_EL1_MASKED; + unsigned long pmr = 0; + + asm volatile(ALTERNATIVE( "mrs %0, daif // arch_local_irq_save\n" - "msr daifset, #2" - : "=r" (flags) - : + "msr daifset, #2\n" + "mov %1, #" __stringify(ICC_PMR_EL1_UNMASKED), + /* --- */ + "mrs %0, daif\n" + "mrs_s %1, " __stringify(SYS_ICC_PMR_EL1) "\n" + "msr_s " __stringify(SYS_ICC_PMR_EL1) ", %2", + ARM64_HAS_IRQ_PRIO_MASKING) + : "=&r" (flags), "=&r" (pmr) + : "r" (masked) : "memory"); - return flags; + + return MAKE_ARCH_FLAGS(flags, pmr); } static inline void arch_local_irq_enable(void) { - asm volatile( - "msr daifclr, #2 // arch_local_irq_enable" - : + unsigned long unmasked = ICC_PMR_EL1_UNMASKED; + + asm volatile(ALTERNATIVE( + "msr daifclr, #2 // arch_local_irq_enable\n" + "nop", + "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%0\n" + "dsb sy", + ARM64_HAS_IRQ_PRIO_MASKING) : + : "r" (unmasked) : "memory"); } static inline void arch_local_irq_disable(void) { - asm volatile( - "msr daifset, #2 // arch_local_irq_disable" - : + unsigned long masked = ICC_PMR_EL1_MASKED; + + asm volatile(ALTERNATIVE( + "msr daifset, #2 // arch_local_irq_disable", + "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%0", + ARM64_HAS_IRQ_PRIO_MASKING) : + : "r" (masked) : "memory"); } @@ -72,12 +111,19 @@ static inline void arch_local_irq_disable(void) static inline unsigned long arch_local_save_flags(void) { unsigned long flags; - asm volatile( - "mrs %0, daif // arch_local_save_flags" - : "=r" (flags) + unsigned long pmr = 0; + + asm volatile(ALTERNATIVE( + "mrs %0, daif // arch_local_save_flags\n" + "mov %1, #" __stringify(ICC_PMR_EL1_UNMASKED), + "mrs %0, daif\n" + "mrs_s %1, " __stringify(SYS_ICC_PMR_EL1), + ARM64_HAS_IRQ_PRIO_MASKING) + : "=r" (flags), "=r" (pmr) : : "memory"); - return flags; + + return MAKE_ARCH_FLAGS(flags, pmr); } /* @@ -85,16 +131,27 @@ static inline unsigned long arch_local_save_flags(void) */ static inline void arch_local_irq_restore(unsigned long flags) { - asm volatile( - "msr daif, %0 // arch_local_irq_restore" + unsigned long pmr = ARCH_FLAGS_GET_PMR(flags); + + flags = ARCH_FLAGS_GET_DAIF(flags); + + asm volatile(ALTERNATIVE( + "msr daif, %0 // arch_local_irq_restore\n" + "nop\n" + "nop", + "msr daif, %0\n" + "msr_s " __stringify(SYS_ICC_PMR_EL1) ",%1\n" + "dsb sy", + ARM64_HAS_IRQ_PRIO_MASKING) : - : "r" (flags) + : "r" (flags), "r" (pmr) : "memory"); } static inline int arch_irqs_disabled_flags(unsigned long flags) { - return flags & PSR_I_BIT; + return (ARCH_FLAGS_GET_DAIF(flags) & (PSR_I_BIT)) | + !(ARCH_FLAGS_GET_PMR(flags) & ICC_PMR_EL1_EN_BIT); } #endif #endif diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 29ec217..67df46e 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -25,8 +25,11 @@ #define CurrentEL_EL1 (1 << 2) #define CurrentEL_EL2 (2 << 2) -/* PMR value use to unmask interrupts */ +/* PMR values used to mask/unmask interrupts */ #define ICC_PMR_EL1_UNMASKED 0xf0 +#define ICC_PMR_EL1_EN_SHIFT 6 +#define ICC_PMR_EL1_EN_BIT (1 << ICC_PMR_EL1_EN_SHIFT) // PMR IRQ enable +#define ICC_PMR_EL1_MASKED (ICC_PMR_EL1_UNMASKED ^ ICC_PMR_EL1_EN_BIT) /* AArch32-specific ptrace requests */ #define COMPAT_PTRACE_GETREGS 12 @@ -201,8 +204,9 @@ static inline void forget_syscall(struct pt_regs *regs) #define processor_mode(regs) \ ((regs)->pstate & PSR_MODE_MASK) -#define interrupts_enabled(regs) \ - (!((regs)->pstate & PSR_I_BIT)) +#define interrupts_enabled(regs) \ + ((!((regs)->pstate & PSR_I_BIT)) && \ + ((regs)->pmr_save & ICC_PMR_EL1_EN_BIT)) #define fast_interrupts_enabled(regs) \ (!((regs)->pstate & PSR_F_BIT)) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 79b06af..91e1e3d 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -912,7 +912,7 @@ work_pending: * "slow" syscall return path. */ ret_to_user: - disable_irq // disable interrupts + disable_irq x21 // disable interrupts ldr x1, [tsk, #TSK_TI_FLAGS] and x2, x1, #_TIF_WORK_MASK cbnz x2, work_pending From patchwork Tue Aug 28 15:51:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8EF29139B for ; Tue, 28 Aug 2018 16:04:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B1722A660 for ; Tue, 28 Aug 2018 16:04:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6FA6F2A663; Tue, 28 Aug 2018 16:04:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E89282A660 for ; Tue, 28 Aug 2018 16:04:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2XsSTWrFyEywlzMTcgUTxNt0kGg4kjo6QbhQWBwS0jI=; b=MJvytsGC9pVPmwinad5bAF6tG0 nWC+9jAZF5JqUqRKsDferBux0f2fkT4G6V+HKZxEifb62ZVHvZUnUg8dg5EMc05ugoBZSUKABBo2+ FRnuEr6MEBQrkkHQc16i6BL/Fl4ZzcScYYMtpw7dIn6WC/+3alvexA2lhzbeMcEBe0nYCNZ0Tci7Q XHgsqd8+Cl55kB5JOWBNFHCVZSgygpMupcshK3rrFYblgsXLMISjD+7DLy6hsqAeMjcD9U4Of6eml ltU5b21g8Dpoj8+QJ/d5HGBaf0dPdeki+93mjg/eokBu/fwzhsjtrwIaNrFlkkJAFDlHq+ngWn+S2 Wu+RjoyA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugTj-0004Mt-2w; Tue, 28 Aug 2018 16:04:27 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugI5-0005CD-ST for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:27 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CC74915BF; Tue, 28 Aug 2018 08:52:25 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DE7143F557; Tue, 28 Aug 2018 08:52:23 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 16/27] arm64: daifflags: Include PMR in daifflags restore operations Date: Tue, 28 Aug 2018 16:51:26 +0100 Message-Id: <1535471497-38854-17-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085226_092998_0A352F54 X-CRM114-Status: GOOD ( 11.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The addition of PMR should not bypass the semantics of daifflags. When DA_F are set, I bit is also set as no interrupts (even of higher priority) is allowed. When DA_F are cleared, I bit is cleared and interrupt enabling/disabling goes through ICC_PMR_EL1. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: James Morse --- arch/arm64/include/asm/daifflags.h | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 8d91f22..ccd95e8 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -18,8 +18,14 @@ #include -#define DAIF_PROCCTX 0 -#define DAIF_PROCCTX_NOIRQ PSR_I_BIT +#include + +#define DAIF_PROCCTX MAKE_ARCH_FLAGS(0, ICC_PMR_EL1_UNMASKED) + +#define DAIF_PROCCTX_NOIRQ \ + (gic_prio_masking_enabled() ? \ + MAKE_ARCH_FLAGS(0, ICC_PMR_EL1_MASKED) : \ + MAKE_ARCH_FLAGS(PSR_I_BIT, ICC_PMR_EL1_UNMASKED)) /* mask/save/unmask/restore all exceptions, including interrupts. */ static inline void local_daif_mask(void) @@ -51,6 +57,10 @@ static inline void local_daif_unmask(void) : : : "memory"); + + /* Unmask IRQs in PMR if needed */ + if (gic_prio_masking_enabled()) + arch_local_irq_enable(); } static inline void local_daif_restore(unsigned long flags) From patchwork Tue Aug 28 15:51:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578823 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92BA7920 for ; Tue, 28 Aug 2018 16:06:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E6522A666 for ; Tue, 28 Aug 2018 16:06:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 70A0E2A669; Tue, 28 Aug 2018 16:06:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C10DB2A666 for ; Tue, 28 Aug 2018 16:06:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=TZI7QC4mkdVESo/3deMpAC+cQUtxQ0ENXeroMKSg/3Y=; b=MwBtimw03e7GzbivYPy0s+cLFh rl3NKWuF4VsDJNpxHiSUTvvRR4hbo60AXe6luLQGMzc+BkJ89Fkwvg1wmFDBy84Bu6Fq6PGEROQl8 FgpfyWHSai/ahEfkLpeKY70zkwkBdwb7xUVbwNJyIt/aN9fmkD3hDHM/vlh8aXNtGy6AYLcCPeSsg Ubt7NQHMO698ry2aYQYnWA16tJxTkoZ8H3DGEYm4IcgzLmMTFnlU6hrdEQYZz4+SbzCrwo8IfOWoH 1IFbJOlbUkNB5MiRpB4vnN0iKavHM6DI/iFOpSHS/SYjTd32i84N+Ov+wYNKylhq5wce3eneDXWDw U5i4+qoA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugVw-0006XR-IM; Tue, 28 Aug 2018 16:06:44 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugII-0005Jj-LQ for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4C9B980D; Tue, 28 Aug 2018 08:52:28 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 17EFC3F557; Tue, 28 Aug 2018 08:52:25 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 17/27] irqchip/gic-v3: Factor group0 detection into functions Date: Tue, 28 Aug 2018 16:51:27 +0100 Message-Id: <1535471497-38854-18-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085238_789863_848D2AF0 X-CRM114-Status: GOOD ( 15.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The code to detect whether Linux has access to group0 interrupts can prove useful in other parts of the driver. Provide a separate function to do this. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 55 +++++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 19 deletions(-) -- 1.9.1 diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index d5912f1..fef6688 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -392,6 +392,39 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs } while (irqnr != ICC_IAR1_EL1_SPURIOUS); } +static u32 gic_get_pribits(void) +{ + u32 pribits; + + pribits = gic_read_ctlr(); + pribits &= ICC_CTLR_EL1_PRI_BITS_MASK; + pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT; + pribits++; + + return pribits; +} + +static bool gic_has_group0(void) +{ + u32 val; + + /* + * Let's find out if Group0 is under control of EL3 or not by + * setting the highest possible, non-zero priority in PMR. + * + * If SCR_EL3.FIQ is set, the priority gets shifted down in + * order for the CPU interface to set bit 7, and keep the + * actual priority in the non-secure range. In the process, it + * looses the least significant bit and the actual priority + * becomes 0x80. Reading it back returns 0, indicating that + * we're don't have access to Group0. + */ + gic_write_pmr(BIT(8 - gic_get_pribits())); + val = gic_read_pmr(); + + return val != 0; +} + static void __init gic_dist_init(void) { unsigned int i; @@ -533,7 +566,7 @@ static void gic_cpu_sys_reg_init(void) u64 mpidr = cpu_logical_map(cpu); u64 need_rss = MPIDR_RS(mpidr); bool group0; - u32 val, pribits; + u32 pribits; /* * Need to check that the SRE bit has actually been set. If @@ -545,25 +578,9 @@ static void gic_cpu_sys_reg_init(void) if (!gic_enable_sre()) pr_err("GIC: unable to set SRE (disabled at EL2), panic ahead\n"); - pribits = gic_read_ctlr(); - pribits &= ICC_CTLR_EL1_PRI_BITS_MASK; - pribits >>= ICC_CTLR_EL1_PRI_BITS_SHIFT; - pribits++; + pribits = gic_get_pribits(); - /* - * Let's find out if Group0 is under control of EL3 or not by - * setting the highest possible, non-zero priority in PMR. - * - * If SCR_EL3.FIQ is set, the priority gets shifted down in - * order for the CPU interface to set bit 7, and keep the - * actual priority in the non-secure range. In the process, it - * looses the least significant bit and the actual priority - * becomes 0x80. Reading it back returns 0, indicating that - * we're don't have access to Group0. - */ - write_gicreg(BIT(8 - pribits), ICC_PMR_EL1); - val = read_gicreg(ICC_PMR_EL1); - group0 = val != 0; + group0 = gic_has_group0(); /* Set priority mask register */ write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1); From patchwork Tue Aug 28 15:51:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578821 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B55B920 for ; Tue, 28 Aug 2018 16:06:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46D2D29566 for ; Tue, 28 Aug 2018 16:06:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 382982A656; Tue, 28 Aug 2018 16:06:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9DAF629566 for ; Tue, 28 Aug 2018 16:06:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Img0aid0AE0wdZkb4pqdkFl/Vai6MeAMKS47wPZm/FI=; b=VYvdpBMFty+Pe2il/hnAWulNHk 1tqtThgK+7D32t/QawAydNWd/dS2UrrakQVoZu9SaE5Na91D/Sc3P6ZvZCPFX+Beo4YUdyXw2yUq0 PPat7Mk+OUqX3eF0d0XGPiZdNKxu0tNCk9slmBbgaXeoYoSjlIZY0YVUuT4GkGD3Z54tD5ByjhwuT U6vuWsTsIv3KXmARKvL6Yssrmj4+8e+ApcB1ai0RKIzf3twCgpiVFjoLxBFAOsf5/3GlMu8+OtJO/ 5hC4gVScuHf99Z8uwqXIokfDGUbgSOhuDiz/n72uf+29F81GmUCRWH85ZpXWfpLwtoiXWjFFKLUrT 7iIMn/WQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugVG-0006Dx-Fz; Tue, 28 Aug 2018 16:06:02 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugII-0005KV-Lk for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C0EA5ED1; Tue, 28 Aug 2018 08:52:30 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8C34E3F557; Tue, 28 Aug 2018 08:52:28 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 18/27] irqchip/gic-v3: Do not overwrite PMR value Date: Tue, 28 Aug 2018 16:51:28 +0100 Message-Id: <1535471497-38854-19-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085238_834475_C5E2D2B4 X-CRM114-Status: GOOD ( 13.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP If the architecture is using ICC_PMR_EL1 to mask IRQs, do not overwrite that value. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) -- 1.9.1 diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index fef6688..162c49c 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -407,6 +407,9 @@ static u32 gic_get_pribits(void) static bool gic_has_group0(void) { u32 val; + u32 old_pmr; + + old_pmr = gic_read_pmr(); /* * Let's find out if Group0 is under control of EL3 or not by @@ -422,6 +425,8 @@ static bool gic_has_group0(void) gic_write_pmr(BIT(8 - gic_get_pribits())); val = gic_read_pmr(); + gic_write_pmr(old_pmr); + return val != 0; } @@ -583,7 +588,8 @@ static void gic_cpu_sys_reg_init(void) group0 = gic_has_group0(); /* Set priority mask register */ - write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1); + if (!gic_prio_masking_enabled()) + write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1); /* * Some firmwares hand over to the kernel with the BPR changed from From patchwork Tue Aug 28 15:51:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578819 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB714139B for ; Tue, 28 Aug 2018 16:05:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98D8829566 for ; Tue, 28 Aug 2018 16:05:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8CED62A656; Tue, 28 Aug 2018 16:05:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D75B629566 for ; Tue, 28 Aug 2018 16:05:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=lqApKP1Y10ZJqtH5xGUEPyIWLChoWw54s8kweKsu7t8=; b=TNNRpL8+ucUA+qyN6HuTmLNAzy XGqRyd5SCu5TdgJuTURKOJV6dw1+zI4LUMYiU3q2k5JBKlN5feV3IaFVPRZrBDGtJsbLD9G5/Jjnw jyOqlLY/wW+l1qx2+R3oADKxJQX+IZpFDWoV0McfSAqs9JDOTsW17gilD3DaHAZcRzrbi0RtGual+ 3t027JdPc1M/pYGZqJHNmzlfYgRl3XMUtSQUAH5QYikerZYtK/MERW1zJeqYSAd2mSapNKxeMjY3A wZBJSvK6Dfi+nI8Gdf0KiLfGGrX5Z0TR4FOwcoAdnIPDccogzMnDsQwGXFyOIcJ2IpNto7l/QUM7g r1zNQ3wg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugUR-0004mt-PJ; Tue, 28 Aug 2018 16:05:11 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugII-0005Lg-LY for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:44 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4130615BF; Tue, 28 Aug 2018 08:52:33 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0C8473F557; Tue, 28 Aug 2018 08:52:30 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 19/27] irqchip/gic-v3: Remove acknowledge loop Date: Tue, 28 Aug 2018 16:51:29 +0100 Message-Id: <1535471497-38854-20-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085238_795552_F5BD0088 X-CRM114-Status: GOOD ( 16.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Multiple interrupts pending for a CPU is actually rare. Doing an acknowledge loop does not give much better performance or even can deteriorate them. Do not loop when an interrupt has been acknowledged, just return from interrupt and wait for another one to be raised. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 65 +++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 34 deletions(-) -- 1.9.1 diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 162c49c..a467fcf 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -348,48 +348,45 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs { u32 irqnr; - do { - irqnr = gic_read_iar(); + irqnr = gic_read_iar(); - if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { - int err; + if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { + int err; - if (static_branch_likely(&supports_deactivate_key)) + if (static_branch_likely(&supports_deactivate_key)) + gic_write_eoir(irqnr); + else + isb(); + + err = handle_domain_irq(gic_data.domain, irqnr, regs); + if (err) { + WARN_ONCE(true, "Unexpected interrupt received!\n"); + if (static_branch_likely(&supports_deactivate_key)) { + if (irqnr < 8192) + gic_write_dir(irqnr); + } else { gic_write_eoir(irqnr); - else - isb(); - - err = handle_domain_irq(gic_data.domain, irqnr, regs); - if (err) { - WARN_ONCE(true, "Unexpected interrupt received!\n"); - if (static_branch_likely(&supports_deactivate_key)) { - if (irqnr < 8192) - gic_write_dir(irqnr); - } else { - gic_write_eoir(irqnr); - } } - continue; } - if (irqnr < 16) { - gic_write_eoir(irqnr); - if (static_branch_likely(&supports_deactivate_key)) - gic_write_dir(irqnr); + return; + } + if (irqnr < 16) { + gic_write_eoir(irqnr); + if (static_branch_likely(&supports_deactivate_key)) + gic_write_dir(irqnr); #ifdef CONFIG_SMP - /* - * Unlike GICv2, we don't need an smp_rmb() here. - * The control dependency from gic_read_iar to - * the ISB in gic_write_eoir is enough to ensure - * that any shared data read by handle_IPI will - * be read after the ACK. - */ - handle_IPI(irqnr, regs); + /* + * Unlike GICv2, we don't need an smp_rmb() here. + * The control dependency from gic_read_iar to + * the ISB in gic_write_eoir is enough to ensure + * that any shared data read by handle_IPI will + * be read after the ACK. + */ + handle_IPI(irqnr, regs); #else - WARN_ONCE(true, "Unexpected SGI received!\n"); + WARN_ONCE(true, "Unexpected SGI received!\n"); #endif - continue; - } - } while (irqnr != ICC_IAR1_EL1_SPURIOUS); + } } static u32 gic_get_pribits(void) From patchwork Tue Aug 28 15:51:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578825 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 41AB0920 for ; Tue, 28 Aug 2018 16:07:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2DAEA2A669 for ; Tue, 28 Aug 2018 16:07:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 20EB62A673; Tue, 28 Aug 2018 16:07:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9446C2A669 for ; Tue, 28 Aug 2018 16:07:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=byhfN7sbvzpLpCAnDI10odUj5I1K/V1odbAxAYfroN8=; b=fwztkrYMO2vueuMjdPdKQOvfpj TxqTsfos8ug2YUbPvPQJ2ITT8IV+OmloqHfnX+h7qY06LuGBERfNB0sofdl/jFzpzB1A3SUqQdJjT BOfiLNbUm70qyBQKhy70jNXAzW2oEpAJU4Zh7Wyc6RSdsjPVLHlhMXAldQPL8v31xncW9wnDmQ1gB 1t8ydrrqTTUCoeXeo+CXY9TUNULEA36Tz2z9AQ+S2XkmctdvR9j5Yaaz11K6KENctdMAspWhU8B9H UK2Tc2SpmXZ6qIbvmHORVM8hagQaUxGT3nOtag3x4q3UZuGSqdtYzz3H5nvBXKt09zZTcOxZlbwKV wbJo61Ww==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugWd-0006qW-7f; Tue, 28 Aug 2018 16:07:27 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugII-0005Mq-Ll for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:52 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D9C4C1682; Tue, 28 Aug 2018 08:52:35 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 818083F557; Tue, 28 Aug 2018 08:52:33 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 20/27] irqchip/gic-v3: Switch to PMR masking after IRQ acknowledge Date: Tue, 28 Aug 2018 16:51:30 +0100 Message-Id: <1535471497-38854-21-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085238_829998_F8FCA129 X-CRM114-Status: GOOD ( 13.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Russell King , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP After an interrupt has been acknowledged, mask the IRQ priority through PMR and clear PSR.I bit, allowing higher priority interrupts to be received during interrupt handling. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- arch/arm/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/include/asm/arch_gicv3.h | 6 ++++++ drivers/irqchip/irq-gic-v3.c | 8 +++++++- 3 files changed, 19 insertions(+), 1 deletion(-) -- 1.9.1 diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h index 58d5d3e..b39d620 100644 --- a/arch/arm/include/asm/arch_gicv3.h +++ b/arch/arm/include/asm/arch_gicv3.h @@ -368,5 +368,11 @@ static inline bool gic_prio_masking_enabled(void) return false; } +static inline void gic_start_pmr_masking(void) +{ + /* Should not get called */ + WARN_ON(true); +} + #endif /* !__ASSEMBLY__ */ #endif /* !__ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 19a5b1f..eb55da8 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -161,5 +161,11 @@ static inline bool gic_prio_masking_enabled(void) && cpus_have_const_cap(ARM64_HAS_IRQ_PRIO_MASKING); } +static inline void gic_start_pmr_masking(void) +{ + gic_write_pmr(ICC_PMR_EL1_MASKED); + asm volatile ("msr daifclr, #2" : : : "memory"); +} + #endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */ diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index a467fcf..56d1fb9 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -350,12 +350,18 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs irqnr = gic_read_iar(); + if (gic_prio_masking_enabled()) { + isb(); + /* Masking IRQs earlier would prevent to ack the current IRQ */ + gic_start_pmr_masking(); + } + if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { int err; if (static_branch_likely(&supports_deactivate_key)) gic_write_eoir(irqnr); - else + else if (!gic_prio_masking_enabled()) isb(); err = handle_domain_irq(gic_data.domain, irqnr, regs); From patchwork Tue Aug 28 15:51:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11F10920 for ; Tue, 28 Aug 2018 16:08:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 003C22A4DB for ; Tue, 28 Aug 2018 16:08:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E879B2A67C; Tue, 28 Aug 2018 16:08:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7BA3B2A4DB for ; Tue, 28 Aug 2018 16:08:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4lYiKuUS+AkvehZopoYJznEuLYCpF46qkvTtdRs4gR4=; b=oYbrvEM6jA6t1WtokPETB+Prmx aEhxMJAm8QkJgbdAskrn/6a9IlPcz+pedmGjgvy8FkHq5AxW6299+9z3POvCYhOBfHTVuAqH/0Nnl wkG0IZ3iMxjTECK4HpihEj32Erqu/7JmQ0JVKdUiPvj9QOxIVTuWxlG/ANMufy4UGVv1yWAHDSE3i q3mgxmMf4/sDBDBfHIGLyFleUyGB5SRz+h9cWTCdiKhaYkj2549m7grmpIIo7Qc1vsi3z4CYTEAip /uz2rW+LxHbN9STLILNqkPARdSl28INcydzzA8XZO2d4nMSfO2c0fpVZM+uY8wPEQZpdW/ZDqLDKs tfebRV/A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugXG-0007B4-Ly; Tue, 28 Aug 2018 16:08:06 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugII-0005O4-MF for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:52:52 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1359C168F; Tue, 28 Aug 2018 08:52:38 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 256A93F557; Tue, 28 Aug 2018 08:52:36 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 21/27] arm64: Switch to PMR masking when starting CPUs Date: Tue, 28 Aug 2018 16:51:31 +0100 Message-Id: <1535471497-38854-22-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085238_881892_00EE999C X-CRM114-Status: GOOD ( 15.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Once the boot CPU has been prepared or a new secondary CPU has been brought up, use ICC_PMR_EL1 to mask interrupts on that CPU and clear PSR.I bit. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Catalin Marinas Cc: Will Deacon Cc: James Morse Cc: Marc Zyngier --- arch/arm64/include/asm/irqflags.h | 3 +++ arch/arm64/kernel/head.S | 35 +++++++++++++++++++++++++++++++++++ arch/arm64/kernel/smp.c | 5 +++++ 3 files changed, 43 insertions(+) -- 1.9.1 diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index 193cfd0..d31e9b6 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -153,5 +153,8 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) return (ARCH_FLAGS_GET_DAIF(flags) & (PSR_I_BIT)) | !(ARCH_FLAGS_GET_PMR(flags) & ICC_PMR_EL1_EN_BIT); } + +void maybe_switch_to_sysreg_gic_cpuif(void); + #endif #endif diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index b085306..ba73690 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -648,6 +648,41 @@ set_cpu_boot_mode_flag: ENDPROC(set_cpu_boot_mode_flag) /* + * void maybe_switch_to_sysreg_gic_cpuif(void) + * + * Enable interrupt controller system register access if this feature + * has been detected by the alternatives system. + * + * Before we jump into generic code we must enable interrupt controller system + * register access because this is required by the irqflags macros. We must + * also mask interrupts at the PMR and unmask them within the PSR. That leaves + * us set up and ready for the kernel to make its first call to + * arch_local_irq_enable(). + * + */ +ENTRY(maybe_switch_to_sysreg_gic_cpuif) +alternative_if_not ARM64_HAS_IRQ_PRIO_MASKING + b 1f +alternative_else + mrs_s x0, SYS_ICC_SRE_EL1 +alternative_endif + orr x0, x0, #1 + msr_s SYS_ICC_SRE_EL1, x0 // Set ICC_SRE_EL1.SRE==1 + isb // Make sure SRE is now set + mrs x0, daif + tbz x0, #7, no_mask_pmr // Are interrupts on? + mov x0, ICC_PMR_EL1_MASKED + msr_s SYS_ICC_PMR_EL1, x0 // Prepare for unmask of I bit + msr daifclr, #2 // Clear the I bit + b 1f +no_mask_pmr: + mov x0, ICC_PMR_EL1_UNMASKED + msr_s SYS_ICC_PMR_EL1, x0 +1: + ret +ENDPROC(maybe_switch_to_sysreg_gic_cpuif) + +/* * These values are written with the MMU off, but read with the MMU on. * Writers will invalidate the corresponding address, discarding up to a * 'Cache Writeback Granule' (CWG) worth of data. The linker script ensures diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 22c9a0a..443fa2b 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -185,6 +185,8 @@ asmlinkage notrace void secondary_start_kernel(void) struct mm_struct *mm = &init_mm; unsigned int cpu; + maybe_switch_to_sysreg_gic_cpuif(); + cpu = task_cpu(current); set_my_cpu_offset(per_cpu_offset(cpu)); @@ -421,6 +423,9 @@ void __init smp_prepare_boot_cpu(void) * and/or scheduling is enabled. */ apply_boot_alternatives(); + + /* Conditionally switch to GIC PMR for interrupt masking */ + maybe_switch_to_sysreg_gic_cpuif(); } static u64 __init of_get_cpu_mpidr(struct device_node *dn) From patchwork Tue Aug 28 15:51:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578833 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C7A46920 for ; Tue, 28 Aug 2018 16:10:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B3A192A6F7 for ; Tue, 28 Aug 2018 16:10:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B157B2A717; Tue, 28 Aug 2018 16:10:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 331732A72E for ; Tue, 28 Aug 2018 16:10:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dbXBylB9SRP08S6S4ODWrHaxm+POl8PPlSmh8iS8TSU=; b=BN9rs0mVLH9qW2Q/xJDAN88tI8 ndV5X5DMzH/TFCm9x/UAoaIY+W2EJNlckaWIHBiap7RT9SORw/q2AGZUb+A17H92OGAnS6HMcCCgb DkuNE6FsIX/J2R3EORMhcEq1yLX8IcRBdgBX3vjzP9YNT99mkQnLlnIHsa2vWuDE8mdmXzsuWmkkR JQTuKQtBszm/81G8Hcyox/UcOqB0iFJ2L4GTkxNl8pc1kQHmJWnA/brHZ7wcC/DNU04GO/DDPaRiG 2kLAB+xnbfWue54SPz1OG04vzfvRJ90Y8zIcwSp8ZJ0tyLuXud703Lo/A15H1nqpozZESwsDHxIPL 1AyK5L2g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugZj-0000nA-Cy; Tue, 28 Aug 2018 16:10:39 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005RZ-Sc for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:08 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 40CAA1684; Tue, 28 Aug 2018 08:52:40 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 52A6E3F557; Tue, 28 Aug 2018 08:52:38 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 22/27] arm64: Add build option for IRQ masking via priority Date: Tue, 28 Aug 2018 16:51:32 +0100 Message-Id: <1535471497-38854-23-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_116672_94F3BB26 X-CRM114-Status: GOOD ( 12.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Provide a build option to enable using GICv3 priorities to enable/disable interrupts. Tested-by: Daniel Thompson Signed-off-by: Julien Thierry Suggested-by: Daniel Thompson Cc: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 15 +++++++++++++++ 1 file changed, 15 insertions(+) -- 1.9.1 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 29e75b4..d09c6ff 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -944,6 +944,21 @@ config ARM64_SSBD If unsure, say Y. +config USE_ICC_SYSREGS_FOR_IRQFLAGS + bool "Use ICC system registers for IRQ masking" + select CONFIG_ARM_GIC_V3 + help + Using the ICC system registers for IRQ masking makes it possible + to simulate NMI on ARM64 systems. This allows several interesting + features (especially debug features) to be used on these systems. + + Say Y here to implement IRQ masking using ICC system + registers when the GIC System Registers are available. The changes + are applied dynamically using the alternatives system so it is safe + to enable this option on systems with older interrupt controllers. + + If unsure, say N + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT From patchwork Tue Aug 28 15:51:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6E09139B for ; Tue, 28 Aug 2018 16:10:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93A302A684 for ; Tue, 28 Aug 2018 16:10:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 854B52A68B; Tue, 28 Aug 2018 16:10:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 09E482A684 for ; Tue, 28 Aug 2018 16:10:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Xg/nbLRjdEWRWWSZE9wsmwcPESLSOCCU4HGVK4rt42E=; b=g+eT8u0OntQlzbOpbBORRBRX4Z PelPXCPHbVaaGpybgO8thEWIpgT0c2SYGQmi+iYf1HkBbwpfXsnDU/FXZqJ41XMRkUyYp86N+3ATr G9v2UPAFehEBxNsAkucPWSW/NBVMKLNzHLEOh2BqiD0c+beDf7en1+NOMWsi2bTK1a9DZpTUvglu9 JDCS9gRUXwD6RNRHe+aFaB4hI1u4vVFM2i9J3LGND2b7rHpfccFFAUPtg5e4frUHJnQgek3L+pRVn ip4aUDO0QwtsDaatLHf/5JLaqDtu6Zebl7l3pUutazT/DB3L91miwUH1dxyfdm9uPMieGzuMBAr5M sEiGOjQg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugYy-0007yv-Ut; Tue, 28 Aug 2018 16:09:52 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005RY-Sd for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 927B2174E; Tue, 28 Aug 2018 08:52:42 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 80CC23F557; Tue, 28 Aug 2018 08:52:40 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 23/27] arm64: Handle serror in NMI context Date: Tue, 28 Aug 2018 16:51:33 +0100 Message-Id: <1535471497-38854-24-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_111485_5B7C9F65 X-CRM114-Status: GOOD ( 10.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Dave Martin MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Per definition of the daifflags, Serrors can occur during any interrupt context, that includes NMI contexts. Trying to nmi_enter in an nmi context will crash. Skip nmi_enter/nmi_exit when serror occurred during an NMI. Suggested-by: James Morse Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Dave Martin Cc: James Morse --- arch/arm64/kernel/traps.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 039e9ff..4955ec6 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -713,13 +713,17 @@ bool arm64_is_fatal_ras_serror(struct pt_regs *regs, unsigned int esr) asmlinkage void do_serror(struct pt_regs *regs, unsigned int esr) { - nmi_enter(); + const bool was_in_nmi = in_nmi(); + + if (!was_in_nmi) + nmi_enter(); /* non-RAS errors are not containable */ if (!arm64_is_ras_serror(esr) || arm64_is_fatal_ras_serror(regs, esr)) arm64_serror_panic(regs, esr); - nmi_exit(); + if (!was_in_nmi) + nmi_exit(); } void __pte_error(const char *file, int line, unsigned long val) From patchwork Tue Aug 28 15:51:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578839 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A4A9139B for ; Tue, 28 Aug 2018 16:12:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46CCE212D8 for ; Tue, 28 Aug 2018 16:12:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A904288FD; Tue, 28 Aug 2018 16:12:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6E458212D8 for ; Tue, 28 Aug 2018 16:12:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=UYtOzXcW1OWrAL3upkIgCxcbeRdAVKSSUcoqLy44m80=; b=VOCmERrRYwSwUNN16+V75ubkzJ QWq2vIGvdz2jP2Rscp6bqjoR2VuoiSCbTNnN//8286EsQPILyYSG8wOcWiCyewFK6IDI5rKOL7HEf wuDuQH5Mfr4E71gg1cI2s8hp6v5Mwf3R2nh9248+A3FMLYLAU/sFb5wmP0eeaQ6d1wQQXQT3yj8F4 lDK60C7/Ae9xVm7A1NSU6TCO5ebQXAwqzvvzlBryrRNblX7QW8avnZMzt9l9mB5/3X3lx68tShuPc rUFp0xgYtPV7fxrC1l73QbMfPrx/+UwW3yLtdTEMu0sF2Ai4lDmFDO27ysCCAXRiEfmHdEkc9gCKg 9ogBQIKw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugbl-00028f-AC; Tue, 28 Aug 2018 16:12:45 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005T2-VH for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:12 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 13A7115BF; Tue, 28 Aug 2018 08:52:45 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D2A483F557; Tue, 28 Aug 2018 08:52:42 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 24/27] irqchip/gic-v3: Detect current view of GIC priorities Date: Tue, 28 Aug 2018 16:51:34 +0100 Message-Id: <1535471497-38854-25-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_291373_AD39E36F X-CRM114-Status: GOOD ( 19.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, Jonathan Corbet , will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The values non secure EL1 needs to use for PMR and RPR registers depends on the value of SCR_EL3.FIQ. The values non secure EL1 sees from the distributor and redistributor depend on whether security is enabled for the GIC or not. Figure out what values we are dealing with to know if the values we use for PMR and RPR match the priority values that have been configured in the distributor and redistributors. Also, add firmware requirements related to SCR_EL3. Signed-off-by: Julien Thierry Cc: Catalin Marinas Cc: Will Deacon Cc: Jonathan Corbet Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- Documentation/arm64/booting.txt | 5 +++++ drivers/irqchip/irq-gic-v3.c | 42 +++++++++++++++++++++++++++++++++++------ 2 files changed, 41 insertions(+), 6 deletions(-) diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt index 8d0df62..e387938 100644 --- a/Documentation/arm64/booting.txt +++ b/Documentation/arm64/booting.txt @@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met: the kernel image will be entered must be initialised by software at a higher exception level to prevent execution in an UNKNOWN state. + - SCR_EL3.FIQ must have the same value across all CPUs the kernel is + executing on. + - The value of SCR_EL3.FIQ must be the same as the one present at boot + time whenever the kernel is executing. + For systems with a GICv3 interrupt controller to be used in v3 mode: - If EL3 is present: ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1. diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 56d1fb9..ffe98e8 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -63,6 +63,28 @@ struct gic_chip_data { static struct gic_chip_data gic_data __read_mostly; static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key); +/* + * The behaviours of RPR and PMR registers differ depending on the value of + * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the + * distributor and redistributors depends on whether security is enabled in the + * GIC. + * + * When security is enabled, non-secure priority values from the (re)distributor + * are presented to the GIC CPUIF as follow: + * (GIC_(R)DIST_PRI[irq] >> 1) | 0x80; + * + * If SCR_EL3.FIQ == 1, the values writen to/read from PMR and RPR at non-secure + * EL1 are subject to a similar operation thus matching the priorities presented + * from the (re)distributor when security is enabled. + * + * see GICv3/GICv4 Architecture Specification (IHI0069D): + * - section 4.8.1 Non-secure accesses to register fields for Secure interrupt + * priorities. + * - Figure 4-7 Secure read of the priority field for a Non-secure Group 1 + * interrupt. + */ +DEFINE_STATIC_KEY_FALSE(have_non_secure_prio_view); + static struct gic_kvm_info gic_v3_kvm_info; static DEFINE_PER_CPU(bool, has_rss); @@ -568,6 +590,12 @@ static void gic_update_vlpi_properties(void) !gic_data.rdists.has_direct_lpi ? "no " : ""); } +/* Check whether it's single security state view */ +static inline bool gic_dist_security_disabled(void) +{ + return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS; +} + static void gic_cpu_sys_reg_init(void) { int i, cpu = smp_processor_id(); @@ -593,6 +621,9 @@ static void gic_cpu_sys_reg_init(void) /* Set priority mask register */ if (!gic_prio_masking_enabled()) write_gicreg(DEFAULT_PMR_VALUE, ICC_PMR_EL1); + else if (static_branch_likely(&have_non_secure_prio_view) && group0) + /* Mismatch configuration with boot CPU */ + WARN_ON(group0 && !gic_dist_security_disabled()); /* * Some firmwares hand over to the kernel with the BPR changed from @@ -845,12 +876,6 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, #endif #ifdef CONFIG_CPU_PM -/* Check whether it's single security state view */ -static bool gic_dist_security_disabled(void) -{ - return readl_relaxed(gic_data.dist_base + GICD_CTLR) & GICD_CTLR_DS; -} - static int gic_cpu_pm_notifier(struct notifier_block *self, unsigned long cmd, void *v) { @@ -1161,6 +1186,11 @@ static int __init gic_init_bases(void __iomem *dist_base, gic_cpu_init(); gic_cpu_pm_init(); + if (gic_prio_masking_enabled()) { + if (!gic_has_group0() || gic_dist_security_disabled()) + static_branch_enable(&have_non_secure_prio_view); + } + return 0; out_free: From patchwork Tue Aug 28 15:51:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578829 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66EDB920 for ; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 53FE72A68B for ; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4856D2A68D; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 08F0C2A68B for ; Tue, 28 Aug 2018 16:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4py2SewDCvKrzO+2gwxjiSI8NbdeuasGrNxF+RSgeis=; b=ohUIN+C/i9efQJch9pgmH5oQn8 1mkQUumEiWg6U788HyJQCwM+tsIHrQieaXJram7DVXWyGz0IvSeOp/5erQWNyB5YFfs23IcD+z8Qe Ff+Lm/agtMYWSqvSPh7TUldeDb/VYaItrh/5mw3f01AHq/7kIemZteNHB79ji5HHrD1OSYMzWvNXK 5Z8SLHMUh1MfKNuQebdiNKw7fswxXjKAoMhifL9XvQn/z3SPXUdnBWoOID1RGu3tE9/x+m9CNS7X2 yl0nUsR0+H0dlXtOz0kbqPSe8iez1fK8A0y2ia0hdlEgjy5Y78vOX9mo8qu9t+6HN/ZCQZ+0DJpYU wcqO3rlA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugY8-0007dS-7o; Tue, 28 Aug 2018 16:09:00 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005T3-Sb for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 893731993; Tue, 28 Aug 2018 08:52:47 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 53CDF3F557; Tue, 28 Aug 2018 08:52:45 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 25/27] irqchip/gic-v3: Add base support for pseudo-NMI Date: Tue, 28 Aug 2018 16:51:35 +0100 Message-Id: <1535471497-38854-26-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_049785_9CCB244B X-CRM114-Status: GOOD ( 17.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Russell King , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Provide a higher priority to be used for pseudo-NMIs. When such an interrupt is received, enter the NMI state and prevent other NMIs to be raised. When returning from a pseudo-NMI, skip preemption and tracing if the interrupted context has interrupts disabled. Signed-off-by: Julien Thierry Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- arch/arm/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/kernel/entry.S | 43 +++++++++++++++++++++++++++++++++++++ drivers/irqchip/irq-gic-v3.c | 34 ++++++++++++++++++++++++++++- 4 files changed, 88 insertions(+), 1 deletion(-) diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h index b39d620..1ed0476 100644 --- a/arch/arm/include/asm/arch_gicv3.h +++ b/arch/arm/include/asm/arch_gicv3.h @@ -374,5 +374,11 @@ static inline void gic_start_pmr_masking(void) WARN_ON(true); } +static inline void gic_set_nmi_active(void) +{ + /* Should not get called */ + WARN_ON(true); +} + #endif /* !__ASSEMBLY__ */ #endif /* !__ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index eb55da8..6213d6f 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -167,5 +167,11 @@ static inline void gic_start_pmr_masking(void) asm volatile ("msr daifclr, #2" : : : "memory"); } +/* Notify an NMI is active, blocking other NMIs */ +static inline void gic_set_nmi_active(void) +{ + asm volatile ("msr daifset, #2" : : : "memory"); +} + #endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 91e1e3d..39d6ef0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -411,6 +411,16 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 mov sp, x19 .endm + /* Should be checked on return from irq handlers */ + .macro branch_if_was_nmi, tmp, target + alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs \tmp, daif + alternative_else + mov \tmp, #0 + alternative_endif + tbnz \tmp, #7, \target // Exiting an NMI + .endm + /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - x0 to x6. @@ -631,12 +641,30 @@ ENDPROC(el1_sync) el1_irq: kernel_entry 1 enable_da_f + #ifdef CONFIG_TRACE_IRQFLAGS +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + ldr x20, [sp, #S_PMR_SAVE] + /* Irqs were disabled, don't trace */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, 1f +#endif bl trace_hardirqs_off +1: #endif irq_handler +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + /* + * Irqs were disabled, we have an nmi. + * We might have interrupted a context with interrupt disabled that set + * NEED_RESCHED flag. + * Skip preemption and irq tracing if needed. + */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, untraced_irq_exit + branch_if_was_nmi x0, skip_preempt +#endif + #ifdef CONFIG_PREEMPT ldr w24, [tsk, #TSK_TI_PREEMPT] // get preempt count cbnz w24, 1f // preempt count != 0 @@ -645,9 +673,13 @@ el1_irq: bl el1_preempt 1: #endif + +skip_preempt: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + +untraced_irq_exit: kernel_exit 1 ENDPROC(el1_irq) @@ -873,6 +905,9 @@ el0_irq_naked: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + + branch_if_was_nmi x2, nmi_ret_to_user + b ret_to_user ENDPROC(el0_irq) @@ -1269,3 +1304,11 @@ alternative_else_nop_endif ENDPROC(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) #endif /* CONFIG_ARM_SDE_INTERFACE */ + +/* + * NMI return path to EL0 + */ +nmi_ret_to_user: + ldr x1, [tsk, #TSK_TI_FLAGS] + b finish_ret_to_user +ENDPROC(nmi_ret_to_user) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index ffe98e8..1af2fcc 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -41,6 +41,8 @@ #include "irq-gic-common.h" +#define GICD_INT_NMI_PRI 0xa0 + struct redist_region { void __iomem *redist_base; phys_addr_t phys_base; @@ -248,6 +250,12 @@ static void gic_unmask_irq(struct irq_data *d) gic_poke_irq(d, GICD_ISENABLER); } +static inline bool gic_supports_nmi(void) +{ + return gic_prio_masking_enabled() + && static_branch_likely(&have_non_secure_prio_view); +} + static int gic_irq_set_irqchip_state(struct irq_data *d, enum irqchip_irq_state which, bool val) { @@ -381,6 +389,23 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { int err; + if (gic_supports_nmi() + && unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) { + /* + * We need to prevent other NMIs to occur even after a + * priority drop. + * We keep I flag set until cpsr is restored from + * kernel_exit. + */ + gic_set_nmi_active(); + + if (static_branch_likely(&supports_deactivate_key)) + gic_write_eoir(irqnr); + + err = handle_domain_nmi(gic_data.domain, irqnr, regs); + return; + } + if (static_branch_likely(&supports_deactivate_key)) gic_write_eoir(irqnr); else if (!gic_prio_masking_enabled()) @@ -1119,6 +1144,11 @@ static int partition_domain_translate(struct irq_domain *d, .select = gic_irq_domain_select, }; +static void gic_enable_nmi_support(void) +{ + static_branch_enable(&have_non_secure_prio_view); +} + static int __init gic_init_bases(void __iomem *dist_base, struct redist_region *rdist_regs, u32 nr_redist_regions, @@ -1188,7 +1218,9 @@ static int __init gic_init_bases(void __iomem *dist_base, if (gic_prio_masking_enabled()) { if (!gic_has_group0() || gic_dist_security_disabled()) - static_branch_enable(&have_non_secure_prio_view); + gic_enable_nmi_support(); + else + pr_warn("SCR_EL3.FIQ set, cannot enable use of pseudo-NMIs\n"); } return 0; From patchwork Tue Aug 28 15:51:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578837 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26F93920 for ; Tue, 28 Aug 2018 16:12:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 158452A684 for ; Tue, 28 Aug 2018 16:12:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0916F2A68E; Tue, 28 Aug 2018 16:12:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EB2BC2A684 for ; Tue, 28 Aug 2018 16:12:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1/0nt7wPWvJLU68nCq80vVAS8ApBA954zM2m+RLvDR4=; b=h8vKmRONqhnNPmje5rKaWoPNlH VukqHD6o4sAWllREd1i6E510ezPLRM6yxKok4jic11bPwPQxfOM5s9zfLanwgq90X/XNcUPoaI6KK 6rhuqIFduzuxICEjS26JgQe30zyRAc6dWx+yN11eZOPtTyP8FEdZNNwmZ1Dlaogbj4MKyTmIfnt90 y/zjgUDPJpHTEAhMahmzwvaNzylOtRzbQCvqfNVqy09JX5Hm1TNghy32ABwCmtz5pQgxb8GvnkkS2 LiSbymCD7jhlL37mjKv6voj4yg8NCix+MyyVzV2Dkew+G2Wl7FqJRfmzGdlZTYUuZ4/dyhZULgSQv MallzD1w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugb5-0001gw-4d; Tue, 28 Aug 2018 16:12:03 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005Tc-TJ for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0A1C580D; Tue, 28 Aug 2018 08:52:50 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C8EB33F557; Tue, 28 Aug 2018 08:52:47 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 26/27] irqchip/gic: Add functions to access irq priorities Date: Tue, 28 Aug 2018 16:51:36 +0100 Message-Id: <1535471497-38854-27-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_149378_FEFAC7D6 X-CRM114-Status: GOOD ( 11.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add accessors to the GIC distributor/redistributors priority registers. Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-common.c | 10 ++++++++++ drivers/irqchip/irq-gic-common.h | 2 ++ 2 files changed, 12 insertions(+) diff --git a/drivers/irqchip/irq-gic-common.c b/drivers/irqchip/irq-gic-common.c index 01e673c..910746f 100644 --- a/drivers/irqchip/irq-gic-common.c +++ b/drivers/irqchip/irq-gic-common.c @@ -98,6 +98,16 @@ int gic_configure_irq(unsigned int irq, unsigned int type, return ret; } +void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio) +{ + writeb_relaxed(prio, base + GIC_DIST_PRI + irq); +} + +u8 gic_get_irq_prio(unsigned int irq, void __iomem *base) +{ + return readb_relaxed(base + GIC_DIST_PRI + irq); +} + void gic_dist_config(void __iomem *base, int gic_irqs, void (*sync_access)(void)) { diff --git a/drivers/irqchip/irq-gic-common.h b/drivers/irqchip/irq-gic-common.h index 3919cd7..1586dbd 100644 --- a/drivers/irqchip/irq-gic-common.h +++ b/drivers/irqchip/irq-gic-common.h @@ -35,6 +35,8 @@ void gic_dist_config(void __iomem *base, int gic_irqs, void gic_cpu_config(void __iomem *base, void (*sync_access)(void)); void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks, void *data); +void gic_set_irq_prio(unsigned int irq, void __iomem *base, u8 prio); +u8 gic_get_irq_prio(unsigned int irq, void __iomem *base); void gic_set_kvm_info(const struct gic_kvm_info *info); From patchwork Tue Aug 28 15:51:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E278139B for ; Tue, 28 Aug 2018 16:11:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C5BC2A72F for ; Tue, 28 Aug 2018 16:11:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 692D52A72A; Tue, 28 Aug 2018 16:11:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8CA9A2A74A for ; Tue, 28 Aug 2018 16:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=c8ehiJR6mVPCCIfwJ3BnhmimXYKDpADQxSVfdpCjDIM=; b=rTWp5ZALYFScHF13kPNI29+XTt 4OvGf4d/auGx3BGZ07ljtAz9CCbhle/SF8veYMztgP7oNq96csYQeTMnZ/QYushPyMwt3BSdCiFrE jL0xkzzGEaAc7B0VqZ3pywhRjmVpbXrUCM53S6KhVaAjzFphO4I4GBGRWj0ywFppaYmih4Wgmp4ct 1D0BzEF5QSb3gZl0bz+ONRY682Q+bRTZEBbjMAedPVKoIGt4v/EmAIwWA2vrDwPwCb1ugqe2Ee2nR oQL3fqCmoT1qNv/wSKhPpowz60xYjjk+TLqp8KDeY+TfkWDhydrQZFyL3sKkwi+tepo7dH6F4F6Vf QW4rnppg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugaP-0001PP-37; Tue, 28 Aug 2018 16:11:21 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005UT-SO for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:08 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7E9F1ED1; Tue, 28 Aug 2018 08:52:52 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 49A823F557; Tue, 28 Aug 2018 08:52:50 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 27/27] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI Date: Tue, 28 Aug 2018 16:51:37 +0100 Message-Id: <1535471497-38854-28-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_099480_92B574CF X-CRM114-Status: GOOD ( 14.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Implement NMI callbacks for GICv3 irqchip. Install NMI safe handlers when setting up interrupt line as NMI. Only SPIs and PPIs are allowed to be set up as NMI. Signed-off-by: Julien Thierry Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- drivers/irqchip/irq-gic-v3.c | 73 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 1af2fcc..b1b255a 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -311,6 +311,70 @@ static int gic_irq_get_irqchip_state(struct irq_data *d, return 0; } +static int gic_irq_set_irqchip_prio(struct irq_data *d, u8 prio) +{ + struct irq_desc *desc = irq_to_desc(d->irq); + /* Number of CPUs having PPI (idx + 16) setup as NMI */ + static uint32_t nb_ppinmi_refs[16] = { 0 }; + + if (gic_peek_irq(d, GICD_ISENABLER)) { + pr_err("Cannot set NMI property of enabled IRQ %u\n", d->irq); + return -EINVAL; + } + + /* + * A secondary irq_chip should be in charge of LPI request, + * it should not be possible to get there + */ + if (WARN_ON(gic_irq(d) >= 8192)) + return -EINVAL; + + /* desc lock should already be held */ + if (prio == GICD_INT_NMI_PRI) { + if (gic_irq(d) < 32) { + /* Setting up NMI, only switch handler for first NMI */ + if (nb_ppinmi_refs[gic_irq(d) - 16] == 0) + desc->handle_irq = handle_percpu_devid_fasteoi_nmi; + + nb_ppinmi_refs[gic_irq(d) - 16]++; + } else { + desc->handle_irq = handle_fasteoi_nmi; + } + } else if (prio == GICD_INT_DEF_PRI) { + if (gic_irq(d) < 32) { + /* Tearing down NMI, only switch handler for last NMI */ + if (nb_ppinmi_refs[gic_irq(d) - 16] == 1) + desc->handle_irq = handle_percpu_devid_irq; + + nb_ppinmi_refs[gic_irq(d) - 16]--; + } else { + desc->handle_irq = handle_fasteoi_irq; + } + } else { + return -EINVAL; + } + + gic_set_irq_prio(gic_irq(d), gic_dist_base(d), prio); + + return 0; +} + +static int gic_irq_nmi_setup(struct irq_data *d) +{ + if (!gic_supports_nmi()) + return -EINVAL; + + return gic_irq_set_irqchip_prio(d, GICD_INT_NMI_PRI); +} + +static void gic_irq_nmi_teardown(struct irq_data *d) +{ + if (WARN_ON(!gic_supports_nmi())) + return; + + gic_irq_set_irqchip_prio(d, GICD_INT_DEF_PRI); +} + static void gic_eoi_irq(struct irq_data *d) { gic_write_eoir(gic_irq(d)); @@ -937,6 +1001,8 @@ static inline void gic_cpu_pm_init(void) { } .irq_set_affinity = gic_set_affinity, .irq_get_irqchip_state = gic_irq_get_irqchip_state, .irq_set_irqchip_state = gic_irq_set_irqchip_state, + .irq_nmi_setup = gic_irq_nmi_setup, + .irq_nmi_teardown = gic_irq_nmi_teardown, .flags = IRQCHIP_SET_TYPE_MASKED | IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND, @@ -952,6 +1018,8 @@ static inline void gic_cpu_pm_init(void) { } .irq_get_irqchip_state = gic_irq_get_irqchip_state, .irq_set_irqchip_state = gic_irq_set_irqchip_state, .irq_set_vcpu_affinity = gic_irq_set_vcpu_affinity, + .irq_nmi_setup = gic_irq_nmi_setup, + .irq_nmi_teardown = gic_irq_nmi_teardown, .flags = IRQCHIP_SET_TYPE_MASKED | IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND, @@ -1147,6 +1215,11 @@ static int partition_domain_translate(struct irq_domain *d, static void gic_enable_nmi_support(void) { static_branch_enable(&have_non_secure_prio_view); + + if (static_branch_likely(&supports_deactivate_key)) + gic_eoimode1_chip.flags |= IRQCHIP_SUPPORTS_NMI; + else + gic_chip.flags |= IRQCHIP_SUPPORTS_NMI; } static int __init gic_init_bases(void __iomem *dist_base,