From patchwork Wed Sep 6 16:02:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Doug Anderson X-Patchwork-Id: 13375779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2801CEE14A0 for ; Wed, 6 Sep 2023 16:06:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=N9Uq94jBub3I743D7Ydf6ORFzhuMTmi4zWd3KyKuCSA=; b=JMLgvPWS3UegBF vk6fK7nKxEC8F75BKJSniRABCMLRGAwL4N0ZC+neeDd+RUnVs49vXpClnYgVl0sElb4oL6Nbdr0BA CEBHYo6LBg0ZlvCkzC8q9tPGACAyFSV8xmTS8KLDiG4f68exVwXzyoqJTXi/39MD9JHGu+zAJLOuS X5dI0G1Vkgnx4lvHPxwMQUN+RLdbv82j6XLJ+3nck00gciZgRVdIxHFdKsH7SPMJ+2BNWY3l0MGf1 hr+JB0BTuREZ/cWMESgNylzhv7ZJa10hk4WiJMq18ojDfmu3ru2cCiO82PkZvK0ufQL6ak5xyai8Z 7NN4EyGBGU7CjkfJxlXg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qdv2h-00ATZV-33; Wed, 06 Sep 2023 16:06:11 +0000 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qdv2c-00ATWG-1P for linux-arm-kernel@lists.infradead.org; Wed, 06 Sep 2023 16:06:10 +0000 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bf078d5f33so30445285ad.3 for ; Wed, 06 Sep 2023 09:06:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1694016363; x=1694621163; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9AStTiUn2wkKE7iLM15Ptbf7v9P7cN3w/9Lr/OinoUk=; b=aviz/7GSkIX1vQa/14vYxrG3GdFdPqTRusm3GzOozvZdOwR+vbTKw2HS//IfPyN/V0 krbsWrBZni9FBWx/8nq+IiTKDvcE6bfYaNNyu49hZrv7c4qBKb4HnHbm5qePC0Lcuuft HV7kVslcbLZcWQzpDtGvPhsjFY1riQMXT3//0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1694016363; x=1694621163; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9AStTiUn2wkKE7iLM15Ptbf7v9P7cN3w/9Lr/OinoUk=; b=RTT39vapQzdbhaW0vZm8HauS5dXqsJfm8ewjPCgFGR9QIwqNl7GBMUidKJZAySouel OSrIjjF7nhBu35J5youFfVS1viAIiAa1l39huXCVa9KNyoW06EVBo1aog3/FXMofQrIg YbeAO1NVElqh7Wntnqc4MttUAgBrNcliLGn3ObEnAyJx3xZaur+QS3Ys2VwWnERVNhLR KI0cCIM/S7sxMfIOGcf2vBVzi1YcmGiE+WiqXRuRjF11LxIlnohkGBAM7hlBOxatorBz /5fU05F9RULLCx15fDfxhGEXuEFpnEGXoTx0+nTa2+Kp8966XFHhwcBrajllpGRKDBl1 l2dQ== X-Gm-Message-State: AOJu0Yw+re/i67t9qIwT9KZes+1icubbjUs6NqGiqSJ8p20qF+CB5i32 OiXq1v8iPyrg6Apo/3GhyRR9Bg== X-Google-Smtp-Source: AGHT+IHvnL/4JaRFUJ+xW/5TxV7a56fOHoi2Ui181sO6LOlIeDV0WX7gAscP3gPGe2rpmUXdId80sQ== X-Received: by 2002:a17:902:ea01:b0:1bf:205e:fe5d with SMTP id s1-20020a170902ea0100b001bf205efe5dmr20963955plg.7.1694016363355; Wed, 06 Sep 2023 09:06:03 -0700 (PDT) Received: from tictac2.mtv.corp.google.com ([2620:15c:9d:2:4a07:e00a:fdae:750b]) by smtp.gmail.com with ESMTPSA id ju19-20020a170903429300b001b8c689060dsm11338859plb.28.2023.09.06.09.06.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Sep 2023 09:06:02 -0700 (PDT) From: Douglas Anderson To: Mark Rutland , Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier Cc: linux-arm-kernel@lists.infradead.org, "Rafael J . Wysocki" , Lecopzer Chen , Chen-Yu Tsai , Tomohiro Misono , Peter Zijlstra , Masayoshi Mizuma , Stephane Eranian , Ard Biesheuvel , kgdb-bugreport@lists.sourceforge.net, Stephen Boyd , linux-perf-users@vger.kernel.org, Thomas Gleixner , ito-yuichi@fujitsu.com, Douglas Anderson , Chen-Yu Tsai , linux-kernel@vger.kernel.org Subject: [PATCH v13 1/7] irqchip/gic-v3: Enable support for SGIs to act as NMIs Date: Wed, 6 Sep 2023 09:02:56 -0700 Message-ID: <20230906090246.v13.1.I1223c11c88937bd0cbd9b086d4ef216985797302@changeid> X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog In-Reply-To: <20230906160505.2431857-1-dianders@chromium.org> References: <20230906160505.2431857-1-dianders@chromium.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230906_090606_479414_4FC98576 X-CRM114-Status: GOOD ( 24.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As of commit 6abbd6988971 ("irqchip/gic, gic-v3: Make SGIs use handle_percpu_devid_irq()") SGIs are treated the same as PPIs/EPPIs and use handle_percpu_devid_irq() by default. Unfortunately, handle_percpu_devid_irq() isn't NMI safe, and so to run in an NMI context those should use handle_percpu_devid_fasteoi_nmi(). In order to accomplish this, we just have to make room for SGIs in the array of refcounts that keeps track of which interrupts are set as NMI. We also rename the array and create a new indexing scheme that accounts for SGIs. Also, enable NMI support prior to gic_smp_init() as allocation of SGIs as IRQs/NMIs happen as part of this routine. Co-developed-by: Sumit Garg Signed-off-by: Sumit Garg Acked-by: Mark Rutland Tested-by: Chen-Yu Tsai Signed-off-by: Douglas Anderson Acked-by: Marc Zyngier --- I'll note that this change is a little more black magic to me than others in this series. I don't have a massive amounts of familiarity with all the moving parts of gic-v3, so I mostly just followed Mark Rutland's advice [1]. As per discussion [2], the hope is that this patch could get Acked by Marc Zyngier and then land through the arm64 tree. If this isn't a good idea for some reason, I'd love suggestions for alternate ways for this series to land. [1] https://lore.kernel.org/r/ZNC-YRQopO0PaIIo@FVFF77S0Q05N.cambridge.arm.com [2] https://lore.kernel.org/r/ZPC1nUw3qKWrC85l@FVFF77S0Q05N.cambridge.arm.com Changes in v13: - s/_idx/_index/ on the patch to make function names consistent. Changes in v12: - Added a comment about why we account for 16 SGIs when Linux uses 8. Changes in v10: - Rewrite as needed for 5.11+ as per Mark Rutland and Sumit. drivers/irqchip/irq-gic-v3.c | 59 +++++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 18 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index eedfa8e9f077..787ccc880b22 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -78,6 +78,13 @@ static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key); #define GIC_LINE_NR min(GICD_TYPER_SPIS(gic_data.rdists.gicd_typer), 1020U) #define GIC_ESPI_NR GICD_TYPER_ESPIS(gic_data.rdists.gicd_typer) +/* + * There are 16 SGIs, though we only actually use 8 in Linux. The other 8 SGIs + * are potentially stolen by the secure side. Some code, especially code dealing + * with hwirq IDs, is simplified by accounting for all 16. + */ +#define SGI_NR 16 + /* * The behaviours of RPR and PMR registers differ depending on the value of * SCR_EL3.FIQ, and the behaviour of non-secure priority registers of the @@ -125,8 +132,8 @@ EXPORT_SYMBOL(gic_nonsecure_priorities); __priority; \ }) -/* ppi_nmi_refs[n] == number of cpus having ppi[n + 16] set as NMI */ -static refcount_t *ppi_nmi_refs; +/* rdist_nmi_refs[n] == number of cpus having the rdist interrupt n set as NMI */ +static refcount_t *rdist_nmi_refs; static struct gic_kvm_info gic_v3_kvm_info __initdata; static DEFINE_PER_CPU(bool, has_rss); @@ -519,9 +526,22 @@ static u32 __gic_get_ppi_index(irq_hw_number_t hwirq) } } -static u32 gic_get_ppi_index(struct irq_data *d) +static u32 __gic_get_rdist_index(irq_hw_number_t hwirq) +{ + switch (__get_intid_range(hwirq)) { + case SGI_RANGE: + case PPI_RANGE: + return hwirq; + case EPPI_RANGE: + return hwirq - EPPI_BASE_INTID + 32; + default: + unreachable(); + } +} + +static u32 gic_get_rdist_index(struct irq_data *d) { - return __gic_get_ppi_index(d->hwirq); + return __gic_get_rdist_index(d->hwirq); } static int gic_irq_nmi_setup(struct irq_data *d) @@ -545,11 +565,14 @@ static int gic_irq_nmi_setup(struct irq_data *d) /* desc lock should already be held */ if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + u32 idx = gic_get_rdist_index(d); - /* Setting up PPI as NMI, only switch handler for first NMI */ - if (!refcount_inc_not_zero(&ppi_nmi_refs[idx])) { - refcount_set(&ppi_nmi_refs[idx], 1); + /* + * Setting up a percpu interrupt as NMI, only switch handler + * for first NMI + */ + if (!refcount_inc_not_zero(&rdist_nmi_refs[idx])) { + refcount_set(&rdist_nmi_refs[idx], 1); desc->handle_irq = handle_percpu_devid_fasteoi_nmi; } } else { @@ -582,10 +605,10 @@ static void gic_irq_nmi_teardown(struct irq_data *d) /* desc lock should already be held */ if (gic_irq_in_rdist(d)) { - u32 idx = gic_get_ppi_index(d); + u32 idx = gic_get_rdist_index(d); /* Tearing down NMI, only switch handler for last NMI */ - if (refcount_dec_and_test(&ppi_nmi_refs[idx])) + if (refcount_dec_and_test(&rdist_nmi_refs[idx])) desc->handle_irq = handle_percpu_devid_irq; } else { desc->handle_irq = handle_fasteoi_irq; @@ -1279,10 +1302,10 @@ static void gic_cpu_init(void) rbase = gic_data_rdist_sgi_base(); /* Configure SGIs/PPIs as non-secure Group-1 */ - for (i = 0; i < gic_data.ppi_nr + 16; i += 32) + for (i = 0; i < gic_data.ppi_nr + SGI_NR; i += 32) writel_relaxed(~0, rbase + GICR_IGROUPR0 + i / 8); - gic_cpu_config(rbase, gic_data.ppi_nr + 16, gic_redist_wait_for_rwp); + gic_cpu_config(rbase, gic_data.ppi_nr + SGI_NR, gic_redist_wait_for_rwp); /* initialise system registers */ gic_cpu_sys_reg_init(); @@ -1939,12 +1962,13 @@ static void gic_enable_nmi_support(void) return; } - ppi_nmi_refs = kcalloc(gic_data.ppi_nr, sizeof(*ppi_nmi_refs), GFP_KERNEL); - if (!ppi_nmi_refs) + rdist_nmi_refs = kcalloc(gic_data.ppi_nr + SGI_NR, + sizeof(*rdist_nmi_refs), GFP_KERNEL); + if (!rdist_nmi_refs) return; - for (i = 0; i < gic_data.ppi_nr; i++) - refcount_set(&ppi_nmi_refs[i], 0); + for (i = 0; i < gic_data.ppi_nr + SGI_NR; i++) + refcount_set(&rdist_nmi_refs[i], 0); pr_info("Pseudo-NMIs enabled using %s ICC_PMR_EL1 synchronisation\n", gic_has_relaxed_pmr_sync() ? "relaxed" : "forced"); @@ -2061,6 +2085,7 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gic_dist_init(); gic_cpu_init(); + gic_enable_nmi_support(); gic_smp_init(); gic_cpu_pm_init(); @@ -2073,8 +2098,6 @@ static int __init gic_init_bases(phys_addr_t dist_phys_base, gicv2m_init(handle, gic_data.domain); } - gic_enable_nmi_support(); - return 0; out_free: