From patchwork Mon Oct 21 04:22:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13843549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2BAC1D3C93D for ; Mon, 21 Oct 2024 04:28:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rF+TQYGEDP6+t2V2Dte+w8ImgbG43FZ1I4yOJyMV9jY=; b=dA6J630dOYYvVABvXfmfnf+lFh OqHlk1o531QSbcclqKPktR8TBuzjbgAmBducnBT9NhwYAjGu+AC8iTrg2dP1bmEw4WfnupUubVgAI PAaEdZOjhlCBgZypzNFDFyoW6whdPzl10fcgie/1b77IjKeEtbqLai4AijPtXuus6R9LUjG4DO1Cy 5ef73zHYD6Y1NiT5IPGTDIbCQY9f3ZXtcHWzlXvGG+isyMqK/+nUQH9jCu20SrAHCMOnbtsJJGboR y7BVNTp3sGIS6lepAvnM6f4CzmQayskpmqkYEiCUWnEpOVQOVI5EuRG5Wb2PjZlae7VXBg2+bNVzs 5IUE9Ywg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t2k1y-000000061fX-20Wv; Mon, 21 Oct 2024 04:28:34 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t2jw8-000000060sD-0Oeu for linux-arm-kernel@lists.infradead.org; Mon, 21 Oct 2024 04:22:34 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e291ac8dcf3so6556973276.3 for ; Sun, 20 Oct 2024 21:22:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729484550; x=1730089350; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rF+TQYGEDP6+t2V2Dte+w8ImgbG43FZ1I4yOJyMV9jY=; b=E9iJmfkK83iR+rX5gkCDs7/ppR2rpAxNimIHmzO7GEdsm1zFTgFSL1ds+lb1yGcljp mH7xyDTaYxAXdShyPyTa416Ly3kqiLIq4ujvk29qjxvBoQKp248MFYotP/rvdtZNCK7r nVbvWI2EAaG6atSqV0GjXItlG9seOFSGR7gB7XtiBdFHAMQQzPowkFTNBd8bY4lJy6QR hdseeVHdTMSqcLh5Qb1Bgt/eEgnsEBLBP9mQJfwDuWQGLolC3CiNJHtMdiQ6hPm8eb1c pW05R9fPHyNfzIObyxIZbwmFIKTfZi2p8aYM1BuXCjHZZcY6TmGk2PQ4qpbtrmDX6i6v nNxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729484550; x=1730089350; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rF+TQYGEDP6+t2V2Dte+w8ImgbG43FZ1I4yOJyMV9jY=; b=f5feOHHGOf/p96aoK0ha1kqVETBstZsoi9oEDEPl/cZRSmy8eNb9+E74rwElTJyDG7 B1bXMi2qSyGrxr4/GKfS0CbZ0ylMffUwsrefl8lVFwO94YjrL4I8aSCegaO45uSJLQ/U 1gCMtvWrTUOfMihhTbHCCmoWsENCSJjcxMSLM/VvLLbVbH7CuNqMI4AsBZ7H+Jen3vPD 9x/mEuz6nYrPXro2qhvx9KjsOTqh9SdpiK+i0LcMR/hlftH/YVHgTYJp03IE0xyqEs16 Sg/pwREZKj1JLm7qIRUYa9klIJ+Ln6slh9usfD2rzUxLZJ4jNjfoSyMiQcqUpd5IOuu6 b2ew== X-Forwarded-Encrypted: i=1; AJvYcCUQJCAnNJPzxyjwTi0L2o8vZsbVMASrMgFQiet4wQyTfYX0mFOWsjSoKKcm93zxIYxch802gFlPMIUezV56E0UX@lists.infradead.org X-Gm-Message-State: AOJu0YwCknMDLG9DXMhLTVhGcBmNDksthJserM0IJASrs/zwJ0zo9MRZ 8t6f9t5apuw6KroyaZBMTrpf2ZR5pIH1+pKWb3Typ5T3b0VNrb0f8phOviQ/7C0V+ki589y/XKu sQQ== X-Google-Smtp-Source: AGHT+IEeIIe5Xg98lfzs9VJWYOJ07F6ek87y8El+d0cpjxlVvqWrNUTXfkoh0Qdzc1+S/vFogSJ2meafeF0= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:1569:9ef4:20ab:abf9]) (user=yuzhao job=sendgmr) by 2002:a25:800f:0:b0:e05:6532:166 with SMTP id 3f1490d57ef6-e2bb11cccaemr14439276.1.1729484550141; Sun, 20 Oct 2024 21:22:30 -0700 (PDT) Date: Sun, 20 Oct 2024 22:22:15 -0600 In-Reply-To: <20241021042218.746659-1-yuzhao@google.com> Mime-Version: 1.0 References: <20241021042218.746659-1-yuzhao@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241021042218.746659-4-yuzhao@google.com> Subject: [PATCH v1 3/6] irqchip/gic-v3: support SGI broadcast From: Yu Zhao To: Andrew Morton , Catalin Marinas , Marc Zyngier , Muchun Song , Thomas Gleixner , Will Deacon Cc: Douglas Anderson , Mark Rutland , Nanyong Sun , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yu Zhao X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241020_212232_168273_A1C2F560 X-CRM114-Status: GOOD ( 13.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org GIC v3 and later support SGI broadcast, i.e., the mode that routes interrupts to all PEs in the system excluding the local CPU. Supporting this mode can avoid looping through all the remote CPUs when broadcasting SGIs, especially for systems with 200+ CPUs. The performance improvement can be measured with the rest of this series booted with "hugetlb_free_vmemmap=on irqchip.gicv3_pseudo_nmi=1": cd /sys/kernel/mm/hugepages/ echo 600 >hugepages-1048576kB/nr_hugepages echo 2048kB >hugepages-1048576kB/demote_size perf record -g -- bash -c "echo 600 >hugepages-1048576kB/demote" gic_ipi_send_mask() bash sys time Before: 38.14% 0m10.513s After: 0.20% 0m5.132s Signed-off-by: Yu Zhao --- drivers/irqchip/irq-gic-v3.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index ce87205e3e82..42c39385e1b9 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -1394,9 +1394,20 @@ static void gic_send_sgi(u64 cluster_id, u16 tlist, unsigned int irq) gic_write_sgi1r(val); } +static void gic_broadcast_sgi(unsigned int irq) +{ + u64 val; + + val = BIT(ICC_SGI1R_IRQ_ROUTING_MODE_BIT) | (irq << ICC_SGI1R_SGI_ID_SHIFT); + + pr_devel("CPU %d: broadcasting SGI %u\n", smp_processor_id(), irq); + gic_write_sgi1r(val); +} + static void gic_ipi_send_mask(struct irq_data *d, const struct cpumask *mask) { int cpu; + cpumask_t broadcast; if (WARN_ON(d->hwirq >= 16)) return; @@ -1407,6 +1418,13 @@ static void gic_ipi_send_mask(struct irq_data *d, const struct cpumask *mask) */ dsb(ishst); + cpumask_copy(&broadcast, cpu_present_mask); + cpumask_clear_cpu(smp_processor_id(), &broadcast); + if (cpumask_equal(&broadcast, mask)) { + gic_broadcast_sgi(d->hwirq); + goto done; + } + for_each_cpu(cpu, mask) { u64 cluster_id = MPIDR_TO_SGI_CLUSTER_ID(gic_cpu_to_affinity(cpu)); u16 tlist; @@ -1414,7 +1432,7 @@ static void gic_ipi_send_mask(struct irq_data *d, const struct cpumask *mask) tlist = gic_compute_target_list(&cpu, mask, cluster_id); gic_send_sgi(cluster_id, tlist, d->hwirq); } - +done: /* Force the above writes to ICC_SGI1R_EL1 to be executed */ isb(); }