From patchwork Fri May 18 09:47:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shawn Lin X-Patchwork-Id: 10409393 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1864E602CB for ; Fri, 18 May 2018 10:14:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0786F288F7 for ; Fri, 18 May 2018 10:14:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F0A24288FE; Fri, 18 May 2018 10:14:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, MAILING_LIST_MULTI, RCVD_IN_SORBS_WEB autolearn=no version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E580A288F7 for ; Fri, 18 May 2018 10:14:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=VjcTioVFrWubPiY5KxGNMlrajboQsNczQqgOCgJn6eM=; b=fwe Ts+aCiGbuORY4n2An/n2517aOS+/REtBg5f3wsNTVlP679reu1uac+00a5Gas8TCcudVgwKzVsS6l 5KMtuuI/yV34jQhChgGPuKipcQenE7a50gflACuwqpDas+bdPwjTpWVsathzuZDyit/2XNlH+gWci iXyCfryfsr+JkKpnVoyiauVScMVulb/iiBTylzg5n8wKtQhuEtAduTz/DywTmWnJrseEbt8ON34IS 2ou5YMwJlYbJedYww4ih3enD5BCXDRkj81GWlrXpzpTvEWyvXu9Kj1UjM0vdWAmDFQJy7jBWIIo49 n3nH9U4BdZM+WwcEq1SoLxpe8dSgLRw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fJcOx-0007Ma-In; Fri, 18 May 2018 10:14:19 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fJcGh-0002S2-Av for linux-rockchip@bombadil.infradead.org; Fri, 18 May 2018 10:05:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=LZZibb6ngh1/FAN7jJ5h7hBZu0P8ESOuVLMmnuXoP34=; b=cDYzt2iY2n2Kp2L5wOYsg60/0 mbc7+dHShSeCte4ikxNqZGtDVwQwlA6/1bvlG0ps5Cs5IxkmgatYWK94U2tSuqFpb5dr/eNMGfvoV dMNYA+mBj2ymkdmw+QneQObWnvLHtFaPxlEsB9ANw5FR0vcmP9jHXbT7Ntx5JVyLSkGOshhEIdVFC MX1lHGIE9/Epzpd5L2yIuy6jfKz7Oiyj8d1D64avbGqDyHvstvs6PVXGxl1BLr7lK9yxYcqR2Qck3 6Xw7Ngd/yu0Wt9J2jIJNDzT8/SQmIaxatJmMCde8W+z24UdfCW59eXYb3QI58XEmIDm1XOQ/YuvGi ht6UmLHqA==; Received: from lucky1.263xmail.com ([211.157.147.133]) by merlin.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fJbzy-0002Nv-9C for linux-rockchip@lists.infradead.org; Fri, 18 May 2018 09:48:32 +0000 Received: from shawn.lin?rock-chips.com (unknown [192.168.167.42]) by lucky1.263xmail.com (Postfix) with ESMTP id AF235955B4; Fri, 18 May 2018 17:48:00 +0800 (CST) X-263anti-spam: KSV:0; X-MAIL-GRAY: 1 X-MAIL-DELIVERY: 0 X-KSVirus-check: 0 X-ABS-CHECKED: 4 Received: from localhost.localdomain (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTPA id 7F3F93C1; Fri, 18 May 2018 17:48:01 +0800 (CST) X-IP-DOMAINF: 1 X-RL-SENDER: shawn.lin@rock-chips.com X-FST-TO: marc.zyngier@arm.com X-SENDER-IP: 58.22.7.114 X-LOGIN-NAME: shawn.lin@rock-chips.com X-UNIQUE-TAG: <8f6906dedef259b894dc8dd98df7227c> X-ATTACHMENT-NUM: 0 X-SENDER: lintao@rock-chips.com X-DNS-TYPE: 0 Received: from localhost.localdomain (unknown [58.22.7.114]) by smtp.263.net (Postfix) whith ESMTP id 1273224RX4Z; Fri, 18 May 2018 17:48:02 +0800 (CST) From: Shawn Lin To: Marc Zyngier Subject: [RFC PATCH] irqchip/gic-v3: Try to distribute irq affinity to the less distributed CPU Date: Fri, 18 May 2018 17:47:46 +0800 Message-Id: <1526636866-209210-1-git-send-email-shawn.lin@rock-chips.com> X-Mailer: git-send-email 1.9.1 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180518_054830_805175_80A3F757 X-CRM114-Status: GOOD ( 23.05 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-rockchip@lists.infradead.org, Shawn Lin , Jeffy Chen , Heiko Stuebner MIME-Version: 1.0 Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+patchwork-linux-rockchip=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP gic-v3 seems only suppot distribute hwirq to one CPU in dispite of setting it via /proc/irq/*/smp_affinity. My RK3399 platform has 6 CPUs and I was trying to bind the emmc irq, whose hwirq is 43 and virq is 30, to all cores echo 3f > /proc/irq/30/smp_affinity but the I/O test still shows the irq was fired to CPU0. For really user case, we may try to distribute different hwirqs to different cores, with the hope of distributing to a less irq-binded core as possible. Otherwise, as current implementation, gic-v3 always distribute it to the first masked cpu, which is what cpumask_any_and actually did in practice now on my platform. So I was thinking to record how much hwirqs are distributed to each core and try to pick up the least used one. This patch is rather rough with slightly test on my board. Just for asking advice from wisdom of your. :) Signed-off-by: Shawn Lin --- drivers/irqchip/irq-gic-v3.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 5a67ec0..b838fda 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -65,6 +65,7 @@ struct gic_chip_data { static struct gic_kvm_info gic_v3_kvm_info; static DEFINE_PER_CPU(bool, has_rss); +static DEFINE_PER_CPU(int, bind_irq_nr); #define MPIDR_RS(mpidr) (((mpidr) & 0xF0UL) >> 4) #define gic_data_rdist() (this_cpu_ptr(gic_data.rdists.rdist)) @@ -340,7 +341,7 @@ static u64 gic_mpidr_to_affinity(unsigned long mpidr) MPIDR_AFFINITY_LEVEL(mpidr, 2) << 16 | MPIDR_AFFINITY_LEVEL(mpidr, 1) << 8 | MPIDR_AFFINITY_LEVEL(mpidr, 0)); - + per_cpu(bind_irq_nr, mpidr) += 1; return aff; } @@ -774,15 +775,31 @@ static void gic_smp_init(void) static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, bool force) { - unsigned int cpu; + unsigned int cpu = 0, min_irq_nr_cpu; void __iomem *reg; int enabled; u64 val; + cpumask_t local; + u8 aff; - if (force) + if (force) { cpu = cpumask_first(mask_val); - else - cpu = cpumask_any_and(mask_val, cpu_online_mask); + } else { + cpu = cpumask_and(&local, mask_val, cpu_online_mask); + if (cpu) { + min_irq_nr_cpu = cpumask_first(&local); + for_each_cpu(cpu, &local) { + if (per_cpu(bind_irq_nr, cpu) < + per_cpu(bind_irq_nr, min_irq_nr_cpu)) + min_irq_nr_cpu = cpu; + } + + cpu = min_irq_nr_cpu; + + } else { + cpu = cpumask_any_and(mask_val, cpu_online_mask); + } + } if (cpu >= nr_cpu_ids) return -EINVAL; @@ -796,6 +813,9 @@ static int gic_set_affinity(struct irq_data *d, const struct cpumask *mask_val, gic_mask_irq(d); reg = gic_dist_base(d) + GICD_IROUTER + (gic_irq(d) * 8); + aff = readq_relaxed(reg) & 0xff; //arch_gicv3.h + if (per_cpu(bind_irq_nr, aff)) + per_cpu(bind_irq_nr, aff) -= 1; val = gic_mpidr_to_affinity(cpu_logical_map(cpu)); gic_write_irouter(val, reg);