From patchwork Mon Jul 6 04:15:41 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 6719811 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id C7F2CC05AC for ; Mon, 6 Jul 2015 04:19:13 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 94EAB2069A for ; Mon, 6 Jul 2015 04:19:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 60D1A2069E for ; Mon, 6 Jul 2015 04:19:11 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZBxpp-0008T8-Ir; Mon, 06 Jul 2015 04:16:49 +0000 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZBxpb-0008Pv-K7 for linux-arm-kernel@lists.infradead.org; Mon, 06 Jul 2015 04:16:37 +0000 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.15.0.59/8.15.0.59) with SMTP id t664ETWK031497; Sun, 5 Jul 2015 21:16:00 -0700 Received: from sc-owa03.marvell.com ([199.233.58.149]) by mx0b-0016f401.pphosted.com with ESMTP id 1vef0d393t-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Sun, 05 Jul 2015 21:16:00 -0700 Received: from maili.marvell.com (10.93.76.83) by SC-OWA03.marvell.com (10.93.76.24) with Microsoft SMTP Server id 8.3.327.1; Sun, 5 Jul 2015 21:15:59 -0700 Received: from xhacker.marvell.com (unknown [10.37.135.134]) by maili.marvell.com (Postfix) with ESMTP id 6DCA93F703F; Sun, 5 Jul 2015 21:15:58 -0700 (PDT) From: Jisheng Zhang To: , , Subject: [PATCH v3 2/2] irqchip: dw-apb-ictl: add irq_set_affinity support Date: Mon, 6 Jul 2015 12:15:41 +0800 Message-ID: <1436156141-3674-3-git-send-email-jszhang@marvell.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1436156141-3674-1-git-send-email-jszhang@marvell.com> References: <1436156141-3674-1-git-send-email-jszhang@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2015-07-06_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 kscore.is_bulkscore=0 kscore.compositescore=1 compositescore=0.9 suspectscore=2 malwarescore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 rbsscore=0.9 spamscore=0 urlsuspectscore=0.9 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1506180000 definitions=main-1507060076 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150705_211635_807386_BCB76D76 X-CRM114-Status: GOOD ( 16.35 ) X-Spam-Score: -2.6 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jisheng Zhang , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Marvell Berlin SoCs, the cpu's local timer is shutdown when the cpu goes to a deep idle state, then the timer framework will be notified to use a broadcast timer instead. In our case, the broadcast timer uses dw-apb-ictl as interrupt chip. This patch adds irq_set_affinity support so that the going to deep idle state cpu can set the interrupt affinity of the broadcast interrupt to avoid unnecessary wakeups and IPIs. NOTE: We achieved this by changing the parent interrupt affinity of a chained interrupt, so it migrates every interrupt on the child interrupt controller which isn't a good thing to do as pointed out by Russell King. Thomas pointed out "we should at least make that an opt-in behaviour and not enabled by default", this patch adds a device tree option for this purpose. If "migrates every interrupt on the child interrupt controller" doesn't matter, turning on the option would add irq_set_affinity support which might be useful in some cases (EG: can save reduce power consumption on Marvell Berlin SoCs). A simple test: ~ # rm /tmp/test.sh ~ # cat > /tmp/test.sh cat /proc/interrupts for i in `seq 10` ; do sleep $i; done cat /proc/interrupts ~ # chmod +x /tmp/test.sh ~ # taskset 0x2 /tmp/test.sh without the patch: CPU0 CPU1 27: 115 36 GIC 27 arch_timer 45: 62 0 GIC 45 mmc0 160: 88 0 interrupt-controller 8 timer 227: 0 0 interrupt-controller 4 f7e81400.i2c 228: 0 0 interrupt-controller 5 f7e81800.i2c 229: 0 0 interrupt-controller 7 dw_spi65535 230: 0 0 interrupt-controller 21 f7e84000.i2c 231: 0 0 interrupt-controller 20 f7e84800.i2c 265: 445 0 interrupt-controller 8 serial IPI0: 0 0 CPU wakeup interrupts IPI1: 0 11 Timer broadcast interrupts IPI2: 56 104 Rescheduling interrupts IPI3: 0 0 Function call interrupts IPI4: 0 4 Single function call interrupts IPI5: 0 0 CPU stop interrupts IPI6: 25 27 IRQ work interrupts IPI7: 0 0 completion interrupts IPI8: 0 0 CPU backtrace Err: 0 CPU0 CPU1 27: 115 38 GIC 27 arch_timer 45: 62 0 GIC 45 mmc0 160: 160 0 interrupt-controller 8 timer 227: 0 0 interrupt-controller 4 f7e81400.i2c 228: 0 0 interrupt-controller 5 f7e81800.i2c 229: 0 0 interrupt-controller 7 dw_spi65535 230: 0 0 interrupt-controller 21 f7e84000.i2c 231: 0 0 interrupt-controller 20 f7e84800.i2c 265: 514 0 interrupt-controller 8 serial IPI0: 0 0 CPU wakeup interrupts IPI1: 0 83 Timer broadcast interrupts IPI2: 56 104 Rescheduling interrupts IPI3: 0 0 Function call interrupts IPI4: 0 4 Single function call interrupts IPI5: 0 0 CPU stop interrupts IPI6: 25 46 IRQ work interrupts IPI7: 0 0 completion interrupts IPI8: 0 0 CPU backtrace Err: 0 cpu0 get 160-88=72 timer interrupts, CPU1 got 83-11=72 broadcast timer IPIs. So, overall system got 72+72=144 wake ups and 72 broadcast timer IPIs With the patch: CPU0 CPU1 27: 107 37 GIC 27 arch_timer 45: 62 0 GIC 45 mmc0 160: 66 7 interrupt-controller 8 timer 227: 0 0 interrupt-controller 4 f7e81400.i2c 228: 0 0 interrupt-controller 5 f7e81800.i2c 229: 0 0 interrupt-controller 7 dw_spi65535 230: 0 0 interrupt-controller 21 f7e84000.i2c 231: 0 0 interrupt-controller 20 f7e84800.i2c 265: 311 0 interrupt-controller 8 serial IPI0: 0 0 CPU wakeup interrupts IPI1: 2 4 Timer broadcast interrupts IPI2: 58 100 Rescheduling interrupts IPI3: 0 0 Function call interrupts IPI4: 0 4 Single function call interrupts IPI5: 0 0 CPU stop interrupts IPI6: 21 24 IRQ work interrupts IPI7: 0 0 completion interrupts IPI8: 0 0 CPU backtrace Err: 0 CPU0 CPU1 27: 107 39 GIC 27 arch_timer 45: 62 0 GIC 45 mmc0 160: 69 75 interrupt-controller 8 timer 227: 0 0 interrupt-controller 4 f7e81400.i2c 228: 0 0 interrupt-controller 5 f7e81800.i2c 229: 0 0 interrupt-controller 7 dw_spi65535 230: 0 0 interrupt-controller 21 f7e84000.i2c 231: 0 0 interrupt-controller 20 f7e84800.i2c 265: 380 0 interrupt-controller 8 serial IPI0: 0 0 CPU wakeup interrupts IPI1: 3 6 Timer broadcast interrupts IPI2: 60 100 Rescheduling interrupts IPI3: 0 0 Function call interrupts IPI4: 0 4 Single function call interrupts IPI5: 0 0 CPU stop interrupts IPI6: 21 45 IRQ work interrupts IPI7: 0 0 completion interrupts IPI8: 0 0 CPU backtrace Err: 0 cpu0 got 69-66=3, cpu1 got 75-7=68 timer interrupts. cpu0 got 3-2=1 broadcast timer IPIs, cpu1 got 6-4=2 broadcast timer IPIs. So, overall system got 3+68+1+2=74 wakeups and 1+2=3 broadcast timer IPIs. This patch removes 50% wakeups and almost 100% broadcast timer IPIs! Signed-off-by: Jisheng Zhang --- .../interrupt-controller/snps,dw-apb-ictl.txt | 5 +++++ drivers/irqchip/irq-dw-apb-ictl.c | 25 ++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt b/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt index 4929117..1dd1786 100644 --- a/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt +++ b/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt @@ -13,6 +13,11 @@ Required properties: - interrupts: interrupt reference to primary interrupt controller - interrupt-parent: (optional) reference specific primary interrupt controller +Optional properties: +- irq-set-affinity: If present, the ictl will be assumed to be able to set + the affinity of every interrupt on the ictl to the same one by changing the + parent interrupt affinity. + The interrupt sources map to the corresponding bits in the interrupt registers, i.e. - 0 maps to bit 0 of low interrupts, diff --git a/drivers/irqchip/irq-dw-apb-ictl.c b/drivers/irqchip/irq-dw-apb-ictl.c index 8bef7f7..4be88cb 100644 --- a/drivers/irqchip/irq-dw-apb-ictl.c +++ b/drivers/irqchip/irq-dw-apb-ictl.c @@ -29,6 +29,7 @@ struct dw_apb_ictl_priv { struct irq_domain *domain; + unsigned int parent_irq; }; static void dw_apb_ictl_handler(unsigned int irq, struct irq_desc *desc) @@ -56,6 +57,21 @@ static void dw_apb_ictl_handler(unsigned int irq, struct irq_desc *desc) chained_irq_exit(chip, desc); } +static int dw_apb_ictl_set_affinity(struct irq_data *d, + const struct cpumask *mask_val, + bool force) +{ + struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d); + struct dw_apb_ictl_priv *priv = gc->private; + struct irq_chip *chip = irq_get_chip(priv->parent_irq); + struct irq_data *data = irq_get_irq_data(priv->parent_irq); + + if (chip && chip->irq_set_affinity) + return chip->irq_set_affinity(data, mask_val, force); + else + return -EINVAL; +} + #ifdef CONFIG_PM static void dw_apb_ictl_resume(struct irq_data *d) { @@ -82,6 +98,7 @@ static int __init dw_apb_ictl_init(struct device_node *np, void __iomem *iobase; int ret, nrirqs, irq; u32 reg; + bool can_set_affinity = of_property_read_bool(np, "irq-set-affinity"); priv = kzalloc(sizeof(*priv), GFP_KERNEL); if (!priv) @@ -95,6 +112,8 @@ static int __init dw_apb_ictl_init(struct device_node *np, goto err_free; } + priv->parent_irq = irq; + ret = of_address_to_resource(np, 0, &r); if (ret) { pr_err("%s: unable to get resource\n", np->full_name); @@ -160,6 +179,9 @@ static int __init dw_apb_ictl_init(struct device_node *np, gc->chip_types[0].chip.irq_mask = irq_gc_mask_set_bit; gc->chip_types[0].chip.irq_unmask = irq_gc_mask_clr_bit; gc->chip_types[0].chip.irq_resume = dw_apb_ictl_resume; + if (can_set_affinity) + gc->chip_types[0].chip.irq_set_affinity = + dw_apb_ictl_set_affinity; if (nrirqs > 32) { gc->chip_types[1].regs.mask = APB_INT_MASK_H; @@ -167,6 +189,9 @@ static int __init dw_apb_ictl_init(struct device_node *np, gc->chip_types[1].chip.irq_mask = irq_gc_mask_set_bit; gc->chip_types[1].chip.irq_unmask = irq_gc_mask_clr_bit; gc->chip_types[1].chip.irq_resume = dw_apb_ictl_resume; + if (can_set_affinity) + gc->chip_types[1].chip.irq_set_affinity = + dw_apb_ictl_set_affinity; } irq_set_handler_data(irq, gc);