From patchwork Thu Feb 11 16:20:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 8282361 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 61E30BEEE5 for ; Thu, 11 Feb 2016 16:20:51 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 762A8203AD for ; Thu, 11 Feb 2016 16:20:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C37320398 for ; Thu, 11 Feb 2016 16:20:49 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aTtxE-0000m1-HC; Thu, 11 Feb 2016 16:18:52 +0000 Received: from mail-pa0-x243.google.com ([2607:f8b0:400e:c03::243]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aTtwf-0000Ca-Mu for linux-arm-kernel@lists.infradead.org; Thu, 11 Feb 2016 16:18:20 +0000 Received: by mail-pa0-x243.google.com with SMTP id yy13so2624733pab.1 for ; Thu, 11 Feb 2016 08:17:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gTfc5I9fdK4fhZ9EVBMPTMy4VXfVoKy+8SuyA+Azr6k=; b=B3rl5PxKwezw4thM6XPBzkQPL8aW7CSzRHnP9xFAK1a9KS5RILL1BVEe8J5fQHnALM F4KuQVZ4i4B6aJCeWTIZN9ECQUzdUxe0ew7EJvz08xFr7e7RD37bHM/RBA5PwnGWL3Zl 7/hLEF1qhiRyAotuy1JfMaS5PneanVqUHoZA7aDPLL9dcZQxa93JaDf/RoH1j+vUb4tn O05A9NDl9FpSWtQgUi4lHb+ylXY2ZamE6/+b/yLx+bX36wNEYQeKN8mNbT6SjyAuA910 CjTWyMUvzj/GDbQbNcU8SwjWpeAXxIkhDI7YW/i8Bb5ATgrHO3LnO6My6ZXskgUCj2qA q/FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gTfc5I9fdK4fhZ9EVBMPTMy4VXfVoKy+8SuyA+Azr6k=; b=DHxel5mznkcd/tkqrEVEMvdfHkogGae3MwkTVEy/+jFWQMS+7j3MkdFQAhcs2ZDuSr YHPXmUIg6deZefr7VBJqkIdejKBoPWsbC6BSJUMRN+7Nzs3GhTS0Xni/FOrnUkVeuPpT ls9Eta2TBIFsw2CnlTNZT7q+tGdI1nPYGhk0BvBhQsqSTEHKaWy26pwitnbMRvkRNYiG qAYsLMdiTlwkOb2ZZZVD+QwgFQCwX5eAvsz7lkekya+URT1H6VuqQlnwQL2DU2wDvoVo yTkPOBvZUrSdosu5VP4jWH3YW80JN/PC8V4To/xhzn3IofN1t9J6+cr3QLwME7H/JwwT /dkQ== X-Gm-Message-State: AG10YOTnYJT23RKCeAeDUMMC6osTMA3KpPqzPAtoj12sGpAlqZDVz9JN7SIhz3kydAmnTA== X-Received: by 10.66.119.71 with SMTP id ks7mr68535562pab.151.1455207477045; Thu, 11 Feb 2016 08:17:57 -0800 (PST) Received: from localhost.localdomain ([14.140.2.178]) by smtp.googlemail.com with ESMTPSA id s90sm13355894pfa.49.2016.02.11.08.17.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Feb 2016 08:17:56 -0800 (PST) From: sunil.kovvuri@gmail.com To: netdev@vger.kernel.org Subject: [PATCH v2 3/6] net: thunderx: Assign affinity hints to vf's interrupts Date: Thu, 11 Feb 2016 21:50:23 +0530 Message-Id: <1455207626-35488-4-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1455207626-35488-1-git-send-email-sunil.kovvuri@gmail.com> References: <1455207626-35488-1-git-send-email-sunil.kovvuri@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160211_081818_287649_0B54FA7A X-CRM114-Status: GOOD ( 15.48 ) X-Spam-Score: -1.4 (-) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: sgoutham@cavium.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, robert.richter@caviumnetworks.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, T_DKIM_INVALID, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sunil Goutham This affinity hint can be used by user space irqbalance tool to set preferred CPU mask for irqs registered by this VF. Irqbalance needs to be in 'exact' mode to set irq affinity same as indicated by affinity hint. Signed-off-by: Sunil Goutham --- drivers/net/ethernet/cavium/thunder/nic.h | 1 + drivers/net/ethernet/cavium/thunder/nicvf_main.c | 37 ++++++++++++++++++++- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nic.h b/drivers/net/ethernet/cavium/thunder/nic.h index 8af363a..00cc915 100644 --- a/drivers/net/ethernet/cavium/thunder/nic.h +++ b/drivers/net/ethernet/cavium/thunder/nic.h @@ -309,6 +309,7 @@ struct nicvf { struct msix_entry msix_entries[NIC_VF_MSIX_VECTORS]; char irq_name[NIC_VF_MSIX_VECTORS][20]; bool irq_allocated[NIC_VF_MSIX_VECTORS]; + cpumask_var_t affinity_mask[NIC_VF_MSIX_VECTORS]; /* VF <-> PF mailbox communication */ bool pf_acked; diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c index c6f146c..90ce93e 100644 --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c @@ -899,6 +899,31 @@ static void nicvf_disable_msix(struct nicvf *nic) } } +static void nicvf_set_irq_affinity(struct nicvf *nic) +{ + int vec, cpu; + int irqnum; + + for (vec = 0; vec < nic->num_vec; vec++) { + if (!nic->irq_allocated[vec]) + continue; + + if (!zalloc_cpumask_var(&nic->affinity_mask[vec], GFP_KERNEL)) + return; + /* CQ interrupts */ + if (vec < NICVF_INTR_ID_SQ) + /* Leave CPU0 for RBDR and other interrupts */ + cpu = nicvf_netdev_qidx(nic, vec) + 1; + else + cpu = 0; + + cpumask_set_cpu(cpumask_local_spread(cpu, nic->node), + nic->affinity_mask[vec]); + irqnum = nic->msix_entries[vec].vector; + irq_set_affinity_hint(irqnum, nic->affinity_mask[vec]); + } +} + static int nicvf_register_interrupts(struct nicvf *nic) { int irq, ret = 0; @@ -944,8 +969,13 @@ static int nicvf_register_interrupts(struct nicvf *nic) ret = request_irq(nic->msix_entries[irq].vector, nicvf_qs_err_intr_handler, 0, nic->irq_name[irq], nic); - if (!ret) - nic->irq_allocated[irq] = true; + if (ret) + goto err; + + nic->irq_allocated[irq] = true; + + /* Set IRQ affinities */ + nicvf_set_irq_affinity(nic); err: if (ret) @@ -963,6 +993,9 @@ static void nicvf_unregister_interrupts(struct nicvf *nic) if (!nic->irq_allocated[irq]) continue; + irq_set_affinity_hint(nic->msix_entries[irq].vector, NULL); + free_cpumask_var(nic->affinity_mask[irq]); + if (irq < NICVF_INTR_ID_SQ) free_irq(nic->msix_entries[irq].vector, nic->napi[irq]); else