From patchwork Fri Jul 15 12:58:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sunil Kovvuri X-Patchwork-Id: 9232009 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2107360865 for ; Fri, 15 Jul 2016 13:02:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1043027F86 for ; Fri, 15 Jul 2016 13:02:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 03B7027F8F; Fri, 15 Jul 2016 13:02:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 692D42832D for ; Fri, 15 Jul 2016 13:02:37 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bO2jg-0001tn-6i; Fri, 15 Jul 2016 13:00:56 +0000 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bO2ie-0008G0-Ef for linux-arm-kernel@lists.infradead.org; Fri, 15 Jul 2016 12:59:59 +0000 Received: by mail-pf0-x244.google.com with SMTP id i6so5993124pfe.0 for ; Fri, 15 Jul 2016 05:59:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2WL0OvuMqiMfcdEa55tGAlCy6nSpN3EWLWJQGkljv1E=; b=vXb+Jzejf3apOg7nwqx6A5jK/FSRoPi4hpZh6GaMph3TrwJ3FutPvYDIP+jofhfMwy 2Aa2VYQ/4m3DKNIyOH4vTzq+ZXSOMUJGFbJMHPwfWsMVygZxpGJibLmwvMjLVSHW9e/l bEunCVS5w8VI3SvNA23gjfrKHUQAxSscUDUHK674uwfaw4EunhRflZ3o34Mw1s5RVAG7 9ZSqvmBy0bFHLEIbA8zxp037+hXjaEp9FH2ht65J+HMv9iGAwwzPGgW4MKAG9BF569xu Fb+pBFH2DXaYXWq9RsgyhODZAeH7Ma8kaZQJMjcQaS9WlKXioLFUu2ct7aqJKIx1s6hk k7AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2WL0OvuMqiMfcdEa55tGAlCy6nSpN3EWLWJQGkljv1E=; b=SMN3CInKkf/MEFb3ZXeXesx2IJLDTxShYiZ86HNfVBv7rs9LgYbJMCW2ZuEJXmpcEi 9pFNuq07QBMpmbClzlrqboOYc0PSq0A7JZXt2xK0yX8V7RbOpN9uACwCXeZcspzk6xJl VU+pyj/s9RDYyFVHDIA7Ty5U1JhWZvYUnGoCH0juXB+9Ul6oyqsO6O+sgZ1Lt76pEvUY S4KAiMFoaS7v0HKYpq+IQnLDQoTTj6L63c3BTGQVQQNMRiRJy3K4WZ7AhObWsJueF2fF YnXpqm3qzAR2w4qlFXOFyR7ZD4WKF8cV6y+x058AE1j/CfP8UZXMeeEtgf4e21M93dUv uZpg== X-Gm-Message-State: ALyK8tKjwWr6uOMbdE+SlJhKCcsTOnpJB1LsL0zYNsBsNRaWK+CfvpI7IqF5DB23u3cujQ== X-Received: by 10.98.111.138 with SMTP id k132mr21259852pfc.105.1468587571508; Fri, 15 Jul 2016 05:59:31 -0700 (PDT) Received: from machine421.in.caveonetworks.com ([14.140.2.178]) by smtp.googlemail.com with ESMTPSA id t80sm5365880pfj.38.2016.07.15.05.59.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 15 Jul 2016 05:59:30 -0700 (PDT) From: sunil.kovvuri@gmail.com To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 06/15] net: thunderx: Enable mailbox interrupts on 81xx/83xx Date: Fri, 15 Jul 2016 18:28:11 +0530 Message-Id: <1468587500-32103-7-git-send-email-sunil.kovvuri@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1468587500-32103-1-git-send-email-sunil.kovvuri@gmail.com> References: <1468587500-32103-1-git-send-email-sunil.kovvuri@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160715_055952_728183_D201B6E7 X-CRM114-Status: GOOD ( 18.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sunil Goutham MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sunil Goutham 88xx has 128 VFs, 81xx has 8 VFs and 83xx will have 32VFs. Made changes to PF driver such that mailbox interrupt enable registers are configuired based on number of VFs HW supports. Also cleanedup mailbox irq handler registration code. Signed-off-by: Sunil Goutham --- drivers/net/ethernet/cavium/thunder/nic_main.c | 88 +++++++++++++++----------- 1 file changed, 50 insertions(+), 38 deletions(-) diff --git a/drivers/net/ethernet/cavium/thunder/nic_main.c b/drivers/net/ethernet/cavium/thunder/nic_main.c index 3f52b36..955c522 100644 --- a/drivers/net/ethernet/cavium/thunder/nic_main.c +++ b/drivers/net/ethernet/cavium/thunder/nic_main.c @@ -66,8 +66,9 @@ struct nicpf { /* MSI-X */ bool msix_enabled; u8 num_vec; - struct msix_entry msix_entries[NIC_PF_MSIX_VECTORS]; + struct msix_entry *msix_entries; bool irq_allocated[NIC_PF_MSIX_VECTORS]; + char irq_name[NIC_PF_MSIX_VECTORS][20]; }; /* Supported devices */ @@ -105,9 +106,22 @@ static u64 nic_reg_read(struct nicpf *nic, u64 offset) /* PF -> VF mailbox communication APIs */ static void nic_enable_mbx_intr(struct nicpf *nic) { - /* Enable mailbox interrupt for all 128 VFs */ - nic_reg_write(nic, NIC_PF_MAILBOX_ENA_W1S, ~0ull); - nic_reg_write(nic, NIC_PF_MAILBOX_ENA_W1S + sizeof(u64), ~0ull); + int vf_cnt = pci_sriov_get_totalvfs(nic->pdev); + +#define INTR_MASK(vfs) ((vfs < 64) ? (BIT_ULL(vfs) - 1) : (~0ull)) + + /* Clear it, to avoid spurious interrupts (if any) */ + nic_reg_write(nic, NIC_PF_MAILBOX_INT, INTR_MASK(vf_cnt)); + + /* Enable mailbox interrupt for all VFs */ + nic_reg_write(nic, NIC_PF_MAILBOX_ENA_W1S, INTR_MASK(vf_cnt)); + /* One mailbox intr enable reg per 64 VFs */ + if (vf_cnt > 64) { + nic_reg_write(nic, NIC_PF_MAILBOX_INT + sizeof(u64), + INTR_MASK(vf_cnt - 64)); + nic_reg_write(nic, NIC_PF_MAILBOX_ENA_W1S + sizeof(u64), + INTR_MASK(vf_cnt - 64)); + } } static void nic_clear_mbx_intr(struct nicpf *nic, int vf, int mbx_reg) @@ -894,11 +908,18 @@ unlock: nic->mbx_lock[vf] = false; } -static void nic_mbx_intr_handler (struct nicpf *nic, int mbx) +static irqreturn_t nic_mbx_intr_handler(int irq, void *nic_irq) { + struct nicpf *nic = (struct nicpf *)nic_irq; + int mbx; u64 intr; u8 vf, vf_per_mbx_reg = 64; + if (irq == nic->msix_entries[NIC_PF_INTR_ID_MBOX0].vector) + mbx = 0; + else + mbx = 1; + intr = nic_reg_read(nic, NIC_PF_MAILBOX_INT + (mbx << 3)); dev_dbg(&nic->pdev->dev, "PF interrupt Mbox%d 0x%llx\n", mbx, intr); for (vf = 0; vf < vf_per_mbx_reg; vf++) { @@ -910,23 +931,6 @@ static void nic_mbx_intr_handler (struct nicpf *nic, int mbx) nic_clear_mbx_intr(nic, vf, mbx); } } -} - -static irqreturn_t nic_mbx0_intr_handler (int irq, void *nic_irq) -{ - struct nicpf *nic = (struct nicpf *)nic_irq; - - nic_mbx_intr_handler(nic, 0); - - return IRQ_HANDLED; -} - -static irqreturn_t nic_mbx1_intr_handler (int irq, void *nic_irq) -{ - struct nicpf *nic = (struct nicpf *)nic_irq; - - nic_mbx_intr_handler(nic, 1); - return IRQ_HANDLED; } @@ -934,7 +938,13 @@ static int nic_enable_msix(struct nicpf *nic) { int i, ret; - nic->num_vec = NIC_PF_MSIX_VECTORS; + nic->num_vec = pci_msix_vec_count(nic->pdev); + + nic->msix_entries = kmalloc_array(nic->num_vec, + sizeof(struct msix_entry), + GFP_KERNEL); + if (!nic->msix_entries) + return -ENOMEM; for (i = 0; i < nic->num_vec; i++) nic->msix_entries[i].entry = i; @@ -942,8 +952,9 @@ static int nic_enable_msix(struct nicpf *nic) ret = pci_enable_msix(nic->pdev, nic->msix_entries, nic->num_vec); if (ret) { dev_err(&nic->pdev->dev, - "Request for #%d msix vectors failed\n", - nic->num_vec); + "Request for #%d msix vectors failed, returned %d\n", + nic->num_vec, ret); + kfree(nic->msix_entries); return ret; } @@ -955,6 +966,7 @@ static void nic_disable_msix(struct nicpf *nic) { if (nic->msix_enabled) { pci_disable_msix(nic->pdev); + kfree(nic->msix_entries); nic->msix_enabled = 0; nic->num_vec = 0; } @@ -973,27 +985,26 @@ static void nic_free_all_interrupts(struct nicpf *nic) static int nic_register_interrupts(struct nicpf *nic) { - int ret; + int i, ret; /* Enable MSI-X */ ret = nic_enable_msix(nic); if (ret) return ret; - /* Register mailbox interrupt handlers */ - ret = request_irq(nic->msix_entries[NIC_PF_INTR_ID_MBOX0].vector, - nic_mbx0_intr_handler, 0, "NIC Mbox0", nic); - if (ret) - goto fail; + /* Register mailbox interrupt handler */ + for (i = NIC_PF_INTR_ID_MBOX0; i < nic->num_vec; i++) { + sprintf(nic->irq_name[i], + "NICPF Mbox%d", (i - NIC_PF_INTR_ID_MBOX0)); - nic->irq_allocated[NIC_PF_INTR_ID_MBOX0] = true; + ret = request_irq(nic->msix_entries[i].vector, + nic_mbx_intr_handler, 0, + nic->irq_name[i], nic); + if (ret) + goto fail; - ret = request_irq(nic->msix_entries[NIC_PF_INTR_ID_MBOX1].vector, - nic_mbx1_intr_handler, 0, "NIC Mbox1", nic); - if (ret) - goto fail; - - nic->irq_allocated[NIC_PF_INTR_ID_MBOX1] = true; + nic->irq_allocated[i] = true; + } /* Enable mailbox interrupt */ nic_enable_mbx_intr(nic); @@ -1002,6 +1013,7 @@ static int nic_register_interrupts(struct nicpf *nic) fail: dev_err(&nic->pdev->dev, "Request irq failed\n"); nic_free_all_interrupts(nic); + nic_disable_msix(nic); return ret; }