From patchwork Thu Sep 20 15:02:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10607915 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FEC514DA for ; Thu, 20 Sep 2018 15:03:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50E5E29242 for ; Thu, 20 Sep 2018 15:03:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4EB1129263; Thu, 20 Sep 2018 15:03:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E703429242 for ; Thu, 20 Sep 2018 15:03:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731305AbeITUrC (ORCPT ); Thu, 20 Sep 2018 16:47:02 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:45029 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730758AbeITUrB (ORCPT ); Thu, 20 Sep 2018 16:47:01 -0400 Received: by mail-pl1-f196.google.com with SMTP id ba4-v6so4475938plb.11 for ; Thu, 20 Sep 2018 08:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xolYOnnkUmGu8WpUGGN47W+wxGLhP0GlE44v2okXiIQ=; b=cTzVIjmwJ0ms2ghozP1isf1EtHO7/NvltGGML2VFUXqAJwez+vT/ZEwFjv2RVzkb6m Qgu2ntUN/hSI6kQtBgxkMw32pMrDqgrc4v4Ogene6uu7lkfpFSTbtJsQnKICRqUexxcP 4hs2xDBLjtdvtInFTmvNDeOReaRWQb3W6WYJQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xolYOnnkUmGu8WpUGGN47W+wxGLhP0GlE44v2okXiIQ=; b=k9jDDcc0cvIkpNpW8ThCSLP7SSXCmkTCCxQWF561ot065wmqKaepww/i4D4/KrR09A r9O9vrBvSWCiaWlxO41GHNUbXagMbs9JfrvYJJb+AS76A//nU+TZeL/nxa3mA8Fu098P qyBDwg9ArfM+miBNZaR6QEpbWnQs25UAy7MRgik9QZcTfWAEPI2hx61zYNzCFYsBt8hE JxWwgtiVW0EihM0MmCRthK9JJbK1w/ImY3oD14LJlCerRiKGixrm/CTT0EM+TGs62FlZ HKgZ4tdv5Mi6tZRLj+ZJp5nrLk5iCzFLZQUbaD78z2V6XHhsMgcXfOil9smu4we3Lnrj kCfg== X-Gm-Message-State: APzg51BCUmk2mTY0cydMh37bWO8nrPlvApLK1Vrl/3SyA4TG48x9C5KV TI6/Iv90hk8CA7LfzWpdhQLuaxLefRo= X-Google-Smtp-Source: ANB0VdZhh02MoefAn8GSe6IXEaXpcnNjXfDJ4XJXqjn+tUmbI2UiDUusjyWbGCiPxVlaaVKcu07E5Q== X-Received: by 2002:a17:902:a413:: with SMTP id p19-v6mr39791583plq.220.1537455787919; Thu, 20 Sep 2018 08:03:07 -0700 (PDT) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 9-v6sm66271043pfc.20.2018.09.20.08.03.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 08:03:06 -0700 (PDT) From: Selvin Xavier To: linux-rdma@vger.kernel.org Cc: dledford@redhat.com, jgg@mellanox.com, Selvin Xavier Subject: [PATCH for-rc 1/4] RDMA/bnxt_re: Avoid null check after accessing the pointer Date: Thu, 20 Sep 2018 08:02:32 -0700 Message-Id: <1537455755-22808-2-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> References: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is reported by smatch check. rcfw->creq_bar_reg_iomem is accessed in bnxt_qplib_rcfw_stop_irq and this variable check afterwards doesn't make sense. Also, rcfw->creq_bar_reg_iomem will never be NULL. So Removing this check. Reported-by: Dan Carpenter Fixes: 6e04b1035689 ("RDMA/bnxt_re: Fix broken RoCE driver due to recent L2 driver changes") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index 2852d35..eed9d56 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -605,13 +605,8 @@ void bnxt_qplib_disable_rcfw_channel(struct bnxt_qplib_rcfw *rcfw) bnxt_qplib_rcfw_stop_irq(rcfw, true); - if (rcfw->cmdq_bar_reg_iomem) - iounmap(rcfw->cmdq_bar_reg_iomem); - rcfw->cmdq_bar_reg_iomem = NULL; - - if (rcfw->creq_bar_reg_iomem) - iounmap(rcfw->creq_bar_reg_iomem); - rcfw->creq_bar_reg_iomem = NULL; + iounmap(rcfw->cmdq_bar_reg_iomem); + iounmap(rcfw->creq_bar_reg_iomem); indx = find_first_bit(rcfw->cmdq_bitmap, rcfw->bmap_size); if (indx != rcfw->bmap_size) @@ -620,6 +615,8 @@ void bnxt_qplib_disable_rcfw_channel(struct bnxt_qplib_rcfw *rcfw) kfree(rcfw->cmdq_bitmap); rcfw->bmap_size = 0; + rcfw->cmdq_bar_reg_iomem = NULL; + rcfw->creq_bar_reg_iomem = NULL; rcfw->aeq_handler = NULL; rcfw->vector = 0; } @@ -705,6 +702,8 @@ int bnxt_qplib_enable_rcfw_channel(struct pci_dev *pdev, dev_err(&rcfw->pdev->dev, "QPLIB: CREQ BAR region %d mapping failed", rcfw->creq_bar_reg); + iounmap(rcfw->cmdq_bar_reg_iomem); + rcfw->cmdq_bar_reg_iomem = NULL; return -ENOMEM; } rcfw->creq_qp_event_processed = 0; From patchwork Thu Sep 20 15:02:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10607919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D140161F for ; Thu, 20 Sep 2018 15:03:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4D75428CDC for ; Thu, 20 Sep 2018 15:03:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4B99229328; Thu, 20 Sep 2018 15:03:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0447B28CDC for ; Thu, 20 Sep 2018 15:03:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733148AbeITUrF (ORCPT ); Thu, 20 Sep 2018 16:47:05 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:39124 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730758AbeITUrE (ORCPT ); Thu, 20 Sep 2018 16:47:04 -0400 Received: by mail-pf1-f195.google.com with SMTP id j8-v6so4505898pff.6 for ; Thu, 20 Sep 2018 08:03:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Ftd0YhfWarv5uygAM939sPKYaAeZFtriz2iYeb3LhcY=; b=G8XKq7ewtzPKdJKIHl+b1EF2rWwhx2Cce3/dvoYjH7RMPG/b1iF59NlBdz/u8afLmi ZL3otRNTRFq70pTSpoa76ZXjgffKRFdPcO5cHoMFEigRAPtmu7jTq3GMzxag3BSiUuE3 /8oBMiehf0bjKjUaDgdo/DtZp/7mwF29W6uek= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Ftd0YhfWarv5uygAM939sPKYaAeZFtriz2iYeb3LhcY=; b=S0TRBCy73bSVFTUtnstqSCznEExVtKVlc2HpjiZdjLp92OawVatMFoazrEG6hhnn/F ejyp2GxzlVvsxOinVY3spxfaV+pul7+TDnegNh1M4QmLUe5pG//EPknWRoBkEv8fyO5k 6FdinqqUk7Pc9qbQfQciRKcHz8EW5WriWpMP+qvG9KqMgwAKc7kRsP1c5WdkYqVLZcGx X0XqyXd76Hbl2beGHmgJn7zXQwmIifmY9ymadgIuH6OT5ifahYEEpWWloRDIv+XkXNv7 JyZgkzlsk9U8tSMJ/COnXAVjm00ncVSvmrEH/d4hOttGplEyIH7sTKr2kBWpGl5n1WBZ 5bwA== X-Gm-Message-State: APzg51AxWQOoac1aTX47p6EJf5Y3WV8qXbXsidgrSlwDs2BsbdUbj4DD 44o0K3F5s/hv98BGVHPFYBsPaGGYyPQ= X-Google-Smtp-Source: ANB0VdZqng6GZNuNsDgqd5EaSEApjT4z/yP7nfxtf7+S68SmmvYLOJ7goTQqJz3xnk8NA3ifeG7y5g== X-Received: by 2002:a62:1815:: with SMTP id 21-v6mr41344792pfy.227.1537455790809; Thu, 20 Sep 2018 08:03:10 -0700 (PDT) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 9-v6sm66271043pfc.20.2018.09.20.08.03.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 08:03:09 -0700 (PDT) From: Selvin Xavier To: linux-rdma@vger.kernel.org Cc: dledford@redhat.com, jgg@mellanox.com, Selvin Xavier Subject: [PATCH for-rc 2/4] RDMA/bnxt_re: Avoid accessing nq->bar_reg_iomem in failure case Date: Thu, 20 Sep 2018 08:02:33 -0700 Message-Id: <1537455755-22808-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> References: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In the failure path, nq->bar_reg_iomem gets accessed without initializing. Avoid this by calling the bnxt_qplib_nq_stop_irq only if the initialization is complete. Reported-by: Dan Carpenter Fixes: 1ac5a4047975 ("RDMA/bnxt_re: Add bnxt_re RoCE driver") Fixes: 6e04b1035689 ("RDMA/bnxt_re: Fix broken RoCE driver due to recent L2 driver changes") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c index 6ad0d46..249efa0 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c @@ -360,7 +360,8 @@ void bnxt_qplib_disable_nq(struct bnxt_qplib_nq *nq) } /* Make sure the HW is stopped! */ - bnxt_qplib_nq_stop_irq(nq, true); + if (nq->requested) + bnxt_qplib_nq_stop_irq(nq, true); if (nq->bar_reg_iomem) iounmap(nq->bar_reg_iomem); From patchwork Thu Sep 20 15:02:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10607921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC64B1390 for ; Thu, 20 Sep 2018 15:03:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B5E0128DD2 for ; Thu, 20 Sep 2018 15:03:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1DAF28F67; Thu, 20 Sep 2018 15:03:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8A81F2927E for ; Thu, 20 Sep 2018 15:03:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733153AbeITUrI (ORCPT ); Thu, 20 Sep 2018 16:47:08 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:42573 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730758AbeITUrI (ORCPT ); Thu, 20 Sep 2018 16:47:08 -0400 Received: by mail-pf1-f194.google.com with SMTP id l9-v6so4501719pff.9 for ; Thu, 20 Sep 2018 08:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=h3OQCPPddQWCtEsFwNKV9TfRqxnvcLas2Qc/xzN8tck=; b=YI4ViATMuFlIU2ZGuQnCv6P/ZXkb8flzH9BcsP8z4Ldp6GPdXAEnhd5NTYFgAt0cY9 EV3MdlGNK6qMIbGreUXwksobmNOm+8dsHcQ5eu65rZUBjLlXMRLXW8Z7gzZTCHmZmHhs SiKnkK35rLBkmWFWyM18HVx9UWnySL5ZQzRRk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=h3OQCPPddQWCtEsFwNKV9TfRqxnvcLas2Qc/xzN8tck=; b=l+SpHWTrJH+SmiwX1M1DU/6mkkFJyAx4nZdD+CZUUxrm474ofSxXImN2lZqKjhsnDQ +kkDPi7fC0v/zA3TMCRMg+00PzNqWOrF9RWMZ9qV/C3UQgocNVJJClcj5aH9W1pFLGbM awpkPjpuNbwAvqsfdliwHGtEGYQy5X/MmlYR3Ii9hIdwH6Zv8BRytNMo9XzZa/yAXCw7 WQV2PEQXdIEd6dOs8aw+TqCmk+VuxxscXh6FGjlqe4xfrGh/QoYoYQwZP0aJJw4w4/Qz 6I7lQ+IKC1lECo1K07VWXUPRUL/z34jIhvjsOograpaJdZKAdr8G16XaoBIw44vrKvdf 75kg== X-Gm-Message-State: APzg51BbP1Hy6OxWi/Gr5zWhcmGOEKL+/155bTenpxZC76TNrn30T4Dm 8GE0SfMIe2rDaaCFSDHfUDB1uBtljpo= X-Google-Smtp-Source: ANB0Vdb3+LKDh3VjDdDbJgtBae+W2Nty0VD9ztXePT6JDtisXJIpJhnmZB3oJs+/gAqiuD9vuSruBQ== X-Received: by 2002:a62:c4da:: with SMTP id h87-v6mr41922646pfk.39.1537455793725; Thu, 20 Sep 2018 08:03:13 -0700 (PDT) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 9-v6sm66271043pfc.20.2018.09.20.08.03.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 08:03:12 -0700 (PDT) From: Selvin Xavier To: linux-rdma@vger.kernel.org Cc: dledford@redhat.com, jgg@mellanox.com, Selvin Xavier Subject: [PATCH for-rc 3/4] RDMA/bnxt_re: Avoid resource leak in case the NQ registration fails Date: Thu, 20 Sep 2018 08:02:34 -0700 Message-Id: <1537455755-22808-4-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> References: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In case the NQ alloc/enable fails, free up the already allocated/enabled NQ before reporting failure. Also, track the alloc/enable using proper state checking. Fixes: 6a5df91baf25 ("RDMA/bnxt_re: Allocate multiple notification queues") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/bnxt_re.h | 2 ++ drivers/infiniband/hw/bnxt_re/main.c | 31 ++++++++++++++++++++++--------- 2 files changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/bnxt_re.h b/drivers/infiniband/hw/bnxt_re/bnxt_re.h index 96f7689..802942a 100644 --- a/drivers/infiniband/hw/bnxt_re/bnxt_re.h +++ b/drivers/infiniband/hw/bnxt_re/bnxt_re.h @@ -120,6 +120,8 @@ struct bnxt_re_dev { #define BNXT_RE_FLAG_HAVE_L2_REF 3 #define BNXT_RE_FLAG_RCFW_CHANNEL_EN 4 #define BNXT_RE_FLAG_QOS_WORK_REG 5 +#define BNXT_RE_FLAG_RESOURCES_ALLOCATED 7 +#define BNXT_RE_FLAG_RESOURCES_INITIALIZED 8 #define BNXT_RE_FLAG_ISSUE_ROCE_STATS 29 struct net_device *netdev; unsigned int version, major, minor; diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 20b9f31..4628a93 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -896,10 +896,8 @@ static void bnxt_re_cleanup_res(struct bnxt_re_dev *rdev) { int i; - if (rdev->nq[0].hwq.max_elements) { - for (i = 1; i < rdev->num_msix; i++) - bnxt_qplib_disable_nq(&rdev->nq[i - 1]); - } + for (i = 1; i < rdev->num_msix; i++) + bnxt_qplib_disable_nq(&rdev->nq[i - 1]); if (rdev->qplib_res.rcfw) bnxt_qplib_cleanup_res(&rdev->qplib_res); @@ -908,6 +906,7 @@ static void bnxt_re_cleanup_res(struct bnxt_re_dev *rdev) static int bnxt_re_init_res(struct bnxt_re_dev *rdev) { int rc = 0, i; + int num_vec_enabled = 0; bnxt_qplib_init_res(&rdev->qplib_res); @@ -923,9 +922,13 @@ static int bnxt_re_init_res(struct bnxt_re_dev *rdev) "Failed to enable NQ with rc = 0x%x", rc); goto fail; } + num_vec_enabled++; } return 0; fail: + for (i = num_vec_enabled; i >= 0; i--) + bnxt_qplib_disable_nq(&rdev->nq[i]); + return rc; } @@ -957,6 +960,7 @@ static void bnxt_re_free_res(struct bnxt_re_dev *rdev, bool lock_wait) static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) { int rc = 0, i; + int num_vec_created = 0; /* Configure and allocate resources for qplib */ rdev->qplib_res.rcfw = &rdev->rcfw; @@ -983,7 +987,7 @@ static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) if (rc) { dev_err(rdev_to_dev(rdev), "Alloc Failed NQ%d rc:%#x", i, rc); - goto dealloc_dpi; + goto free_nq; } rc = bnxt_re_net_ring_alloc (rdev, rdev->nq[i].hwq.pbl[PBL_LVL_0].pg_map_arr, @@ -996,14 +1000,17 @@ static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) dev_err(rdev_to_dev(rdev), "Failed to allocate NQ fw id with rc = 0x%x", rc); + bnxt_qplib_free_nq(&rdev->nq[i]); goto free_nq; } + num_vec_created++; } return 0; free_nq: - for (i = 0; i < rdev->num_msix - 1; i++) + for (i = num_vec_created; i >= 0; i--) { + bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id, true); bnxt_qplib_free_nq(&rdev->nq[i]); -dealloc_dpi: + } bnxt_qplib_dealloc_dpi(&rdev->qplib_res, &rdev->qplib_res.dpi_tbl, &rdev->dpi_privileged); @@ -1233,8 +1240,11 @@ static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) if (test_and_clear_bit(BNXT_RE_FLAG_QOS_WORK_REG, &rdev->flags)) cancel_delayed_work(&rdev->worker); - bnxt_re_cleanup_res(rdev); - bnxt_re_free_res(rdev, lock_wait); + if (test_and_clear_bit(BNXT_RE_FLAG_RESOURCES_INITIALIZED, + &rdev->flags)) + bnxt_re_cleanup_res(rdev); + if (test_and_clear_bit(BNXT_RE_FLAG_RESOURCES_ALLOCATED, &rdev->flags)) + bnxt_re_free_res(rdev, lock_wait); if (test_and_clear_bit(BNXT_RE_FLAG_RCFW_CHANNEL_EN, &rdev->flags)) { rc = bnxt_qplib_deinit_rcfw(&rdev->rcfw); @@ -1358,12 +1368,15 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) pr_err("Failed to allocate resources: %#x\n", rc); goto fail; } + set_bit(BNXT_RE_FLAG_RESOURCES_ALLOCATED, &rdev->flags); rc = bnxt_re_init_res(rdev); if (rc) { pr_err("Failed to initialize resources: %#x\n", rc); goto fail; } + set_bit(BNXT_RE_FLAG_RESOURCES_INITIALIZED, &rdev->flags); + if (!rdev->is_virtfn) { rc = bnxt_re_setup_qos(rdev); if (rc) From patchwork Thu Sep 20 15:02:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10607923 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8B3D14DA for ; Thu, 20 Sep 2018 15:03:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9810728D6C for ; Thu, 20 Sep 2018 15:03:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 960A728E64; Thu, 20 Sep 2018 15:03:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F678293D1 for ; Thu, 20 Sep 2018 15:03:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733171AbeITUrL (ORCPT ); Thu, 20 Sep 2018 16:47:11 -0400 Received: from mail-pl1-f196.google.com ([209.85.214.196]:42861 "EHLO mail-pl1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730758AbeITUrL (ORCPT ); Thu, 20 Sep 2018 16:47:11 -0400 Received: by mail-pl1-f196.google.com with SMTP id g23-v6so4474932plq.9 for ; Thu, 20 Sep 2018 08:03:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cYx1t/vOQVD9xDTALwOy1JRcy9KCQHU7A45ErKchMk8=; b=EI//F+xYBGi/TtJLLsHUgUKLMfrXFGs3/RBnezjG/uVLCoFlNHBeu7PWWpKpS/fWgm DQIpuFP4VahCYI10ddMdJ/I+Bp6CbnOF2sLAD4iys1f+uPUjB6BNLLXTLvPcuLXtB8et CmN0JF6s+0dp9+yL2H4CzjBcbVUUEgNER4Vvk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cYx1t/vOQVD9xDTALwOy1JRcy9KCQHU7A45ErKchMk8=; b=RlNlHldNzeRJ+yexS0Aov6YUVuAVKihaEOpJg472C8BQhQ72Rw4uKd4SSA6dfWYcev HTod1rB1gV8Ox5BFAgeWwd17BMKY8gfeWHqDZ/U0a3JmGuXoOPSIEKl1OZ1E9AaKd6DI J2VtOx8lwXyYu/ovfe/5fSx7zG9UGKQO8ji3vj8Z2VMB0MvPbdqtorUOR/HZ98OmU+pX rHUxsprafhNiQaDv6ybWRfUcPnm/6MpsZGsRQHRBl10Fhiej7GfyAQLZakxeQPl1wsri R5i9OSasGdJbUuKFyhnA5TR9Q3oNoyOTchOmCUaBm2Wrm2CNIrH+ms1xkgMuDeoQLeM0 35Dg== X-Gm-Message-State: APzg51DJVAaqcg2dU4wucpjF0GzP4ABJMuokbfH6cdTuK000R8r+J4zT YtX5o2o+L/vs3nN4B/AmiG8V+Up2B90= X-Google-Smtp-Source: ANB0VdZ9ztKdJhBjBo/RRp4WphQSY//Ahuttl9XEhHSRjN8jfmwle9mEDaOA1gUrmiMnyp2E8wt2lw== X-Received: by 2002:a17:902:b702:: with SMTP id d2-v6mr40427886pls.12.1537455796895; Thu, 20 Sep 2018 08:03:16 -0700 (PDT) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 9-v6sm66271043pfc.20.2018.09.20.08.03.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 08:03:15 -0700 (PDT) From: Selvin Xavier To: linux-rdma@vger.kernel.org Cc: dledford@redhat.com, jgg@mellanox.com, Selvin Xavier Subject: [PATCH for-rc 4/4] RDMA/bnxt_re: Handle all RDMA resource initialization under rtnl_lock Date: Thu, 20 Sep 2018 08:02:35 -0700 Message-Id: <1537455755-22808-5-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> References: <1537455755-22808-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP bnxt_re_ib_reg acquires and releases the rtnl lock whenever it accesses the L2 driver. First step in this function registers some callbacks with L2 driver. L2 driver assumes that roce device is initialized at this point. RoCE device registration continues after releasing rtnl lock. L2 driver call back before completing the resource initialization may cause a system crash. Avoid the failure/system crash due to a call back by acquiring the lock for the entire duration of device initialization. Re-factor the code to remove the rtnl lock from the individual function and acquire and release it from the caller. Fixes: 1ac5a4047975 ("RDMA/bnxt_re: Add bnxt_re RoCE driver") Fixes: 6e04b1035689 ("RDMA/bnxt_re: Fix broken RoCE driver due to recent L2 driver changes") Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 95 +++++++++++++++--------------------- 1 file changed, 39 insertions(+), 56 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 4628a93..d0b79a0 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -78,7 +78,7 @@ static struct list_head bnxt_re_dev_list = LIST_HEAD_INIT(bnxt_re_dev_list); /* Mutex to protect the list of bnxt_re devices added */ static DEFINE_MUTEX(bnxt_re_dev_lock); static struct workqueue_struct *bnxt_re_wq; -static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait); +static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev); /* SR-IOV helper functions */ @@ -182,7 +182,7 @@ static void bnxt_re_shutdown(void *p) if (!rdev) return; - bnxt_re_ib_unreg(rdev, false); + bnxt_re_ib_unreg(rdev); } static void bnxt_re_stop_irq(void *handle) @@ -251,7 +251,7 @@ static struct bnxt_ulp_ops bnxt_re_ulp_ops = { /* Driver registration routines used to let the networking driver (bnxt_en) * to know that the RoCE driver is now installed */ -static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev, bool lock_wait) +static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; int rc; @@ -260,14 +260,9 @@ static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev, bool lock_wait) return -EINVAL; en_dev = rdev->en_dev; - /* Acquire rtnl lock if it is not invokded from netdev event */ - if (lock_wait) - rtnl_lock(); rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev, BNXT_ROCE_ULP); - if (lock_wait) - rtnl_unlock(); return rc; } @@ -281,14 +276,12 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rtnl_lock(); rc = en_dev->en_ops->bnxt_register_device(en_dev, BNXT_ROCE_ULP, &bnxt_re_ulp_ops, rdev); - rtnl_unlock(); return rc; } -static int bnxt_re_free_msix(struct bnxt_re_dev *rdev, bool lock_wait) +static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; int rc; @@ -298,13 +291,9 @@ static int bnxt_re_free_msix(struct bnxt_re_dev *rdev, bool lock_wait) en_dev = rdev->en_dev; - if (lock_wait) - rtnl_lock(); rc = en_dev->en_ops->bnxt_free_msix(rdev->en_dev, BNXT_ROCE_ULP); - if (lock_wait) - rtnl_unlock(); return rc; } @@ -320,7 +309,6 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); - rtnl_lock(); num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, BNXT_ROCE_ULP, rdev->msix_entries, num_msix_want); @@ -335,7 +323,6 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) } rdev->num_msix = num_msix_got; done: - rtnl_unlock(); return rc; } @@ -358,24 +345,18 @@ static void bnxt_re_fill_fw_msg(struct bnxt_fw_msg *fw_msg, void *msg, fw_msg->timeout = timeout; } -static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, - bool lock_wait) +static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id) { struct bnxt_en_dev *en_dev = rdev->en_dev; struct hwrm_ring_free_input req = {0}; struct hwrm_ring_free_output resp; struct bnxt_fw_msg fw_msg; - bool do_unlock = false; int rc = -EINVAL; if (!en_dev) return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - if (lock_wait) { - rtnl_lock(); - do_unlock = true; - } bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_RING_FREE, -1, -1); req.ring_type = RING_ALLOC_REQ_RING_TYPE_L2_CMPL; @@ -386,8 +367,6 @@ static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, if (rc) dev_err(rdev_to_dev(rdev), "Failed to free HW ring:%d :%#x", req.ring_id, rc); - if (do_unlock) - rtnl_unlock(); return rc; } @@ -405,7 +384,6 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, dma_addr_t *dma_arr, return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - rtnl_lock(); bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_RING_ALLOC, -1, -1); req.enables = 0; req.page_tbl_addr = cpu_to_le64(dma_arr[0]); @@ -426,27 +404,21 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, dma_addr_t *dma_arr, if (!rc) *fw_ring_id = le16_to_cpu(resp.ring_id); - rtnl_unlock(); return rc; } static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, - u32 fw_stats_ctx_id, bool lock_wait) + u32 fw_stats_ctx_id) { struct bnxt_en_dev *en_dev = rdev->en_dev; struct hwrm_stat_ctx_free_input req = {0}; struct bnxt_fw_msg fw_msg; - bool do_unlock = false; int rc = -EINVAL; if (!en_dev) return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - if (lock_wait) { - rtnl_lock(); - do_unlock = true; - } bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_STAT_CTX_FREE, -1, -1); req.stat_ctx_id = cpu_to_le32(fw_stats_ctx_id); @@ -457,8 +429,6 @@ static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, dev_err(rdev_to_dev(rdev), "Failed to free HW stats context %#x", rc); - if (do_unlock) - rtnl_unlock(); return rc; } @@ -478,7 +448,6 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - rtnl_lock(); bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_STAT_CTX_ALLOC, -1, -1); req.update_period_ms = cpu_to_le32(1000); @@ -490,7 +459,6 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, if (!rc) *fw_stats_ctx_id = le32_to_cpu(resp.stat_ctx_id); - rtnl_unlock(); return rc; } @@ -932,19 +900,19 @@ static int bnxt_re_init_res(struct bnxt_re_dev *rdev) return rc; } -static void bnxt_re_free_nq_res(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_free_nq_res(struct bnxt_re_dev *rdev) { int i; for (i = 0; i < rdev->num_msix - 1; i++) { - bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id, lock_wait); + bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id); bnxt_qplib_free_nq(&rdev->nq[i]); } } -static void bnxt_re_free_res(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_free_res(struct bnxt_re_dev *rdev) { - bnxt_re_free_nq_res(rdev, lock_wait); + bnxt_re_free_nq_res(rdev); if (rdev->qplib_res.dpi_tbl.max) { bnxt_qplib_dealloc_dpi(&rdev->qplib_res, @@ -1008,7 +976,7 @@ static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) return 0; free_nq: for (i = num_vec_created; i >= 0; i--) { - bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id, true); + bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id); bnxt_qplib_free_nq(&rdev->nq[i]); } bnxt_qplib_dealloc_dpi(&rdev->qplib_res, @@ -1226,7 +1194,7 @@ static int bnxt_re_setup_qos(struct bnxt_re_dev *rdev) return 0; } -static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev) { int i, rc; @@ -1244,28 +1212,27 @@ static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) &rdev->flags)) bnxt_re_cleanup_res(rdev); if (test_and_clear_bit(BNXT_RE_FLAG_RESOURCES_ALLOCATED, &rdev->flags)) - bnxt_re_free_res(rdev, lock_wait); + bnxt_re_free_res(rdev); if (test_and_clear_bit(BNXT_RE_FLAG_RCFW_CHANNEL_EN, &rdev->flags)) { rc = bnxt_qplib_deinit_rcfw(&rdev->rcfw); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to deinitialize RCFW: %#x", rc); - bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id, - lock_wait); + bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id); bnxt_qplib_free_ctx(rdev->en_dev->pdev, &rdev->qplib_ctx); bnxt_qplib_disable_rcfw_channel(&rdev->rcfw); - bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id, lock_wait); + bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id); bnxt_qplib_free_rcfw_channel(&rdev->rcfw); } if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) { - rc = bnxt_re_free_msix(rdev, lock_wait); + rc = bnxt_re_free_msix(rdev); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to free MSI-X vectors: %#x", rc); } if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) { - rc = bnxt_re_unregister_netdev(rdev, lock_wait); + rc = bnxt_re_unregister_netdev(rdev); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to unregister with netdev: %#x", rc); @@ -1286,6 +1253,12 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) { int i, j, rc; + bool locked; + + /* Acquire rtnl lock through out this function */ + rtnl_lock(); + locked = true; + /* Registered a new RoCE device instance to netdev */ rc = bnxt_re_register_netdev(rdev); if (rc) { @@ -1387,12 +1360,16 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); } + rtnl_unlock(); + locked = false; + /* Register ib dev */ rc = bnxt_re_register_ib(rdev); if (rc) { pr_err("Failed to register with IB: %#x\n", rc); goto fail; } + set_bit(BNXT_RE_FLAG_IBDEV_REGISTERED, &rdev->flags); dev_info(rdev_to_dev(rdev), "Device registered successfully"); for (i = 0; i < ARRAY_SIZE(bnxt_re_attributes); i++) { rc = device_create_file(&rdev->ibdev.dev, @@ -1408,7 +1385,6 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) goto fail; } } - set_bit(BNXT_RE_FLAG_IBDEV_REGISTERED, &rdev->flags); ib_get_eth_speed(&rdev->ibdev, 1, &rdev->active_speed, &rdev->active_width); set_bit(BNXT_RE_FLAG_ISSUE_ROCE_STATS, &rdev->flags); @@ -1417,17 +1393,21 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) return 0; free_sctx: - bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id, true); + bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id); free_ctx: bnxt_qplib_free_ctx(rdev->en_dev->pdev, &rdev->qplib_ctx); disable_rcfw: bnxt_qplib_disable_rcfw_channel(&rdev->rcfw); free_ring: - bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id, true); + bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id); free_rcfw: bnxt_qplib_free_rcfw_channel(&rdev->rcfw); fail: - bnxt_re_ib_unreg(rdev, true); + if (!locked) + rtnl_lock(); + bnxt_re_ib_unreg(rdev); + rtnl_unlock(); + return rc; } @@ -1580,7 +1560,7 @@ static int bnxt_re_netdev_event(struct notifier_block *notifier, */ if (atomic_read(&rdev->sched_count) > 0) goto exit; - bnxt_re_ib_unreg(rdev, false); + bnxt_re_ib_unreg(rdev); bnxt_re_remove_one(rdev); bnxt_re_dev_unreg(rdev); break; @@ -1659,7 +1639,10 @@ static void __exit bnxt_re_mod_exit(void) */ flush_workqueue(bnxt_re_wq); bnxt_re_dev_stop(rdev); - bnxt_re_ib_unreg(rdev, true); + /* Acquire the rtnl_lock as the L2 resources are freed here */ + rtnl_lock(); + bnxt_re_ib_unreg(rdev); + rtnl_unlock(); bnxt_re_remove_one(rdev); bnxt_re_dev_unreg(rdev); }