From patchwork Mon Dec 5 06:38:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 9460405 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A3DE96074E for ; Mon, 5 Dec 2016 06:46:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8DA3124EE5 for ; Mon, 5 Dec 2016 06:46:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 828BE25223; Mon, 5 Dec 2016 06:46:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A37725EA6 for ; Mon, 5 Dec 2016 06:46:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751212AbcLEGqG (ORCPT ); Mon, 5 Dec 2016 01:46:06 -0500 Received: from lpdvsmtp01.broadcom.com ([192.19.211.62]:33828 "EHLO relay.smtp.broadcom.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751320AbcLEGpo (ORCPT ); Mon, 5 Dec 2016 01:45:44 -0500 Received: from dhcp-10-192-206-197.iig.avagotech.net (dhcp-10-192-206-132.iig.avagotech.net [10.192.206.132]) by relay.smtp.broadcom.com (Postfix) with ESMTP id 9BE15280548; Sun, 4 Dec 2016 22:38:57 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.10.3 relay.smtp.broadcom.com 9BE15280548 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=broadcom.com; s=dkimrelay; t=1480919938; bh=Cs+KdE0/Zin3Crx8bMqZi4yeptl/bjQxbcj9curE45U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GxbvFCGE6GNmHGDFWAyE4SKPvESRKSdpjvhxtTT4b2iUudHKnyK7d2aCCHgwlaOXW 62LF79iGgQ9YsDBsZHQMRmLYVo3e7yk7YEbLpOc3UYUO1+sBcg3Nn5B5CR7NCbUhhk Bw5LftiC1TI9/SBc+vU6dS8So26uamBIU8S8lqWI= From: Selvin Xavier To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Selvin Xavier , Eddie Wai , Devesh Sharma , Somnath Kotur , Sriharsha Basavapatna Subject: [PATCH 24/28] bnxt_re: Handling dispatching of events to IB stack and cleanup during unload Date: Sun, 4 Dec 2016 22:38:28 -0800 Message-Id: <1480919912-1079-25-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1480919912-1079-1-git-send-email-selvin.xavier@broadcom.com> References: <1480919912-1079-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch implements handling dispatch of appropriate event to the IB stack based on NETDEV events received. Also implements cleanup of the resources during driver unload. Signed-off-by: Eddie Wai Signed-off-by: Devesh Sharma Signed-off-by: Somnath Kotur Signed-off-by: Sriharsha Basavapatna Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxtre/bnxt_re_main.c | 77 +++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) diff --git a/drivers/infiniband/hw/bnxtre/bnxt_re_main.c b/drivers/infiniband/hw/bnxtre/bnxt_re_main.c index a138741..3f55548 100644 --- a/drivers/infiniband/hw/bnxtre/bnxt_re_main.c +++ b/drivers/infiniband/hw/bnxtre/bnxt_re_main.c @@ -714,6 +714,60 @@ static int bnxt_re_alloc_res(struct bnxt_re_dev *rdev) return rc; } +static void bnxt_re_dispatch_event(struct ib_device *ibdev, struct ib_qp *qp, + u8 port_num, enum ib_event_type event) +{ + struct ib_event ib_event; + + ib_event.device = ibdev; + if (qp) + ib_event.element.qp = qp; + else + ib_event.element.port_num = port_num; + ib_event.event = event; + ib_dispatch_event(&ib_event); +} + +static bool bnxt_re_is_qp1_or_shadow_qp(struct bnxt_re_dev *rdev, + struct bnxt_re_qp *qp) +{ + return (qp->ib_qp.qp_type == IB_QPT_GSI) || (qp == rdev->qp1_sqp); +} + +static void bnxt_re_dev_stop(struct bnxt_re_dev *rdev, bool qp_wait) +{ + int mask = IB_QP_STATE, qp_count, count = 1; + struct ib_qp_attr qp_attr; + struct bnxt_re_qp *qp; + + qp_attr.qp_state = IB_QPS_ERR; + mutex_lock(&rdev->qp_lock); + list_for_each_entry(qp, &rdev->qp_list, list) { + /* Modify the state of all QPs except QP1/Shadow QP */ + if (qp && !bnxt_re_is_qp1_or_shadow_qp(rdev, qp)) { + if (qp->qplib_qp.state != + CMDQ_MODIFY_QP_NEW_STATE_RESET || + qp->qplib_qp.state != + CMDQ_MODIFY_QP_NEW_STATE_ERR) { + bnxt_re_dispatch_event(&rdev->ibdev, &qp->ib_qp, + 1, IB_EVENT_QP_FATAL); + bnxt_re_modify_qp(&qp->ib_qp, &qp_attr, mask, + NULL); + } + } + } + + mutex_unlock(&rdev->qp_lock); + if (qp_wait) { + /* Give the application some time to clean up */ + do { + qp_count = atomic_read(&rdev->qp_count); + msleep(100); + } while ((qp_count != atomic_read(&rdev->qp_count)) && + count--); + } +} + static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) { int i, rc; @@ -872,6 +926,8 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) } } set_bit(BNXT_RE_FLAG_IBDEV_REGISTERED, &rdev->flags); + bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, IB_EVENT_PORT_ACTIVE); + bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, IB_EVENT_GID_CHANGE); return 0; free_sctx: @@ -950,10 +1006,18 @@ static void bnxt_re_task(struct work_struct *work) "Failed to register with IB: %#x", rc); break; case NETDEV_UP: + bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, + IB_EVENT_PORT_ACTIVE); break; case NETDEV_DOWN: + bnxt_re_dev_stop(rdev, false); break; case NETDEV_CHANGE: + if (!netif_carrier_ok(rdev->netdev)) + bnxt_re_dev_stop(rdev, false); + else if (netif_carrier_ok(rdev->netdev)) + bnxt_re_dispatch_event(&rdev->ibdev, NULL, 1, + IB_EVENT_PORT_ACTIVE); break; default: break; @@ -1071,6 +1135,7 @@ static int __init bnxt_re_mod_init(void) } static void __exit bnxt_re_mod_exit(void) { + struct bnxt_re_dev *rdev, *next; LIST_HEAD(to_be_deleted); /* Free all adapter allocated resources */ @@ -1079,6 +1144,18 @@ static void __exit bnxt_re_mod_exit(void) list_splice_init_rcu(&bnxt_re_dev_list, &to_be_deleted, synchronize_rcu); mutex_unlock(&bnxt_re_dev_lock); + + /* Can use the new list without protection */ + /* Cleanup the devices in reverse order so that the VF device + * cleanup is done before PF cleanup + */ + list_for_each_entry_safe_reverse(rdev, next, &to_be_deleted, list) { + bnxt_re_dev_stop(rdev, true); + bnxt_re_ib_unreg(rdev, true); + bnxt_re_remove_one(rdev); + bnxt_re_dev_unreg(rdev); + } + unregister_netdevice_notifier(&bnxt_re_netdev_notifier); if (bnxt_re_wq) destroy_workqueue(bnxt_re_wq);