From patchwork Sat Feb 9 17:28:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Parvi Kaustubhi (pkaustub)" X-Patchwork-Id: 10804535 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9790213BF for ; Sat, 9 Feb 2019 17:28:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 71EBE2873C for ; Sat, 9 Feb 2019 17:28:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 61FA629561; Sat, 9 Feb 2019 17:28:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 000A12873C for ; Sat, 9 Feb 2019 17:28:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727726AbfBIR2l (ORCPT ); Sat, 9 Feb 2019 12:28:41 -0500 Received: from rcdn-iport-5.cisco.com ([173.37.86.76]:3583 "EHLO rcdn-iport-5.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727655AbfBIR2l (ORCPT ); Sat, 9 Feb 2019 12:28:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3220; q=dns/txt; s=iport; t=1549733320; x=1550942920; h=from:to:cc:subject:date:message-id; bh=1pZE/LTdcVb5b//iDTd+9vfgTdMW9B3tFd7G3URJ1EQ=; b=dCVvh3NrRpR0OCTInw89wxrnkWyrBr8b0LojIJYl7Yd0IaF5ylBItB+2 aMfrFFbrpvueiBRBzoRfi6FBo3lvy8HRfEa3ReCOl/lyW06O0i2YbdrSX neT0c6kVYCLszgL4iyqX7fGHX/o8IOFHHKOO+6wyanmbbi2le3LcUluBY c=; X-IronPort-AV: E=Sophos;i="5.58,352,1544486400"; d="scan'208";a="297008342" Received: from alln-core-11.cisco.com ([173.36.13.133]) by rcdn-iport-5.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Feb 2019 17:28:39 +0000 Received: from cisco.com (savbu-usnic-a.cisco.com [10.193.184.48]) by alln-core-11.cisco.com (8.15.2/8.15.2) with ESMTP id x19HSc1C006241; Sat, 9 Feb 2019 17:28:39 GMT Received: by cisco.com (Postfix, from userid 478433) id CF18020F2001; Sat, 9 Feb 2019 09:28:38 -0800 (PST) From: Parvi Kaustubhi To: linux-rdma@vger.kernel.org Cc: gvaradar@cisco.com, benve@cisco.com, tinamdar@cisco.com, neescoba@cisco.com, jsquyres@cisco.com, ravianan@cisco.com, pkaustub@cisco.com Subject: [PATCH for-next v4] IB/usnic: fix deadlock Date: Sat, 9 Feb 2019 09:28:30 -0800 Message-Id: <1549733310-14966-1-git-send-email-pkaustub@cisco.com> X-Mailer: git-send-email 2.7.0 X-Outbound-SMTP-Client: 10.193.184.48, savbu-usnic-a.cisco.com X-Outbound-Node: alln-core-11.cisco.com Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There is a dead lock in usnic ib_register and netdev_notify path. usnic_ib_discover_pf() | mutex_lock(&usnic_ib_ibdev_list_lock); | usnic_ib_device_add(); | ib_register_device() | usnic_ib_query_port() | mutex_lock(&us_ibdev->usdev_lock); | ib_get_eth_speed() | rtnl_lock() order of lock: &usnic_ib_ibdev_list_lock -> usdev_lock -> rtnl_lock rtnl_lock() | usnic_ib_netdevice_event() | mutex_lock(&usnic_ib_ibdev_list_lock); order of lock: rtnl_lock -> &usnic_ib_ibdev_list_lock Solution is to use ib_device_get_by_netdev() to lookup ib_dev while handling netdev/ inet events. Signed-off-by: Parvi Kaustubhi Reviewed-by: Govindarajulu Varadarajan Reviewed-by: Tanmay Inamdar --- Changelog: v3->v4 * Added missing ib_device_put* calls. * Fixed wonky control flow v2->v3: * Jason: drivers should not hold any locks while calling ib_unregister_device() * Jason: use https://github.com/jgunthorpe/linux/commits/device_locking_cleanup v1->v2: * Have notifier blocks in usnic_ib_dev instead of using workqueue to defer event handling. --- drivers/infiniband/hw/usnic/usnic_ib_main.c | 30 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c index 9529a08..d0ddbec 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_main.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c @@ -216,18 +216,17 @@ static int usnic_ib_netdevice_event(struct notifier_block *notifier, unsigned long event, void *ptr) { struct usnic_ib_dev *us_ibdev; + struct ib_device *ibdev; struct net_device *netdev = netdev_notifier_info_to_dev(ptr); - mutex_lock(&usnic_ib_ibdev_list_lock); - list_for_each_entry(us_ibdev, &usnic_ib_ibdev_list, ib_dev_link) { - if (us_ibdev->netdev == netdev) { - usnic_ib_handle_usdev_event(us_ibdev, event); - break; - } - } - mutex_unlock(&usnic_ib_ibdev_list_lock); + ibdev = ib_device_get_by_netdev(netdev, RDMA_DRIVER_USNIC); + if (!ibdev) + return NOTIFY_DONE; + us_ibdev = container_of(ibdev, struct usnic_ib_dev, ib_dev); + usnic_ib_handle_usdev_event(us_ibdev, event); + ib_device_put(ibdev); return NOTIFY_DONE; } @@ -282,16 +281,15 @@ static int usnic_ib_inetaddr_event(struct notifier_block *notifier, struct usnic_ib_dev *us_ibdev; struct in_ifaddr *ifa = ptr; struct net_device *netdev = ifa->ifa_dev->dev; + struct ib_device *ibdev; - mutex_lock(&usnic_ib_ibdev_list_lock); - list_for_each_entry(us_ibdev, &usnic_ib_ibdev_list, ib_dev_link) { - if (us_ibdev->netdev == netdev) { - usnic_ib_handle_inet_event(us_ibdev, event, ptr); - break; - } - } - mutex_unlock(&usnic_ib_ibdev_list_lock); + ibdev = ib_device_get_by_netdev(netdev, RDMA_DRIVER_USNIC); + if (!ibdev) + return NOTIFY_DONE; + us_ibdev = container_of(ibdev, struct usnic_ib_dev, ib_dev); + usnic_ib_handle_inet_event(us_ibdev, event, ptr); + ib_device_put(ibdev); return NOTIFY_DONE; } static struct notifier_block usnic_ib_inetaddr_notifier = {