From patchwork Fri Nov 7 04:50:31 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hariprasad S X-Patchwork-Id: 5249501 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 04FDDC11AC for ; Fri, 7 Nov 2014 04:44:56 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0FD6020145 for ; Fri, 7 Nov 2014 04:44:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F1B6920138 for ; Fri, 7 Nov 2014 04:44:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751523AbaKGEox (ORCPT ); Thu, 6 Nov 2014 23:44:53 -0500 Received: from 99-65-72-227.uvs.sntcca.sbcglobal.net ([99.65.72.227]:33655 "EHLO stargate3.asicdesigners.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751348AbaKGEow (ORCPT ); Thu, 6 Nov 2014 23:44:52 -0500 Received: from silicon.asicdesigners.com (silicon.blr.asicdesigners.com [10.193.185.90]) by stargate3.asicdesigners.com (8.13.8/8.13.8) with ESMTP id sA74ihYV022446; Thu, 6 Nov 2014 20:44:46 -0800 From: Hariprasad Shenai To: linux-rdma@vger.kernel.org Cc: roland@purestorage.com, swise@opengridcomputing.com, nirranjan@chelsio.com, kumaras@chelsio.com, Hariprasad Shenai Subject: [PATCH 1/2] iw_cxgb4: Fixes locking issue in process_mpa_request Date: Fri, 7 Nov 2014 10:20:31 +0530 Message-Id: <1415335832-11413-2-git-send-email-hariprasad@chelsio.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1415335832-11413-1-git-send-email-hariprasad@chelsio.com> References: <1415335832-11413-1-git-send-email-hariprasad@chelsio.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP ============================================= [ INFO: possible recursive locking detected ] 3.17.0+ #3 Tainted: G E --------------------------------------------- kworker/u64:3/299 is trying to acquire lock: (&epc->mutex){+.+.+.}, at: [] process_mpa_request+0x1aa/0x3e0 [iw_cxgb4] but task is already holding lock: (&epc->mutex){+.+.+.}, at: [] rx_data+0x9e/0x1f0 [iw_cxgb4] other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock(&epc->mutex); lock(&epc->mutex); *** DEADLOCK *** May be due to missing lock nesting notation 3 locks held by kworker/u64:3/299: #0: ("%s""iw_cxgb4"){.+.+.+}, at: [] process_one_work+0x13d/0x4d0 #1: (skb_work){+.+.+.}, at: [] process_one_work+0x13d/0x4d0 #2: (&epc->mutex){+.+.+.}, at: [] rx_data+0x9e/0x1f0 [iw_cxgb4] stack backtrace: CPU: 2 PID: 299 Comm: kworker/u64:3 Tainted: G E 3.17.0+ #3 Hardware name: Dell Inc. PowerEdge T110/0X744K, BIOS 1.2.1 01/28/2010 Workqueue: iw_cxgb4 process_work [iw_cxgb4] ffff8800b91593d0 ffff8800b8a2f9f8 ffffffff815df107 0000000000000001 ffff8800b9158750 ffff8800b8a2fa28 ffffffff8109f0e2 ffff8800bb768a00 ffff8800b91593d0 ffff8800b9158750 0000000000000000 ffff8800b8a2fa88 Call Trace: [] dump_stack+0x49/0x62 [] print_deadlock_bug+0xf2/0x100 [] validate_chain+0x454/0x700 [] __lock_acquire+0x3c4/0x580 [] ? process_mpa_request+0x1aa/0x3e0 [iw_cxgb4] [] lock_acquire+0x9c/0x110 [] ? process_mpa_request+0x1aa/0x3e0 [iw_cxgb4] [] mutex_lock_nested+0x4b/0x360 [] ? process_mpa_request+0x1aa/0x3e0 [iw_cxgb4] [] ? del_timer_sync+0xaa/0xd0 [] ? try_to_del_timer_sync+0x70/0x70 [] process_mpa_request+0x1aa/0x3e0 [iw_cxgb4] [] ? update_rx_credits+0xec/0x140 [iw_cxgb4] [] rx_data+0xd1/0x1f0 [iw_cxgb4] [] ? mark_held_locks+0x73/0xa0 [] ? _raw_spin_unlock_irqrestore+0x40/0x70 [] ? trace_hardirqs_on_caller+0xfd/0x1c0 [] ? trace_hardirqs_on+0xd/0x10 [] process_work+0x51/0x80 [iw_cxgb4] [] process_one_work+0x1b8/0x4d0 [] ? process_one_work+0x13d/0x4d0 [] worker_thread+0x120/0x3c0 [] ? process_one_work+0x4d0/0x4d0 [] kthread+0xde/0x100 [] ? _raw_spin_unlock_irq+0x30/0x40 [] ? __init_kthread_worker+0x70/0x70 [] ret_from_fork+0x7c/0xb0 [] ? __init_kthread_worker+0x70/0x70 =============================================== Based on original work by Steve Wise Signed-off-by: Steve Wise Signed-off-by: Hariprasad Shenai --- drivers/infiniband/hw/cxgb4/cm.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c index fb61f66..ce87fd3 100644 --- a/drivers/infiniband/hw/cxgb4/cm.c +++ b/drivers/infiniband/hw/cxgb4/cm.c @@ -1640,7 +1640,8 @@ static void process_mpa_request(struct c4iw_ep *ep, struct sk_buff *skb) __state_set(&ep->com, MPA_REQ_RCVD); /* drive upcall */ - mutex_lock(&ep->parent_ep->com.mutex); + mutex_lock_nested(&ep->parent_ep->com.mutex, + SINGLE_DEPTH_NESTING); if (ep->parent_ep->com.state != DEAD) { if (connect_request_upcall(ep)) abort_connection(ep, skb, GFP_KERNEL);