From patchwork Mon Oct 8 10:27:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 10630349 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 653CA112B for ; Mon, 8 Oct 2018 10:28:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93F56286EE for ; Mon, 8 Oct 2018 10:28:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 885942873B; Mon, 8 Oct 2018 10:28:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E6CB286EE for ; Mon, 8 Oct 2018 10:28:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726511AbeJHRji (ORCPT ); Mon, 8 Oct 2018 13:39:38 -0400 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35283 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726330AbeJHRji (ORCPT ); Mon, 8 Oct 2018 13:39:38 -0400 Received: by mail-wm1-f68.google.com with SMTP id e187-v6so7771440wmf.0 for ; Mon, 08 Oct 2018 03:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fJlINjU1A+kDzXIL11bL055E7CAseYOFFIPcioGp8+0=; b=btcE9pTQt/qBeK5PApyWb0B47+PSvXR+7yGlIp2UadVVGo4atzZe1qYOuw7yVN2xQs SBo9yZ2mCbVrGvQEBE2XIRtae1IRk/IavnHNQMf1HNt1Gr4C+1Dwr0S2RCIs/mx8bxEd 6NsDNlLZtPj1rpY7Kt8l1G4c8gAoR/LJmVj/M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fJlINjU1A+kDzXIL11bL055E7CAseYOFFIPcioGp8+0=; b=hMAYDcLqkv69d7IGRZ9KKJJ3XL7uMpddbfQ4+XdFfiATK83pl/oribTy5aGkEH+g81 eZ4SIFNrAr5OvONUaYsVDJgqsKhlzMkwZmGAzkGIWlex8KNio9XBMoeTyDU5tAATVrJx p+2eDrUjbDsQ3+OaqTsjQElRa2/kRs80ATjblPJEUDn9am3/CJl8VCn7ODl62A3oa1Yi VXYafFEydpbP+fsJFtIOtxi3C4ttHcf0S6TFW2LG/CP0PegyoZ9GHdYhRVPPo2X6kngj UPHixBWdyqOLWGvoOLoRirRuJp5oNdkYjZeQADWCQ4kIuWo12xD0VLkGDsPiobT7dvH9 W//Q== X-Gm-Message-State: ABuFfoi6a1O8z5JA1opdDrp63HMQ/6MEj9QTzgrsGaa+/C6iXd7ed2pn QgHiCI3KOAU2fgFdqGTNKBlalg== X-Google-Smtp-Source: ACcGV61Rlc7h5FDQyTx/8NcrpyzTc6x3Hr2qJj+TTzA/20ozYtBgTNK//u2c2hOtYaePwkqruf/nvA== X-Received: by 2002:a1c:c683:: with SMTP id w125-v6mr14674849wmf.117.1538994514995; Mon, 08 Oct 2018 03:28:34 -0700 (PDT) Received: from dhcp-10-192-206-197.dhcp.broadcom.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 143-v6sm12674897wmv.6.2018.10.08.03.28.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 08 Oct 2018 03:28:34 -0700 (PDT) From: Selvin Xavier To: dledford@redhat.com, jgg@mellanox.com Cc: linux-rdma@vger.kernel.org, Selvin Xavier Subject: [PATCH rdma-next 02/13] RDMA/bnxt_re: Fix recursive lock warning in debug kernel Date: Mon, 8 Oct 2018 03:27:53 -0700 Message-Id: <1538994484-21390-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1538994484-21390-1-git-send-email-selvin.xavier@broadcom.com> References: <1538994484-21390-1-git-send-email-selvin.xavier@broadcom.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Fix possible recursive lock warning. Its a false warning as the locks are part of two differnt HW Queue data structure - cmdq and creq. Debug kernel is throwing the following warning and stack trace. [ 783.914967] ============================================ [ 783.914970] WARNING: possible recursive locking detected [ 783.914973] 4.19.0-rc2+ #33 Not tainted [ 783.914976] -------------------------------------------- [ 783.914979] swapper/2/0 is trying to acquire lock: [ 783.914982] 000000002aa3949d (&(&hwq->lock)->rlock){..-.}, at: bnxt_qplib_service_creq+0x232/0x350 [bnxt_re] [ 783.914999] but task is already holding lock: [ 783.915002] 00000000be73920d (&(&hwq->lock)->rlock){..-.}, at: bnxt_qplib_service_creq+0x2a/0x350 [bnxt_re] [ 783.915013] other info that might help us debug this: [ 783.915016] Possible unsafe locking scenario: [ 783.915019] CPU0 [ 783.915021] ---- [ 783.915034] lock(&(&hwq->lock)->rlock); [ 783.915035] lock(&(&hwq->lock)->rlock); [ 783.915037] *** DEADLOCK *** [ 783.915038] May be due to missing lock nesting notation [ 783.915039] 1 lock held by swapper/2/0: [ 783.915040] #0: 00000000be73920d (&(&hwq->lock)->rlock){..-.}, at: bnxt_qplib_service_creq+0x2a/0x350 [bnxt_re] [ 783.915044] stack backtrace: [ 783.915046] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.19.0-rc2+ #33 [ 783.915047] Hardware name: Dell Inc. PowerEdge R730/0599V5, BIOS 1.0.4 08/28/2014 [ 783.915048] Call Trace: [ 783.915049] [ 783.915054] dump_stack+0x90/0xe3 [ 783.915058] __lock_acquire+0x106c/0x1080 [ 783.915061] ? sched_clock+0x5/0x10 [ 783.915063] lock_acquire+0xbd/0x1a0 [ 783.915065] ? bnxt_qplib_service_creq+0x232/0x350 [bnxt_re] [ 783.915069] _raw_spin_lock_irqsave+0x4a/0x90 [ 783.915071] ? bnxt_qplib_service_creq+0x232/0x350 [bnxt_re] [ 783.915073] bnxt_qplib_service_creq+0x232/0x350 [bnxt_re] [ 783.915078] tasklet_action_common.isra.17+0x197/0x1b0 [ 783.915081] __do_softirq+0xcb/0x3a6 [ 783.915084] irq_exit+0xe9/0x100 [ 783.915085] do_IRQ+0x6a/0x120 [ 783.915087] common_interrupt+0xf/0xf [ 783.915088] Use nested notation for the spin_lock to avoid this warning. Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index 1410500..0c6a3ac 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -311,8 +311,17 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, rcfw->aeq_handler(rcfw, qp_event, qp); break; default: - /* Command Response */ - spin_lock_irqsave(&cmdq->lock, flags); + /* + * Command Response + * cmdq->lock needs to be acquired to synchronie + * the command send and completion reaping. This function + * is always called with creq->lock held. Using + * the nested variant of spin_lock. + * + */ + + spin_lock_irqsave_nested(&cmdq->lock, flags, + SINGLE_DEPTH_NESTING); cookie = le16_to_cpu(qp_event->cookie); mcookie = qp_event->cookie; blocked = cookie & RCFW_CMD_IS_BLOCKING;