From patchwork Thu Jun 8 10:24:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13271920 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89F61C7EE23 for ; Thu, 8 Jun 2023 10:37:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236062AbjFHKhK (ORCPT ); Thu, 8 Jun 2023 06:37:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235994AbjFHKhI (ORCPT ); Thu, 8 Jun 2023 06:37:08 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1AC3E4D for ; Thu, 8 Jun 2023 03:37:02 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1b01bd7093aso1883995ad.1 for ; Thu, 08 Jun 2023 03:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1686220622; x=1688812622; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=gLutcpfZkuIDhI7geDLcW1Kg2azRF+u1Y0FLWhLWpEk=; b=TfQm9WTlj0sec30yeVfRQvfC0v0PGUJKyDoyQQ1rIUfFKWdWBN0ydzULrrxJLlVKkz jrL3vSyZrKDeNrgLmdBl8xnr/Mn9OrXuQuSlbsMUvir6ldkBmg6i64bGtA83HTKdpq/D 7l3p1CqR6zwaBJj0LSlLP5lYsE7ESlkYZ0Lc0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686220622; x=1688812622; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=gLutcpfZkuIDhI7geDLcW1Kg2azRF+u1Y0FLWhLWpEk=; b=AEGKz9VBZ7Lo5lXTTwFqW4x+a261+Kx/qW1FIsMIloOh3kh2ikOxxnAdIcBwi6o4Hp MVwX/LcI6knsyoHyocW8eSJG2oGTfmmxGRrPzpFCjB82gtkl3AYeY6KXcLn52b0hB0zU OFCwEneC7znaNDKpx17h+Q6slLobNhkvrQvDMx2UCkFikZCOdYzESvxP/OVU2FiJPUfI gfkaQ+6ezuWSp9EsZ1aCYGdeTIvYED5b/GibeGELSYuhoj3ItMT3fEzW/Q2aN9XIFe1w FJsLyxa/+4CD3PJLEUfbTJWzntnJJzKKvXp3k1aBLpztojeHjDQZK0n2ihYp9OEqW8ok soEA== X-Gm-Message-State: AC+VfDyhES4ELaYDaqovQDFGqHmj9/nTBVBjEwj6G4fMv3OO/Qe1FniH vFYDLNP+WSPkT+rX3BgOkHGgBA== X-Google-Smtp-Source: ACHHUZ6AjQjteJ795O1PfMU5v0UvQSt4kzOicM+8lfDyIBwFHjVAamvvceGk9NYClWP67doLN976oA== X-Received: by 2002:a17:902:7b96:b0:1ac:451d:34b with SMTP id w22-20020a1709027b9600b001ac451d034bmr4876321pll.9.1686220622048; Thu, 08 Jun 2023 03:37:02 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id jj11-20020a170903048b00b001a980a23802sm1128510plb.111.2023.06.08.03.36.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jun 2023 03:37:01 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Kashyap Desai , Selvin Xavier Subject: [PATCH for-next 02/17] RDMA/bnxt_re: Avoid calling wake_up threads from spin_lock context Date: Thu, 8 Jun 2023 03:24:53 -0700 Message-Id: <1686219908-11181-3-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1686219908-11181-1-git-send-email-selvin.xavier@broadcom.com> References: <1686219908-11181-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Kashyap Desai bnxt_qplib_service_creq can be called from interrupt or tasklet or process context. So the function take irq variant of spin_lock. But when wake_up is invoked with the lock held, it is putting the calling context to sleep. [exception RIP: __wake_up_common+190] RIP: ffffffffb7539d7e RSP: ffffa73300207ad8 RFLAGS: 00000083 RAX: 0000000000000001 RBX: ffff91fa295f69b8 RCX: dead000000000200 RDX: ffffa733344af940 RSI: ffffa73336527940 RDI: ffffa73336527940 RBP: 000000000000001c R8: 0000000000000002 R9: 00000000000299c0 R10: 0000017230de82c5 R11: 0000000000000002 R12: ffffa73300207b28 R13: 0000000000000000 R14: ffffa733341bf928 R15: 0000000000000000 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 Call the wakeup after releasing the lock. Fixes: 1ac5a4047975 ("RDMA/bnxt_re: Add bnxt_re RoCE driver") Signed-off-by: Kashyap Desai Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index d4ce82b..c11b8e7 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -295,7 +295,8 @@ static int bnxt_qplib_process_func_event(struct bnxt_qplib_rcfw *rcfw, } static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, - struct creq_qp_event *qp_event) + struct creq_qp_event *qp_event, + u32 *num_wait) { struct creq_qp_error_notification *err_event; struct bnxt_qplib_hwq *hwq = &rcfw->cmdq.hwq; @@ -304,6 +305,7 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, u16 cbit, blocked = 0; struct pci_dev *pdev; unsigned long flags; + u32 wait_cmds = 0; __le16 mcookie; u16 cookie; int rc = 0; @@ -363,9 +365,10 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, crsqe->req_size = 0; if (!blocked) - wake_up(&rcfw->cmdq.waitq); + wait_cmds++; spin_unlock_irqrestore(&hwq->lock, flags); } + *num_wait += wait_cmds; return rc; } @@ -379,6 +382,7 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t) struct creq_base *creqe; u32 sw_cons, raw_cons; unsigned long flags; + u32 num_wakeup = 0; /* Service the CREQ until budget is over */ spin_lock_irqsave(&hwq->lock, flags); @@ -397,7 +401,8 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t) switch (type) { case CREQ_BASE_TYPE_QP_EVENT: bnxt_qplib_process_qp_event - (rcfw, (struct creq_qp_event *)creqe); + (rcfw, (struct creq_qp_event *)creqe, + &num_wakeup); creq->stats.creq_qp_event_processed++; break; case CREQ_BASE_TYPE_FUNC_EVENT: @@ -425,6 +430,8 @@ static void bnxt_qplib_service_creq(struct tasklet_struct *t) rcfw->res->cctx, true); } spin_unlock_irqrestore(&hwq->lock, flags); + if (num_wakeup) + wake_up_nr(&rcfw->cmdq.waitq, num_wakeup); } static irqreturn_t bnxt_qplib_creq_irq(int irq, void *dev_instance)