From patchwork Thu May 11 07:26:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13237587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EE00C77B7F for ; Thu, 11 May 2023 07:38:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237402AbjEKHia (ORCPT ); Thu, 11 May 2023 03:38:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44994 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237235AbjEKHi2 (ORCPT ); Thu, 11 May 2023 03:38:28 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 284952115 for ; Thu, 11 May 2023 00:38:27 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-643a1656b79so5673408b3a.3 for ; Thu, 11 May 2023 00:38:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1683790706; x=1686382706; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=vRPzR3hysjgBXHiDIpMGYdQB4V+B83jsUHDv1pqm4+w=; b=cRKR9Sresp76gCKme8KyZtThmhIHminUapJSkcvtmahhRauM4smuquByslZe62WjER IBjPFoPV0CD2vHR7MksDWrneBWcyN0eP2MnENTfKWTvalJeZt2zmVHCUOw+9IXRWuOPo OR5iSJYq5Z9xKM3PHGWViUVyn0HfgvQag+eu0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683790706; x=1686382706; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=vRPzR3hysjgBXHiDIpMGYdQB4V+B83jsUHDv1pqm4+w=; b=Stfjv8PQv2d6UULFvzqyL3Ec0rr5NdyPS5TS+wYIRCXr0MHZrWUIJIal9cMIqt7/rX iRgDQKZy7MHSXy/zoer+UMfsMX3bV2ZEQYUQ56hoJnAN2U1w7m5+yVp5VMIpe4HEkv8C X8nEhd2eQarxQY+z99+VDXtn3I+bBfW+700skGTnTvuFQqzTja+WRA2SkeoOca2IEG/p 9NomP6dEED2GiwsjA5I+ugsunKytqA3JLpRPjPdtjAZ+cci+shg7tKZU657T9dBAUyKb /hwz7elk0d3HdB0Gnh8E0eGpLtsynOWgNMiuWuxRJOYW8Hk4YwxL6RfAIEqrCOWlIuD/ F8rQ== X-Gm-Message-State: AC+VfDw/UZtRAEVjnk2seJpZnSEHvOkEetLS8ZulYGbLOSTOYX6tnCdA I24G6r9OoHu4SAiDqzCdSmXsOw== X-Google-Smtp-Source: ACHHUZ6MtA5m6CrJkClnTMy9/YanrBz3tC8POvPfLTSOi5fEVkVSC0WJOoGu9yjd0IjVZRg1FI2uDQ== X-Received: by 2002:a05:6a20:12cd:b0:101:7ccd:e197 with SMTP id v13-20020a056a2012cd00b001017ccde197mr9877244pzg.52.1683790706557; Thu, 11 May 2023 00:38:26 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id n14-20020a65488e000000b00502f4c62fd3sm4174332pgs.33.2023.05.11.00.38.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 May 2023 00:38:25 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Selvin Xavier , Kalesh AP Subject: [PATCH for-rc 03/10] RDMA/bnxt_re: Disable/kill tasklet only if it is enabled Date: Thu, 11 May 2023 00:26:18 -0700 Message-Id: <1683789985-22917-4-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1683789985-22917-1-git-send-email-selvin.xavier@broadcom.com> References: <1683789985-22917-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org When the ulp hook to start the IRQ fails because the rings are not available, tasklets are not enabled. In this case when the driver is unloaded, driver calls CREQ tasklet_kill. This causes an indefinite hang as the tasklet is not enabled. Driver shouldn't call tasklet_kill if it is not enabled. So using the creq->requested and nq->requested flags to identify if both tasklets/irqs are registered. Checking this flag while scheduling the tasklet from ISR. Also, added a cleanup for disabling tasklet, in case request_irq fails during start_irq. Check for return value for bnxt_qplib_rcfw_start_irq and in case the bnxt_qplib_rcfw_start_irq fails, return bnxt_re_start_irq without attempting to start NQ IRQs. Fixes: 1ac5a4047975 ("RDMA/bnxt_re: Add bnxt_re RoCE driver") Signed-off-by: Kalesh AP Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/main.c | 12 +++++++++--- drivers/infiniband/hw/bnxt_re/qplib_fp.c | 16 ++++++++++------ drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 14 +++++++++----- 3 files changed, 28 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index b9e2f89..a44f290 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -283,15 +283,21 @@ static void bnxt_re_start_irq(void *handle, struct bnxt_msix_entry *ent) for (indx = 0; indx < rdev->num_msix; indx++) rdev->en_dev->msix_entries[indx].vector = ent[indx].vector; - bnxt_qplib_rcfw_start_irq(rcfw, msix_ent[BNXT_RE_AEQ_IDX].vector, - false); + rc = bnxt_qplib_rcfw_start_irq(rcfw, msix_ent[BNXT_RE_AEQ_IDX].vector, + false); + if (rc) { + ibdev_warn(&rdev->ibdev, "Failed to reinit CREQ\n"); + return; + } for (indx = BNXT_RE_NQ_IDX ; indx < rdev->num_msix; indx++) { nq = &rdev->nq[indx - 1]; rc = bnxt_qplib_nq_start_irq(nq, indx - 1, msix_ent[indx].vector, false); - if (rc) + if (rc) { ibdev_warn(&rdev->ibdev, "Failed to reinit NQ index %d\n", indx - 1); + return; + } } } diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c index e42c756..3c4a3bb 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c @@ -399,6 +399,9 @@ static irqreturn_t bnxt_qplib_nq_irq(int irq, void *dev_instance) void bnxt_qplib_nq_stop_irq(struct bnxt_qplib_nq *nq, bool kill) { + if (!nq->requested) + return; + tasklet_disable(&nq->nq_tasklet); /* Mask h/w interrupt */ bnxt_qplib_ring_nq_db(&nq->nq_db.dbinfo, nq->res->cctx, false); @@ -406,11 +409,10 @@ void bnxt_qplib_nq_stop_irq(struct bnxt_qplib_nq *nq, bool kill) synchronize_irq(nq->msix_vec); if (kill) tasklet_kill(&nq->nq_tasklet); - if (nq->requested) { - irq_set_affinity_hint(nq->msix_vec, NULL); - free_irq(nq->msix_vec, nq); - nq->requested = false; - } + + irq_set_affinity_hint(nq->msix_vec, NULL); + free_irq(nq->msix_vec, nq); + nq->requested = false; } void bnxt_qplib_disable_nq(struct bnxt_qplib_nq *nq) @@ -449,8 +451,10 @@ int bnxt_qplib_nq_start_irq(struct bnxt_qplib_nq *nq, int nq_indx, snprintf(nq->name, sizeof(nq->name), "bnxt_qplib_nq-%d", nq_indx); rc = request_irq(nq->msix_vec, bnxt_qplib_nq_irq, 0, nq->name, nq); - if (rc) + if (rc) { + tasklet_disable(&nq->nq_tasklet); return rc; + } cpumask_clear(&nq->mask); cpumask_set_cpu(nq_indx, &nq->mask); diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index de90691..a668f87 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -636,6 +636,10 @@ void bnxt_qplib_rcfw_stop_irq(struct bnxt_qplib_rcfw *rcfw, bool kill) struct bnxt_qplib_creq_ctx *creq; creq = &rcfw->creq; + + if (!creq->requested) + return; + tasklet_disable(&creq->creq_tasklet); /* Mask h/w interrupts */ bnxt_qplib_ring_nq_db(&creq->creq_db.dbinfo, rcfw->res->cctx, false); @@ -644,10 +648,8 @@ void bnxt_qplib_rcfw_stop_irq(struct bnxt_qplib_rcfw *rcfw, bool kill) if (kill) tasklet_kill(&creq->creq_tasklet); - if (creq->requested) { - free_irq(creq->msix_vec, rcfw); - creq->requested = false; - } + free_irq(creq->msix_vec, rcfw); + creq->requested = false; } void bnxt_qplib_disable_rcfw_channel(struct bnxt_qplib_rcfw *rcfw) @@ -693,8 +695,10 @@ int bnxt_qplib_rcfw_start_irq(struct bnxt_qplib_rcfw *rcfw, int msix_vector, tasklet_enable(&creq->creq_tasklet); rc = request_irq(creq->msix_vec, bnxt_qplib_creq_irq, 0, "bnxt_qplib_creq", rcfw); - if (rc) + if (rc) { + tasklet_disable(&creq->creq_tasklet); return rc; + } creq->requested = true; bnxt_qplib_ring_nq_db(&creq->creq_db.dbinfo, rcfw->res->cctx, true);