From patchwork Thu Jun 8 10:25:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13271928 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC8D2C7EE23 for ; Thu, 8 Jun 2023 10:37:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235695AbjFHKhk (ORCPT ); Thu, 8 Jun 2023 06:37:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236155AbjFHKhh (ORCPT ); Thu, 8 Jun 2023 06:37:37 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F79E2D4A for ; Thu, 8 Jun 2023 03:37:24 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-65055aa4ed7so218175b3a.3 for ; Thu, 08 Jun 2023 03:37:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1686220644; x=1688812644; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=tq7eOSDAeLLk5LknjFHp5iJKCx5zSSFDlnOSqSoGxxQ=; b=D+BIkPGN0v1la3Oje7+Ym9PodL8nr/mqUyx2K2thkJOGQbKburYkMKEhnZrye6HXBF 35nAIj1dUnJrFQwhuZYydJk92FvWx20X2v2Qp3/N2S/nyk24tPfVzdfpzyrmko70q5Xx HdYv7ZZxXD0yVGQSZvZJpHLQ9LkyXuiI9Hjl0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686220644; x=1688812644; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tq7eOSDAeLLk5LknjFHp5iJKCx5zSSFDlnOSqSoGxxQ=; b=Po+S7KTIjAZVt5HsKwJJPhReIMzambu1v6m2VrfXsHeBxu3ru76m5t/vF3G8ynqp5T NvrywDoKqVDcjWhTNdAPF5baq/jKDezbuJxNScctuPXJ9VZimPNyvg9PXl/uEMIERkH7 62co3FwYXTuV5dkAQsb3LBJ8OKyW/9Chtj1bC45Z8RgRfoMvirjsW2Zfc5A0IZ30NHxb K0GPRO0K/B+JqBj7Wm7Nm2Gunsvz6lP0XtvkOvox9QfyOvIuK1CkuPTK9ijKegTDOJN0 dZ1UbRRDs2+bVACWQ+uOWLpr4GJyIe50LqGrsyx31xV1s0fUjWCUJ3CMYHeBsMul7uDN 26BQ== X-Gm-Message-State: AC+VfDyBqMgC7xqtaBLylP8uguy/msEnByYGAXeD18pp7q7UiOzEojmR SRa5e8Hbxxhr4fC2gSJ7H/FAsw== X-Google-Smtp-Source: ACHHUZ6nHO7EUYVXNabAN880qe47lfqLfX/AOkbefHePfqb8Jt5zQBVyUitE3qGtnl3yyYWCZo8JRw== X-Received: by 2002:a05:6a20:42a3:b0:112:be13:5a54 with SMTP id o35-20020a056a2042a300b00112be135a54mr3850053pzj.50.1686220643983; Thu, 08 Jun 2023 03:37:23 -0700 (PDT) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id jj11-20020a170903048b00b001a980a23802sm1128510plb.111.2023.06.08.03.37.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jun 2023 03:37:23 -0700 (PDT) From: Selvin Xavier To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Kashyap Desai , Selvin Xavier Subject: [PATCH for-next 10/17] RDMA/bnxt_re: handle command completions after driver detect a timedout Date: Thu, 8 Jun 2023 03:25:01 -0700 Message-Id: <1686219908-11181-11-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1686219908-11181-1-git-send-email-selvin.xavier@broadcom.com> References: <1686219908-11181-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Kashyap Desai If calling context detect command timeout, associated memory stored on stack will not be valid. If firmware complete the same command later, this causes incorrect memory access by driver. Added is_waiter_alive to handle delayed completion by firmware. is_waiter_alive is set and reset under command queue lock. Signed-off-by: Kashyap Desai Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/qplib_rcfw.c | 46 +++++++++++++++++++----------- drivers/infiniband/hw/bnxt_re/qplib_rcfw.h | 1 + 2 files changed, 30 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c index 3215f8a..e105961 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.c @@ -216,6 +216,7 @@ static int __send_message(struct bnxt_qplib_rcfw *rcfw, crsqe->free_slots = free_slots; crsqe->resp = (struct creq_qp_event *)msg->resp; crsqe->resp->cookie = cpu_to_le16(cookie); + crsqe->is_waiter_alive = true; crsqe->req_size = __get_cmdq_base_cmd_size(msg->req, msg->req_sz); if (__get_cmdq_base_resp_size(msg->req, msg->req_sz) && msg->sb) { struct bnxt_qplib_rcfw_sbuf *sbuf = msg->sb; @@ -347,7 +348,9 @@ static int __bnxt_qplib_rcfw_send_message(struct bnxt_qplib_rcfw *rcfw, struct bnxt_qplib_cmdqmsg *msg) { struct creq_qp_event *evnt = (struct creq_qp_event *)msg->resp; - u16 cookie; + struct bnxt_qplib_crsqe *crsqe; + unsigned long flags; + u16 cookie, cbit; int rc = 0; u8 opcode; @@ -363,6 +366,7 @@ static int __bnxt_qplib_rcfw_send_message(struct bnxt_qplib_rcfw *rcfw, cookie = le16_to_cpu(__get_cmdq_base_cookie(msg->req, msg->req_sz)) & RCFW_MAX_COOKIE_VALUE; + cbit = cookie % rcfw->cmdq_depth; if (msg->block) rc = __block_for_resp(rcfw, cookie, opcode); @@ -378,6 +382,14 @@ static int __bnxt_qplib_rcfw_send_message(struct bnxt_qplib_rcfw *rcfw, return rc; } + if (rc) { + spin_lock_irqsave(&rcfw->cmdq.hwq.lock, flags); + crsqe = &rcfw->crsqe_tbl[cbit]; + crsqe->is_waiter_alive = false; + spin_unlock_irqrestore(&rcfw->cmdq.hwq.lock, flags); + return -ETIMEDOUT; + } + if (evnt->status) { /* failed with status */ dev_err(&rcfw->pdev->dev, "cmdq[%#x]=%#x status %#x\n", @@ -480,15 +492,16 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, struct creq_qp_error_notification *err_event; struct bnxt_qplib_hwq *hwq = &rcfw->cmdq.hwq; struct bnxt_qplib_crsqe *crsqe; + u32 qp_id, tbl_indx, req_size; struct bnxt_qplib_qp *qp; u16 cbit, blocked = 0; + bool is_waiter_alive; struct pci_dev *pdev; unsigned long flags; u32 wait_cmds = 0; __le16 mcookie; u16 cookie; int rc = 0; - u32 qp_id, tbl_indx; pdev = rcfw->pdev; switch (qp_event->event) { @@ -525,26 +538,25 @@ static int bnxt_qplib_process_qp_event(struct bnxt_qplib_rcfw *rcfw, cookie &= RCFW_MAX_COOKIE_VALUE; cbit = cookie % rcfw->cmdq_depth; crsqe = &rcfw->crsqe_tbl[cbit]; - if (crsqe->resp && - crsqe->resp->cookie == mcookie) { - memcpy(crsqe->resp, qp_event, sizeof(*qp_event)); - crsqe->resp = NULL; - } else { - if (crsqe->resp && crsqe->resp->cookie) - dev_err(&pdev->dev, - "CMD %s cookie sent=%#x, recd=%#x\n", - crsqe->resp ? "mismatch" : "collision", - crsqe->resp ? crsqe->resp->cookie : 0, - mcookie); - } if (!test_and_clear_bit(cbit, rcfw->cmdq.cmdq_bitmap)) dev_warn(&pdev->dev, "CMD bit %d was not requested\n", cbit); - hwq->cons += crsqe->req_size; + + if (crsqe->is_waiter_alive) { + if (crsqe->resp) + memcpy(crsqe->resp, qp_event, sizeof(*qp_event)); + if (!blocked) + wait_cmds++; + } + + req_size = crsqe->req_size; + is_waiter_alive = crsqe->is_waiter_alive; + crsqe->req_size = 0; + if (!is_waiter_alive) + crsqe->resp = NULL; - if (!blocked) - wait_cmds++; + hwq->cons += req_size; spin_unlock_irqrestore(&hwq->lock, flags); } *num_wait += wait_cmds; diff --git a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h index 089e616..6ed81c1 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h +++ b/drivers/infiniband/hw/bnxt_re/qplib_rcfw.h @@ -152,6 +152,7 @@ struct bnxt_qplib_crsqe { u32 req_size; /* Free slots at the time of submission */ u32 free_slots; + bool is_waiter_alive; }; struct bnxt_qplib_rcfw_sbuf {