From patchwork Tue Jan 23 04:54:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Selvin Xavier X-Patchwork-Id: 13526831 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 379DD23BD for ; Tue, 23 Jan 2024 05:11:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705986668; cv=none; b=V3ze6gG1kxtl6XkJyoBsSWci6mz5PhEeNz6ytsXL3jMDZVeLub8/8GBY2wksmPoYF0BIzXL6fhxIcLAX4tTCP+nxdn4twYeejF5FV3xDo2QAt0At5id/kXzPHdlEJ1nUhD92ozRWj/svg1nec86sij+VUgcgjlkCkDvW2WNtYg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705986668; c=relaxed/simple; bh=5cVuwtaVLgoYIVnYKAKnWBKSfZiFw/ZtUhHN9FAKXaA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: Content-Type; b=KKtlxTxd0Ke+eOJ+TShAWbc2CCC+ECmOZgB744V9Cjao22etCCfpVq4UaB+hXDwDFJCYpcVJ693FAue9g8uZf/y4C4gQMLRaU4ULQwFz9DP3u4KcP+dS25iY8SCFdaefDI07/zBs8tGYIB1IGb8wJbpqXhXkBWdcyXyt8pwOHQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=e+BB92kQ; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="e+BB92kQ" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-6d9f94b9186so3843921b3a.0 for ; Mon, 22 Jan 2024 21:11:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1705986666; x=1706591466; darn=vger.kernel.org; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=4YaZ0YdzS8TUbiDv9DkUyCiRYGLiis7g8Jj1PIYd/dY=; b=e+BB92kQ9xynzbtR23esjxlk1BzrqAtH1+eDjkDV2lURlWnQ7ZzHfbHVpGvUJhU5WF HUN4CaxTKoFfmu0sqVfFZejT6T8mioJI2mXiLvKt1rMKWd/Ub2lGqiNTOlnI3QkJjW58 7w0Yr2y3ipacN6/TKTocyCcwqDlb6VxTQ4TYY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705986666; x=1706591466; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=4YaZ0YdzS8TUbiDv9DkUyCiRYGLiis7g8Jj1PIYd/dY=; b=OXk0zNcM1ho3D6RhPonNklubXVlZgTRAkVkLhlSouCjr/OriPgtDWGCuGm9vmt0pI+ cNxaSEFtB5dFa0g3UnKPVH+S7KWEC7NNDbXFSqk69QE8XREwSDslfy/GqDlOlec7jK7U nD8LQmS75PcgGNBrH++TqEWcLGeOfeKHL0u28OO926ioP6kL6VHasGeBJ4lB4/FZeagH 1KYdrIsbchXGMHFGVj6iXlkUEKofbKadoY2Mf+nYufvpIVRgf6a7nd1tPvSO9eheMrLW txF8buMvmFYpt+QdSucOSBFi/d6jLkfYEHNYbYSFHwMKjJZQdOYu9z7qFBWo0lf6Iws5 CJAg== X-Gm-Message-State: AOJu0YwzMkgJW0b1QNpJ7LHIN2vXUiygCEE0xil5cfccbq0OaaH+jQFT NMM2GBun6YpgYEtNWsnUBgSoFsFd6KFMCoO0ZFYwC1nbGnpJyqbd+9qNBOEcRQ== X-Google-Smtp-Source: AGHT+IEYrPym4wXIN0g/+6QIqy6A/UELL+IwJVzoG+Pv/wXe46qSdJSIUG26sblLPg55qfjrY3mpGg== X-Received: by 2002:a05:6a20:548a:b0:199:a2a8:da72 with SMTP id i10-20020a056a20548a00b00199a2a8da72mr6691894pzk.10.1705986666503; Mon, 22 Jan 2024 21:11:06 -0800 (PST) Received: from dhcp-10-192-206-197.iig.avagotech.net.net ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id lp4-20020a056a003d4400b006dce766903dsm672949pfb.90.2024.01.22.21.11.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Jan 2024 21:11:05 -0800 (PST) From: Selvin Xavier To: leon@kernel.org, jgg@ziepe.ca Cc: linux-rdma@vger.kernel.org, andrew.gospodarek@broadcom.com, Kalesh AP , Kashyap Desai , Selvin Xavier Subject: [PATCH for-rc 3/5] RDMA/bnxt_re: Fix unconditional fence for newer adapters Date: Mon, 22 Jan 2024 20:54:35 -0800 Message-Id: <1705985677-15551-4-git-send-email-selvin.xavier@broadcom.com> X-Mailer: git-send-email 2.5.5 In-Reply-To: <1705985677-15551-1-git-send-email-selvin.xavier@broadcom.com> References: <1705985677-15551-1-git-send-email-selvin.xavier@broadcom.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Kalesh AP Older adapters required an unconditional fence for non-wire memory operations. Newer adapters doesn't require this and therefore, disabling the unconditional fence. Fixes: 1801d87b3598 ("RDMA/bnxt_re: Support new 5760X P7 devices") Signed-off-by: Kashyap Desai Signed-off-by: Kalesh AP Signed-off-by: Selvin Xavier --- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index e1ea492..fdb4fde 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -2566,11 +2566,6 @@ static int bnxt_re_build_inv_wqe(const struct ib_send_wr *wr, wqe->type = BNXT_QPLIB_SWQE_TYPE_LOCAL_INV; wqe->local_inv.inv_l_key = wr->ex.invalidate_rkey; - /* Need unconditional fence for local invalidate - * opcode to work as expected. - */ - wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE; - if (wr->send_flags & IB_SEND_SIGNALED) wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_SIGNAL_COMP; if (wr->send_flags & IB_SEND_SOLICITED) @@ -2593,12 +2588,6 @@ static int bnxt_re_build_reg_wqe(const struct ib_reg_wr *wr, wqe->frmr.levels = qplib_frpl->hwq.level; wqe->type = BNXT_QPLIB_SWQE_TYPE_REG_MR; - /* Need unconditional fence for reg_mr - * opcode to function as expected. - */ - - wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE; - if (wr->wr.send_flags & IB_SEND_SIGNALED) wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_SIGNAL_COMP; @@ -2729,6 +2718,18 @@ static int bnxt_re_post_send_shadow_qp(struct bnxt_re_dev *rdev, return rc; } +static void bnxt_re_legacy_set_uc_fence(struct bnxt_qplib_swqe *wqe) +{ + /* Need unconditional fence for non-wire memory opcode + * to work as expected. + */ + if (wqe->type == BNXT_QPLIB_SWQE_TYPE_LOCAL_INV || + wqe->type == BNXT_QPLIB_SWQE_TYPE_FAST_REG_MR || + wqe->type == BNXT_QPLIB_SWQE_TYPE_REG_MR || + wqe->type == BNXT_QPLIB_SWQE_TYPE_BIND_MW) + wqe->flags |= BNXT_QPLIB_SWQE_FLAGS_UC_FENCE; +} + int bnxt_re_post_send(struct ib_qp *ib_qp, const struct ib_send_wr *wr, const struct ib_send_wr **bad_wr) { @@ -2808,8 +2809,11 @@ int bnxt_re_post_send(struct ib_qp *ib_qp, const struct ib_send_wr *wr, rc = -EINVAL; goto bad; } - if (!rc) + if (!rc) { + if (!bnxt_qplib_is_chip_gen_p5_p7(qp->rdev->chip_ctx)) + bnxt_re_legacy_set_uc_fence(&wqe); rc = bnxt_qplib_post_send(&qp->qplib_qp, &wqe); + } bad: if (rc) { ibdev_err(&qp->rdev->ibdev,