From patchwork Thu Aug 25 02:39:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cheng Xu X-Patchwork-Id: 12954154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A80C8C25B06 for ; Thu, 25 Aug 2022 02:39:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231289AbiHYCjX (ORCPT ); Wed, 24 Aug 2022 22:39:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231166AbiHYCjN (ORCPT ); Wed, 24 Aug 2022 22:39:13 -0400 Received: from out30-132.freemail.mail.aliyun.com (out30-132.freemail.mail.aliyun.com [115.124.30.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EC089A9C9 for ; Wed, 24 Aug 2022 19:39:10 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R961e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=chengyou@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0VNBL-a2_1661395147; Received: from localhost(mailfrom:chengyou@linux.alibaba.com fp:SMTPD_---0VNBL-a2_1661395147) by smtp.aliyun-inc.com; Thu, 25 Aug 2022 10:39:07 +0800 From: Cheng Xu To: jgg@ziepe.ca, leon@kernel.org Cc: linux-rdma@vger.kernel.org, KaiShen@linux.alibaba.com Subject: [PATCH for-next v2 1/2] RDMA/erdma: Introduce internal post_send/post_recv for qp drain Date: Thu, 25 Aug 2022 10:39:04 +0800 Message-Id: <20220825023905.28274-2-chengyou@linux.alibaba.com> X-Mailer: git-send-email 2.37.0 In-Reply-To: <20220825023905.28274-1-chengyou@linux.alibaba.com> References: <20220825023905.28274-1-chengyou@linux.alibaba.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org For erdma, hardware won't process newly posted send WRs or recv WRs after QP state changed to error, and no flush cqes will generated for them. So, internal post_send and post_recv functions are introduced to prevent the new send WRs or recv WRs. Reviewed-by: Leon Romanovsky Signed-off-by: Cheng Xu --- drivers/infiniband/hw/erdma/erdma_main.c | 4 +- drivers/infiniband/hw/erdma/erdma_qp.c | 45 ++++++++++++++++++++--- drivers/infiniband/hw/erdma/erdma_verbs.h | 17 +++++++-- 3 files changed, 55 insertions(+), 11 deletions(-) diff --git a/drivers/infiniband/hw/erdma/erdma_main.c b/drivers/infiniband/hw/erdma/erdma_main.c index 07e743d24847..4921ebc1286d 100644 --- a/drivers/infiniband/hw/erdma/erdma_main.c +++ b/drivers/infiniband/hw/erdma/erdma_main.c @@ -460,8 +460,8 @@ static const struct ib_device_ops erdma_device_ops = { .mmap = erdma_mmap, .mmap_free = erdma_mmap_free, .modify_qp = erdma_modify_qp, - .post_recv = erdma_post_recv, - .post_send = erdma_post_send, + .post_recv = erdma_post_recv_nodrain, + .post_send = erdma_post_send_nodrain, .poll_cq = erdma_poll_cq, .query_device = erdma_query_device, .query_gid = erdma_query_gid, diff --git a/drivers/infiniband/hw/erdma/erdma_qp.c b/drivers/infiniband/hw/erdma/erdma_qp.c index bc3ec22a62c5..abf8b134d076 100644 --- a/drivers/infiniband/hw/erdma/erdma_qp.c +++ b/drivers/infiniband/hw/erdma/erdma_qp.c @@ -475,8 +475,8 @@ static void kick_sq_db(struct erdma_qp *qp, u16 pi) writeq(db_data, qp->kern_qp.hw_sq_db); } -int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr, - const struct ib_send_wr **bad_send_wr) +static int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr, + const struct ib_send_wr **bad_send_wr, bool drain) { struct erdma_qp *qp = to_eqp(ibqp); int ret = 0; @@ -488,6 +488,16 @@ int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr, return -EINVAL; spin_lock_irqsave(&qp->lock, flags); + + if (unlikely(qp->flags & ERDMA_QP_FLAGS_TX_STOPPED)) { + *bad_send_wr = send_wr; + ret = -EINVAL; + goto out; + } + + if (unlikely(drain)) + qp->flags |= ERDMA_QP_FLAGS_TX_STOPPED; + sq_pi = qp->kern_qp.sq_pi; while (wr) { @@ -507,11 +517,19 @@ int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr, wr = wr->next; } - spin_unlock_irqrestore(&qp->lock, flags); +out: + spin_unlock_irqrestore(&qp->lock, flags); return ret; } +int erdma_post_send_nodrain(struct ib_qp *ibqp, + const struct ib_send_wr *send_wr, + const struct ib_send_wr **bad_send_wr) +{ + return erdma_post_send(ibqp, send_wr, bad_send_wr, false); +} + static int erdma_post_recv_one(struct erdma_qp *qp, const struct ib_recv_wr *recv_wr) { @@ -542,8 +560,8 @@ static int erdma_post_recv_one(struct erdma_qp *qp, return 0; } -int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, - const struct ib_recv_wr **bad_recv_wr) +static int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, + const struct ib_recv_wr **bad_recv_wr, bool drain) { const struct ib_recv_wr *wr = recv_wr; struct erdma_qp *qp = to_eqp(ibqp); @@ -552,6 +570,15 @@ int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, spin_lock_irqsave(&qp->lock, flags); + if (unlikely(qp->flags & ERDMA_QP_FLAGS_RX_STOPPED)) { + ret = -EINVAL; + *bad_recv_wr = recv_wr; + goto out; + } + + if (unlikely(drain)) + qp->flags |= ERDMA_QP_FLAGS_RX_STOPPED; + while (wr) { ret = erdma_post_recv_one(qp, wr); if (ret) { @@ -561,6 +588,14 @@ int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, wr = wr->next; } +out: spin_unlock_irqrestore(&qp->lock, flags); return ret; } + +int erdma_post_recv_nodrain(struct ib_qp *ibqp, + const struct ib_recv_wr *recv_wr, + const struct ib_recv_wr **bad_recv_wr) +{ + return erdma_post_recv(ibqp, recv_wr, bad_recv_wr, false); +} diff --git a/drivers/infiniband/hw/erdma/erdma_verbs.h b/drivers/infiniband/hw/erdma/erdma_verbs.h index c7baddb1f292..f4148fbac878 100644 --- a/drivers/infiniband/hw/erdma/erdma_verbs.h +++ b/drivers/infiniband/hw/erdma/erdma_verbs.h @@ -195,6 +195,12 @@ struct erdma_qp_attrs { u8 pd_len; }; +enum erdma_qp_flags { + ERDMA_QP_FLAGS_DRAIN_ISSUED = (1 << 0), + ERDMA_QP_FLAGS_TX_STOPPED = (1 << 1), + ERDMA_QP_FLAGS_RX_STOPPED = (1 << 2), +}; + struct erdma_qp { struct ib_qp ibqp; struct kref ref; @@ -202,6 +208,7 @@ struct erdma_qp { struct erdma_dev *dev; struct erdma_cep *cep; struct rw_semaphore state_lock; + unsigned long flags; union { struct erdma_kqp kern_qp; @@ -328,10 +335,12 @@ void erdma_mmap_free(struct rdma_user_mmap_entry *rdma_entry); void erdma_qp_get_ref(struct ib_qp *ibqp); void erdma_qp_put_ref(struct ib_qp *ibqp); struct ib_qp *erdma_get_ibqp(struct ib_device *dev, int id); -int erdma_post_send(struct ib_qp *ibqp, const struct ib_send_wr *send_wr, - const struct ib_send_wr **bad_send_wr); -int erdma_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *recv_wr, - const struct ib_recv_wr **bad_recv_wr); +int erdma_post_send_nodrain(struct ib_qp *ibqp, + const struct ib_send_wr *send_wr, + const struct ib_send_wr **bad_send_wr); +int erdma_post_recv_nodrain(struct ib_qp *ibqp, + const struct ib_recv_wr *recv_wr, + const struct ib_recv_wr **bad_recv_wr); int erdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); struct ib_mr *erdma_ib_alloc_mr(struct ib_pd *ibpd, enum ib_mr_type mr_type, u32 max_num_sg);