From patchwork Wed Apr 5 04:26:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13201135 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB05BC7619A for ; Wed, 5 Apr 2023 04:26:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236696AbjDEE0o (ORCPT ); Wed, 5 Apr 2023 00:26:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236689AbjDEE0n (ORCPT ); Wed, 5 Apr 2023 00:26:43 -0400 Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B74BE172C for ; Tue, 4 Apr 2023 21:26:42 -0700 (PDT) Received: by mail-oi1-x233.google.com with SMTP id l18so25813167oic.13 for ; Tue, 04 Apr 2023 21:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680668802; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=tZw/e3Zy7PQFJtPm84yb5P9q2Zu9EmpszPG7N5mmwsM=; b=kC/zWCEoLUUfPEgQjHXvic0O962EKDbygGgk4kr/KK74ySFL0197xE+S11QA4/QK13 99qK7HlKeiA8g8u0wtp3BCdPyujgrKT5oZ5xxhWnhop9E7lmtwu9I11vaCBO6+uuT/7h SGcxlC7/QLXKcRLDx8L943cF3S2B1wzJoPc6mkWh7JhXpIVVl1DMUftbwSlJYiJzN4Fv /VBCuOVfFt3Q0GOmcrpMBoChLWdmRWxYs3YmWo9yq1AW+GUxWzPKU5Pvcnhbch/ZWfxU ItJhEkzc/rPztX5vdoubh8/aznw+uCOfULPMWSgLwVwo6vh/F0gTHLwaqbE/kIR1qrG1 d0+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680668802; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=tZw/e3Zy7PQFJtPm84yb5P9q2Zu9EmpszPG7N5mmwsM=; b=cQgiluc2s55q+4ejslL8cgm4OFYU3l/Y7c/VXGsY+zBUw18V50V07ihnAmtnToqYxn Kai9FzmZfmc534SDsFFBHAZ16iHfeYA3pVTIWJ8ycOPA1Y6ZtCsmv4I/Rushqng0tUxe QcbUaH5v6SDP9QfQG27DZNMbZiPitK0t4eG9qF4u5yiiXqJtbS6Mhq7K4Z8jQxt3zlLC 5hQF6JuCDn1nYCtMmjfbMvcIiKLnmcSwm5MSbq1k74KFiJ1J3Nm/VXSLvdPm9yPHSNhU A2+picJ3td9rdFt5SSNN0gwKl8h3hYpxYKtUGl9KPELuwvoWeOvcum/HBxINpJYOxzqA dVnw== X-Gm-Message-State: AAQBX9f0T8KjjRkHujKAVG5t37LAPd2joYrUsQrSNPx4H1gZTuIIspNh cGi/WmiONODxzVpiFzOcbhw= X-Google-Smtp-Source: AKy350Y9btaLinUAchtcubGtqMDW7Ro17rVNdPp1KLoEWM+4pJiZ2mmTxXO0glW1+Vo7LD7o1nOE+w== X-Received: by 2002:a05:6808:3a96:b0:386:eec2:957a with SMTP id fb22-20020a0568083a9600b00386eec2957amr2296195oib.9.1680668802038; Tue, 04 Apr 2023 21:26:42 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.gmail.com with ESMTPSA id x80-20020a4a4153000000b0053d9be4be68sm6321326ooa.19.2023.04.04.21.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 21:26:27 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 1/5] RDMA/rxe: Remove qp->resp.state Date: Tue, 4 Apr 2023 23:26:07 -0500 Message-Id: <20230405042611.6467-1-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.37.2 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The rxe driver has four different QP state variables, qp->attr.qp_state, qp->req.state, qp->comp.state, and qp->resp.state. All of these basically carry the same information. This patch replaces uses of qp->resp.state by qp->attr.qp_state. This is the first of three patches which will remove all but the qp->attr.qp_state variable. This will bring the driver closer to the IBA description. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_net.c | 2 +- drivers/infiniband/sw/rxe/rxe_qp.c | 5 ----- drivers/infiniband/sw/rxe/rxe_recv.c | 2 +- drivers/infiniband/sw/rxe/rxe_resp.c | 10 +++++----- drivers/infiniband/sw/rxe/rxe_verbs.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 6 files changed, 8 insertions(+), 14 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index a2ace42e9536..2be2425083ce 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -414,7 +414,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, struct rxe_dev *rxe = to_rdev(qp->ibqp.device); if ((is_request && (qp->req.state != QP_STATE_READY)) || - (!is_request && (qp->resp.state != QP_STATE_READY))) { + (!is_request && (qp_state(qp) <= IB_QPS_RTR))) { rxe_dbg_qp(qp, "Packet dropped. QP is not in ready state\n"); goto drop; } diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 49891f8ed4e6..38694b355bed 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -287,7 +287,6 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, qp->resp.opcode = OPCODE_NONE; qp->resp.msn = 0; - qp->resp.state = QP_STATE_RESET; return 0; } @@ -479,7 +478,6 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* move qp to the reset state */ qp->req.state = QP_STATE_RESET; qp->comp.state = QP_STATE_RESET; - qp->resp.state = QP_STATE_RESET; /* drain work and packet queuesc */ rxe_requester(qp); @@ -532,7 +530,6 @@ static void rxe_qp_drain(struct rxe_qp *qp) void rxe_qp_error(struct rxe_qp *qp) { qp->req.state = QP_STATE_ERROR; - qp->resp.state = QP_STATE_ERROR; qp->comp.state = QP_STATE_ERROR; qp->attr.qp_state = IB_QPS_ERR; @@ -663,13 +660,11 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_INIT: rxe_dbg_qp(qp, "state -> INIT\n"); qp->req.state = QP_STATE_INIT; - qp->resp.state = QP_STATE_INIT; qp->comp.state = QP_STATE_INIT; break; case IB_QPS_RTR: rxe_dbg_qp(qp, "state -> RTR\n"); - qp->resp.state = QP_STATE_READY; break; case IB_QPS_RTS: diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index 434a693cd4a5..ac42ceccf71f 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -39,7 +39,7 @@ static int check_type_state(struct rxe_dev *rxe, struct rxe_pkt_info *pkt, } if (pkt->mask & RXE_REQ_MASK) { - if (unlikely(qp->resp.state != QP_STATE_READY)) + if (unlikely(qp_state(qp) <= IB_QPS_RTR)) return -EINVAL; } else if (unlikely(qp->req.state < QP_STATE_READY || qp->req.state > QP_STATE_DRAINED)) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 01e3cbea8445..67eac616235c 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1137,7 +1137,7 @@ static enum resp_states do_complete(struct rxe_qp *qp, return RESPST_ERR_CQ_OVERFLOW; finish: - if (unlikely(qp->resp.state == QP_STATE_ERROR)) + if (unlikely(qp_state(qp) == IB_QPS_ERR)) return RESPST_CHK_RESOURCE; if (unlikely(!pkt)) return RESPST_DONE; @@ -1464,10 +1464,10 @@ int rxe_responder(struct rxe_qp *qp) struct rxe_pkt_info *pkt = NULL; int ret; - if (!qp->valid || qp->resp.state == QP_STATE_ERROR || - qp->resp.state == QP_STATE_RESET) { - bool notify = qp->valid && - (qp->resp.state == QP_STATE_ERROR); + if (!qp->valid || qp_state(qp) == IB_QPS_ERR || + qp_state(qp) == IB_QPS_RESET) { + bool notify = qp->valid && (qp_state(qp) == IB_QPS_ERR); + drain_req_pkts(qp); flush_recv_queue(qp, notify); goto exit; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 4e2db7c2e4ed..2a5da3160db9 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -1012,7 +1012,7 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, spin_unlock_irqrestore(&rq->producer_lock, flags); - if (qp->resp.state == QP_STATE_ERROR) + if (qp_state(qp) == IB_QPS_ERR) rxe_sched_task(&qp->resp.task); err_out: diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index d812093a3916..12594cb2a9cf 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -173,7 +173,6 @@ struct resp_res { }; struct rxe_resp_info { - enum rxe_qp_state state; u32 msn; u32 psn; u32 ack_psn; From patchwork Wed Apr 5 04:26:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13201136 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E580C7619A for ; Wed, 5 Apr 2023 04:26:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236689AbjDEE0y (ORCPT ); Wed, 5 Apr 2023 00:26:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229587AbjDEE0x (ORCPT ); Wed, 5 Apr 2023 00:26:53 -0400 Received: from mail-oi1-x234.google.com (mail-oi1-x234.google.com [IPv6:2607:f8b0:4864:20::234]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D283310CF for ; Tue, 4 Apr 2023 21:26:52 -0700 (PDT) Received: by mail-oi1-x234.google.com with SMTP id y184so25827633oiy.8 for ; Tue, 04 Apr 2023 21:26:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680668812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9O+HD33FVdquyuVKwfAdYxW6gFbaxtWKtNZ/Gth7z5k=; b=bTFk4IaGSUAfSzBUcUR2+ZQdZ8EiuSf9+8V3Mvdn5zY4WVM4Gmw7wKwhwi0BBxFijw TB/01VwoMIw+0KYUIPZK2pjjC305+DeP4d9ABQOe0vtgkzIIwPtNlRwxfNqptRoMK5zz WN95K9iZFqdKFkkB1S2U4/bZ3QzdqyCt7tcXa5WlGFyFnMuqkYQsQLuVuSpEf12lxgX5 qBuIzOhDYg73lPIdoTVhvgzJQkN37iOXZSI/KriIgOY/g4nBb7Wy/cYL0fJjIBoTPbGD efdhBdD695eBeS9eTS9trH3zK4vx48OLJdM0F6s0btX+ZOJtlcRefDiOUBUhA1sdVRFp E5ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680668812; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9O+HD33FVdquyuVKwfAdYxW6gFbaxtWKtNZ/Gth7z5k=; b=xeZbQCDTBFPtiehUyKOBBHkrNZOPiv1UfOCnFh/aauc5647Y/4QV/Dfp+tZradbWkB q15QBOGEtQM94rh5kvPTCRBPrGa7xIOojCxzUAR9X10Sa8SRCRsOH3H6fh/lXAHpW6kq rmZSjmRm91ApA7WbezBIaQ6qEmg0j++vOkfjOMy9fNJSmZVR8HH5yYkxpjWqxcjbNsn4 ULXaxyVonanbnPC1CqQ0G7BYCQv3wBh8U2NuvYE3VyONSUWGmaEZaAm8Xej2iSzgvwAp txopFN3NuduyFmNWvqikbeXPsvIB+Tl8JSW7hi+KI2kXowVhlEzL096V/r87l9PhbibN VrMw== X-Gm-Message-State: AAQBX9eKrIfBt9edzhocGd22GDVrGMNWpyHI4DFew4gNg3BJ5As7whwX vdIP/bkf3g9f6Ci00SOdnoR8IAaTBaA= X-Google-Smtp-Source: AKy350Zn14eDwyTf8SLFld70XnvyIPyFZ+3cPtrNjWRuWLvJLZJEsjJwYQhvBcHBoF6r/kUmwqzdkA== X-Received: by 2002:a05:6808:46:b0:389:7b6b:7a2c with SMTP id v6-20020a056808004600b003897b6b7a2cmr2260320oic.44.1680668812161; Tue, 04 Apr 2023 21:26:52 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.gmail.com with ESMTPSA id x80-20020a4a4153000000b0053d9be4be68sm6321326ooa.19.2023.04.04.21.26.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 21:26:45 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 2/5] RDMA/rxe: Remove qp->comp.state Date: Tue, 4 Apr 2023 23:26:08 -0500 Message-Id: <20230405042611.6467-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230405042611.6467-1-rpearsonhpe@gmail.com> References: <20230405042611.6467-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The rxe driver has four different QP state variables, qp->attr.qp_state, qp->req.state, qp->comp.state, and qp->resp.state. All of these basically carry the same information. This patch replaces uses of qp->comp.state by qp->attr.qp_state. This is the second of three patches which will remove all but the qp->attr.qp_state variable. This will bring the driver closer to the IBA description. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 8 ++++---- drivers/infiniband/sw/rxe/rxe_qp.c | 5 ----- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 3 files changed, 4 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 2c70cdcd55dc..173ebfe784e6 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -619,10 +619,10 @@ int rxe_completer(struct rxe_qp *qp) enum comp_state state; int ret; - if (!qp->valid || qp->comp.state == QP_STATE_ERROR || - qp->comp.state == QP_STATE_RESET) { - bool notify = qp->valid && - (qp->comp.state == QP_STATE_ERROR); + if (!qp->valid || qp_state(qp) == IB_QPS_ERR || + qp_state(qp) == IB_QPS_RESET) { + bool notify = qp->valid && (qp_state(qp) == IB_QPS_ERR); + drain_resp_pkts(qp); flush_send_queue(qp, notify); goto exit; diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 38694b355bed..a4486d9c540e 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -232,7 +232,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, QUEUE_TYPE_FROM_CLIENT); qp->req.state = QP_STATE_RESET; - qp->comp.state = QP_STATE_RESET; qp->req.opcode = -1; qp->comp.opcode = -1; @@ -477,7 +476,6 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* move qp to the reset state */ qp->req.state = QP_STATE_RESET; - qp->comp.state = QP_STATE_RESET; /* drain work and packet queuesc */ rxe_requester(qp); @@ -530,7 +528,6 @@ static void rxe_qp_drain(struct rxe_qp *qp) void rxe_qp_error(struct rxe_qp *qp) { qp->req.state = QP_STATE_ERROR; - qp->comp.state = QP_STATE_ERROR; qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ @@ -660,7 +657,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_INIT: rxe_dbg_qp(qp, "state -> INIT\n"); qp->req.state = QP_STATE_INIT; - qp->comp.state = QP_STATE_INIT; break; case IB_QPS_RTR: @@ -670,7 +666,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_RTS: rxe_dbg_qp(qp, "state -> RTS\n"); qp->req.state = QP_STATE_READY; - qp->comp.state = QP_STATE_READY; break; case IB_QPS_SQD: diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 12594cb2a9cf..1ae8dfd0ce7b 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -127,7 +127,6 @@ struct rxe_req_info { }; struct rxe_comp_info { - enum rxe_qp_state state; u32 psn; int opcode; int timeout; From patchwork Wed Apr 5 04:26:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13201137 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB843C761AF for ; Wed, 5 Apr 2023 04:27:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236852AbjDEE1E (ORCPT ); Wed, 5 Apr 2023 00:27:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229587AbjDEE1D (ORCPT ); Wed, 5 Apr 2023 00:27:03 -0400 Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AA98172C for ; Tue, 4 Apr 2023 21:27:02 -0700 (PDT) Received: by mail-oi1-x236.google.com with SMTP id r14so20157444oiw.12 for ; Tue, 04 Apr 2023 21:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680668822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lZnZA0C6aViBKBdFotqbjonQKG7mf5YChVdnOfdrNcc=; b=NyypYq1fLbZASD/yTOWIA4KIkk8/ECYatM9rTOCq0gLKTF0yqknV1xGFpJmQfEyO0Q UxJjsOOfqsUE/EwIXKXEPkxGHvXz330nbtJRSMtQl3OU0TTOt8enrI9PzdtJgK3j6YjL o6PFLegV1FE2H4SgnPi0lqSVrxQY2t4nima3keZ2bqpUNbLZUej2YbGer0n+NFYwf1X9 Rkjw1APqWUHp7EKLSfovXA+c3AjVMncfsda6Yc5S3/qvHvWEV+npIVHNOrTBNft8tIxm ZPhOnfvW6V1qfmD4XV9/q0zSQJOvyXNCN6aywc5LCWo+CKqIJRSmr5kCJgn+d3+17aTt I7TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680668822; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lZnZA0C6aViBKBdFotqbjonQKG7mf5YChVdnOfdrNcc=; b=3OxEA6pbMomDSrDpPIX/eNpSJKOJutXancK7JWvDAoP9yGJuXlpmlcWVrHHTD8q0pX +D8C91+T27CXTDONUSFbQH8otsYXQqh9LLK2ltPeV+2mcd8gQ4ZEF5on2GNlAlRsewmk JS1OQwKrl/IWGC59Tp6r+6WfSBtLjkPUEnxomNVSZe2zcndeiq/p620uWGP5Z4b14M0Z sUjnlIqaEAeCQLb9YN9xRelk811tfx0t0pRtn8HNmVnOYYeMLNsB/IcqjpdxuhkbPeL/ pUKYj8wlfFTNGIJJipy86t+IX3FETVxLGHgT3PF0ND2fIQ/gmksB9u9H52iPzo5fjoMf Q/ig== X-Gm-Message-State: AAQBX9cxK54WngUDG1KXue2Xo/U1ev0Yqdjr5z9c2cCRBYzRr+2o08Mp CSZUk5maOaLMVWtLO1RegpY= X-Google-Smtp-Source: AKy350Yz1EIW/GAMAeXtuQhNsCojEfAz4wcPEvh6qPeM7tXkr45VmKrsNOafc8g9BT8haHgDB1Dn7w== X-Received: by 2002:a05:6808:5cc:b0:386:db46:f6fb with SMTP id d12-20020a05680805cc00b00386db46f6fbmr1662404oij.17.1680668821717; Tue, 04 Apr 2023 21:27:01 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.gmail.com with ESMTPSA id x80-20020a4a4153000000b0053d9be4be68sm6321326ooa.19.2023.04.04.21.26.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 21:26:55 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 3/5] RDMA/rxe: Remove qp->req.state Date: Tue, 4 Apr 2023 23:26:09 -0500 Message-Id: <20230405042611.6467-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230405042611.6467-1-rpearsonhpe@gmail.com> References: <20230405042611.6467-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The rxe driver has four different QP state variables, qp->attr.qp_state, qp->req.state, qp->comp.state, and qp->resp.state. All of these basically carry the same information. This patch replaces uses of qp->req.state by qp->attr.qp_state and enum rxe_qp_state. This is the third of three patches which will remove all but the qp->attr.qp_state variable. This will bring the driver closer to the IBA description. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 9 +++-- drivers/infiniband/sw/rxe/rxe_net.c | 4 +-- drivers/infiniband/sw/rxe/rxe_qp.c | 49 ++++++++------------------- drivers/infiniband/sw/rxe/rxe_recv.c | 9 ++--- drivers/infiniband/sw/rxe/rxe_req.c | 15 ++++---- drivers/infiniband/sw/rxe/rxe_verbs.c | 4 +-- drivers/infiniband/sw/rxe/rxe_verbs.h | 10 ------ 7 files changed, 34 insertions(+), 66 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 173ebfe784e6..979990734e0c 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -491,12 +491,11 @@ static inline enum comp_state complete_ack(struct rxe_qp *qp, } } - if (unlikely(qp->req.state == QP_STATE_DRAIN)) { + if (unlikely(qp_state(qp) == IB_QPS_SQD)) { /* state_lock used by requester & completer */ spin_lock_bh(&qp->state_lock); - if ((qp->req.state == QP_STATE_DRAIN) && - (qp->comp.psn == qp->req.psn)) { - qp->req.state = QP_STATE_DRAINED; + if (qp->attr.sq_draining && qp->comp.psn == qp->req.psn) { + qp->attr.sq_draining = 0; spin_unlock_bh(&qp->state_lock); if (qp->ibqp.event_handler) { @@ -723,7 +722,7 @@ int rxe_completer(struct rxe_qp *qp) * (4) the timeout parameter is set */ if ((qp_type(qp) == IB_QPT_RC) && - (qp->req.state == QP_STATE_READY) && + (qp_state(qp) >= IB_QPS_RTS) && (psn_compare(qp->req.psn, qp->comp.psn) > 0) && qp->qp_timeout_jiffies) mod_timer(&qp->retrans_timer, diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 2be2425083ce..9ed81d0bd25c 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -413,8 +413,8 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, int is_request = pkt->mask & RXE_REQ_MASK; struct rxe_dev *rxe = to_rdev(qp->ibqp.device); - if ((is_request && (qp->req.state != QP_STATE_READY)) || - (!is_request && (qp_state(qp) <= IB_QPS_RTR))) { + if ((is_request && (qp_state(qp) < IB_QPS_RTS)) || + (!is_request && (qp_state(qp) < IB_QPS_RTR))) { rxe_dbg_qp(qp, "Packet dropped. QP is not in ready state\n"); goto drop; } diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index a4486d9c540e..b9e40ad02ac6 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -231,7 +231,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, qp->req.wqe_index = queue_get_producer(qp->sq.queue, QUEUE_TYPE_FROM_CLIENT); - qp->req.state = QP_STATE_RESET; qp->req.opcode = -1; qp->comp.opcode = -1; @@ -394,12 +393,9 @@ int rxe_qp_chk_attr(struct rxe_dev *rxe, struct rxe_qp *qp, goto err1; } - if (mask & IB_QP_STATE) { - if (cur_state == IB_QPS_SQD) { - if (qp->req.state == QP_STATE_DRAIN && - new_state != IB_QPS_ERR) - goto err1; - } + if (mask & IB_QP_STATE && cur_state == IB_QPS_SQD) { + if (qp->attr.sq_draining && new_state != IB_QPS_ERR) + goto err1; } if (mask & IB_QP_PORT) { @@ -474,9 +470,6 @@ static void rxe_qp_reset(struct rxe_qp *qp) rxe_disable_task(&qp->comp.task); rxe_disable_task(&qp->req.task); - /* move qp to the reset state */ - qp->req.state = QP_STATE_RESET; - /* drain work and packet queuesc */ rxe_requester(qp); rxe_completer(qp); @@ -512,22 +505,9 @@ static void rxe_qp_reset(struct rxe_qp *qp) rxe_enable_task(&qp->req.task); } -/* drain the send queue */ -static void rxe_qp_drain(struct rxe_qp *qp) -{ - if (qp->sq.queue) { - if (qp->req.state != QP_STATE_DRAINED) { - qp->req.state = QP_STATE_DRAIN; - rxe_sched_task(&qp->comp.task); - rxe_sched_task(&qp->req.task); - } - } -} - /* move the qp to the error state */ void rxe_qp_error(struct rxe_qp *qp) { - qp->req.state = QP_STATE_ERROR; qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ @@ -540,6 +520,8 @@ void rxe_qp_error(struct rxe_qp *qp) int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, struct ib_udata *udata) { + enum ib_qp_state cur_state = (mask & IB_QP_CUR_STATE) ? + attr->cur_qp_state : qp->attr.qp_state; int err; if (mask & IB_QP_MAX_QP_RD_ATOMIC) { @@ -656,7 +638,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_INIT: rxe_dbg_qp(qp, "state -> INIT\n"); - qp->req.state = QP_STATE_INIT; break; case IB_QPS_RTR: @@ -665,12 +646,15 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, case IB_QPS_RTS: rxe_dbg_qp(qp, "state -> RTS\n"); - qp->req.state = QP_STATE_READY; break; case IB_QPS_SQD: rxe_dbg_qp(qp, "state -> SQD\n"); - rxe_qp_drain(qp); + if (cur_state != IB_QPS_SQD) { + qp->attr.sq_draining = 1; + rxe_sched_task(&qp->comp.task); + rxe_sched_task(&qp->req.task); + } break; case IB_QPS_SQE: @@ -708,16 +692,11 @@ int rxe_qp_to_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask) rxe_av_to_attr(&qp->pri_av, &attr->ah_attr); rxe_av_to_attr(&qp->alt_av, &attr->alt_ah_attr); - if (qp->req.state == QP_STATE_DRAIN) { - attr->sq_draining = 1; - /* applications that get this state - * typically spin on it. yield the - * processor - */ + /* Applications that get this state typically spin on it. + * Yield the processor + */ + if (qp->attr.sq_draining) cond_resched(); - } else { - attr->sq_draining = 0; - } rxe_dbg_qp(qp, "attr->sq_draining = %d\n", attr->sq_draining); diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index ac42ceccf71f..ca17ac6c5878 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -39,11 +39,12 @@ static int check_type_state(struct rxe_dev *rxe, struct rxe_pkt_info *pkt, } if (pkt->mask & RXE_REQ_MASK) { - if (unlikely(qp_state(qp) <= IB_QPS_RTR)) + if (unlikely(qp_state(qp) < IB_QPS_RTR)) return -EINVAL; - } else if (unlikely(qp->req.state < QP_STATE_READY || - qp->req.state > QP_STATE_DRAINED)) - return -EINVAL; + } else { + if (unlikely(qp_state(qp) < IB_QPS_RTS)) + return -EINVAL; + } return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 745731140a54..8a8242512f2a 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -120,13 +120,13 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) cons = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); prod = queue_get_producer(q, QUEUE_TYPE_FROM_CLIENT); - if (unlikely(qp->req.state == QP_STATE_DRAIN)) { + if (unlikely(qp_state(qp) == IB_QPS_SQD)) { /* check to see if we are drained; * state_lock used by requester and completer */ spin_lock_bh(&qp->state_lock); do { - if (qp->req.state != QP_STATE_DRAIN) { + if (!qp->attr.sq_draining) { /* comp just finished */ spin_unlock_bh(&qp->state_lock); break; @@ -139,7 +139,7 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) break; } - qp->req.state = QP_STATE_DRAINED; + qp->attr.sq_draining = 0; spin_unlock_bh(&qp->state_lock); if (qp->ibqp.event_handler) { @@ -159,8 +159,7 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) wqe = queue_addr_from_index(q, index); - if (unlikely((qp->req.state == QP_STATE_DRAIN || - qp->req.state == QP_STATE_DRAINED) && + if (unlikely((qp_state(qp) == IB_QPS_SQD) && (wqe->state != wqe_state_processing))) return NULL; @@ -656,7 +655,7 @@ int rxe_requester(struct rxe_qp *qp) if (unlikely(!qp->valid)) goto exit; - if (unlikely(qp->req.state == QP_STATE_ERROR)) { + if (unlikely(qp_state(qp) == IB_QPS_ERR)) { wqe = req_next_wqe(qp); if (wqe) /* @@ -667,7 +666,7 @@ int rxe_requester(struct rxe_qp *qp) goto exit; } - if (unlikely(qp->req.state == QP_STATE_RESET)) { + if (unlikely(qp_state(qp) == IB_QPS_RESET)) { qp->req.wqe_index = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); qp->req.opcode = -1; @@ -836,7 +835,7 @@ int rxe_requester(struct rxe_qp *qp) /* update wqe_index for each wqe completion */ qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; - qp->req.state = QP_STATE_ERROR; + qp->attr.qp_state = IB_QPS_ERR; rxe_sched_task(&qp->comp.task); exit: ret = -EAGAIN; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 2a5da3160db9..28799f1cacb8 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -881,7 +881,7 @@ static int rxe_post_send_kernel(struct rxe_qp *qp, const struct ib_send_wr *wr, if (!err) rxe_sched_task(&qp->req.task); - if (unlikely(qp->req.state == QP_STATE_ERROR)) + if (unlikely(qp_state(qp) == IB_QPS_ERR)) rxe_sched_task(&qp->comp.task); return err; @@ -900,7 +900,7 @@ static int rxe_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, goto err_out; } - if (unlikely(qp->req.state < QP_STATE_READY)) { + if (unlikely(qp_state(qp) < IB_QPS_RTS)) { *bad_wr = wr; err = -EINVAL; rxe_dbg_qp(qp, "qp not ready to send"); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 1ae8dfd0ce7b..26a20f088692 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -102,17 +102,7 @@ struct rxe_srq { int error; }; -enum rxe_qp_state { - QP_STATE_RESET, - QP_STATE_INIT, - QP_STATE_READY, - QP_STATE_DRAIN, /* req only */ - QP_STATE_DRAINED, /* req only */ - QP_STATE_ERROR -}; - struct rxe_req_info { - enum rxe_qp_state state; int wqe_index; u32 psn; int opcode; From patchwork Wed Apr 5 04:26:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13201138 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99116C761AF for ; Wed, 5 Apr 2023 04:27:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236862AbjDEE1M (ORCPT ); Wed, 5 Apr 2023 00:27:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229587AbjDEE1J (ORCPT ); Wed, 5 Apr 2023 00:27:09 -0400 Received: from mail-oo1-xc2b.google.com (mail-oo1-xc2b.google.com [IPv6:2607:f8b0:4864:20::c2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA93B3599 for ; Tue, 4 Apr 2023 21:27:08 -0700 (PDT) Received: by mail-oo1-xc2b.google.com with SMTP id f7-20020a4ab647000000b0054101f316c7so2078592ooo.13 for ; Tue, 04 Apr 2023 21:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680668828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F8+SKO55qGGMajqsSuK/gkh4B4t1XPBiRdlnbt9yI2k=; b=h4FNdmRj5J0ZYhANlkJPZQVUS4A0S6sJG3xtY9wXxIqJirfnvvtl0RqwmkOpcFkmqS Rk1wLeBY0czEQ390aTC4X5HirW7ovxdIqZAjUWRAQ6i0jNMM9Hoy2A4k1PTO5wbsEPt3 0O5xCmhEyKq4GD+ZBuSsYOoS+t0Bct4+XtZWIzU1VJChQqkyhe5MA5JfyzxRN0wl7MGX Qii4K5iSjHSsPgMvAFIGmIm3oUSGiCz84Wh1qGAZxeX3i49c4YikhWEn2HMwYexy39QB LGnXtS6Lg4UjWSzWJH3bB1HQfEQfRSOV7g9J0l3LlJtPJO9pE4uNJrtAsei8qEw1kio5 EYdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680668828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F8+SKO55qGGMajqsSuK/gkh4B4t1XPBiRdlnbt9yI2k=; b=1OnSx2MA4nBZ2azXmEa+xLaSXVH9bK5kXsg9B8szSvSVeozIpiDqkgn3VOWS68EYGM KMskRhyLOoBiykxA9lfmlLg21UZlT7fRFvqPPUqdIlvdkgJfLLt+sDxoY1WG3PQk5wjS UCxetl8k0pC3IgJ6D/FuxXybwYHMoyNkU1KT6/3tabPSTIW/cArZcUc2/DOEjKHLBaz2 cPltpPXUjV89WMXK8WycgDiQBPwjKoYWzLiBfUnOBPDFXGrLJcffR8jKw3UNYKzO/Ls2 2TR0ECNcVrNrSbDnT5tEbZgqXdSmGd8LIK6UTEu/sBDLIQOcBhDPtw4U8QGR7+P8V6so DX1Q== X-Gm-Message-State: AAQBX9esMfdoJR8OeMEkWISQUmlHfdZ2oxFNy1GLcjU6gtFhElIAr94G XqPgxPjOMXXfbGcis3u9m7U= X-Google-Smtp-Source: AKy350YYHJAYQBnACIDcGUm+HQs0ttd94jLGw21+auMBSYyUVOP30eJgxRslXMzd+II6beV2ZMjsnQ== X-Received: by 2002:a4a:5241:0:b0:537:b1ad:1c7c with SMTP id d62-20020a4a5241000000b00537b1ad1c7cmr2460277oob.0.1680668828083; Tue, 04 Apr 2023 21:27:08 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.gmail.com with ESMTPSA id x80-20020a4a4153000000b0053d9be4be68sm6321326ooa.19.2023.04.04.21.27.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 21:27:05 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 4/5] RDMA/rxe: Move code to check if drained to subroutine Date: Tue, 4 Apr 2023 23:26:10 -0500 Message-Id: <20230405042611.6467-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230405042611.6467-1-rpearsonhpe@gmail.com> References: <20230405042611.6467-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Move two blocks of code in rxe_comp.c and rxe_req.c to subroutines that check if draining is complete in the SQD state and, if so, generate a SQ_DRAINED event. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 35 ++++++++++++++++------------ drivers/infiniband/sw/rxe/rxe_req.c | 32 ++++++++++++++----------- 2 files changed, 38 insertions(+), 29 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 979990734e0c..1ccae8cff359 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -477,20 +477,8 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe) } } -static inline enum comp_state complete_ack(struct rxe_qp *qp, - struct rxe_pkt_info *pkt, - struct rxe_send_wqe *wqe) +static void comp_check_sq_drain_done(struct rxe_qp *qp) { - if (wqe->has_rd_atomic) { - wqe->has_rd_atomic = 0; - atomic_inc(&qp->req.rd_atomic); - if (qp->req.need_rd_atomic) { - qp->comp.timeout_retry = 0; - qp->req.need_rd_atomic = 0; - rxe_sched_task(&qp->req.task); - } - } - if (unlikely(qp_state(qp) == IB_QPS_SQD)) { /* state_lock used by requester & completer */ spin_lock_bh(&qp->state_lock); @@ -507,10 +495,27 @@ static inline enum comp_state complete_ack(struct rxe_qp *qp, qp->ibqp.event_handler(&ev, qp->ibqp.qp_context); } - } else { - spin_unlock_bh(&qp->state_lock); + return; } + spin_unlock_bh(&qp->state_lock); } +} + +static inline enum comp_state complete_ack(struct rxe_qp *qp, + struct rxe_pkt_info *pkt, + struct rxe_send_wqe *wqe) +{ + if (wqe->has_rd_atomic) { + wqe->has_rd_atomic = 0; + atomic_inc(&qp->req.rd_atomic); + if (qp->req.need_rd_atomic) { + qp->comp.timeout_retry = 0; + qp->req.need_rd_atomic = 0; + rxe_sched_task(&qp->req.task); + } + } + + comp_check_sq_drain_done(qp); do_complete(qp, wqe); diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 8a8242512f2a..f329038efbc8 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -108,17 +108,12 @@ void rnr_nak_timer(struct timer_list *t) rxe_sched_task(&qp->req.task); } -static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) +static void req_check_sq_drain_done(struct rxe_qp *qp) { - struct rxe_send_wqe *wqe; struct rxe_queue *q = qp->sq.queue; unsigned int index = qp->req.wqe_index; - unsigned int cons; - unsigned int prod; - - wqe = queue_head(q, QUEUE_TYPE_FROM_CLIENT); - cons = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); - prod = queue_get_producer(q, QUEUE_TYPE_FROM_CLIENT); + unsigned int cons = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); + struct rxe_send_wqe *wqe = queue_addr_from_index(q, cons); if (unlikely(qp_state(qp) == IB_QPS_SQD)) { /* check to see if we are drained; @@ -126,18 +121,14 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) */ spin_lock_bh(&qp->state_lock); do { - if (!qp->attr.sq_draining) { + if (!qp->attr.sq_draining) /* comp just finished */ - spin_unlock_bh(&qp->state_lock); break; - } if (wqe && ((index != cons) || - (wqe->state != wqe_state_posted))) { + (wqe->state != wqe_state_posted))) /* comp not done yet */ - spin_unlock_bh(&qp->state_lock); break; - } qp->attr.sq_draining = 0; spin_unlock_bh(&qp->state_lock); @@ -151,9 +142,22 @@ static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) qp->ibqp.event_handler(&ev, qp->ibqp.qp_context); } + return; } while (0); + spin_unlock_bh(&qp->state_lock); } +} +static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) +{ + struct rxe_send_wqe *wqe; + struct rxe_queue *q = qp->sq.queue; + unsigned int index = qp->req.wqe_index; + unsigned int prod; + + req_check_sq_drain_done(qp); + + prod = queue_get_producer(q, QUEUE_TYPE_FROM_CLIENT); if (index == prod) return NULL; From patchwork Wed Apr 5 04:26:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 13201139 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 410E2C7619A for ; Wed, 5 Apr 2023 04:27:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229587AbjDEE10 (ORCPT ); Wed, 5 Apr 2023 00:27:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236840AbjDEE1Y (ORCPT ); Wed, 5 Apr 2023 00:27:24 -0400 Received: from mail-oa1-x2c.google.com (mail-oa1-x2c.google.com [IPv6:2001:4860:4864:20::2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82DB810CF for ; Tue, 4 Apr 2023 21:27:22 -0700 (PDT) Received: by mail-oa1-x2c.google.com with SMTP id 586e51a60fabf-17aa62d0a4aso37087509fac.4 for ; Tue, 04 Apr 2023 21:27:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680668842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0UrGFcsWq3dVrqh1lrExS/7KIzMlxfVpQ1ept8dQ9ic=; b=YA9lLvHvmglJ8NT8VtXZ7G8xx4QHmGl5/PdI7ZWpse8L3tNqAikloQPR7mKR+5stQJ xZQPpTGcWDGA02OKb34RZ7r+FmhwheWTPsAtalcnHkngcWJfGvY/oVuquOiXKLlWR1kK rzLNY+Qb8Gjy26Eb3pY+1g7s+pbkmeFs+18LsiMIuXAQINpbr01cgnnYfnOlrNk1OhZJ 8K2vNhsuguH8n3qIPx3FYw6TwRQo/y7FEfcYG8+8Nmcuo/R7Zw0hRFD+I0ZPgge7JSkW vhThs7aMCclsy+7Ad1ePTZZgC7jioURq8Gw+jz1AuHCIIkIyT9ydToD6bFtAnLrlB+Bt h76A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680668842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0UrGFcsWq3dVrqh1lrExS/7KIzMlxfVpQ1ept8dQ9ic=; b=DEFBPsEGWtlJ4VB3ZkP4VKIAZV9k/K1B7YvkNMPcY6MAecjdHXVmNlO+icQEF9AQ0J 8xihhvxGNoBAFb4wDM7k6bQd5jhFn2im+M3AZPTPSPLET1iiX5dwLgPzjz6Di1NwWSW1 ZcpAxlOJFbeLe31d4cS45bfIIzYTvV5EDcEbgV78L9+OgW7wp9WSp+mQljPPpN50mPrl gfncPRYCTtDGTpj1yGKHLr7IkFDNVI0lxd9Zh3TdsogKi8hJa74QdiGy5Y82ovz3s4JP lgtMJGJ8XPBuc+bHrr11FWdtww8wCLiPe7Txhe+zpezytEB75cZF/hLQyxawe+yz7Rz8 mZhw== X-Gm-Message-State: AAQBX9eJidoUJl7Rs70bagRXw8L8eV59inHcEMAfb0e2osQErBRrew2Z 3VecU04kmefHT/QltuVJhek= X-Google-Smtp-Source: AKy350ZvOARNye67wMpUVxLNMm9y7dgnxn2gYHW6rkD83DDa+tMWZJl9QmqLoX5BKicvq9Cx3sexHw== X-Received: by 2002:a05:6870:d192:b0:17f:7dca:8926 with SMTP id a18-20020a056870d19200b0017f7dca8926mr2662030oac.20.1680668841688; Tue, 04 Apr 2023 21:27:21 -0700 (PDT) Received: from rpearson-X570-AORUS-PRO-WIFI.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.gmail.com with ESMTPSA id x80-20020a4a4153000000b0053d9be4be68sm6321326ooa.19.2023.04.04.21.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Apr 2023 21:27:11 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, leon@kernel.org, jhack@hpe.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next 5/5] RDMA/rxe: Protect QP state with qp->state_lock Date: Tue, 4 Apr 2023 23:26:11 -0500 Message-Id: <20230405042611.6467-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230405042611.6467-1-rpearsonhpe@gmail.com> References: <20230405042611.6467-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Currently the rxe driver makes little effort to make the changes to qp state (which includes qp->attr.qp_state, qp->attr.sq_draining and qp->valid) atomic between different client threads and IO threads. In particular a common template is for an RDMA application to call ib_modify_qp() to move a qp to ERR state and then wait until all the packet and work queues have drained before calling ib_destroy_qp(). None of these state changes are protected by locks to assure that the changes are executed atomically and that memory barriers are included. This has been observed to lead to incorrect behavior around qp cleanup. This patch continues the work of the previous patches in this series and adds locking code around qp state changes and lookups. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_comp.c | 48 ++++-- drivers/infiniband/sw/rxe/rxe_net.c | 3 + drivers/infiniband/sw/rxe/rxe_qp.c | 153 +++++++++-------- drivers/infiniband/sw/rxe/rxe_recv.c | 10 +- drivers/infiniband/sw/rxe/rxe_req.c | 71 +++++--- drivers/infiniband/sw/rxe/rxe_resp.c | 12 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 238 +++++++++++++++----------- 7 files changed, 317 insertions(+), 218 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index 1ccae8cff359..db18ace74d2b 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -118,10 +118,12 @@ void retransmit_timer(struct timer_list *t) rxe_dbg_qp(qp, "retransmit timer fired\n"); + spin_lock_bh(&qp->state_lock); if (qp->valid) { qp->comp.timeout = 1; rxe_sched_task(&qp->comp.task); } + spin_unlock_bh(&qp->state_lock); } void rxe_comp_queue_pkt(struct rxe_qp *qp, struct sk_buff *skb) @@ -479,9 +481,8 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe) static void comp_check_sq_drain_done(struct rxe_qp *qp) { + spin_lock_bh(&qp->state_lock); if (unlikely(qp_state(qp) == IB_QPS_SQD)) { - /* state_lock used by requester & completer */ - spin_lock_bh(&qp->state_lock); if (qp->attr.sq_draining && qp->comp.psn == qp->req.psn) { qp->attr.sq_draining = 0; spin_unlock_bh(&qp->state_lock); @@ -497,8 +498,8 @@ static void comp_check_sq_drain_done(struct rxe_qp *qp) } return; } - spin_unlock_bh(&qp->state_lock); } + spin_unlock_bh(&qp->state_lock); } static inline enum comp_state complete_ack(struct rxe_qp *qp, @@ -614,6 +615,26 @@ static void free_pkt(struct rxe_pkt_info *pkt) ib_device_put(dev); } +/* reset the retry timer if + * - QP is type RC + * - there is a packet sent by the requester that + * might be acked (we still might get spurious + * timeouts but try to keep them as few as possible) + * - the timeout parameter is set + * - the QP is alive + */ +static void reset_retry_timer(struct rxe_qp *qp) +{ + if (qp_type(qp) == IB_QPT_RC && qp->qp_timeout_jiffies) { + spin_lock_bh(&qp->state_lock); + if (qp_state(qp) >= IB_QPS_RTS && + psn_compare(qp->req.psn, qp->comp.psn) > 0) + mod_timer(&qp->retrans_timer, + jiffies + qp->qp_timeout_jiffies); + spin_unlock_bh(&qp->state_lock); + } +} + int rxe_completer(struct rxe_qp *qp) { struct rxe_dev *rxe = to_rdev(qp->ibqp.device); @@ -623,14 +644,17 @@ int rxe_completer(struct rxe_qp *qp) enum comp_state state; int ret; + spin_lock_bh(&qp->state_lock); if (!qp->valid || qp_state(qp) == IB_QPS_ERR || - qp_state(qp) == IB_QPS_RESET) { + qp_state(qp) == IB_QPS_RESET) { bool notify = qp->valid && (qp_state(qp) == IB_QPS_ERR); drain_resp_pkts(qp); flush_send_queue(qp, notify); + spin_unlock_bh(&qp->state_lock); goto exit; } + spin_unlock_bh(&qp->state_lock); if (qp->comp.timeout) { qp->comp.timeout_retry = 1; @@ -718,20 +742,7 @@ int rxe_completer(struct rxe_qp *qp) break; } - /* re reset the timeout counter if - * (1) QP is type RC - * (2) the QP is alive - * (3) there is a packet sent by the requester that - * might be acked (we still might get spurious - * timeouts but try to keep them as few as possible) - * (4) the timeout parameter is set - */ - if ((qp_type(qp) == IB_QPT_RC) && - (qp_state(qp) >= IB_QPS_RTS) && - (psn_compare(qp->req.psn, qp->comp.psn) > 0) && - qp->qp_timeout_jiffies) - mod_timer(&qp->retrans_timer, - jiffies + qp->qp_timeout_jiffies); + reset_retry_timer(qp); goto exit; case COMPST_ERROR_RETRY: @@ -793,6 +804,7 @@ int rxe_completer(struct rxe_qp *qp) */ qp->req.wait_for_rnr_timer = 1; rxe_dbg_qp(qp, "set rnr nak timer\n"); + // TODO who protects from destroy_qp?? mod_timer(&qp->rnr_nak_timer, jiffies + rnrnak_jiffies(aeth_syn(pkt) & ~AETH_TYPE_MASK)); diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 9ed81d0bd25c..2bc7361152ea 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -413,11 +413,14 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, int is_request = pkt->mask & RXE_REQ_MASK; struct rxe_dev *rxe = to_rdev(qp->ibqp.device); + spin_lock_bh(&qp->state_lock); if ((is_request && (qp_state(qp) < IB_QPS_RTS)) || (!is_request && (qp_state(qp) < IB_QPS_RTR))) { + spin_unlock_bh(&qp->state_lock); rxe_dbg_qp(qp, "Packet dropped. QP is not in ready state\n"); goto drop; } + spin_unlock_bh(&qp->state_lock); rxe_icrc_generate(skb, pkt); diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index b9e40ad02ac6..762597f01b5f 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -325,8 +325,10 @@ int rxe_qp_from_init(struct rxe_dev *rxe, struct rxe_qp *qp, struct rxe_pd *pd, if (err) goto err2; + spin_lock_bh(&qp->state_lock); qp->attr.qp_state = IB_QPS_RESET; qp->valid = 1; + spin_unlock_bh(&qp->state_lock); return 0; @@ -377,27 +379,9 @@ int rxe_qp_to_init(struct rxe_qp *qp, struct ib_qp_init_attr *init) return 0; } -/* called by the modify qp verb, this routine checks all the parameters before - * making any changes - */ int rxe_qp_chk_attr(struct rxe_dev *rxe, struct rxe_qp *qp, struct ib_qp_attr *attr, int mask) { - enum ib_qp_state cur_state = (mask & IB_QP_CUR_STATE) ? - attr->cur_qp_state : qp->attr.qp_state; - enum ib_qp_state new_state = (mask & IB_QP_STATE) ? - attr->qp_state : cur_state; - - if (!ib_modify_qp_is_ok(cur_state, new_state, qp_type(qp), mask)) { - rxe_dbg_qp(qp, "invalid mask or state\n"); - goto err1; - } - - if (mask & IB_QP_STATE && cur_state == IB_QPS_SQD) { - if (qp->attr.sq_draining && new_state != IB_QPS_ERR) - goto err1; - } - if (mask & IB_QP_PORT) { if (!rdma_is_port_valid(&rxe->ib_dev, attr->port_num)) { rxe_dbg_qp(qp, "invalid port %d\n", attr->port_num); @@ -508,22 +492,96 @@ static void rxe_qp_reset(struct rxe_qp *qp) /* move the qp to the error state */ void rxe_qp_error(struct rxe_qp *qp) { + spin_lock_bh(&qp->state_lock); qp->attr.qp_state = IB_QPS_ERR; /* drain work and packet queues */ rxe_sched_task(&qp->resp.task); rxe_sched_task(&qp->comp.task); rxe_sched_task(&qp->req.task); + spin_unlock_bh(&qp->state_lock); +} + +static void rxe_qp_sqd(struct rxe_qp *qp, struct ib_qp_attr *attr, + int mask) +{ + spin_lock_bh(&qp->state_lock); + qp->attr.sq_draining = 1; + rxe_sched_task(&qp->comp.task); + rxe_sched_task(&qp->req.task); + spin_unlock_bh(&qp->state_lock); +} + +/* caller should hold qp->state_lock */ +static int __qp_chk_state(struct rxe_qp *qp, struct ib_qp_attr *attr, + int mask) +{ + enum ib_qp_state cur_state; + enum ib_qp_state new_state; + + cur_state = (mask & IB_QP_CUR_STATE) ? + attr->cur_qp_state : qp->attr.qp_state; + new_state = (mask & IB_QP_STATE) ? + attr->qp_state : cur_state; + + if (!ib_modify_qp_is_ok(cur_state, new_state, qp_type(qp), mask)) + return -EINVAL; + + if (mask & IB_QP_STATE && cur_state == IB_QPS_SQD) { + if (qp->attr.sq_draining && new_state != IB_QPS_ERR) + return -EINVAL; + } + + return 0; } +static const char *qps2str[] = { + [IB_QPS_RESET] = "RESET", + [IB_QPS_INIT] = "INIT", + [IB_QPS_RTR] = "RTR", + [IB_QPS_RTS] = "RTS", + [IB_QPS_SQD] = "SQD", + [IB_QPS_SQE] = "SQE", + [IB_QPS_ERR] = "ERR", +}; + /* called by the modify qp verb */ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, struct ib_udata *udata) { - enum ib_qp_state cur_state = (mask & IB_QP_CUR_STATE) ? - attr->cur_qp_state : qp->attr.qp_state; int err; + if (mask & IB_QP_CUR_STATE) + qp->attr.cur_qp_state = attr->qp_state; + + if (mask & IB_QP_STATE) { + spin_lock_bh(&qp->state_lock); + err = __qp_chk_state(qp, attr, mask); + if (!err) { + qp->attr.qp_state = attr->qp_state; + rxe_dbg_qp(qp, "state -> %s\n", + qps2str[attr->qp_state]); + } + spin_unlock_bh(&qp->state_lock); + + if (err) + return err; + + switch (attr->qp_state) { + case IB_QPS_RESET: + rxe_qp_reset(qp); + break; + case IB_QPS_SQD: + rxe_qp_sqd(qp, attr, mask); + break; + case IB_QPS_ERR: + rxe_qp_error(qp); + break; + default: + break; + } + } + if (mask & IB_QP_MAX_QP_RD_ATOMIC) { int max_rd_atomic = attr->max_rd_atomic ? roundup_pow_of_two(attr->max_rd_atomic) : 0; @@ -545,9 +603,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, return err; } - if (mask & IB_QP_CUR_STATE) - qp->attr.cur_qp_state = attr->qp_state; - if (mask & IB_QP_EN_SQD_ASYNC_NOTIFY) qp->attr.en_sqd_async_notify = attr->en_sqd_async_notify; @@ -627,48 +682,6 @@ int rxe_qp_from_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask, if (mask & IB_QP_DEST_QPN) qp->attr.dest_qp_num = attr->dest_qp_num; - if (mask & IB_QP_STATE) { - qp->attr.qp_state = attr->qp_state; - - switch (attr->qp_state) { - case IB_QPS_RESET: - rxe_dbg_qp(qp, "state -> RESET\n"); - rxe_qp_reset(qp); - break; - - case IB_QPS_INIT: - rxe_dbg_qp(qp, "state -> INIT\n"); - break; - - case IB_QPS_RTR: - rxe_dbg_qp(qp, "state -> RTR\n"); - break; - - case IB_QPS_RTS: - rxe_dbg_qp(qp, "state -> RTS\n"); - break; - - case IB_QPS_SQD: - rxe_dbg_qp(qp, "state -> SQD\n"); - if (cur_state != IB_QPS_SQD) { - qp->attr.sq_draining = 1; - rxe_sched_task(&qp->comp.task); - rxe_sched_task(&qp->req.task); - } - break; - - case IB_QPS_SQE: - rxe_dbg_qp(qp, "state -> SQE !!?\n"); - /* Not possible from modify_qp. */ - break; - - case IB_QPS_ERR: - rxe_dbg_qp(qp, "state -> ERR\n"); - rxe_qp_error(qp); - break; - } - } - return 0; } @@ -695,10 +708,12 @@ int rxe_qp_to_attr(struct rxe_qp *qp, struct ib_qp_attr *attr, int mask) /* Applications that get this state typically spin on it. * Yield the processor */ - if (qp->attr.sq_draining) + spin_lock_bh(&qp->state_lock); + if (qp->attr.sq_draining) { + spin_unlock_bh(&qp->state_lock); cond_resched(); - - rxe_dbg_qp(qp, "attr->sq_draining = %d\n", attr->sq_draining); + } + spin_unlock_bh(&qp->state_lock); return 0; } @@ -722,7 +737,9 @@ static void rxe_qp_do_cleanup(struct work_struct *work) { struct rxe_qp *qp = container_of(work, typeof(*qp), cleanup_work.work); + spin_lock_bh(&qp->state_lock); qp->valid = 0; + spin_unlock_bh(&qp->state_lock); qp->qp_timeout_jiffies = 0; if (qp_type(qp) == IB_QPT_RC) { diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c index ca17ac6c5878..2f953cc74256 100644 --- a/drivers/infiniband/sw/rxe/rxe_recv.c +++ b/drivers/infiniband/sw/rxe/rxe_recv.c @@ -38,13 +38,19 @@ static int check_type_state(struct rxe_dev *rxe, struct rxe_pkt_info *pkt, return -EINVAL; } + spin_lock_bh(&qp->state_lock); if (pkt->mask & RXE_REQ_MASK) { - if (unlikely(qp_state(qp) < IB_QPS_RTR)) + if (unlikely(qp_state(qp) < IB_QPS_RTR)) { + spin_unlock_bh(&qp->state_lock); return -EINVAL; + } } else { - if (unlikely(qp_state(qp) < IB_QPS_RTS)) + if (unlikely(qp_state(qp) < IB_QPS_RTS)) { + spin_unlock_bh(&qp->state_lock); return -EINVAL; + } } + spin_unlock_bh(&qp->state_lock); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index f329038efbc8..8e50d116d273 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -102,24 +102,33 @@ void rnr_nak_timer(struct timer_list *t) rxe_dbg_qp(qp, "nak timer fired\n"); - /* request a send queue retry */ - qp->req.need_retry = 1; - qp->req.wait_for_rnr_timer = 0; - rxe_sched_task(&qp->req.task); + spin_lock_bh(&qp->state_lock); + if (qp->valid) { + /* request a send queue retry */ + qp->req.need_retry = 1; + qp->req.wait_for_rnr_timer = 0; + rxe_sched_task(&qp->req.task); + } + spin_unlock_bh(&qp->state_lock); } static void req_check_sq_drain_done(struct rxe_qp *qp) { - struct rxe_queue *q = qp->sq.queue; - unsigned int index = qp->req.wqe_index; - unsigned int cons = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); - struct rxe_send_wqe *wqe = queue_addr_from_index(q, cons); + struct rxe_queue *q; + unsigned int index; + unsigned int cons; + struct rxe_send_wqe *wqe; + + spin_lock_bh(&qp->state_lock); + if (qp_state(qp) == IB_QPS_SQD) { + q = qp->sq.queue; + index = qp->req.wqe_index; + cons = queue_get_consumer(q, QUEUE_TYPE_FROM_CLIENT); + wqe = queue_addr_from_index(q, cons); - if (unlikely(qp_state(qp) == IB_QPS_SQD)) { /* check to see if we are drained; * state_lock used by requester and completer */ - spin_lock_bh(&qp->state_lock); do { if (!qp->attr.sq_draining) /* comp just finished */ @@ -144,28 +153,40 @@ static void req_check_sq_drain_done(struct rxe_qp *qp) } return; } while (0); - spin_unlock_bh(&qp->state_lock); } + spin_unlock_bh(&qp->state_lock); } -static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) +static struct rxe_send_wqe *__req_next_wqe(struct rxe_qp *qp) { - struct rxe_send_wqe *wqe; struct rxe_queue *q = qp->sq.queue; unsigned int index = qp->req.wqe_index; unsigned int prod; - req_check_sq_drain_done(qp); - prod = queue_get_producer(q, QUEUE_TYPE_FROM_CLIENT); if (index == prod) return NULL; + else + return queue_addr_from_index(q, index); +} - wqe = queue_addr_from_index(q, index); +static struct rxe_send_wqe *req_next_wqe(struct rxe_qp *qp) +{ + struct rxe_send_wqe *wqe; + + req_check_sq_drain_done(qp); + + wqe = __req_next_wqe(qp); + if (wqe == NULL) + return NULL; + spin_lock(&qp->state_lock); if (unlikely((qp_state(qp) == IB_QPS_SQD) && - (wqe->state != wqe_state_processing))) + (wqe->state != wqe_state_processing))) { + spin_unlock(&qp->state_lock); return NULL; + } + spin_unlock(&qp->state_lock); wqe->mask = wr_opcode_mask(wqe->wr.opcode, qp); return wqe; @@ -656,15 +677,16 @@ int rxe_requester(struct rxe_qp *qp) struct rxe_ah *ah; struct rxe_av *av; - if (unlikely(!qp->valid)) + spin_lock_bh(&qp->state_lock); + if (unlikely(!qp->valid)) { + spin_unlock_bh(&qp->state_lock); goto exit; + } if (unlikely(qp_state(qp) == IB_QPS_ERR)) { - wqe = req_next_wqe(qp); + wqe = __req_next_wqe(qp); + spin_unlock_bh(&qp->state_lock); if (wqe) - /* - * Generate an error completion for error qp state - */ goto err; else goto exit; @@ -678,8 +700,10 @@ int rxe_requester(struct rxe_qp *qp) qp->req.wait_psn = 0; qp->req.need_retry = 0; qp->req.wait_for_rnr_timer = 0; + spin_unlock_bh(&qp->state_lock); goto exit; } + spin_unlock_bh(&qp->state_lock); /* we come here if the retransmit timer has fired * or if the rnr timer has fired. If the retransmit @@ -839,8 +863,7 @@ int rxe_requester(struct rxe_qp *qp) /* update wqe_index for each wqe completion */ qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index); wqe->state = wqe_state_error; - qp->attr.qp_state = IB_QPS_ERR; - rxe_sched_task(&qp->comp.task); + rxe_qp_error(qp); exit: ret = -EAGAIN; out: diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 67eac616235c..68f6cd188d8e 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1137,8 +1137,13 @@ static enum resp_states do_complete(struct rxe_qp *qp, return RESPST_ERR_CQ_OVERFLOW; finish: - if (unlikely(qp_state(qp) == IB_QPS_ERR)) + spin_lock_bh(&qp->state_lock); + if (unlikely(qp_state(qp) == IB_QPS_ERR)) { + spin_unlock_bh(&qp->state_lock); return RESPST_CHK_RESOURCE; + } + spin_unlock_bh(&qp->state_lock); + if (unlikely(!pkt)) return RESPST_DONE; if (qp_type(qp) == IB_QPT_RC) @@ -1464,14 +1469,17 @@ int rxe_responder(struct rxe_qp *qp) struct rxe_pkt_info *pkt = NULL; int ret; + spin_lock_bh(&qp->state_lock); if (!qp->valid || qp_state(qp) == IB_QPS_ERR || - qp_state(qp) == IB_QPS_RESET) { + qp_state(qp) == IB_QPS_RESET) { bool notify = qp->valid && (qp_state(qp) == IB_QPS_ERR); drain_req_pkts(qp); flush_recv_queue(qp, notify); + spin_unlock_bh(&qp->state_lock); goto exit; } + spin_unlock_bh(&qp->state_lock); qp->resp.aeth_syndrome = AETH_ACK_UNLIMITED; diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 28799f1cacb8..2bf4c7431382 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -660,42 +660,70 @@ static int rxe_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) } /* send wr */ + +/* sanity check incoming send work request */ static int validate_send_wr(struct rxe_qp *qp, const struct ib_send_wr *ibwr, - unsigned int mask, unsigned int length) + unsigned int *maskp, unsigned int *lengthp) { int num_sge = ibwr->num_sge; struct rxe_sq *sq = &qp->sq; + unsigned int mask = 0; + unsigned long length = 0; + int err = -EINVAL; + int i; - if (unlikely(num_sge > sq->max_sge)) { - rxe_dbg_qp(qp, "num_sge > max_sge"); - goto err_out; - } + do { + mask = wr_opcode_mask(ibwr->opcode, qp); + if (!mask) { + rxe_err_qp(qp, "bad wr opcode for qp type"); + break; + } - if (unlikely(mask & WR_ATOMIC_MASK)) { - if (length != 8) { - rxe_dbg_qp(qp, "atomic length != 8"); - goto err_out; + if (num_sge > sq->max_sge) { + rxe_err_qp(qp, "num_sge > max_sge"); + break; } - if (atomic_wr(ibwr)->remote_addr & 0x7) { - rxe_dbg_qp(qp, "misaligned atomic address"); - goto err_out; + length = 0; + for (i = 0; i < ibwr->num_sge; i++) + length += ibwr->sg_list[i].length; + + if (length > (1UL << 31)) { + rxe_err_qp(qp, "message length too long"); + break; } - } - if (unlikely((ibwr->send_flags & IB_SEND_INLINE) && - (length > sq->max_inline))) { - rxe_dbg_qp(qp, "inline length too big"); - goto err_out; - } + if (mask & WR_ATOMIC_MASK) { + if (length != 8) { + rxe_err_qp(qp, "atomic length != 8"); + break; + } + if (atomic_wr(ibwr)->remote_addr & 0x7) { + rxe_err_qp(qp, "misaligned atomic address"); + break; + } + } + if (ibwr->send_flags & IB_SEND_INLINE) { + if (!(mask & WR_INLINE_MASK)) { + rxe_err_qp(qp, "opcode doesn't support inline data"); + break; + } + if (length > sq->max_inline) { + rxe_err_qp(qp, "inline length too big"); + break; + } + } - return 0; + err = 0; + } while (0); -err_out: - return -EINVAL; + *maskp = mask; + *lengthp = (int)length; + + return err; } -static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, +static int init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, const struct ib_send_wr *ibwr) { wr->wr_id = ibwr->wr_id; @@ -711,8 +739,18 @@ static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, wr->wr.ud.ah_num = to_rah(ibah)->ah_num; if (qp_type(qp) == IB_QPT_GSI) wr->wr.ud.pkey_index = ud_wr(ibwr)->pkey_index; - if (wr->opcode == IB_WR_SEND_WITH_IMM) + + switch (wr->opcode) { + case IB_WR_SEND_WITH_IMM: wr->ex.imm_data = ibwr->ex.imm_data; + break; + case IB_WR_SEND: + break; + default: + rxe_err_qp(qp, "bad wr opcode %d for UD/GSI QP", + wr->opcode); + return -EINVAL; + } } else { switch (wr->opcode) { case IB_WR_RDMA_WRITE_WITH_IMM: @@ -729,6 +767,11 @@ static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, case IB_WR_SEND_WITH_INV: wr->ex.invalidate_rkey = ibwr->ex.invalidate_rkey; break; + case IB_WR_RDMA_READ_WITH_INV: + wr->ex.invalidate_rkey = ibwr->ex.invalidate_rkey; + wr->wr.rdma.remote_addr = rdma_wr(ibwr)->remote_addr; + wr->wr.rdma.rkey = rdma_wr(ibwr)->rkey; + break; case IB_WR_ATOMIC_CMP_AND_SWP: case IB_WR_ATOMIC_FETCH_AND_ADD: wr->wr.atomic.remote_addr = @@ -746,10 +789,20 @@ static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, wr->wr.reg.key = reg_wr(ibwr)->key; wr->wr.reg.access = reg_wr(ibwr)->access; break; + case IB_WR_SEND: + case IB_WR_BIND_MW: + case IB_WR_FLUSH: + case IB_WR_ATOMIC_WRITE: + break; default: + rxe_err_qp(qp, "unsupported wr opcode %d", + wr->opcode); + return -EINVAL; break; } } + + return 0; } static void copy_inline_data_to_wqe(struct rxe_send_wqe *wqe, @@ -765,19 +818,22 @@ static void copy_inline_data_to_wqe(struct rxe_send_wqe *wqe, } } -static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, +static int init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, unsigned int mask, unsigned int length, struct rxe_send_wqe *wqe) { int num_sge = ibwr->num_sge; + int err; - init_send_wr(qp, &wqe->wr, ibwr); + err = init_send_wr(qp, &wqe->wr, ibwr); + if (err) + return err; /* local operation */ if (unlikely(mask & WR_LOCAL_OP_MASK)) { wqe->mask = mask; wqe->state = wqe_state_posted; - return; + return 0; } if (unlikely(ibwr->send_flags & IB_SEND_INLINE)) @@ -796,93 +852,62 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, wqe->dma.sge_offset = 0; wqe->state = wqe_state_posted; wqe->ssn = atomic_add_return(1, &qp->ssn); + + return 0; } -static int post_one_send(struct rxe_qp *qp, const struct ib_send_wr *ibwr, - unsigned int mask, u32 length) +static int post_one_send(struct rxe_qp *qp, const struct ib_send_wr *ibwr) { int err; struct rxe_sq *sq = &qp->sq; struct rxe_send_wqe *send_wqe; - unsigned long flags; + unsigned int mask; + unsigned int length; int full; - err = validate_send_wr(qp, ibwr, mask, length); + err = validate_send_wr(qp, ibwr, &mask, &length); if (err) return err; - spin_lock_irqsave(&qp->sq.sq_lock, flags); - full = queue_full(sq->queue, QUEUE_TYPE_FROM_ULP); if (unlikely(full)) { - spin_unlock_irqrestore(&qp->sq.sq_lock, flags); - rxe_dbg_qp(qp, "queue full"); + rxe_err_qp(qp, "send queue full"); return -ENOMEM; } send_wqe = queue_producer_addr(sq->queue, QUEUE_TYPE_FROM_ULP); - init_send_wqe(qp, ibwr, mask, length, send_wqe); - - queue_advance_producer(sq->queue, QUEUE_TYPE_FROM_ULP); - - spin_unlock_irqrestore(&qp->sq.sq_lock, flags); + err = init_send_wqe(qp, ibwr, mask, length, send_wqe); + if (!err) + queue_advance_producer(sq->queue, QUEUE_TYPE_FROM_ULP); - return 0; + return err; } -static int rxe_post_send_kernel(struct rxe_qp *qp, const struct ib_send_wr *wr, +static int rxe_post_send_kernel(struct rxe_qp *qp, + const struct ib_send_wr *ibwr, const struct ib_send_wr **bad_wr) { int err = 0; - unsigned int mask; - unsigned int length = 0; - int i; - struct ib_send_wr *next; - - while (wr) { - mask = wr_opcode_mask(wr->opcode, qp); - if (unlikely(!mask)) { - rxe_dbg_qp(qp, "bad wr opcode for qp"); - err = -EINVAL; - *bad_wr = wr; - break; - } - - if (unlikely((wr->send_flags & IB_SEND_INLINE) && - !(mask & WR_INLINE_MASK))) { - rxe_dbg_qp(qp, "opcode doesn't support inline data"); - err = -EINVAL; - *bad_wr = wr; - break; - } - - next = wr->next; - - length = 0; - for (i = 0; i < wr->num_sge; i++) - length += wr->sg_list[i].length; - if (length > 1<<31) { - err = -EINVAL; - rxe_dbg_qp(qp, "message length too long"); - *bad_wr = wr; - break; - } + unsigned long flags; - err = post_one_send(qp, wr, mask, length); + spin_lock_irqsave(&qp->sq.sq_lock, flags); + while (ibwr) { + err = post_one_send(qp, ibwr); if (err) { - *bad_wr = wr; + *bad_wr = ibwr; break; } - - wr = next; + ibwr = ibwr->next; } + spin_unlock_irqrestore(&qp->sq.sq_lock, flags); - /* if we didn't post anything there's nothing to do */ if (!err) rxe_sched_task(&qp->req.task); - if (unlikely(qp_state(qp) == IB_QPS_ERR)) + spin_lock_bh(&qp->state_lock); + if (qp_state(qp) == IB_QPS_ERR) rxe_sched_task(&qp->comp.task); + spin_unlock_bh(&qp->state_lock); return err; } @@ -893,19 +918,21 @@ static int rxe_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, struct rxe_qp *qp = to_rqp(ibqp); int err; - if (unlikely(!qp->valid)) { - *bad_wr = wr; - err = -EINVAL; - rxe_dbg_qp(qp, "qp destroyed"); - goto err_out; + spin_lock_bh(&qp->state_lock); + /* caller has already called destroy_qp */ + if (WARN_ON_ONCE(!qp->valid)) { + spin_unlock_bh(&qp->state_lock); + rxe_err_qp(qp, "qp has been destroyed"); + return -EINVAL; } if (unlikely(qp_state(qp) < IB_QPS_RTS)) { + spin_unlock_bh(&qp->state_lock); *bad_wr = wr; - err = -EINVAL; - rxe_dbg_qp(qp, "qp not ready to send"); - goto err_out; + rxe_err_qp(qp, "qp not ready to send"); + return -EINVAL; } + spin_unlock_bh(&qp->state_lock); if (qp->is_user) { /* Utilize process context to do protocol processing */ @@ -913,14 +940,10 @@ static int rxe_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, } else { err = rxe_post_send_kernel(qp, wr, bad_wr); if (err) - goto err_out; + return err; } return 0; - -err_out: - rxe_err_qp(qp, "returned err = %d", err); - return err; } /* recv wr */ @@ -985,18 +1008,27 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, struct rxe_rq *rq = &qp->rq; unsigned long flags; - if (unlikely((qp_state(qp) < IB_QPS_INIT) || !qp->valid)) { + spin_lock_bh(&qp->state_lock); + /* caller has already called destroy_qp */ + if (WARN_ON_ONCE(!qp->valid)) { + spin_unlock_bh(&qp->state_lock); + rxe_err_qp(qp, "qp has been destroyed"); + return -EINVAL; + } + + /* see C10-97.2.1 */ + if (unlikely((qp_state(qp) < IB_QPS_INIT))) { + spin_unlock_bh(&qp->state_lock); *bad_wr = wr; - err = -EINVAL; - rxe_dbg_qp(qp, "qp destroyed or not ready to post recv"); - goto err_out; + rxe_dbg_qp(qp, "qp not ready to post recv"); + return -EINVAL; } + spin_unlock_bh(&qp->state_lock); if (unlikely(qp->srq)) { *bad_wr = wr; - err = -EINVAL; - rxe_dbg_qp(qp, "use post_srq_recv instead"); - goto err_out; + rxe_dbg_qp(qp, "qp has srq, use post_srq_recv instead"); + return -EINVAL; } spin_lock_irqsave(&rq->producer_lock, flags); @@ -1012,12 +1044,10 @@ static int rxe_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, spin_unlock_irqrestore(&rq->producer_lock, flags); + spin_lock_bh(&qp->state_lock); if (qp_state(qp) == IB_QPS_ERR) rxe_sched_task(&qp->resp.task); - -err_out: - if (err) - rxe_err_qp(qp, "returned err = %d", err); + spin_unlock_bh(&qp->state_lock); return err; }