From patchwork Mon Jun 6 14:38:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12870437 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2419C433EF for ; Mon, 6 Jun 2022 14:39:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239732AbiFFOj3 (ORCPT ); Mon, 6 Jun 2022 10:39:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239831AbiFFOj0 (ORCPT ); Mon, 6 Jun 2022 10:39:26 -0400 Received: from mail-oa1-x30.google.com (mail-oa1-x30.google.com [IPv6:2001:4860:4864:20::30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B91CD150A29 for ; Mon, 6 Jun 2022 07:39:25 -0700 (PDT) Received: by mail-oa1-x30.google.com with SMTP id 586e51a60fabf-e656032735so19429492fac.0 for ; Mon, 06 Jun 2022 07:39:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hNCWlLHg2GISb9iFySN3QSGSoJ3+7SrsBn0Xrg6VcdE=; b=bScqZHssX3UlS+ZyQLDeEL8kUFm4Ykl7GGJFD/CsEskODP/zuTC0BlJs5VqrBnLnNi flt5fE8rHgdEBwNxk92lYxfXpnCKMGi/Dl2myLqJoYOLH4CTFFXftr5WepFo7bwa5p3U TEEnNNJ934sWoKYjtO36yBeSLbA8lnKDTHQWmYsizSonyZwRzyrc+RI1hSZuoU5KfFe7 c9xWkaa9s2PZmDbPq2IURqNeKRtU2Ct9aFpE1oqGjhocjBRlYSxF613StvTEH13TWuwv 3Qkl9Cw/fTiL3R5B9z+kl3/8jgz62dnWjSbdqpCUZbz/8JjfNyWvLWPuXOCrdeKgDDIl tLZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hNCWlLHg2GISb9iFySN3QSGSoJ3+7SrsBn0Xrg6VcdE=; b=CszRH5Qn0CwdR89JCy80KVYN/4/RxZVgtkuyUemNQvJRg08qYpH1VMkAbgpieAftkH C7ERnPkXnRYDBoxlqNzRjzTs4XDN1faUcEA0fjw846lPnIY6PcOKzs0MryUqJhtAnuoN hUoJcX62WG1hadj1GThcjDnQ5I2tlLlXiH+PdvOV/LkisSVrPqvPOezRbiLgmxeUWXEk wz+5FiBkSiPghFSqck7XE7yEZMR266o+YdKZn5DOuphbwtyUgL7IFMTnolApga149Nuw DxFU7BxiprnwH2KT1yML00RZSW9RB/zeIOorD4uh6Rz+jEhxjKOBw0ZK3q1BKa2h+yAl WoWA== X-Gm-Message-State: AOAM531wEZJIblBuUi3I/+SHvl67cV9xM89QsCUFAArCT3qqZMxutITp ft3OPNLG8DAnchFNjoI0HAM= X-Google-Smtp-Source: ABdhPJyi/seBzieyOrCTSuum+BIt9iQkRGPpAkT12n9/Wlh4hSs5AAwBu144wyW2LRDaPSHKhccQAA== X-Received: by 2002:a05:6871:7a1:b0:f1:b33d:7875 with SMTP id o33-20020a05687107a100b000f1b33d7875mr13691013oap.272.1654526365002; Mon, 06 Jun 2022 07:39:25 -0700 (PDT) Received: from u-22.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.googlemail.com with ESMTPSA id q28-20020a05683022dc00b0060c00c3fde5sm658335otc.72.2022.06.06.07.39.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 07:39:24 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, frank.zago@hpe.com, jhack@hpe.com Cc: Bob Pearson Subject: [PATCH 1/5] RDMA/rxe: Move code to rxe_prepare_atomic_res() Date: Mon, 6 Jun 2022 09:38:33 -0500 Message-Id: <20220606143836.3323-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220606143836.3323-1-rpearsonhpe@gmail.com> References: <20220606143836.3323-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Separate the code that prepares the atomic responder resource into a subroutine. This is preparation for merging the normal and retry atomic responder flows. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 37 ++++++++++++++++++---------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index f4f6ee5d81fe..69723bc1a071 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -1019,10 +1019,27 @@ static int send_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, return err; } +static struct resp_res *rxe_prepare_atomic_res(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + struct resp_res *res; + + res = &qp->resp.resources[qp->resp.res_head]; + rxe_advance_resp_resource(qp); + free_rd_atomic_resource(qp, res); + + res->type = RXE_ATOMIC_MASK; + res->first_psn = pkt->psn; + res->last_psn = pkt->psn; + res->cur_psn = pkt->psn; + + return res; +} + static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, u8 syndrome) { - int rc = 0; + int err = 0; struct rxe_pkt_info ack_pkt; struct sk_buff *skb; struct resp_res *res; @@ -1031,28 +1048,22 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE, 0, pkt->psn, syndrome); if (!skb) { - rc = -ENOMEM; + err = -ENOMEM; goto out; } - res = &qp->resp.resources[qp->resp.res_head]; - free_rd_atomic_resource(qp, res); - rxe_advance_resp_resource(qp); - skb_get(skb); - res->type = RXE_ATOMIC_MASK; + + res = rxe_prepare_atomic_res(qp, pkt); res->atomic.skb = skb; - res->first_psn = ack_pkt.psn; - res->last_psn = ack_pkt.psn; - res->cur_psn = ack_pkt.psn; - rc = rxe_xmit_packet(qp, &ack_pkt, skb); - if (rc) { + err = rxe_xmit_packet(qp, &ack_pkt, skb); + if (err) { pr_err_ratelimited("Failed sending ack\n"); rxe_put(qp); } out: - return rc; + return err; } static enum resp_states acknowledge(struct rxe_qp *qp, From patchwork Mon Jun 6 14:38:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12870438 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D9BBC43334 for ; Mon, 6 Jun 2022 14:39:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239822AbiFFOja (ORCPT ); Mon, 6 Jun 2022 10:39:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239791AbiFFOj1 (ORCPT ); Mon, 6 Jun 2022 10:39:27 -0400 Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F8CC153535 for ; Mon, 6 Jun 2022 07:39:26 -0700 (PDT) Received: by mail-oi1-x22e.google.com with SMTP id m82so19886962oif.13 for ; Mon, 06 Jun 2022 07:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z/QYRwqsPC9QQxGxg7qWciRwy16PvFOKYnmFeB9KJKo=; b=LTxu10YPPd7ycHGk2ftF4kfCskcPPnJ9kaP0gq7HzB50NL1Le9D3uiInryTzrMKLD5 qXVrE+F4KVE5OIaCUf42ovh8HX+PXtBH1K1V7wEhCGVH4JZgqkSJG3cD7IMR90Bn6Uql O6WnaRz1cTylPkpDp+Ne70qyNJxcCWvTScShRXpz80vWCDvm52greyaxOZMOje3IWg2y OCEwhkT2WYrbG6olnQQ/Wlh+ZeTBtHOtmZlHPszW+uqOj7C91OzSWpuMbzSYl41dOgLj AJZjafWTYmOxOZmfNTOj9l7A5skcibfFbdpzCA4o5jKlI01Z6KuKs/MEp8w8s4qhXxnW KGXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z/QYRwqsPC9QQxGxg7qWciRwy16PvFOKYnmFeB9KJKo=; b=KHtfCXg0httdBIvsDTRLYB+URrYI3alcyXWZx8JkTM8Q1evQH/xBPuXgdJO31txaxC ODDLShfw7XAtoPzpWeFGNWReSnOUy50aDdtsWPgdbx4uBHXp0gUY4qTcLKQwbwYNdPwI NCmX1SNj+2nr9gEo84S7PHfjdRoLuSNfTLbDIGjI9G7wvOclVyuSQH/U/Pp3dU+M4UAh 2kw0HxIMHqTUz0K4qWfLh2LGO6Aki2bZ04iPR2xDXA5AKW2XnlYW3ATa4miQWPvUVot4 1yNX54I0CcTKKa0mv8eEokZ0EKmChtHX8fmOJ0v8H4WigvkkbRLWU4hN6S8mtyM8wjJ9 0KkA== X-Gm-Message-State: AOAM532xQJNeRU/6cgajB2noUaRC1e2nW9aD+WXRESCmUWjVrCGdaO7O iY77CInAt3iTIDalN4lkO7M= X-Google-Smtp-Source: ABdhPJwJNVjvnDLYiXA9nZQtUGDDNlhNaXSIZnLiw6pYjcoYqsvoDjaDeeSDJbB/EKKEI19v/unK4Q== X-Received: by 2002:a05:6808:17a8:b0:327:9efa:cf1a with SMTP id bg40-20020a05680817a800b003279efacf1amr28642920oib.81.1654526365614; Mon, 06 Jun 2022 07:39:25 -0700 (PDT) Received: from u-22.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.googlemail.com with ESMTPSA id q28-20020a05683022dc00b0060c00c3fde5sm658335otc.72.2022.06.06.07.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 07:39:25 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, frank.zago@hpe.com, jhack@hpe.com Cc: Bob Pearson Subject: [PATCH 2/5] RDMA/rxe: Add a responder state for atomic reply Date: Mon, 6 Jun 2022 09:38:34 -0500 Message-Id: <20220606143836.3323-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220606143836.3323-1-rpearsonhpe@gmail.com> References: <20220606143836.3323-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add a responder state for atomic reply similar to read reply and rename process_atomic() rxe_atomic_reply(). In preparation for merging the normal and retry atomic responder flows. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 69723bc1a071..4babd6fbfefe 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -21,6 +21,7 @@ enum resp_states { RESPST_CHK_RKEY, RESPST_EXECUTE, RESPST_READ_REPLY, + RESPST_ATOMIC_REPLY, RESPST_COMPLETE, RESPST_ACKNOWLEDGE, RESPST_CLEANUP, @@ -55,6 +56,7 @@ static char *resp_state_name[] = { [RESPST_CHK_RKEY] = "CHK_RKEY", [RESPST_EXECUTE] = "EXECUTE", [RESPST_READ_REPLY] = "READ_REPLY", + [RESPST_ATOMIC_REPLY] = "ATOMIC_REPLY", [RESPST_COMPLETE] = "COMPLETE", [RESPST_ACKNOWLEDGE] = "ACKNOWLEDGE", [RESPST_CLEANUP] = "CLEANUP", @@ -552,8 +554,8 @@ static enum resp_states write_data_in(struct rxe_qp *qp, /* Guarantee atomicity of atomic operations at the machine level. */ static DEFINE_SPINLOCK(atomic_ops_lock); -static enum resp_states process_atomic(struct rxe_qp *qp, - struct rxe_pkt_info *pkt) +static enum resp_states rxe_atomic_reply(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) { u64 *vaddr; enum resp_states ret; @@ -585,7 +587,16 @@ static enum resp_states process_atomic(struct rxe_qp *qp, spin_unlock_bh(&atomic_ops_lock); - ret = RESPST_NONE; + qp->resp.msn++; + + /* next expected psn, read handles this separately */ + qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; + qp->resp.ack_psn = qp->resp.psn; + + qp->resp.opcode = pkt->opcode; + qp->resp.status = IB_WC_SUCCESS; + + ret = RESPST_ACKNOWLEDGE; out: return ret; } @@ -858,9 +869,7 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt) qp->resp.msn++; return RESPST_READ_REPLY; } else if (pkt->mask & RXE_ATOMIC_MASK) { - err = process_atomic(qp, pkt); - if (err) - return err; + return RESPST_ATOMIC_REPLY; } else { /* Unreachable */ WARN_ON_ONCE(1); @@ -1327,6 +1336,9 @@ int rxe_responder(void *arg) case RESPST_READ_REPLY: state = read_reply(qp, pkt); break; + case RESPST_ATOMIC_REPLY: + state = rxe_atomic_reply(qp, pkt); + break; case RESPST_ACKNOWLEDGE: state = acknowledge(qp, pkt); break; From patchwork Mon Jun 6 14:38:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12870439 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39A1DC43334 for ; Mon, 6 Jun 2022 14:39:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239791AbiFFOjb (ORCPT ); Mon, 6 Jun 2022 10:39:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239830AbiFFOj1 (ORCPT ); Mon, 6 Jun 2022 10:39:27 -0400 Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E747714C749 for ; Mon, 6 Jun 2022 07:39:26 -0700 (PDT) Received: by mail-oi1-x22d.google.com with SMTP id y69so6128321oia.7 for ; Mon, 06 Jun 2022 07:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0KF+yaCXSyIYckD85Ccvzhib8uV1qADvrtwkcf45tUs=; b=NdwMWgxS3Fl9YTO4Zbks9aus4zS/Qt/g+prIdCP0Yn/xCYwHfNHFTr19HCixXDF3Fl jqI3EMbQl+cs1ipSLbhSYYkJAVXBBKARR7TI+SAmPvgCiy1MRnYNNeUypLdb1sEtujGp 3/W/TMiXyt/WxSuTi+h6QMmAE2O0zmiddVfNoFeCZ+5evpdSwqErlfLCUQ7vdsN6vHqv HW938ZimF6pbNew5k2nfl6nhtVek+qpz0XX4w59z1H+/Tr9ghK1GBivlFPqH8QG/GYRX ClXvVKibheS69jJ/WCyqBHwQB8A7pv70/wqp6st8P+f8etDTfiL15keNzq9hJFX3Q+ns +qFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0KF+yaCXSyIYckD85Ccvzhib8uV1qADvrtwkcf45tUs=; b=SrfgT35k+qqzwWz6Pq31fYTdzN+BhQ+lbv2JdfU6wqjIcq2Qng6wVNp+7payu/o6+A Cw3NjT+YL/hGx25N2UXt5GFvLCCcTVXMjE4hUf6GMnPFTuvhHUedCuIVx7j5U8RufW/P NsHDpXfsnwP5i7VxMhJVdo9rP5TVkDjhewTW53cmvyf59K0Kbib3mr2T0YrHNZ1tofXT Fe0fyGsy2fp4xYyocWBlmcKSaR0VET0nt9PdcYnMEOcG5DZaSU9LxoBaIyIbXen21kTS 6Vy+3VejYhVWiyxxo8nY06ksimQBzq5JqKdsod9YpQN3owZD+h/zxBO7BaQ49KUxJDwj tllQ== X-Gm-Message-State: AOAM5320MW8vc/SQI9AmMzdVAdCoxwtrWabeIOdtEuYOaUbiwh5wl5L1 lg5WviK1sroBUbCi3ZUrRb8= X-Google-Smtp-Source: ABdhPJyGqSffGsZFJDGrDn3KqsqRCzOcL06mteak7JuI29vF5rSMC4E5qbGf7iNfi2DzfxhpUbsiiQ== X-Received: by 2002:a05:6808:191d:b0:32e:8e4e:abbd with SMTP id bf29-20020a056808191d00b0032e8e4eabbdmr3693486oib.263.1654526366256; Mon, 06 Jun 2022 07:39:26 -0700 (PDT) Received: from u-22.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.googlemail.com with ESMTPSA id q28-20020a05683022dc00b0060c00c3fde5sm658335otc.72.2022.06.06.07.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 07:39:26 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, frank.zago@hpe.com, jhack@hpe.com Cc: Bob Pearson Subject: [PATCH 3/5] RDMA/rxe: Move atomic responder res to atomic_reply Date: Mon, 6 Jun 2022 09:38:35 -0500 Message-Id: <20220606143836.3323-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220606143836.3323-1-rpearsonhpe@gmail.com> References: <20220606143836.3323-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Move the allocation of the atomic responder resource up into rxe_atomic_reply() from send_atomic_ack(). In preparation for merging the normal and retry atomic responder flows. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 49 +++++++++++++++++----------- 1 file changed, 30 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 4babd6fbfefe..4908b9fc0204 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -554,12 +554,36 @@ static enum resp_states write_data_in(struct rxe_qp *qp, /* Guarantee atomicity of atomic operations at the machine level. */ static DEFINE_SPINLOCK(atomic_ops_lock); +static struct resp_res *rxe_prepare_atomic_res(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + struct resp_res *res; + + res = &qp->resp.resources[qp->resp.res_head]; + rxe_advance_resp_resource(qp); + free_rd_atomic_resource(qp, res); + + res->type = RXE_ATOMIC_MASK; + res->first_psn = pkt->psn; + res->last_psn = pkt->psn; + res->cur_psn = pkt->psn; + res->replay = 0; + + return res; +} + static enum resp_states rxe_atomic_reply(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { u64 *vaddr; enum resp_states ret; struct rxe_mr *mr = qp->resp.mr; + struct resp_res *res = qp->resp.res; + + if (!res) { + res = rxe_prepare_atomic_res(qp, pkt); + qp->resp.res = res; + } if (mr->state != RXE_MR_STATE_VALID) { ret = RESPST_ERR_RKEY_VIOLATION; @@ -1028,30 +1052,13 @@ static int send_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, return err; } -static struct resp_res *rxe_prepare_atomic_res(struct rxe_qp *qp, - struct rxe_pkt_info *pkt) -{ - struct resp_res *res; - - res = &qp->resp.resources[qp->resp.res_head]; - rxe_advance_resp_resource(qp); - free_rd_atomic_resource(qp, res); - - res->type = RXE_ATOMIC_MASK; - res->first_psn = pkt->psn; - res->last_psn = pkt->psn; - res->cur_psn = pkt->psn; - - return res; -} - static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, u8 syndrome) { int err = 0; struct rxe_pkt_info ack_pkt; struct sk_buff *skb; - struct resp_res *res; + struct resp_res *res = qp->resp.res; skb = prepare_ack_packet(qp, pkt, &ack_pkt, IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE, 0, pkt->psn, @@ -1063,7 +1070,6 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, skb_get(skb); - res = rxe_prepare_atomic_res(qp, pkt); res->atomic.skb = skb; err = rxe_xmit_packet(qp, &ack_pkt, skb); @@ -1071,6 +1077,11 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, pr_err_ratelimited("Failed sending ack\n"); rxe_put(qp); } + + /* have to clear this since it is used to trigger + * long read replies + */ + qp->resp.res = NULL; out: return err; } From patchwork Mon Jun 6 14:38:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12870440 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2C57C433EF for ; Mon, 6 Jun 2022 14:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239821AbiFFOje (ORCPT ); Mon, 6 Jun 2022 10:39:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239781AbiFFOj2 (ORCPT ); Mon, 6 Jun 2022 10:39:28 -0400 Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7ACD115A3C8 for ; Mon, 6 Jun 2022 07:39:27 -0700 (PDT) Received: by mail-oi1-x230.google.com with SMTP id s8so14670788oib.6 for ; Mon, 06 Jun 2022 07:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pASGD7A0Oxrc5XI72YSbByAp9HDA7sfARKnMSfbO3Wc=; b=YNLPSrU0W0PB24DnkH3qvTUh2J1jp+3vX/ifIqMxjtt6DaPwTtscD/uGwDUpr1CuvK gDvDH7EuQQOQtvpWea+R5yUS/CaFXNpmq99sah+ZFZXoTOC6WZTYUxWPshI3OPTfyaog 1NIk8hZsEKgBYyoxdZwi/OHNo5E4zMzr50DqlTDrobYYQrkvJkRyuXlejME5UuRJqQqB rIscudU+gxRC64twBrPC1OucIJZ1aO80I6bW7ecKeWlEChD3TMtk09gyrVNcty8emd/y SEd30uGx5PrxWKj2cHIhDbd4zcNyUqhgdfBZf0VAZzwajUQ5iDKhGf9/ErPWe/+G66sl eqKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pASGD7A0Oxrc5XI72YSbByAp9HDA7sfARKnMSfbO3Wc=; b=Lcf4vUa+tdRYjop3PEQ9xLtI7v7WwhqfpSlQdoUo1+3TZ0I4me28yY4u//0pN4wZGR FuTf2f3UsImHexpnfozNVwDM7NJdUjcJUAyvdTo1+Dr6pwp23M8t1CgE0xUY1JhYEkjy djVn2UusrCEbXgksyT3v387HSg9XA9Ccddv+PGcg1HV8bbTb3Z9Knn9RUQy3vBf2LrAa jtWHlgePriSmx1VSnE+1rf2/rMnuqkzLntuC28rje5GeQj53TQi/PjdqQZVS+SDA2gq8 2pxCCu6j3tPwpOhgL302OnlvF4dNQByukr22e3qi/W+WHtApXhqs9/bAorlxuhIkeHdf hl8g== X-Gm-Message-State: AOAM533iyU5sxQsX0PYfzT+tpMa65OZ0xG9hHKCpip3kK+w/4erwi8qv +xv2S6ZTRQJz/EINNzj4a4H/m4Kb91iY2g== X-Google-Smtp-Source: ABdhPJxz+tr/JpUmg8888F+I1a1xrPwJ1239P31NodDA1verRcdCwFvdO+8WH+rA8AzEXjIoYuJUmw== X-Received: by 2002:a05:6808:221a:b0:32b:9564:c949 with SMTP id bd26-20020a056808221a00b0032b9564c949mr29298658oib.149.1654526366875; Mon, 06 Jun 2022 07:39:26 -0700 (PDT) Received: from u-22.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.googlemail.com with ESMTPSA id q28-20020a05683022dc00b0060c00c3fde5sm658335otc.72.2022.06.06.07.39.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 07:39:26 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, frank.zago@hpe.com, jhack@hpe.com Cc: Bob Pearson Subject: [PATCH 4/5] RDMA/rxe: Move atomic original value to res Date: Mon, 6 Jun 2022 09:38:36 -0500 Message-Id: <20220606143836.3323-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220606143836.3323-1-rpearsonhpe@gmail.com> References: <20220606143836.3323-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Move the saved original value to the atomic responder resource. This replaces saving it in the qp. In preparation for merging the normal and retry atomic responder flows. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_resp.c | 13 +++++++------ drivers/infiniband/sw/rxe/rxe_verbs.h | 2 +- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 4908b9fc0204..320ab7c717cb 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -579,6 +579,7 @@ static enum resp_states rxe_atomic_reply(struct rxe_qp *qp, enum resp_states ret; struct rxe_mr *mr = qp->resp.mr; struct resp_res *res = qp->resp.res; + u64 value; if (!res) { res = rxe_prepare_atomic_res(qp, pkt); @@ -599,16 +600,16 @@ static enum resp_states rxe_atomic_reply(struct rxe_qp *qp, } spin_lock_bh(&atomic_ops_lock); - - qp->resp.atomic_orig = *vaddr; + res->atomic.orig_val = value = *vaddr; if (pkt->opcode == IB_OPCODE_RC_COMPARE_SWAP) { - if (*vaddr == atmeth_comp(pkt)) - *vaddr = atmeth_swap_add(pkt); + if (value == atmeth_comp(pkt)) + value = atmeth_swap_add(pkt); } else { - *vaddr += atmeth_swap_add(pkt); + value += atmeth_swap_add(pkt); } + *vaddr = value; spin_unlock_bh(&atomic_ops_lock); qp->resp.msn++; @@ -664,7 +665,7 @@ static struct sk_buff *prepare_ack_packet(struct rxe_qp *qp, } if (ack->mask & RXE_ATMACK_MASK) - atmack_set_orig(ack, qp->resp.atomic_orig); + atmack_set_orig(ack, qp->resp.res->atomic.orig_val); err = rxe_prepare(&qp->pri_av, ack, skb); if (err) { diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index ac464e68c923..5ee0f2599896 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -156,6 +156,7 @@ struct resp_res { union { struct { struct sk_buff *skb; + u64 orig_val; } atomic; struct { u64 va_org; @@ -189,7 +190,6 @@ struct rxe_resp_info { u32 resid; u32 rkey; u32 length; - u64 atomic_orig; /* SRQ only */ struct { From patchwork Mon Jun 6 14:38:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12870441 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0589C43334 for ; Mon, 6 Jun 2022 14:39:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239781AbiFFOjg (ORCPT ); Mon, 6 Jun 2022 10:39:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239811AbiFFOj3 (ORCPT ); Mon, 6 Jun 2022 10:39:29 -0400 Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com [IPv6:2001:4860:4864:20::36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22D2C150A29 for ; Mon, 6 Jun 2022 07:39:28 -0700 (PDT) Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-f16a3e0529so19400205fac.2 for ; Mon, 06 Jun 2022 07:39:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U49BvOdGgMN5ce3QzHU9Q6ZMWiGBn/QthrIFJCZRmUE=; b=dRMakNC5RhVWlIKZkemu37NCzSVn1LyldZQwVcz1vxULcRYjHT7JUlRN0eTljh85DF RzN2uTwacKxTjYdzYsnhgViGcJ0WKIieOhH26U1aVqepysKGmmRsc4mwsQ0WJRXdMaAF FJWqREXieQmb/MiO4ojn6abmR7Se+JbzGgFx2u6bQDHWEdMayqdR9iBU6cHIcmRB2N/o mT8QB2BAdukJ52vf6LSW3bF3N7baPKld+9f7NE8he0aYSpPfJ3OY9cmjnCl2Tvz73JIC dfSK59s0d07UZoh+a3KG0FB0VpcyR7bsTJGw843wv5qIUBjiIRNzpspspM4vYfSztC4P NfSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U49BvOdGgMN5ce3QzHU9Q6ZMWiGBn/QthrIFJCZRmUE=; b=aGTm50EEjy8NxHxKr7enL5Ry3U1yCMOTiFHhEnaPPwJ5OXfwvEY1uXkcxVGJ8FH8KP /EYy2c6on1EZ4w5AuSBTOUhGLsYudbMA5lwZy1jgNlA0ZWc9xtKgEBFIgb1i//75tZ5m H5rWORmies9r66gBQX+Q3qHohcSYYpDc0gl8DfVD11OU3KDboloxp8f1CRyDorBaZpp9 hpl21XhVfFSKOmMfgL6eiaMzWYLy2gf8KEs1kIBAhZlani2BwzCND/PVTVhxbdS1n9hS SRvkclhf5lOEBfYavoMmN25kKoXvc1GtSaBVwYeEFz3XMG/bD+YTrFrBz0oeaiXgU+g5 3Wmw== X-Gm-Message-State: AOAM532qb1hh3IKdqlMYJ37D+TKiC4wOZ1lCNehXngFtQtQdgdEwr0K2 KnCVMoYn3UUMa3l8yxRwT5k= X-Google-Smtp-Source: ABdhPJzbu9xuDICYCVnIxYc1JtvzVaXjjmj8iq6R9oeQPgEtdtCGxHWalfyVFv8AZbcYKo+6p/6l6g== X-Received: by 2002:a05:6870:eca1:b0:f6:dd:8d8 with SMTP id eo33-20020a056870eca100b000f600dd08d8mr12347047oab.23.1654526367472; Mon, 06 Jun 2022 07:39:27 -0700 (PDT) Received: from u-22.tx.rr.com (097-099-248-255.res.spectrum.com. [97.99.248.255]) by smtp.googlemail.com with ESMTPSA id q28-20020a05683022dc00b0060c00c3fde5sm658335otc.72.2022.06.06.07.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 07:39:27 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org, frank.zago@hpe.com, jhack@hpe.com Cc: Bob Pearson Subject: [PATCH 5/5] RDMA/rxe: Merge normal and retry atomic flows Date: Mon, 6 Jun 2022 09:38:37 -0500 Message-Id: <20220606143836.3323-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220606143836.3323-1-rpearsonhpe@gmail.com> References: <20220606143836.3323-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make the execution of the atomic operation in rxe_atomic_reply() conditional on res->replay and make duplicate_request() call into rxe_atomic_reply() to merge the two flows. This is modeled on the behavior of read reply. Delete the skb from the atomic responder resource since it is no longer used. Adjust the reference counting of the qp in send_atomic_ack() for this flow. Fixes: 8700e3e7c485 ("The software RoCE driver") Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_qp.c | 2 - drivers/infiniband/sw/rxe/rxe_resp.c | 79 ++++++++++++--------------- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 - 3 files changed, 36 insertions(+), 46 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c index 22e9b85344c3..8355a5b1cb60 100644 --- a/drivers/infiniband/sw/rxe/rxe_qp.c +++ b/drivers/infiniband/sw/rxe/rxe_qp.c @@ -129,8 +129,6 @@ static void free_rd_atomic_resources(struct rxe_qp *qp) void free_rd_atomic_resource(struct rxe_qp *qp, struct resp_res *res) { - if (res->type == RXE_ATOMIC_MASK) - kfree_skb(res->atomic.skb); res->type = 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 320ab7c717cb..6888bd9d0bfc 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -586,40 +586,43 @@ static enum resp_states rxe_atomic_reply(struct rxe_qp *qp, qp->resp.res = res; } - if (mr->state != RXE_MR_STATE_VALID) { - ret = RESPST_ERR_RKEY_VIOLATION; - goto out; - } + if (!res->replay) { + if (mr->state != RXE_MR_STATE_VALID) { + ret = RESPST_ERR_RKEY_VIOLATION; + goto out; + } - vaddr = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, sizeof(u64)); + vaddr = iova_to_vaddr(mr, qp->resp.va + qp->resp.offset, + sizeof(u64)); - /* check vaddr is 8 bytes aligned. */ - if (!vaddr || (uintptr_t)vaddr & 7) { - ret = RESPST_ERR_MISALIGNED_ATOMIC; - goto out; - } + /* check vaddr is 8 bytes aligned. */ + if (!vaddr || (uintptr_t)vaddr & 7) { + ret = RESPST_ERR_MISALIGNED_ATOMIC; + goto out; + } - spin_lock_bh(&atomic_ops_lock); - res->atomic.orig_val = value = *vaddr; + spin_lock_bh(&atomic_ops_lock); + res->atomic.orig_val = value = *vaddr; - if (pkt->opcode == IB_OPCODE_RC_COMPARE_SWAP) { - if (value == atmeth_comp(pkt)) - value = atmeth_swap_add(pkt); - } else { - value += atmeth_swap_add(pkt); - } + if (pkt->opcode == IB_OPCODE_RC_COMPARE_SWAP) { + if (value == atmeth_comp(pkt)) + value = atmeth_swap_add(pkt); + } else { + value += atmeth_swap_add(pkt); + } - *vaddr = value; - spin_unlock_bh(&atomic_ops_lock); + *vaddr = value; + spin_unlock_bh(&atomic_ops_lock); - qp->resp.msn++; + qp->resp.msn++; - /* next expected psn, read handles this separately */ - qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; - qp->resp.ack_psn = qp->resp.psn; + /* next expected psn, read handles this separately */ + qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; + qp->resp.ack_psn = qp->resp.psn; - qp->resp.opcode = pkt->opcode; - qp->resp.status = IB_WC_SUCCESS; + qp->resp.opcode = pkt->opcode; + qp->resp.status = IB_WC_SUCCESS; + } ret = RESPST_ACKNOWLEDGE; out: @@ -1059,7 +1062,6 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, int err = 0; struct rxe_pkt_info ack_pkt; struct sk_buff *skb; - struct resp_res *res = qp->resp.res; skb = prepare_ack_packet(qp, pkt, &ack_pkt, IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE, 0, pkt->psn, @@ -1069,15 +1071,9 @@ static int send_atomic_ack(struct rxe_qp *qp, struct rxe_pkt_info *pkt, goto out; } - skb_get(skb); - - res->atomic.skb = skb; - err = rxe_xmit_packet(qp, &ack_pkt, skb); - if (err) { - pr_err_ratelimited("Failed sending ack\n"); - rxe_put(qp); - } + if (err) + pr_err_ratelimited("Failed sending atomic ack\n"); /* have to clear this since it is used to trigger * long read replies @@ -1205,14 +1201,11 @@ static enum resp_states duplicate_request(struct rxe_qp *qp, /* Find the operation in our list of responder resources. */ res = find_resource(qp, pkt->psn); if (res) { - skb_get(res->atomic.skb); - /* Resend the result. */ - rc = rxe_xmit_packet(qp, pkt, res->atomic.skb); - if (rc) { - pr_err("Failed resending result. This flow is not handled - skb ignored\n"); - rc = RESPST_CLEANUP; - goto out; - } + res->replay = 1; + res->cur_psn = pkt->psn; + qp->resp.res = res; + rc = RESPST_ATOMIC_REPLY; + goto out; } /* Resource not found. Class D error. Drop the request. */ diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 5ee0f2599896..0c01c7f58d43 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -155,7 +155,6 @@ struct resp_res { union { struct { - struct sk_buff *skb; u64 orig_val; } atomic; struct {