From patchwork Thu Sep 30 04:25:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527225 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38896C433EF for ; Thu, 30 Sep 2021 04:27:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B8686124A for ; Thu, 30 Sep 2021 04:27:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347928AbhI3E26 (ORCPT ); Thu, 30 Sep 2021 00:28:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347733AbhI3E24 (ORCPT ); Thu, 30 Sep 2021 00:28:56 -0400 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F203FC06176A for ; Wed, 29 Sep 2021 21:27:14 -0700 (PDT) Received: by mail-ot1-x32c.google.com with SMTP id d12-20020a05683025cc00b0054d8486c6b8so5770968otu.0 for ; Wed, 29 Sep 2021 21:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pcxAdiImyK4B9yAh9TbVrtYx8EXTONj/TcOyvXcVc/w=; b=Ix9Nhru8lqlhBJ8TTPpbX/vX6ah0Hf257E5Jf1ZHm9M048BwjqMM5Gz7z7gb/TSPzh xN6dHALQzGB5/t29toP8dmTe9FjHgX2PB8vpiPCyp5EvHlMRgCQgR/aiJhQ9XiXTN6d9 WAs3s/niQj5U6mVklGBsraxQYex+w/p/xc8XNwIYhWGO4CSh7mypgaH7NZXGzSABf44I DXGbUfhuTsRMg4sFZKTO2P/vg/xGMGTVOQI59b/eaZuI6bCE7qXh5zGKf3jLusk04YYh NHaG3nwm9DRmi+QfV5tWmpex9byqSQN1km3TUfvrQ5nDdPTYeFcBQcmeYynX0b1TZtgF I2AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pcxAdiImyK4B9yAh9TbVrtYx8EXTONj/TcOyvXcVc/w=; b=xosNje1AgCZ7nA/1t/XFbtXp44dsv4RajF4AvwVFXKgaJ4OUiy4Nmsvdf+yXC9akxO HqoUbuqArWfRJl/jBVSd+lBbIhqp6YQKlmujiKuvaDak756wIysPkVkExOiFOcX8tYdQ NVPMhd9lDPvmN6xzI3Q7hJv5nzNDBsIxg4Qff6o+uFzxRL9EU+H1ojLp5r+Jk6PvH11M VQXda9HrxMstGSWxJKgT/lGFaaDK7NF1mgSyOzV6dGu2qUiUOWHm/isjzBTOd7M4c7ey GnCmhOR+vD83gHkc9jtwDSbFBd5KkMmwqlVWibuOg8+LF5RVi0ayC2deDZFYK8dL/Rpz ZCmA== X-Gm-Message-State: AOAM5304XgspI6KanYSZJ0WnpvteNL6tCEGrJR4Aiq/Xdyrzx3UfWfkR 1HeV2UDcoS7jBu8WWiDsjpHzQ2UGUYw1tA== X-Google-Smtp-Source: ABdhPJyo991Zf0glgxtdBGq0KQdxiVjIQ/ilnN8lZ8R606ZeYqgr1wC9whqf71ckD0M5o7Thj0gZUA== X-Received: by 2002:a05:6830:112:: with SMTP id i18mr3227513otp.186.1632976034415; Wed, 29 Sep 2021 21:27:14 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:14 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 1/6] RDMA/rxe: Move AV from rxe_send_wqe to rxe_send_wr Date: Wed, 29 Sep 2021 23:25:59 -0500 Message-Id: <20210930042603.4318-2-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Move the struct rxe_av av from struct rxe_send_wqe to struct rxe_send_wr placing it in wr.ud at the same offset as it was previously. This has the effect of increasing the size of struct rxe_send_wr while keeping the size of struct rxe_send_wqe the same. This better reflects the use of this field which is only used for UD sends. This change has no effect on ABI compatibility so the modified rxe driver will operate with older versions of rdma-core. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_av.c | 2 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 3 ++- include/uapi/rdma/rdma_user_rxe.h | 4 +++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c index da2e867a1ed9..85580ea5eed0 100644 --- a/drivers/infiniband/sw/rxe/rxe_av.c +++ b/drivers/infiniband/sw/rxe/rxe_av.c @@ -107,5 +107,5 @@ struct rxe_av *rxe_get_av(struct rxe_pkt_info *pkt) if (qp_type(pkt->qp) == IB_QPT_RC || qp_type(pkt->qp) == IB_QPT_UC) return &pkt->qp->pri_av; - return (pkt->wqe) ? &pkt->wqe->av : NULL; + return (pkt->wqe) ? &pkt->wqe->wr.wr.ud.av : NULL; } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 9d0bb9aa7514..c09e1c25ce66 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -584,7 +584,8 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, if (qp_type(qp) == IB_QPT_UD || qp_type(qp) == IB_QPT_SMI || qp_type(qp) == IB_QPT_GSI) - memcpy(&wqe->av, &to_rah(ud_wr(ibwr)->ah)->av, sizeof(wqe->av)); + memcpy(&wqe->wr.wr.ud.av, &to_rah(ud_wr(ibwr)->ah)->av, + sizeof(struct rxe_av)); if (unlikely(ibwr->send_flags & IB_SEND_INLINE)) copy_inline_data_to_wqe(wqe, ibwr); diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index e283c2220aba..2f1ebbe96434 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -98,6 +98,9 @@ struct rxe_send_wr { __u32 remote_qpn; __u32 remote_qkey; __u16 pkey_index; + __u16 reserved; + __u32 pad[5]; + struct rxe_av av; } ud; struct { __aligned_u64 addr; @@ -148,7 +151,6 @@ struct rxe_dma_info { struct rxe_send_wqe { struct rxe_send_wr wr; - struct rxe_av av; __u32 status; __u32 state; __aligned_u64 iova; From patchwork Thu Sep 30 04:26:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527231 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC6C8C4332F for ; Thu, 30 Sep 2021 04:27:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 888586124A for ; Thu, 30 Sep 2021 04:27:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347935AbhI3E27 (ORCPT ); Thu, 30 Sep 2021 00:28:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347778AbhI3E25 (ORCPT ); Thu, 30 Sep 2021 00:28:57 -0400 Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81A90C06161C for ; Wed, 29 Sep 2021 21:27:15 -0700 (PDT) Received: by mail-oo1-xc2c.google.com with SMTP id b5-20020a4ac285000000b0029038344c3dso1456214ooq.8 for ; Wed, 29 Sep 2021 21:27:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ln2CICo8MaUfdlYX/StcL4ROVFBeevggZ31VqY1hkwM=; b=AdYUQ+L/aOETahwfp8KyKdKjl/m9cE3+KGu9vvbeW3BQBY2ie6HMgEL5nbyUNtQ+gJ c58CG04lE67zWgLoD1luKAd3U3AZ1fAwVzzoPcwHAgnx3SQyy//lA/6hjVjMImMdmmSg ThFZQJWZULiDHl+9+ZVvHn5Lf/tQP/jv9Mv7z2PltUGE/MuEHglLqUQ2vc7g5zUJ8W4N 46DlSJcz6gkIzqBftXDh8QSYDXBKqs6uX/f0XJmA4jeuUZ3MGl6nAzFRykHe+rCPOXlU hIv926d1q3uJ7wTgO0HU/C+yiyD0jAMytITRdK28yZLnn1zUY6d0Vaupt/li32ITVJRO /Ygw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ln2CICo8MaUfdlYX/StcL4ROVFBeevggZ31VqY1hkwM=; b=bi7mBNb9FvVMB1bGeTrxqQdCouiJ1InwUn2htUkFPWGdali0tVR8f9DOG60oOUfzMH +5QP696S5VLvK+phajFYZy/iGV+ZdaR9VsgxEmwTEhp87Czv5qUQ9rjswj2CzWCo6D41 5969LwE35Myt962976Lr/YQelR1pK80d+/1avOrLh8KNfe0IOPD5Q5F/ORyvHcqovmPh 315Vnx002PCxUXdvr3klEacHsu9kh8Nqs6E6QLjcOxb6KzUpVTnxni4D0Ww5brivEWm7 AQqGpSKXwa6qDW956Iw307kmOznYpch0WmJRxhS5vm47Wz5tQIClAx+kTnLrFlGixe1P Rq5g== X-Gm-Message-State: AOAM530nv8Fwq/zNlmyQ/NBhVYyUX6kro7pMUbRGcVFK3Zah9MCJ4/EE bW0l8ALGeElAzDfc94+TBqk= X-Google-Smtp-Source: ABdhPJznPgBENUs41cA7tEpRnfFegGC9esdb5y5gfQNSHPfEQzXadfnl0e5on2QVX+Seq9CN1YgOcA== X-Received: by 2002:a4a:d883:: with SMTP id b3mr2984461oov.82.1632976034903; Wed, 29 Sep 2021 21:27:14 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:14 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 2/6] RDMA/rxe: Change AH objects to indexed Date: Wed, 29 Sep 2021 23:26:00 -0500 Message-Id: <20210930042603.4318-3-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make changes to rxe_param.h and rxe_pool.c to allow indexing of AH objects. Valid indices are non-zero so older providers can be detected. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_param.h | 4 +++- drivers/infiniband/sw/rxe/rxe_pool.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index b5a70cbe94aa..d92d7edd712b 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -67,7 +67,9 @@ enum rxe_device_param { RXE_MAX_MCAST_GRP = 8192, RXE_MAX_MCAST_QP_ATTACH = 56, RXE_MAX_TOT_MCAST_QP_ATTACH = 0x70000, - RXE_MAX_AH = 100, + RXE_MAX_AH = 16383, + RXE_MIN_AH_INDEX = 1, + RXE_MAX_AH_INDEX = 16383, RXE_MAX_SRQ_WR = 0x4000, RXE_MIN_SRQ_WR = 1, RXE_MAX_SRQ_SGE = 27, diff --git a/drivers/infiniband/sw/rxe/rxe_pool.c b/drivers/infiniband/sw/rxe/rxe_pool.c index ffa8420b4765..7b4cb46edfd9 100644 --- a/drivers/infiniband/sw/rxe/rxe_pool.c +++ b/drivers/infiniband/sw/rxe/rxe_pool.c @@ -26,7 +26,9 @@ struct rxe_type_info rxe_type_info[RXE_NUM_TYPES] = { .name = "rxe-ah", .size = sizeof(struct rxe_ah), .elem_offset = offsetof(struct rxe_ah, pelem), - .flags = RXE_POOL_NO_ALLOC, + .flags = RXE_POOL_INDEX | RXE_POOL_NO_ALLOC, + .min_index = RXE_MIN_AH_INDEX, + .max_index = RXE_MAX_AH_INDEX, }, [RXE_TYPE_SRQ] = { .name = "rxe-srq", From patchwork Thu Sep 30 04:26:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527229 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E9F3C43217 for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4916661880 for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347778AbhI3E27 (ORCPT ); Thu, 30 Sep 2021 00:28:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347920AbhI3E26 (ORCPT ); Thu, 30 Sep 2021 00:28:58 -0400 Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 092A6C06176C for ; Wed, 29 Sep 2021 21:27:16 -0700 (PDT) Received: by mail-ot1-x32c.google.com with SMTP id l16-20020a9d6a90000000b0053b71f7dc83so5691794otq.7 for ; Wed, 29 Sep 2021 21:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lBxkpKC/Ie6Rn+yTC69M869crKSuud+tFzqyCacmKP8=; b=LEQGWgUjTN2IM4ykt5wEWjmkF0I12yebGMBEmBIb3KmEbLCKb9fE+3BJOR+exIMF65 yMM1mmrrUaJoubeHpPdLDP0iG5mvwbjNwVgeCglHwFgfmf/LDvLrOQeCewuZskffNtDg MS37KxRM+9BNZePtycT85JgYxb6MlXOh+Lr3ZVSgtzH+WCeloQcsiNFEz33d2jmpG722 Mf8OWMbhERMw7tVjZvTS+8dm5Jl/6NavN5yF+YcBeoQDomrwsS9Sf6GbFAz1xRzJpv1g 0k+DEB0iBSdFU3VxiltmKtGduLQFOMgu9AMqGoc0+2gCv3gmT9qJAxLnBvK9dubSmkYA RdKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lBxkpKC/Ie6Rn+yTC69M869crKSuud+tFzqyCacmKP8=; b=TCJUoUe5PPW3bM4hEYZg4PPECC4hHDn0hzjIdg9DwLiF6wzeIEaBFaiI7BRzds/n93 fLNJLieTTh3ckaaT65veYYA4nG2EEQoOhSssBFGvNKaBQBNOSZP/FQuwW9I0US3gYg0S IoioPiSHA4AIlPyKhs0TtF79Af46nWwJXbwZwiH4R+IS96+b+Fi979H3N4RyHPwTYXRj hZhE1ZAuCe8qTtKp2MUKdzjfJYz8K4pkKASOW9ER/Zba8Bi5UMGg4b38KG4RUTIAN4Eh Gt+11WT7Lzn5iIEITfL7/QrNiacVzdwRXvIlnwd7AtxMNByRd6SEDpRe6/NU61s94oYJ YYHA== X-Gm-Message-State: AOAM532trdU0GtQPKOt9w/zIowPR0aaueP7eoNLXM6YBY2qNXCHF8Mi2 xy+Wqtak2Cwi9Wk9lFNYZP4= X-Google-Smtp-Source: ABdhPJxsz3bPZqA9X717mA1MK4x1rvt+F3LCULChhMqLLzKAuDo6QyznotuDEieJLzvFW6GjOCAbpg== X-Received: by 2002:a05:6830:24a8:: with SMTP id v8mr3134279ots.185.1632976035451; Wed, 29 Sep 2021 21:27:15 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:15 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 3/6] RDMA/rxe: Create AH index and return to user space Date: Wed, 29 Sep 2021 23:26:01 -0500 Message-Id: <20210930042603.4318-4-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Make changes to rdma_user_rxe.h to allow indexing AH objects, passing the index in UD send WRs to the driver and returning the index to the rxe provider. Modify rxe_create_ah() to add an index to AH when created and if called from a new user provider return it to user space. If called from an old provider mark the AH as not having a useful index. Modify rxe_destroy_ah to drop the index before deleting the object. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.c | 31 ++++++++++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_verbs.h | 2 ++ include/uapi/rdma/rdma_user_rxe.h | 8 ++++++- 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index c09e1c25ce66..8854ace63acd 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -158,9 +158,19 @@ static int rxe_create_ah(struct ib_ah *ibah, struct ib_udata *udata) { - int err; struct rxe_dev *rxe = to_rdev(ibah->device); struct rxe_ah *ah = to_rah(ibah); + struct rxe_create_ah_resp __user *uresp = NULL; + int err; + + if (udata) { + /* test if new user provider */ + if (udata->outlen >= sizeof(*uresp)) + uresp = udata->outbuf; + ah->is_user = true; + } else { + ah->is_user = false; + } err = rxe_av_chk_attr(rxe, init_attr->ah_attr); if (err) @@ -170,6 +180,24 @@ static int rxe_create_ah(struct ib_ah *ibah, if (err) return err; + /* create index > 0 */ + rxe_add_index(ah); + ah->ah_num = ah->pelem.index; + + if (uresp) { + /* only if new user provider */ + err = copy_to_user(&uresp->ah_num, &ah->ah_num, + sizeof(uresp->ah_num)); + if (err) { + rxe_drop_index(ah); + rxe_drop_ref(ah); + return -EFAULT; + } + } else if (ah->is_user) { + /* only if old user provider */ + ah->ah_num = 0; + } + rxe_init_av(init_attr->ah_attr, &ah->av); return 0; } @@ -202,6 +230,7 @@ static int rxe_destroy_ah(struct ib_ah *ibah, u32 flags) { struct rxe_ah *ah = to_rah(ibah); + rxe_drop_index(ah); rxe_drop_ref(ah); return 0; } diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index c807639435eb..9cd203f1fa22 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -48,6 +48,8 @@ struct rxe_ah { struct rxe_pool_entry pelem; struct rxe_pd *pd; struct rxe_av av; + bool is_user; + int ah_num; }; struct rxe_cqe { diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index 2f1ebbe96434..dc9f7a5e203a 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -99,7 +99,8 @@ struct rxe_send_wr { __u32 remote_qkey; __u16 pkey_index; __u16 reserved; - __u32 pad[5]; + __u32 ah_num; + __u32 pad[4]; struct rxe_av av; } ud; struct { @@ -170,6 +171,11 @@ struct rxe_recv_wqe { struct rxe_dma_info dma; }; +struct rxe_create_ah_resp { + __u32 ah_num; + __u32 reserved; +}; + struct rxe_create_cq_resp { struct mminfo mi; }; From patchwork Thu Sep 30 04:26:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527227 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79CE7C43219 for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 59E9B61881 for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347920AbhI3E3A (ORCPT ); Thu, 30 Sep 2021 00:29:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346657AbhI3E26 (ORCPT ); Thu, 30 Sep 2021 00:28:58 -0400 Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7F346C06176A for ; Wed, 29 Sep 2021 21:27:16 -0700 (PDT) Received: by mail-oo1-xc34.google.com with SMTP id y16-20020a4ade10000000b002b5dd6f4c8dso1446810oot.12 for ; Wed, 29 Sep 2021 21:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qApN8UnhyEUIjIpM/GmsOjj23Il/VJ1qkJLR9SYk3QA=; b=Wldlagw7thHxcDgiV9GEopPWyPfoXGlvrSP2i1Wq25u4mGHoH4jhyS+jvP4u3ZjXNx SvkIt7Dsui2t4JgLbSUgoZOJQovZT1vaAHa75gfeu6UCi5eDh00A4Gzu0KF9E29dZ5uZ cRxjlg8JsBs66xyXu+y19DQUOTlcUiNKg8kpqb2cxnuBG+2TiFA3zbtb5OMulXiyoY// MN7afehdI6sM8oYasolJXH8EvQHBcXUSZBR+LOdWCqoNQ9ddkZECvxU9SpkGN7x3KpwV ya9CRD24ptQ4Nni3oo2lOTql8bhMuJMyHjdRhYM1Y28GkqRm73gHJtc6Jaymv7GrAjyd 4s5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qApN8UnhyEUIjIpM/GmsOjj23Il/VJ1qkJLR9SYk3QA=; b=m4fH1xkzEfwp0gPgm5LTegKN6x+Zt8P6oUW3cpCAQ4byvl7PKwl1EwncsZKvsBerEp qJzqBX0XTlHPHQuyWynHfEt1hg5O7+GaGhWpda7Y5astfTeT2wLpMde7kb5xV1NLrmWA goM/ZwLDyJDoq4x2vQmmfg3OqF4eRApZ3q9gioYYXbV/A9qLDPernFQ3uHj2d5arD5b6 Vha/Wc1Po5xnxOo5ijgG2aVsqzb70B0aK3PZnLCbPkSfnGZEzX16MRHbn2VbKXXJu9Mp Y+i3MBI/Z/1ye+/17oJ+zGofxWFWcxRiFKLJ0KiydiJ9wiaXsi65for0BQKLnjWlY5jO uwCg== X-Gm-Message-State: AOAM533TXdGczRG8zvCAMf8IilQpYTrpnU27T6kpIrf1jQjqIp3MM5m3 Ii4kiIZ+tKp4xyYTu/fVV5crMfzCsz9sOg== X-Google-Smtp-Source: ABdhPJwP8FsBlkgkGHNpggNF0UUCRI0mhEZVDcIDe8TtbIfvTpWfE2NLuZLpi1CVI4MVWs/KzjM3fw== X-Received: by 2002:a4a:424b:: with SMTP id i11mr3038281ooj.87.1632976035945; Wed, 29 Sep 2021 21:27:15 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:15 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 4/6] RDMA/rxe: Replace ah->pd by ah->ibah.pd Date: Wed, 29 Sep 2021 23:26:02 -0500 Message-Id: <20210930042603.4318-5-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The pd field in struct rxe_ah is redundant with the pd field in the rdma-core's ib_ah. Eliminate the pd field in rxe_ah and add an inline to extract the pd from the ibah field. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 9cd203f1fa22..881a5159a7d0 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -46,7 +46,6 @@ struct rxe_pd { struct rxe_ah { struct ib_ah ibah; struct rxe_pool_entry pelem; - struct rxe_pd *pd; struct rxe_av av; bool is_user; int ah_num; @@ -471,6 +470,11 @@ static inline struct rxe_mw *to_rmw(struct ib_mw *mw) return mw ? container_of(mw, struct rxe_mw, ibmw) : NULL; } +static inline struct rxe_pd *rxe_ah_pd(struct rxe_ah *ah) +{ + return to_rpd(ah->ibah.pd); +} + static inline struct rxe_pd *mr_pd(struct rxe_mr *mr) { return to_rpd(mr->ibmr.pd); From patchwork Thu Sep 30 04:26:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527235 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE3C2C433F5 for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B593D6126A for ; Thu, 30 Sep 2021 04:27:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346657AbhI3E3A (ORCPT ); Thu, 30 Sep 2021 00:29:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347733AbhI3E27 (ORCPT ); Thu, 30 Sep 2021 00:28:59 -0400 Received: from mail-ot1-x330.google.com (mail-ot1-x330.google.com [IPv6:2607:f8b0:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C1E7C06161C for ; Wed, 29 Sep 2021 21:27:17 -0700 (PDT) Received: by mail-ot1-x330.google.com with SMTP id g62-20020a9d2dc4000000b0054752cfbc59so5733873otb.1 for ; Wed, 29 Sep 2021 21:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kxLoSpQBwcYypLUoXkx1nNuISVxwJZaSG5YT/2mAQ9Y=; b=FbFxyC0YLB59LczchZn1vlqLChyo5Q3ylhih1vZFScan++o2tkoQHZkWtrT7R+qSKN ZEbLjDACoxGMVxvb6/u7YWwpcBXd44DkBS2X+sUYUzSTej2dmGVE9PQKheGhz27A9yFT aQMTTYRMHAVwUHz8qLmqS0yoMfq+Q/PYy66U3R67q1I0NvJsgHbcuAWRa0qToot91Hu+ LDvQYtgmnJanUGf4Pk9YsULfJbUxLPvsXFhcCz+NAh+10g1hwJC6IpGYl4s6k9jKyXKl 6WLwNi7kL+8hYOjWmfFiyOp5lTVACjqRi5kaEkOAxB+Dop52dm1btTj8e7+7f1qiARAA xW+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kxLoSpQBwcYypLUoXkx1nNuISVxwJZaSG5YT/2mAQ9Y=; b=xWgBEbZ/yA6AuhnwWllyHPRoaGR/zO3OAnSuZwVwRqf4ESocwI5hFLjiJQI0jDm7gi k0msQ2hUy64qEVT/ik7t8H/vzkiJGI33KGxs1R07doHPjPWM833Ktor1yy7i06QQPl6H ZsckItDLo3GdKuz16eODaB6qQ9F9Lxn6Qg7IZ91zc7osZT22d1EV0aa9ybq4G3gHGQuA HMSR6TWGnSaUc1N1NTdPDbO4mdFGbySUbY4TQy9kr5obC8vOQeark+/Br28x5SZ9hck/ jyO18ZkCHhcHc4qBMJXhIsJ6I29rY8TfRZz6YQeot3az3gv6F4FXcodPnVhwCIiZuLvU qqJg== X-Gm-Message-State: AOAM5305IL4a0w+E8W+KP4QRjRqDXcn+jeVLqhGLx6MQ4ZU5BdybfI6g S/lRS+IoWGJMQRAc0/9qkGrYCSDfwB8fHg== X-Google-Smtp-Source: ABdhPJwQZ7s+WpqgYOwD3T0C69tWbpmR3CkdX12HH4P4dFTXOTL2/0dXX2c1HP6OImK/Da+nGai74g== X-Received: by 2002:a05:6830:25ca:: with SMTP id d10mr3311628otu.27.1632976036468; Wed, 29 Sep 2021 21:27:16 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:16 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 5/6] RDMA/rxe: Lookup kernel AH from ah index in UD WQEs Date: Wed, 29 Sep 2021 23:26:03 -0500 Message-Id: <20210930042603.4318-6-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add code to rxe_get_av in rxe_av.c to use the AH index in UD send WQEs to lookup the kernel AH. For old user providers continue to use the AV passed in WQEs. Move setting pkt->rxe to before the call to rxe_get_av() to get access to the AH pool. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_av.c | 20 +++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++++--- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c index 85580ea5eed0..38c7b6fb39d7 100644 --- a/drivers/infiniband/sw/rxe/rxe_av.c +++ b/drivers/infiniband/sw/rxe/rxe_av.c @@ -101,11 +101,29 @@ void rxe_av_fill_ip_info(struct rxe_av *av, struct rdma_ah_attr *attr) struct rxe_av *rxe_get_av(struct rxe_pkt_info *pkt) { + struct rxe_ah *ah; + u32 ah_num; + if (!pkt || !pkt->qp) return NULL; if (qp_type(pkt->qp) == IB_QPT_RC || qp_type(pkt->qp) == IB_QPT_UC) return &pkt->qp->pri_av; - return (pkt->wqe) ? &pkt->wqe->wr.wr.ud.av : NULL; + if (!pkt->wqe) + return NULL; + + ah_num = pkt->wqe->wr.wr.ud.ah_num; + if (ah_num) { + /* only new user provider or kernel client */ + ah = rxe_pool_get_index(&pkt->rxe->ah_pool, ah_num); + if (!ah || ah->ah_num != ah_num || rxe_ah_pd(ah) != pkt->qp->pd) { + pr_warn("Unable to find AH matching ah_num\n"); + return NULL; + } + return &ah->av; + } + + /* only old user provider for UD sends*/ + return &pkt->wqe->wr.wr.ud.av; } diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 2981b3ef3cc0..06f642f942f7 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -379,9 +379,8 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, /* length from start of bth to end of icrc */ paylen = rxe_opcode[opcode].length + payload + pad + RXE_ICRC_SIZE; - /* pkt->hdr, rxe, port_num and mask are initialized in ifc - * layer - */ + /* pkt->hdr, port_num and mask are initialized in ifc layer */ + pkt->rxe = rxe; pkt->opcode = opcode; pkt->qp = qp; pkt->psn = qp->req.psn; @@ -391,6 +390,9 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, /* init skb */ av = rxe_get_av(pkt); + if (!av) + return NULL; + skb = rxe_init_packet(rxe, av, paylen, pkt); if (unlikely(!skb)) return NULL; From patchwork Thu Sep 30 04:26:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bob Pearson X-Patchwork-Id: 12527233 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E23AC433FE for ; Thu, 30 Sep 2021 04:27:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A0446124A for ; Thu, 30 Sep 2021 04:27:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347733AbhI3E3B (ORCPT ); Thu, 30 Sep 2021 00:29:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347937AbhI3E27 (ORCPT ); Thu, 30 Sep 2021 00:28:59 -0400 Received: from mail-ot1-x32f.google.com (mail-ot1-x32f.google.com [IPv6:2607:f8b0:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94607C06176A for ; Wed, 29 Sep 2021 21:27:17 -0700 (PDT) Received: by mail-ot1-x32f.google.com with SMTP id r43-20020a05683044ab00b0054716b40005so5707641otv.4 for ; Wed, 29 Sep 2021 21:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Fo+XA5ZhNhrA0pZdOhXylGStZB1nDPPpygrSUz+ZKz8=; b=qLClRVegKJJOpqtO0PJ3KHIya6GOv0HA3uaEcOO8il507mNKvkzReF6SOmdpDRItpL 4OhFANN/5XT2DWQFGGLWrog9uW4gl6w7mlFEoOXQjGoY3y70eI9DYtmAQoO5W73A15SV uKf5Jd/FPKJhWrHtLv42zZpu0Lz58buHk/ZvbMkcYgIIqTTQ2SF/PhbeXTBeAcSPzRtW RFKVTs6Wo19TsgMtKFI4LBSBgn0zkdZ7mUTxRiBKks0wk+ry31X0Ugukt2KE9/d+Rd+s LwZasa2+lzOhOuNhXUmpptJ2PZjItP0ZzXshfjjgDp4sF0iDds26SO8XjyZdBBBRBisk TUWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Fo+XA5ZhNhrA0pZdOhXylGStZB1nDPPpygrSUz+ZKz8=; b=QsWpkDkWsklJ1wL7KlIC862EaFyNY5FxePr2qxT1R6d8C5IllGO27B7LfBzsU8AGG9 X+0IMwV0xAZTRzXz/BIY0rLJad1PmJ2PSYUm8bky7OJnE5Xk/SgtpH9ALkzbITlb9anx nZYTqF1U0zTeuvTRlhmlfK9+d7+yEUq2Tqcre+xQOPWJg70HpNRTWdGcWoWnBSAWAANj sLy372dlTkvFLmGZT547GZs/3dSdAbIRIDWoj5dbAJerMWrbO6rJyyQ7/AJj1eWi+/Qe ws/YJs751nEgroOHGWw8AoRaID3E7SXTkOcF5+AuE/VAnMHLggxH2UpQQYdLzIncRBcJ fcEQ== X-Gm-Message-State: AOAM532Apm5BEWfV7cxMEQMptJvYqxzOf1a14w6Uo4TGsGYLg3JUmRm+ hprGxwUBQ7YuU4BNq45Stg0= X-Google-Smtp-Source: ABdhPJwMFHLTVcA/mhoQ9ePpwao4bKju4UhGKh2fqKhBeCMl/7E9SyU++6dg5jfytPnlDWuhI6KzyQ== X-Received: by 2002:a9d:f04:: with SMTP id 4mr3164432ott.307.1632976037009; Wed, 29 Sep 2021 21:27:17 -0700 (PDT) Received: from ubunto-21.tx.rr.com (2603-8081-140c-1a00-48b3-0edc-a395-cab0.res6.spectrum.com. [2603:8081:140c:1a00:48b3:edc:a395:cab0]) by smtp.gmail.com with ESMTPSA id a23sm373661otp.44.2021.09.29.21.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 21:27:16 -0700 (PDT) From: Bob Pearson To: jgg@nvidia.com, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org Cc: Bob Pearson Subject: [PATCH for-next v4 6/6] RDMA/rxe: Convert kernel UD post send to use ah_num Date: Wed, 29 Sep 2021 23:26:04 -0500 Message-Id: <20210930042603.4318-7-rpearsonhpe@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210930042603.4318-1-rpearsonhpe@gmail.com> References: <20210930042603.4318-1-rpearsonhpe@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Modify ib_post_send for kernel UD sends to put the AH index into the WQE instead of the address vector. Signed-off-by: Bob Pearson --- drivers/infiniband/sw/rxe/rxe_verbs.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 8854ace63acd..b808777e2221 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -537,8 +537,11 @@ static void init_send_wr(struct rxe_qp *qp, struct rxe_send_wr *wr, if (qp_type(qp) == IB_QPT_UD || qp_type(qp) == IB_QPT_SMI || qp_type(qp) == IB_QPT_GSI) { + struct ib_ah *ibah = ud_wr(ibwr)->ah; + wr->wr.ud.remote_qpn = ud_wr(ibwr)->remote_qpn; wr->wr.ud.remote_qkey = ud_wr(ibwr)->remote_qkey; + wr->wr.ud.ah_num = to_rah(ibah)->ah_num; if (qp_type(qp) == IB_QPT_GSI) wr->wr.ud.pkey_index = ud_wr(ibwr)->pkey_index; if (wr->opcode == IB_WR_SEND_WITH_IMM) @@ -610,12 +613,6 @@ static void init_send_wqe(struct rxe_qp *qp, const struct ib_send_wr *ibwr, return; } - if (qp_type(qp) == IB_QPT_UD || - qp_type(qp) == IB_QPT_SMI || - qp_type(qp) == IB_QPT_GSI) - memcpy(&wqe->wr.wr.ud.av, &to_rah(ud_wr(ibwr)->ah)->av, - sizeof(struct rxe_av)); - if (unlikely(ibwr->send_flags & IB_SEND_INLINE)) copy_inline_data_to_wqe(wqe, ibwr); else