From patchwork Tue Dec 28 08:07:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700234 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 961ACC4332F for ; Tue, 28 Dec 2021 08:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235570AbhL1IC3 (ORCPT ); Tue, 28 Dec 2021 03:02:29 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:10151 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235528AbhL1ICZ (ORCPT ); Tue, 28 Dec 2021 03:02:25 -0500 IronPort-Data: A9a23:Wxc+CqrtKQLbHi8b6Xft34jZtSxeBmJGZBIvgKrLsJaIsI5as4F+vmQdWGiEOKzYYmakLt91bNyx9UtVupGDzNZlQVRl+3s9QiMRo6IpJ/zDcB6oYHn6wu4v7a5fx5xHLIGGdajYd1eEzvuWGuWn/SkUOZ2gHOKmUbeeYHApHGeIdQ964f5ds79g6mJXqYjha++9kYuaT/z3YDdJ6RYtWo4nw/7rRCdUgRjHkGhwUmrSyhx8lAS2e3E9VPrzLEwqRpfyatE88uWSH44vwFwll1418SvBCvv9+lr6WkYMBLDPPwmSkWcQUK+n6vRAjnVqlP9la7xHMgEK49mKt4kZJNFlsZ2iSQYrP6TKsOoAURhECDw4NqpDkFPCCSHm4JLOkBGfKhMAxN0rVinaJ7Yw4P56CHtV8voYMD0lYRWKhubwy7W+IsF+l8YxPcuxZNtHkn5lxDDdS/0hRPjrR6TD49BH0TEoi8ZBNfbDbtUUaHxkaxGoSxlOJVoWCJs4k8+om3Dgfjweo1WQzYIz7m/V5A98yr7gNJzSYNPibcxVl1yfoGbu+Xr4DhATcteYzFKt93iogeTPtSXlWY4THfuz8fsCqFmSwHEDTR4bT122pdGnhUOkHdFSMUoZ/mwpt6da3EiqSMTtGge0pXesoBERQZxTHvc85QXLzbDbiy6bCWcsXD9McNFgv8ZeeNCA/jdlhPuwXXo27uLTEinbq9+pQfqJEXB9BQc/ieUsFFBtDwHfnbwO IronPort-HdrOrdr: A9a23:ddz9L65rcqxeHRTzVwPXwCzXdLJyesId70hD6qhwISY6TiX+rbHJoB17726MtN9/YhEdcLy7VJVoBEmskKKdgrNhWotKPjOW21dARbsKheCJrgEIWReOktK1vp0AT0ERMrLN5CBB/KTHCReDYqsd6ejC4Ka1nv3f0nsoaQlrbptr5wB/Bh3zKDwMeCB2QYo+CIGH5tdK4x6peXEsZMy9AXUfG8fZodmjruOdXTc2Qw4g9BKVjS6lrJrzEx2j1B8YVD9VhZcOmFK16zDE2g== X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657411" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD05.g08.fujitsu.local (unknown [10.167.33.204]) by cn.fujitsu.com (Postfix) with ESMTP id 193974D15A21; Tue, 28 Dec 2021 16:01:47 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD05.g08.fujitsu.local (10.167.33.204) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:48 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:44 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 01/10] RDMA: mr: Introduce is_pmem Date: Tue, 28 Dec 2021 16:07:08 +0800 Message-ID: <20211228080717.10666-2-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 193974D15A21.AE5CF X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We can use it to indicate whether the registering mr is associated with a pmem/nvdimm or not. Currently, we only assign it in rxe driver, for other device/drivers, they should implement it if needed. RDMA FLUSH will support the persistence feature for a pmem/nvdimm. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_mr.c | 47 ++++++++++++++++++++++++++++++ include/rdma/ib_verbs.h | 1 + 2 files changed, 48 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 7c4cd19a9db2..bcd5e7afa475 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -162,6 +162,50 @@ void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr) mr->type = IB_MR_TYPE_DMA; } +// XXX: the logic is similar with mm/memory-failure.c +static bool page_in_dev_pagemap(struct page *page) +{ + unsigned long pfn; + struct page *p; + struct dev_pagemap *pgmap = NULL; + + pfn = page_to_pfn(page); + if (!pfn) { + pr_err("no such pfn for page %p\n", page); + return false; + } + + p = pfn_to_online_page(pfn); + if (!p) { + if (pfn_valid(pfn)) { + pgmap = get_dev_pagemap(pfn, NULL); + if (pgmap) + put_dev_pagemap(pgmap); + } + } + + return !!pgmap; +} + +static bool iova_in_pmem(struct rxe_mr *mr, u64 iova, int length) +{ + struct page *page = NULL; + char *vaddr = iova_to_vaddr(mr, iova, length); + + if (!vaddr) { + pr_err("not a valid iova %llu\n", iova); + return false; + } + + page = virt_to_page(vaddr); + if (!page) { + pr_err("no such page for vaddr %p\n", vaddr); + return false; + } + + return page_in_dev_pagemap(page); +} + int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int access, struct rxe_mr *mr) { @@ -236,6 +280,9 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, set->va = start; set->offset = ib_umem_offset(umem); + // iova_in_pmem must be called after set is updated + mr->ibmr.is_pmem = iova_in_pmem(mr, iova, length); + return 0; err_release_umem: diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 6e9ad656ecb7..822ebb3425dc 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1807,6 +1807,7 @@ struct ib_mr { unsigned int page_size; enum ib_mr_type type; bool need_inval; + bool is_pmem; union { struct ib_uobject *uobject; /* user */ struct list_head qp_entry; /* FR */ From patchwork Tue Dec 28 08:07:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C480C433EF for ; Tue, 28 Dec 2021 08:02:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235479AbhL1ICT (ORCPT ); Tue, 28 Dec 2021 03:02:19 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:10151 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235464AbhL1ICS (ORCPT ); Tue, 28 Dec 2021 03:02:18 -0500 IronPort-Data: A9a23:dK9/rKwvNA4xbTLWkDh6t+dpxyrEfRIJ4+MujC/XYbTApDor1mMDmGEZWGiPPqyNYGuhKItzat628UwA6JPQm4RnHQtv/xmBbVoQ95OdWo7xwmQcns+qBpSaChohtq3yU/GYRCwPZiKa9kfF3oTJ9yEmjPjSHOqkUYYoBwgqLeNaYHZ44f5cs75h6mJYqYDR7zKl4bsekeWGULOW82Ic3lYv1k62gEgHUMIeF98vlgdWifhj5DcynpSOZX4VDfnZw3DQGuG4EgMmLtsvwo1V/kuBl/ssItij1LjmcEwWWaOUNg+L4pZUc/H6xEEc+WppieBmXBYfQR4/ZzGhm9FjyNRPtJW2YQk0PKzQg/lbWB5de817FfQfqeGafiTn4KR/yGWDKRMA2c5GFlk7NJcD/eB3GWxm+vkRKTRLZReG78qk0bCpW+s23px7BMbuNYIb/HpnyFnxCfcvR5/cTqPS6NlX9Dctj99DHLDVYM9xQT5ucxnBYxRJNX8XFZshkebujX76GxVcpVWTjak6+W7eyEp2yreFGNPVc8aNQ8F9mFiZqmPPuW/+B3kyMdabzjGF2nSyh+POlGXwX4d6PLm58ON6xV6e3GoeDDUIWlah5/q0kEizX5RYMUN80i4vq7UisVanS9DVQRK1ujiHswQaVt4WFPc1gCmPxaX88QeUHmVCRTcpVTCMnKfaXhRzjhnQwYyvXmcp7dWopbum3u/8hVuP1eI9dDRqifc4cDY4 IronPort-HdrOrdr: A9a23:I8ttva4u5M/IRmAdTAPXwCzXdLJyesId70hD6qhwISY6TiX+rbHJoB17726MtN9/YhEdcLy7VJVoBEmskKKdgrNhWotKPjOW21dARbsKheCJrgEIWReOktK1vp0AT0ERMrLN5CBB/KTHCReDYqsd6ejC4Ka1nv3f0nsoaQlrbptr5wB/Bh3zKDwMeCB2QYo+CIGH5tdK4x6peXEsZMy9AXUfG8fZodmjruOdXTc2Qw4g9BKVjS6lrJrzEx2j1B8YVD9VhZcOmFK16zDE2g== X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657409" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD06.g08.fujitsu.local (unknown [10.167.33.206]) by cn.fujitsu.com (Postfix) with ESMTP id 06CDB4D15A22; Tue, 28 Dec 2021 16:01:48 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD06.g08.fujitsu.local (10.167.33.206) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:46 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:45 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 02/10] RDMA: Allow registering MR with flush access flags Date: Tue, 28 Dec 2021 16:07:09 +0800 Message-ID: <20211228080717.10666-3-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 06CDB4D15A22.AF473 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Users can use ibv_reg_mr(3) to register flush access flags. Signed-off-by: Li Zhijian --- include/rdma/ib_verbs.h | 6 +++++- include/uapi/rdma/ib_user_ioctl_verbs.h | 2 ++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 822ebb3425dc..f04d66539879 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1444,10 +1444,14 @@ enum ib_access_flags { IB_ACCESS_ON_DEMAND = IB_UVERBS_ACCESS_ON_DEMAND, IB_ACCESS_HUGETLB = IB_UVERBS_ACCESS_HUGETLB, IB_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_RELAXED_ORDERING, + IB_ACCESS_FLUSH_GLOBAL_VISIBILITY = IB_UVERBS_ACCESS_FLUSH_GLOBAL_VISIBLITY, + IB_ACCESS_FLUSH_PERSISTENT = IB_UVERBS_ACCESS_FLUSH_PERSISTENT, + IB_ACCESS_FLUSH = IB_ACCESS_FLUSH_GLOBAL_VISIBILITY | + IB_ACCESS_FLUSH_PERSISTENT, IB_ACCESS_OPTIONAL = IB_UVERBS_ACCESS_OPTIONAL_RANGE, IB_ACCESS_SUPPORTED = - ((IB_ACCESS_HUGETLB << 1) - 1) | IB_ACCESS_OPTIONAL, + ((IB_ACCESS_FLUSH_PERSISTENT << 1) - 1) | IB_ACCESS_OPTIONAL, }; /* diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h index 3072e5d6b692..4763d3e3ea16 100644 --- a/include/uapi/rdma/ib_user_ioctl_verbs.h +++ b/include/uapi/rdma/ib_user_ioctl_verbs.h @@ -57,6 +57,8 @@ enum ib_uverbs_access_flags { IB_UVERBS_ACCESS_ZERO_BASED = 1 << 5, IB_UVERBS_ACCESS_ON_DEMAND = 1 << 6, IB_UVERBS_ACCESS_HUGETLB = 1 << 7, + IB_UVERBS_ACCESS_FLUSH_GLOBAL_VISIBLITY = 1 << 8, + IB_UVERBS_ACCESS_FLUSH_PERSISTENT = 1 << 9, IB_UVERBS_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_OPTIONAL_FIRST, IB_UVERBS_ACCESS_OPTIONAL_RANGE = From patchwork Tue Dec 28 08:07:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 048A0C4332F for ; Tue, 28 Dec 2021 08:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235552AbhL1IC0 (ORCPT ); Tue, 28 Dec 2021 03:02:26 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47591 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235464AbhL1ICW (ORCPT ); Tue, 28 Dec 2021 03:02:22 -0500 IronPort-Data: A9a23:/yqbtaoh+6phOnsktzIlST90qlFeBmJGZBIvgKrLsJaIsI5as4F+vjYZWzuHMvnfMGv9c4oia4nn8xxUvZ+DnNNhQVA5/C9nQiMRo6IpJ/zDcB6oYHn6wu4v7a5fx5xHLIGGdajYd1eEzvuWGuWn/SkUOZ2gHOKmUbeeYHApHGeIdQ964f5ds79g6mJXqYjha++9kYuaT/z3YDdJ6RYtWo4nw/7rRCdUgRjHkGhwUmrSyhx8lAS2e3E9VPrzLEwqRpfyatE88uWSH44vwFwll1418SvBCvv9+lr6WkYMBLDPPwmSkWcQUK+n6vRAjnVqlP9la7xHMgEK49mKt4kZJNFlsZ2iSQYrP6TKsOoAURhECDw4NqpDkFPCCSHm4JLOkBGfKRMAxN0rVinaJ7Yw4P56CHtV8voYMD0lYRWKhubwy7W+IsF+l8YxPcuxZNtHkn5lxDDdS/0hRPjrR6TD49BH0TEoi8ZBNfbDbtUUaHxkaxGoSxlOJVoWCJs4k8+om3Dgfjweo1WQzYIz7m/V5A98yr7gNJzSYNPibcxVl1yfoGbu+Xr4DhATcteYzFKt93iogeTPtSXlWY4THfuz8fsCqFmSwHEDTR4bT122pdGnhUOkHdFSMUoZ/mwpt6da3EiqSMTtGge0pXesoBERQZxTHvc85QXLzbDbiy6bCWcsXD9McNFgv8ZeeNCA/jdlhPuwXXo27uLTEinbq9+pQfqJEXB9BQc/ieUsFFdtDwHfnbwO IronPort-HdrOrdr: A9a23:J1d0dKCLHqZc767lHelI55DYdb4zR+YMi2TC1yhKJyC9Ffbo8fxG/c5rrCMc5wxwZJhNo7y90ey7MBbhHP1OkO4s1NWZLWrbUQKTRekIh+bfKn/baknDH4VmtJuIHZIQNDSJNykZsS/l2njEL/8QhMmA7LuzhfrTi1NkTQRRYalm6AtjYzzraXFedU1XA4YjDpqA6o5irzqkQ34eacO2HT0rRO7Gzue77q7OUFoXAQI98gmSgXeN4L7+KRKR2RATSHdu7N4ZgBD4rzA= X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657412" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id B95204D15A23; Tue, 28 Dec 2021 16:01:48 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:48 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:46 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 03/10] RDMA/rxe: Allow registering FLUSH flags for supported device only Date: Tue, 28 Dec 2021 16:07:10 +0800 Message-ID: <20211228080717.10666-4-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: B95204D15A23.AE83B X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Device should enable IB_DEVICE_RDMA_FLUSH capability if it want to support RDMA FLUSH. Signed-off-by: Li Zhijian --- include/rdma/ib_verbs.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index f04d66539879..51d58b641201 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -291,6 +291,7 @@ enum ib_device_cap_flags { /* The device supports padding incoming writes to cacheline. */ IB_DEVICE_PCI_WRITE_END_PADDING = (1ULL << 36), IB_DEVICE_ALLOW_USER_UNREG = (1ULL << 37), + IB_DEVICE_RDMA_FLUSH = (1ULL << 38), }; enum ib_atomic_cap { @@ -4319,6 +4320,10 @@ static inline int ib_check_mr_access(struct ib_device *ib_dev, if (flags & IB_ACCESS_ON_DEMAND && !(ib_dev->attrs.device_cap_flags & IB_DEVICE_ON_DEMAND_PAGING)) return -EINVAL; + + if (flags & IB_ACCESS_FLUSH && + !(ib_dev->attrs.device_cap_flags & IB_DEVICE_RDMA_FLUSH)) + return -EINVAL; return 0; } From patchwork Tue Dec 28 08:07:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AD68C433FE for ; Tue, 28 Dec 2021 08:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235523AbhL1ICX (ORCPT ); Tue, 28 Dec 2021 03:02:23 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:10151 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235494AbhL1ICV (ORCPT ); Tue, 28 Dec 2021 03:02:21 -0500 IronPort-Data: A9a23:uzrgya4WU07Bpw2qGEIyHwxRtGbGchMFZxGqfqrLsXjdYENS1zMAyTAXUDiGMvaJNjb1c4h3YYW19xwEvsPTzoRjTVY5pCpnJ55ogZCbXIzGdC8cHM8zwvXrFRsht4NHAjX5BJhcokT0+1H9YtANkVEmjfvRH+CmWLadUsxMbVQMpBkJ2EsLd9ER0tYAbeiRW2thiPuqyyHtEAbNNw1cbgr435m+RCZH55wejt+3UmsWPpintHeG/5Uc4Ql2yauZdxMUSaEMdgK2qnqq8V23wo/Z109F5tKNmbC9fFAIQ6LJIE6FjX8+t6qK20AE/3JtlP1gcqd0hUR/0l1lm/h1ycdNtJ6xQAEBMLDOmfgGTl9TFCQW0ahuoeaWfyXg7ZXOp6HBWz62qxl0N2ksJYAR4P1wB2FW+NQXLTkMalaIgOfe6KCqSPt9hJ57dJHDM4YWu3UmxjbcZd4iQJnFTLrH48dV2jgYht1HAvvfIcEebFJHaBXfbhxPM1EYIJIjlfilhz/0dDgwgFaYoqwf4GXJygF1lr/3P7L9c9yDX8JTmm6euGPK/m2/CRYfXPSawz6C+XKEgPHOkSLyHokVEdWQ8v9snU3WxWEJDhASfUW0rOP/iUOkXd9bbUsO9UIGqak06VzuVNX4VjWmr3Oe+B0RQdxdF6s98g7l4qjb7C6LB2UcQ3hKYbQbWGUeLdAx/gbR2Yq3Wno06/vIIU9xP4y89VuaURX550dbNUfolTc43uQ= IronPort-HdrOrdr: A9a23:J6kF2aOdY4ip78BcTiWjsMiBIKoaSvp037Eqv3oedfUzSL3/qynOpoVj6faaslYssR0b9exofZPwJE80lqQFhrX5X43SPzUO0VHAROoJgLcKgQeQfxEWndQ96U4PScdD4aXLfDpHZNjBkXSFOudl0N+a67qpmOub639sSDthY6Zm4xwRMHfhLmRGABlBGYEiFIeRou5Opz+bc3wRacihQlYfWeyrnaywqLvWJQ4BGwU86BSDyReh6LvBGRCe2RsEFxNjqI1SiVT4rw== X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657415" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id A464E4D15A24; Tue, 28 Dec 2021 16:01:49 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:49 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:47 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 04/10] RDMA/rxe: Enable IB_DEVICE_RDMA_FLUSH for rxe device Date: Tue, 28 Dec 2021 16:07:11 +0800 Message-ID: <20211228080717.10666-5-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: A464E4D15A24.AF05D X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We will enable the RDMA FLUSH on rxe device later. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_param.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index 918270e34a35..5c140f311aa2 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -53,7 +53,8 @@ enum rxe_device_param { | IB_DEVICE_ALLOW_USER_UNREG | IB_DEVICE_MEM_WINDOW | IB_DEVICE_MEM_WINDOW_TYPE_2A - | IB_DEVICE_MEM_WINDOW_TYPE_2B, + | IB_DEVICE_MEM_WINDOW_TYPE_2B + | IB_DEVICE_RDMA_FLUSH, RXE_MAX_SGE = 32, RXE_MAX_WQE_SIZE = sizeof(struct rxe_send_wqe) + sizeof(struct ib_sge) * RXE_MAX_SGE, From patchwork Tue Dec 28 08:07:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B5DAC433FE for ; Tue, 28 Dec 2021 08:02:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235502AbhL1ICW (ORCPT ); Tue, 28 Dec 2021 03:02:22 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47591 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235478AbhL1ICT (ORCPT ); Tue, 28 Dec 2021 03:02:19 -0500 IronPort-Data: A9a23:hCRgkqJPaLKhLizxFE+Rj5QlxSXFcZb7ZxGrkP8bfHCxgjMh0DRSzWEdDWmOOK2DYDSnf4h+a4y0oB9SusLXx4NqS1BcGVNFFSwT8ZWfbTi6wuYcBwvLd4ubChsPA/w2MrEsF+hpCC+MzvuRGuK59yAlj/vYHuOU5NPsYUideyc1EU/Ntjozw4bVsqYw6TSIK1vlVeHa+qUzC3f5s9JACV/43orYwP9ZUFsejxtD1rA2TagjUFYzDBD5BrpHTU26ByOQroW5goeHq+j/ILGRpgs1/j8mDJWrj7T6blYXBLXVOGBiiFIPA+773EcE/Xd0j87XN9JFAatTozGIjdBwytREs7S+VAUoIrbR3u8aVnG0FgknZPwYpeCbeSfXXcu7iheun2HX6+92AUgsJooe+v56KW5L/P0cbjsKa3irlfO00qO5ELE03uwsKcDqOMUUvXQI5TXYBPApXp3FW6jM6vdYwT4vi8EIFvHbD+IdaCVibRTJZRJnOkoeF58/2uyvgxHXdzBfrnqWqLAx7myVyxZ+uJDkMNPPfdqObcNLn0qZryTN+GGRKhQQMNuUyRKD7HOgh+aJliT+MKoWFbul5rtpjUeVy2g7FhIbTx24rOO/h0r4XMhQQ2QQ+ywzve0o+EmiZsfyUgf+o3OeuBMYHd1KHIUS6g6C4rjV7h6UQGMNJgOtwvROWNQeHGRsjwHW2YiyQ2EHjVFcclrFnp/8kN94EXFORYPaWRI5cA== IronPort-HdrOrdr: A9a23:CvgY86xaWGnT48rM8r8SKrPwyL1zdoMgy1knxilNoRw8SKKlfqeV7ZImPH7P+U8ssR4b+exoVJPtfZqYz+8R3WBzB8bEYOCFghrKEGgK1+KLqFeMJ8S9zJ846U4KSclD4bPLYmSS9fyKgjVQDexQveWvweS5g/vE1XdxQUVPY6Fk1Q1wDQGWCSRNNXJ7LKt8BJyB/dBGujblXXwWa/6wDn4DU/OGiMbMkPvdEGQ7Li9i+A+Tlimp9bK/NxCZ2y0VWzRJzaxn0UWtqX2A2pme X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657407" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD05.g08.fujitsu.local (unknown [10.167.33.204]) by cn.fujitsu.com (Postfix) with ESMTP id 70B9C4D15A26; Tue, 28 Dec 2021 16:01:50 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD05.g08.fujitsu.local (10.167.33.204) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:51 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:47 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 05/10] RDMA/rxe: Allow registering persistent flag for pmem MR only Date: Tue, 28 Dec 2021 16:07:12 +0800 Message-ID: <20211228080717.10666-6-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 70B9C4D15A26.AF226 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Memory region should support 2 placement types: IB_ACCESS_FLUSH_PERSISTENT and IB_ACCESS_FLUSH_GLOBAL_VISIBILITY, and only pmem/nvdimm has ability to persist data(IB_ACCESS_FLUSH_PERSISTENT). Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index bcd5e7afa475..21616d058f29 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -206,6 +206,11 @@ static bool iova_in_pmem(struct rxe_mr *mr, u64 iova, int length) return page_in_dev_pagemap(page); } +static bool ib_check_flush_access_flags(struct ib_mr *mr, u32 flags) +{ + return mr->is_pmem || !(flags & IB_ACCESS_FLUSH_PERSISTENT); +} + int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int access, struct rxe_mr *mr) { @@ -282,6 +287,13 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, // iova_in_pmem must be called after set is updated mr->ibmr.is_pmem = iova_in_pmem(mr, iova, length); + if (!ib_check_flush_access_flags(&mr->ibmr, access)) { + pr_err("Cannot set IB_ACCESS_FLUSH_PERSISTENT for non-pmem memory\n"); + mr->state = RXE_MR_STATE_INVALID; + mr->umem = NULL; + err = -EINVAL; + goto err_release_umem; + } return 0; From patchwork Tue Dec 28 08:07:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88411C433EF for ; Tue, 28 Dec 2021 08:02:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235515AbhL1ICY (ORCPT ); Tue, 28 Dec 2021 03:02:24 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47599 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235495AbhL1ICW (ORCPT ); Tue, 28 Dec 2021 03:02:22 -0500 IronPort-Data: A9a23:iGnAeqAlo2CqFRVW/3fiw5YqxClBgxIJ4g17XOLfUFO9gjgj0TYOymsaXmuEP/vYYmCmfo9/b9yx8UhQu5WAx9UxeLYW3SszFioV86IpJjg4wn/YZnrUdouaJK5ex512huLocYZkHhcwmj/3auK49CMmhfnRLlbBILWs1h5ZFFYMpBgJ2UoLd94R2uaEsPDha++/kYqaT/73ZDdJ7wVJ3lc8sMpvnv/AUMPa41v0tnRmDRxCUcS3e3M9VPrzLonpR5f0rxU9IwK0ewrD5OnREmLx9BFrBM6nk6rgbwsBRbu60Qqm0yIQAvb9xEMZ4HFaPqUTbZLwbW9TiieJntJwwdNlu4GySBsyI+vHn+F1vxxwSngvY/AZpOabSZS4mYnJp6HcSFP22/hnFloxO40A9854BGhP8boTLzVlRgKShfCnwujjErFEicEqLc2tN4Qa0llkzDjfAukrR4jORari5cJRwzoxwMtJGJ72Y8sGZDtvZRLPSx1SM0gaCdQ1m+LArn3ydDtwq1+Po6czpW/Jw2RZ2bjkKt3TfvSMW8RZn0/erWXDl0z8CBUdP9y3zySE/nOlwOTImEvTXIMUCa399fNwhlCX7nIcBQdQVlahp/S9zEmkVLp3L00S5zprt6Q3/WS1QdTnGR61uniJulgbQdU4O+815ymfy6fM7kCSDwA5opRpADA9nJZuA2V0iRnSxJW0bQGDeYa9ERq1nop4ZxvraUD59VM/WBI= IronPort-HdrOrdr: A9a23:dt8IOq9tVvkgwGevW2xuk+A8I+orL9Y04lQ7vn2YSXRuHPBw8Pre5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPMxbEpSWZueeMbEDt7eZ3OCnKadc/PC3tLCvmfzFz2pgCSVja6Rb5Q9/DQqBe3cGPzVuNN4oEoaG/Mpbq36FcXQTVM6yAX4IRKztvNvO/aiWGyIuNlo27hWUlzO05PrfGxic5B0XVDRC2vMD3AH+4nTE2pk= X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657414" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id 50FDC4D15A28; Tue, 28 Dec 2021 16:01:51 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:50 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:48 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 06/10] RDMA/rxe: Implement RC RDMA FLUSH service in requester side Date: Tue, 28 Dec 2021 16:07:13 +0800 Message-ID: <20211228080717.10666-7-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 50FDC4D15A28.A04D9 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org a RC FLUSH packet consists of: +----+------+------+ |BTH | FETH | RETH | +----+------+------+ oA19-2: FLUSH shall be single packet message and shall have no payload. oA19-5: FLUSH BTH shall hold the Opcode = 0x1C FLUSH Extended Transport Header(FETH) +-----+-----------+------------------------+----------------------+ |Bits | 31-6 | 5-4 | 3-0 | +-----+-----------+------------------------+----------------------+ | | Reserved | Selectivity Level(SEL) | Placement Type(PLT) | +-----+-----------+------------------------+----------------------+ Selectivity Level (SEL) – defines the memory region scope the FLUSH should apply on. Values are as follows: • b’00 - Memory Region Range: FLUSH applies for all preceding memory updates to the RETH range on this QP. All RETH fields shall be valid in this selectivity mode. RETH:DMALen field shall be between zero and (2 31 -1) bytes (inclusive). • b’01 - Memory Region: FLUSH applies for all preceding memory updates to RETH.R_key on this QP. RETH:DMALen and RETH:VA shall be ignored in this mode. • b'10 - Reserved. • b'11 - Reserved. Placement Type (PLT) – Defines the memory placement guarantee of this FLUSH. Multiple bits may be set in this field. Values are as follows: • Bit 0 if set to '1' indicated that the FLUSH should guarantee Global Visibility. • Bit 1 if set to '1' indicated that the FLUSH should guarantee Persistence. • Bits 3:2 are reserved Signed-off-by: Li Zhijian --- drivers/infiniband/core/uverbs_cmd.c | 16 ++++++++++++++++ drivers/infiniband/sw/rxe/rxe_hdr.h | 24 ++++++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_opcode.c | 13 +++++++++++++ drivers/infiniband/sw/rxe/rxe_opcode.h | 3 +++ drivers/infiniband/sw/rxe/rxe_req.c | 10 ++++++++++ include/rdma/ib_pack.h | 2 ++ include/rdma/ib_verbs.h | 9 +++++++++ include/uapi/rdma/ib_user_verbs.h | 7 +++++++ include/uapi/rdma/rdma_user_rxe.h | 6 ++++++ 9 files changed, 90 insertions(+) diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 6b6393176b3c..ed54fb0fd13d 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -2080,6 +2080,22 @@ static int ib_uverbs_post_send(struct uverbs_attr_bundle *attrs) rdma->rkey = user_wr->wr.rdma.rkey; next = &rdma->wr; + } else if (user_wr->opcode == IB_WR_RDMA_FLUSH) { + struct ib_flush_wr *flush; + + next_size = sizeof(*flush); + flush = alloc_wr(next_size, user_wr->num_sge); + if (!flush) { + ret = -ENOMEM; + goto out_put; + } + + flush->remote_addr = user_wr->wr.flush.remote_addr; + flush->rkey = user_wr->wr.flush.rkey; + flush->type = user_wr->wr.flush.type; + flush->level = user_wr->wr.flush.level; + + next = &flush->wr; } else if (user_wr->opcode == IB_WR_ATOMIC_CMP_AND_SWP || user_wr->opcode == IB_WR_ATOMIC_FETCH_AND_ADD) { struct ib_atomic_wr *atomic; diff --git a/drivers/infiniband/sw/rxe/rxe_hdr.h b/drivers/infiniband/sw/rxe/rxe_hdr.h index e432f9e37795..e37aa1944b18 100644 --- a/drivers/infiniband/sw/rxe/rxe_hdr.h +++ b/drivers/infiniband/sw/rxe/rxe_hdr.h @@ -607,6 +607,29 @@ static inline void reth_set_len(struct rxe_pkt_info *pkt, u32 len) rxe_opcode[pkt->opcode].offset[RXE_RETH], len); } +/* + * FLUSH Extended Transport Header(FETH) + * +-----+-----------+------------------------+----------------------+ + * |Bits | 31-6 | 5-4 | 3-0 | + * +-----+-----------+------------------------+----------------------+ + * | Reserved | Selectivity Level(SEL) | Placement Type(PLT) | + * +-----+-----------+------------------------+----------------------+ + */ +#define FETH_PLT_SHIFT 0UL +#define FETH_SEL_SHIFT 4UL +#define FETH_RESERVED_SHIFT 6UL +#define FETH_PLT_MASK ((1UL << FETH_SEL_SHIFT) - 1UL) +#define FETH_SEL_MASK (~FETH_PLT_MASK & ((1UL << FETH_RESERVED_SHIFT) - 1UL)) + +static inline void feth_init(struct rxe_pkt_info *pkt, u32 type, u32 level) +{ + u32 *p = (u32 *)(pkt->hdr + rxe_opcode[pkt->opcode].offset[RXE_FETH]); + u32 feth = ((level << FETH_SEL_SHIFT) & FETH_SEL_MASK) | + ((type << FETH_PLT_SHIFT) & FETH_PLT_MASK); + + *p = cpu_to_be32(feth); +} + /****************************************************************************** * Atomic Extended Transport Header ******************************************************************************/ @@ -910,6 +933,7 @@ enum rxe_hdr_length { RXE_ATMETH_BYTES = sizeof(struct rxe_atmeth), RXE_IETH_BYTES = sizeof(struct rxe_ieth), RXE_RDETH_BYTES = sizeof(struct rxe_rdeth), + RXE_FETH_BYTES = sizeof(u32), }; static inline size_t header_size(struct rxe_pkt_info *pkt) diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.c b/drivers/infiniband/sw/rxe/rxe_opcode.c index 3ef5a10a6efd..d61c8b354af4 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.c +++ b/drivers/infiniband/sw/rxe/rxe_opcode.c @@ -316,6 +316,19 @@ struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = { + RXE_AETH_BYTES, } }, + [IB_OPCODE_RC_RDMA_FLUSH] = { + .name = "IB_OPCODE_RC_RDMA_FLUSH", + .mask = RXE_FETH_MASK | RXE_RETH_MASK | RXE_FLUSH_MASK + | RXE_START_MASK | RXE_END_MASK | RXE_REQ_MASK, + .length = RXE_BTH_BYTES + RXE_FETH_BYTES + RXE_RETH_BYTES, + .offset = { + [RXE_BTH] = 0, + [RXE_FETH] = RXE_BTH_BYTES, + [RXE_RETH] = RXE_BTH_BYTES + RXE_FETH_BYTES, + [RXE_PAYLOAD] = RXE_BTH_BYTES + + RXE_FETH_BYTES + RXE_RETH_BYTES, + } + }, [IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE] = { .name = "IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE", .mask = RXE_AETH_MASK | RXE_ATMACK_MASK | RXE_ACK_MASK diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.h b/drivers/infiniband/sw/rxe/rxe_opcode.h index 8f9aaaf260f2..dbc2eca8a92c 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.h +++ b/drivers/infiniband/sw/rxe/rxe_opcode.h @@ -48,6 +48,7 @@ enum rxe_hdr_type { RXE_DETH, RXE_IMMDT, RXE_PAYLOAD, + RXE_FETH, NUM_HDR_TYPES }; @@ -63,6 +64,7 @@ enum rxe_hdr_mask { RXE_IETH_MASK = BIT(RXE_IETH), RXE_RDETH_MASK = BIT(RXE_RDETH), RXE_DETH_MASK = BIT(RXE_DETH), + RXE_FETH_MASK = BIT(RXE_FETH), RXE_PAYLOAD_MASK = BIT(RXE_PAYLOAD), RXE_REQ_MASK = BIT(NUM_HDR_TYPES + 0), @@ -80,6 +82,7 @@ enum rxe_hdr_mask { RXE_END_MASK = BIT(NUM_HDR_TYPES + 10), RXE_LOOPBACK_MASK = BIT(NUM_HDR_TYPES + 12), + RXE_FLUSH_MASK = BIT(NUM_HDR_TYPES + 13), RXE_READ_OR_ATOMIC_MASK = (RXE_READ_MASK | RXE_ATOMIC_MASK), RXE_WRITE_OR_SEND_MASK = (RXE_WRITE_MASK | RXE_SEND_MASK), diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index 5eb89052dd66..a3e9351873e2 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -220,6 +220,9 @@ static int next_opcode_rc(struct rxe_qp *qp, u32 opcode, int fits) IB_OPCODE_RC_SEND_ONLY_WITH_IMMEDIATE : IB_OPCODE_RC_SEND_FIRST; + case IB_WR_RDMA_FLUSH: + return IB_OPCODE_RC_RDMA_FLUSH; + case IB_WR_RDMA_READ: return IB_OPCODE_RC_RDMA_READ_REQUEST; @@ -418,6 +421,10 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, reth_set_len(pkt, wqe->dma.resid); } + /* Fill Flush Extension Transport Header */ + if (pkt->mask & RXE_FETH_MASK) + feth_init(pkt, ibwr->wr.flush.type, ibwr->wr.flush.level); + if (pkt->mask & RXE_IMMDT_MASK) immdt_set_imm(pkt, ibwr->ex.imm_data); @@ -477,6 +484,9 @@ static int finish_packet(struct rxe_qp *qp, struct rxe_send_wqe *wqe, memset(pad, 0, bth_pad(pkt)); } + } else if (pkt->mask & RXE_FLUSH_MASK) { + // oA19-2: shall have no payload. + wqe->dma.resid = 0; } return 0; diff --git a/include/rdma/ib_pack.h b/include/rdma/ib_pack.h index a9162f25beaf..d19edb502de6 100644 --- a/include/rdma/ib_pack.h +++ b/include/rdma/ib_pack.h @@ -84,6 +84,7 @@ enum { /* opcode 0x15 is reserved */ IB_OPCODE_SEND_LAST_WITH_INVALIDATE = 0x16, IB_OPCODE_SEND_ONLY_WITH_INVALIDATE = 0x17, + IB_OPCODE_RDMA_FLUSH = 0x1C, /* real constants follow -- see comment about above IB_OPCODE() macro for more details */ @@ -112,6 +113,7 @@ enum { IB_OPCODE(RC, FETCH_ADD), IB_OPCODE(RC, SEND_LAST_WITH_INVALIDATE), IB_OPCODE(RC, SEND_ONLY_WITH_INVALIDATE), + IB_OPCODE(RC, RDMA_FLUSH), /* UC */ IB_OPCODE(UC, SEND_FIRST), diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 51d58b641201..d336f2f8bb69 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1297,6 +1297,7 @@ struct ib_qp_attr { enum ib_wr_opcode { /* These are shared with userspace */ IB_WR_RDMA_WRITE = IB_UVERBS_WR_RDMA_WRITE, + IB_WR_RDMA_FLUSH = IB_UVERBS_WR_RDMA_FLUSH, IB_WR_RDMA_WRITE_WITH_IMM = IB_UVERBS_WR_RDMA_WRITE_WITH_IMM, IB_WR_SEND = IB_UVERBS_WR_SEND, IB_WR_SEND_WITH_IMM = IB_UVERBS_WR_SEND_WITH_IMM, @@ -1391,6 +1392,14 @@ struct ib_atomic_wr { u32 rkey; }; +struct ib_flush_wr { + struct ib_send_wr wr; + u64 remote_addr; + u32 rkey; + u32 type; + u32 level; +}; + static inline const struct ib_atomic_wr *atomic_wr(const struct ib_send_wr *wr) { return container_of(wr, struct ib_atomic_wr, wr); diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index 7ee73a0652f1..4b7093f58259 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -784,6 +784,7 @@ enum ib_uverbs_wr_opcode { IB_UVERBS_WR_RDMA_READ_WITH_INV = 11, IB_UVERBS_WR_MASKED_ATOMIC_CMP_AND_SWP = 12, IB_UVERBS_WR_MASKED_ATOMIC_FETCH_AND_ADD = 13, + IB_UVERBS_WR_RDMA_FLUSH = 14, /* Review enum ib_wr_opcode before modifying this */ }; @@ -797,6 +798,12 @@ struct ib_uverbs_send_wr { __u32 invalidate_rkey; } ex; union { + struct { + __aligned_u64 remote_addr; + __u32 rkey; + __u32 type; + __u32 level; + } flush; struct { __aligned_u64 remote_addr; __u32 rkey; diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index f09c5c9e3dd5..45f1b5b25414 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -82,6 +82,12 @@ struct rxe_send_wr { __u32 invalidate_rkey; } ex; union { + struct { + __aligned_u64 remote_addr; + __u32 rkey; + __u32 type; + __u32 level; + } flush; struct { __aligned_u64 remote_addr; __u32 rkey; From patchwork Tue Dec 28 08:07:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 348E1C4332F for ; Tue, 28 Dec 2021 08:02:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235511AbhL1ICX (ORCPT ); Tue, 28 Dec 2021 03:02:23 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47591 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235491AbhL1ICV (ORCPT ); Tue, 28 Dec 2021 03:02:21 -0500 IronPort-Data: A9a23:H/QsvK+TKEYZjE2zE+AIDrUD3n+TJUtcMsCJ2f8bfWQNrUpz0jMCzzBKWjiOaarca2GhLd51bt609xlT6p6DxoAyQFdlrnsFo1Bi85ScXYvDRqvT04J+FuWaFQQ/qZx2huDodKjYdVeB4Ef9WlTdhSMkj/jRHOCsULes1h1ZHmeIdg9w0HqPpMZp2uaEsfDha++8kYuaT//3YTdJ6BYoWo4g0J9vnTs01BjEVJz0iXRlDRxDlAe2e3D4l/vzL4npR5fzatE88uJX24/+IL+FEmPxp3/BC/uulPD1b08LXqXPewOJjxK6WYD72l4b+HN0if19aZLwam8O49mNt9Rw2tVMt525Ty8nI6/NhP8AFRJfFkmSOIUfoe6ZfCfj6Jf7I0ruNiGEL+9VJFsuMIQC4eFxAXlD3fMdITEJKBuEgoqe0qO5WPhu3Jx7dOHkOYoevjdryjSxJfInSJbMXKjM/dJe0x8wm8lREPeYbM0cARJrbQvNYh1GPFg/CI83g+qpwHL4dlVwrF+So4I07nLVwQg316LiWPLcetWQQsNRtkCGp27H9iLyBRRyHNmVzT2O8lqqmO7DnCq9U4UXfJW89/h3kBiQy3YVBRk+S1S2u7+6h1S4VtYZLFYbkgIqrK4v5AmxQtz0dwO3rWTCvRMGXddUVeog52mlyKXbyxSYC3AJCDVIAOHKHudeqScCjwfPxo22Q2c09uD9dJ5UzZ/MxRvaBMTfBTZSDcPccTY43g== IronPort-HdrOrdr: A9a23:6nAw1Kx30g05ZRlNMBjHKrPwyr1zdoMgy1knxilNoRw8SK2lfqeV7Y0mPH7P+U4ssR4b6LO90cW7Lk80sKQFhbX5Xo3SOjUO2lHYTr2KhLGKq1aLdkHDH6xmpMBdmsNFaOEYY2IVsS+D2njcLz8/+qj6zImYwffZ02x2TRxnL4Vp7wJCAA6dFUFsLTM2f6YRJd6N4NZdvTq8dTAyZsS/PHMMWO/OvJnlj5TjCCR2eiIP2U2fiy+y8r7mH1y91hcaaTlGxrAv6izkvmXCl96ej80= X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657413" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id 42F6E4D15A2A; Tue, 28 Dec 2021 16:01:52 +0800 (CST) Received: from G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:52 +0800 Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:51 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:49 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 07/10] RDMA/rxe: Set BTH's SE to zero for FLUSH packet Date: Tue, 28 Dec 2021 16:07:14 +0800 Message-ID: <20211228080717.10666-8-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 42F6E4D15A2A.A000C X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The SPEC said: oA19-6: FLUSH BTH header field solicited event (SE) indication shall be set to zero. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_req.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index a3e9351873e2..082c5f76f29b 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -401,7 +401,9 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, (pkt->mask & RXE_END_MASK) && ((pkt->mask & (RXE_SEND_MASK)) || (pkt->mask & (RXE_WRITE_MASK | RXE_IMMDT_MASK)) == - (RXE_WRITE_MASK | RXE_IMMDT_MASK)); + (RXE_WRITE_MASK | RXE_IMMDT_MASK)) && + /* oA19-6: always set SE to zero */ + !(pkt->mask & RXE_FETH_MASK); qp_num = (pkt->mask & RXE_DETH_MASK) ? ibwr->wr.ud.remote_qpn : qp->attr.dest_qp_num; From patchwork Tue Dec 28 08:07:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD5DBC4332F for ; Tue, 28 Dec 2021 08:02:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235537AbhL1ICZ (ORCPT ); Tue, 28 Dec 2021 03:02:25 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:10151 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235499AbhL1ICX (ORCPT ); Tue, 28 Dec 2021 03:02:23 -0500 IronPort-Data: A9a23:ercpgKPcO46kp9LvrR37lsFynXyQoLVcMsFnjC/WdQi5gG8qgjUHzWUaUTvSOPrZMDfwLY9/PdzioEoG65GHm99gGjLY11k3ESsS9pCt6fd1j6vIF3rLaJWFFSqL1u1GAjX7BJ1yHi+0SiuFaOC79CAmj/HQH9IQNcafUsxPbV49IMseoUI78wIJqtYAbemRW2thi/uryyHsEAPNNwpPD44hw/nrRCWDExjFkGhwUlQWPZintbJF/pUfJMp3yaqZdxMUTmTId9NWSdovzJnhlo/Y1xwrTN24kLfnaVBMSbnXVeSMoiMOHfH83V4Z/Wpvuko4HKN0hUN/jzSbn9FzydxLnZKtSwY1JbCKk+MYO/VdO3gnbPMbp+OefhBTtuTWlSUqaUDE2e1jBVstOosY4utfDmRH9PheIzcIBjiRluCk0bDhErE0rssmJcjveogYvxlIyTDQC/k5TJbbTqPFzd9F1Sg9h4ZFGvO2T84YdjdubB3GbDVPJ14IBZN4l+Ct7lH7fjpegFGYv6w65y7U1gMZ+LHtOcDSfNiiQ9tUkkeR4GnB+gzRBxseM9ef4Tyb836tj6nEmiaTcIYTEqCos/1nmluewkQNBxAME1i2u/+0jgi5Qd03A0gV/Dc+6LI+8UWDUNbwRVu7rWSCsxpaXMBfe8U45w6l2KvZ+wvfDWFsc9LrQLTKr+dvHXpziADPxIivWFRSXHSuYSr13t+pQfmaYHl9wbc+WBI5 IronPort-HdrOrdr: A9a23:O5RRV6xplBwui8LOMMP0KrPwyL1zdoMgy1knxilNoRw8SKKlfqeV7ZImPH7P+U8ssR4b+exoVJPtfZqYz+8R3WBzB8bEYOCFghrKEGgK1+KLqFeMJ8S9zJ846U4KSclD4bPLYmSS9fyKgjVQDexQveWvweS5g/vE1XdxQUVPY6Fk1Q1wDQGWCSRNNXJ7LKt8BJyB/dBGujblXXwWa/6wDn4DU/OGiMbMkPvdEGQ7Li9i+A+Tlimp9bK/NxCZ2y0VWzRJzaxn0UWtqX2A2pme X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657417" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXMBPEKD06.g08.fujitsu.local (unknown [10.167.33.206]) by cn.fujitsu.com (Postfix) with ESMTP id DC4A84D15A2C; Tue, 28 Dec 2021 16:01:52 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD06.g08.fujitsu.local (10.167.33.206) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:51 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:50 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 08/10] RDMA/rxe: Implement flush execution in responder side Date: Tue, 28 Dec 2021 16:07:15 +0800 Message-ID: <20211228080717.10666-9-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: DC4A84D15A2C.A2D95 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org In contrast to other opcodes, after a series of sanity checking, FLUSH opcode will do a Placement Type checking before it really do the FLUSH operation. Responder will also reply NAK "Remote Access Error" if it found a placement type violation. We will persist data via arch_wb_cache_pmem(), which could be architecture specific. After the execution, responder would reply a responded successfully by RDMA READ response of zero size. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_hdr.h | 28 ++++++ drivers/infiniband/sw/rxe/rxe_loc.h | 2 + drivers/infiniband/sw/rxe/rxe_mr.c | 4 +- drivers/infiniband/sw/rxe/rxe_resp.c | 131 ++++++++++++++++++++++++++- include/uapi/rdma/ib_user_verbs.h | 10 ++ 5 files changed, 169 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_hdr.h b/drivers/infiniband/sw/rxe/rxe_hdr.h index e37aa1944b18..cdfd393b8bd8 100644 --- a/drivers/infiniband/sw/rxe/rxe_hdr.h +++ b/drivers/infiniband/sw/rxe/rxe_hdr.h @@ -630,6 +630,34 @@ static inline void feth_init(struct rxe_pkt_info *pkt, u32 type, u32 level) *p = cpu_to_be32(feth); } +static inline u32 __feth_plt(void *arg) +{ + u32 *fethp = arg; + u32 feth = be32_to_cpu(*fethp); + + return (feth & FETH_PLT_MASK) >> FETH_PLT_SHIFT; +} + +static inline u32 __feth_sel(void *arg) +{ + u32 *fethp = arg; + u32 feth = be32_to_cpu(*fethp); + + return (feth & FETH_SEL_MASK) >> FETH_SEL_SHIFT; +} + +static inline u32 feth_plt(struct rxe_pkt_info *pkt) +{ + return __feth_plt(pkt->hdr + + rxe_opcode[pkt->opcode].offset[RXE_FETH]); +} + +static inline u32 feth_sel(struct rxe_pkt_info *pkt) +{ + return __feth_sel(pkt->hdr + + rxe_opcode[pkt->opcode].offset[RXE_FETH]); +} + /****************************************************************************** * Atomic Extended Transport Header ******************************************************************************/ diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index b1e174afb1d4..73c39ff11e28 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -80,6 +80,8 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, enum rxe_mr_copy_dir dir); int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma, void *addr, int length, enum rxe_mr_copy_dir dir); +void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, + size_t *offset_out); void *iova_to_vaddr(struct rxe_mr *mr, u64 iova, int length); struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, enum rxe_mr_lookup_type type); diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 21616d058f29..2cb530305392 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -325,8 +325,8 @@ int rxe_mr_init_fast(struct rxe_pd *pd, int max_pages, struct rxe_mr *mr) return err; } -static void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, - size_t *offset_out) +void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, + size_t *offset_out) { struct rxe_map_set *set = mr->cur_map_set; size_t offset = iova - set->iova + set->offset; diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index e8f435fa6e4d..6730336037d1 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -5,6 +5,7 @@ */ #include +#include #include "rxe.h" #include "rxe_loc.h" @@ -19,6 +20,7 @@ enum resp_states { RESPST_CHK_RESOURCE, RESPST_CHK_LENGTH, RESPST_CHK_RKEY, + RESPST_CHK_PLT, RESPST_EXECUTE, RESPST_READ_REPLY, RESPST_COMPLETE, @@ -35,6 +37,7 @@ enum resp_states { RESPST_ERR_TOO_MANY_RDMA_ATM_REQ, RESPST_ERR_RNR, RESPST_ERR_RKEY_VIOLATION, + RESPST_ERR_PLT_VIOLATION, RESPST_ERR_INVALIDATE_RKEY, RESPST_ERR_LENGTH, RESPST_ERR_CQ_OVERFLOW, @@ -53,6 +56,7 @@ static char *resp_state_name[] = { [RESPST_CHK_RESOURCE] = "CHK_RESOURCE", [RESPST_CHK_LENGTH] = "CHK_LENGTH", [RESPST_CHK_RKEY] = "CHK_RKEY", + [RESPST_CHK_PLT] = "CHK_PLACEMENT_TYPE", [RESPST_EXECUTE] = "EXECUTE", [RESPST_READ_REPLY] = "READ_REPLY", [RESPST_COMPLETE] = "COMPLETE", @@ -69,6 +73,7 @@ static char *resp_state_name[] = { [RESPST_ERR_TOO_MANY_RDMA_ATM_REQ] = "ERR_TOO_MANY_RDMA_ATM_REQ", [RESPST_ERR_RNR] = "ERR_RNR", [RESPST_ERR_RKEY_VIOLATION] = "ERR_RKEY_VIOLATION", + [RESPST_ERR_PLT_VIOLATION] = "ERR_PLACEMENT_TYPE_VIOLATION", [RESPST_ERR_INVALIDATE_RKEY] = "ERR_INVALIDATE_RKEY_VIOLATION", [RESPST_ERR_LENGTH] = "ERR_LENGTH", [RESPST_ERR_CQ_OVERFLOW] = "ERR_CQ_OVERFLOW", @@ -400,6 +405,30 @@ static enum resp_states check_length(struct rxe_qp *qp, } } +static enum resp_states check_placement_type(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + struct rxe_mr *mr = qp->resp.mr; + u32 plt = feth_plt(pkt); + + // no FLUSH access flag MR + if ((mr->access & IB_ACCESS_FLUSH) == 0) { + pr_err("This mr isn't registered any flush access permission\n"); + return RESPST_ERR_PLT_VIOLATION; + } + + if ((plt & IB_EXT_PLT_GLB_VIS && + !(mr->access & IB_ACCESS_FLUSH_GLOBAL_VISIBILITY)) || + (plt & IB_EXT_PLT_PERSIST && + !(mr->access & IB_ACCESS_FLUSH_PERSISTENT))) { + pr_err("Target MR don't allow this placement type, is_pmem: %d, register flag: %x, request flag: %x\n", + mr->ibmr.is_pmem, mr->access >> 8, plt); + return RESPST_ERR_PLT_VIOLATION; + } + + return RESPST_EXECUTE; +} + static enum resp_states check_rkey(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { @@ -413,7 +442,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, enum resp_states state; int access; - if (pkt->mask & RXE_READ_OR_WRITE_MASK) { + if (pkt->mask & (RXE_READ_OR_WRITE_MASK | RXE_FLUSH_MASK)) { if (pkt->mask & RXE_RETH_MASK) { qp->resp.va = reth_va(pkt); qp->resp.offset = 0; @@ -434,8 +463,10 @@ static enum resp_states check_rkey(struct rxe_qp *qp, } /* A zero-byte op is not required to set an addr or rkey. */ + /* RXE_FETH_MASK carraies zero-byte playload */ if ((pkt->mask & RXE_READ_OR_WRITE_MASK) && (pkt->mask & RXE_RETH_MASK) && + !(pkt->mask & RXE_FETH_MASK) && reth_len(pkt) == 0) { return RESPST_EXECUTE; } @@ -503,7 +534,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, WARN_ON_ONCE(qp->resp.mr); qp->resp.mr = mr; - return RESPST_EXECUTE; + return pkt->mask & RXE_FETH_MASK ? RESPST_CHK_PLT : RESPST_EXECUTE; err: if (mr) @@ -549,6 +580,91 @@ static enum resp_states write_data_in(struct rxe_qp *qp, return rc; } +static int nvdimm_flush_iova(struct rxe_mr *mr, u64 iova, int length) +{ + int err; + int bytes; + u8 *va; + struct rxe_map **map; + struct rxe_phys_buf *buf; + int m; + int i; + size_t offset; + + if (length == 0) + return 0; + + if (mr->type == IB_MR_TYPE_DMA) { + arch_wb_cache_pmem((void *)iova, length); + return 0; + } + + WARN_ON_ONCE(!mr->cur_map_set); + + err = mr_check_range(mr, iova, length); + if (err) { + err = -EFAULT; + goto err1; + } + + lookup_iova(mr, iova, &m, &i, &offset); + + map = mr->cur_map_set->map + m; + buf = map[0]->buf + i; + + while (length > 0) { + va = (u8 *)(uintptr_t)buf->addr + offset; + bytes = buf->size - offset; + + if (bytes > length) + bytes = length; + + arch_wb_cache_pmem(va, bytes); + + length -= bytes; + + offset = 0; + buf++; + i++; + + if (i == RXE_BUF_PER_MAP) { + i = 0; + map++; + buf = map[0]->buf; + } + } + + return 0; + +err1: + return err; +} + +static enum resp_states process_flush(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + u64 length = 0, start = qp->resp.va; + u32 sel = feth_sel(pkt); + u32 plt = feth_plt(pkt); + struct rxe_mr *mr = qp->resp.mr; + + if (sel == IB_EXT_SEL_MR_RANGE) + length = qp->resp.length; + else if (sel == IB_EXT_SEL_MR_WHOLE) + length = mr->cur_map_set->length; + + if (plt == IB_EXT_PLT_PERSIST) { + nvdimm_flush_iova(mr, start, length); + wmb(); // clwb follows by a sfence + } else if (plt == IB_EXT_PLT_GLB_VIS) + wmb(); // sfence is enough + + /* set RDMA READ response of zero */ + qp->resp.resid = 0; + + return RESPST_READ_REPLY; +} + /* Guarantee atomicity of atomic operations at the machine level. */ static DEFINE_SPINLOCK(atomic_ops_lock); @@ -801,6 +917,8 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt) err = process_atomic(qp, pkt); if (err) return err; + } else if (pkt->mask & RXE_FLUSH_MASK) { + return process_flush(qp, pkt); } else { /* Unreachable */ WARN_ON_ONCE(1); @@ -1059,7 +1177,7 @@ static enum resp_states duplicate_request(struct rxe_qp *qp, /* SEND. Ack again and cleanup. C9-105. */ send_ack(qp, pkt, AETH_ACK_UNLIMITED, prev_psn); return RESPST_CLEANUP; - } else if (pkt->mask & RXE_READ_MASK) { + } else if (pkt->mask & RXE_READ_MASK || pkt->mask & RXE_FLUSH_MASK) { struct resp_res *res; res = find_resource(qp, pkt->psn); @@ -1098,7 +1216,7 @@ static enum resp_states duplicate_request(struct rxe_qp *qp, /* Reset the resource, except length. */ res->read.va_org = iova; res->read.va = iova; - res->read.resid = resid; + res->read.resid = pkt->mask & RXE_FLUSH_MASK ? 0 : resid; /* Replay the RDMA read reply. */ qp->resp.res = res; @@ -1244,6 +1362,9 @@ int rxe_responder(void *arg) case RESPST_CHK_RKEY: state = check_rkey(qp, pkt); break; + case RESPST_CHK_PLT: + state = check_placement_type(qp, pkt); + break; case RESPST_EXECUTE: state = execute(qp, pkt); break; @@ -1298,6 +1419,8 @@ int rxe_responder(void *arg) break; case RESPST_ERR_RKEY_VIOLATION: + /* oA19-13 8 */ + case RESPST_ERR_PLT_VIOLATION: if (qp_type(qp) == IB_QPT_RC) { /* Class C */ do_class_ac_error(qp, AETH_NAK_REM_ACC_ERR, diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index 4b7093f58259..efa06f53c7c6 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -105,6 +105,16 @@ enum { IB_USER_VERBS_EX_CMD_MODIFY_CQ }; +enum ib_ext_placement_type { + IB_EXT_PLT_GLB_VIS = 1 << 0, + IB_EXT_PLT_PERSIST = 1 << 1, +}; + +enum ib_ext_selectivity_level { + IB_EXT_SEL_MR_RANGE = 0, + IB_EXT_SEL_MR_WHOLE, +}; + /* * Make sure that all structs defined in this file remain laid out so * that they pack the same way on 32-bit and 64-bit architectures (to From patchwork Tue Dec 28 08:07:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BBFAC433FE for ; Tue, 28 Dec 2021 08:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235599AbhL1IC2 (ORCPT ); Tue, 28 Dec 2021 03:02:28 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47591 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235519AbhL1ICY (ORCPT ); Tue, 28 Dec 2021 03:02:24 -0500 IronPort-Data: A9a23:tAOA9KB5HCWTOhVW/3fiw5YqxClBgxIJ4g17XOLfB1a6hj4q0WBWx2FJD2iFaK6OYDf3fN9+Pojn/E0E7MeAx9UxeLYW3SszFioV86IpJjg4wn/YZnrUdouaJK5ex512huLocYZkHhcwmj/3auK49CMmhfnRLlbBILWs1h5ZFFYMpBgJ2UoLd94R2uaEsPDha++/kYqaT/73ZDdJ7wVJ3lc8sMpvnv/AUMPa41v0tnRmDRxCUcS3e3M9VPrzLonpR5f0rxU9IwK0ewrD5OnREmLx9BFrBM6nk6rgbwsBRbu60Qqm0yIQAvb9xEMZ4HFaPqUTbZLwbW9TiieJntJwwdNlu4GySBsyI+vHn+F1vxxwSngvY/AZpOWbSZS4mYnJp6HcSFP22/hnFloxO40A9854BGhP8boTLzVlRgKShfCnwujjErFEicEqLc2tN4Qa0llkzDjfAukrR4jORari5cJRwzoxwMtJGJ72Y8sGZDtvZRLPSx1SM0gaCdQ1m+LArn3ydDtwq1+Po6czpW/Jw2RZ2bjkKt3TfvSMW8RZn0/erWXDl0z8CBUdP9y3zySE/nOlwOTImEvTXIMUCa399fNwhlCX7nIcBQdQVlahp/S9zEmkVLp3L00S5zprt6Q3/WS1QdTnGR61uniJulgbQdU4O+815ymfy6fM7kCSDwA5opRpADA9nJZuA2V0iRnSxJW0bQGDeYa9ERq1nop4ZxvoYkD59VM/WBI= IronPort-HdrOrdr: A9a23:C35bv6HTqGBBuhHKpLqEDMeALOsnbusQ8zAXP0AYc3Jom6uj5qaTdZUgpGbJYVkqOE3I9ertBEDEewK4yXcX2/h3AV7BZniEhILAFugLhuGO/9SjIVybygc378ZdmsZFZ+EYdWIK7/oS/jPIbuoI8Z2W9ryyn+fCwzNIRQFuUatp6AB0EW+gYzZLbTgDFZwkD4Cd+8YCgzKhfE4cZsO9CmJAcPPEo7Tw5ejbSC9DFxg68xOPkD/tzLb7FiKT1hAYXygK4ZpKyxm8rzDE X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657424" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:59 +0800 Received: from G08CNEXMBPEKD04.g08.fujitsu.local (unknown [10.167.33.201]) by cn.fujitsu.com (Postfix) with ESMTP id 0CD2A4D15A20; Tue, 28 Dec 2021 16:01:54 +0800 (CST) Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXMBPEKD04.g08.fujitsu.local (10.167.33.201) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:53 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:51 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 09/10] RDMA/rxe: Implement flush completion Date: Tue, 28 Dec 2021 16:07:16 +0800 Message-ID: <20211228080717.10666-10-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 0CD2A4D15A20.AE515 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce a new IB_UVERBS_WC_FLUSH code to tell userspace a FLUSH completion. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_comp.c | 4 +++- include/rdma/ib_verbs.h | 1 + include/uapi/rdma/ib_user_verbs.h | 1 + 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index f363fe3fa414..e5b9d07eba93 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -104,6 +104,7 @@ static enum ib_wc_opcode wr_to_wc_opcode(enum ib_wr_opcode opcode) case IB_WR_LOCAL_INV: return IB_WC_LOCAL_INV; case IB_WR_REG_MR: return IB_WC_REG_MR; case IB_WR_BIND_MW: return IB_WC_BIND_MW; + case IB_WR_RDMA_FLUSH: return IB_WC_RDMA_FLUSH; default: return 0xff; @@ -261,7 +262,8 @@ static inline enum comp_state check_ack(struct rxe_qp *qp, */ case IB_OPCODE_RC_RDMA_READ_RESPONSE_MIDDLE: if (wqe->wr.opcode != IB_WR_RDMA_READ && - wqe->wr.opcode != IB_WR_RDMA_READ_WITH_INV) { + wqe->wr.opcode != IB_WR_RDMA_READ_WITH_INV && + wqe->wr.opcode != IB_WR_RDMA_FLUSH) { wqe->status = IB_WC_FATAL_ERR; return COMPST_ERROR; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index d336f2f8bb69..d528f2c4eba0 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -963,6 +963,7 @@ const char *__attribute_const__ ib_wc_status_msg(enum ib_wc_status status); enum ib_wc_opcode { IB_WC_SEND = IB_UVERBS_WC_SEND, IB_WC_RDMA_WRITE = IB_UVERBS_WC_RDMA_WRITE, + IB_WC_RDMA_FLUSH = IB_UVERBS_WC_FLUSH, IB_WC_RDMA_READ = IB_UVERBS_WC_RDMA_READ, IB_WC_COMP_SWAP = IB_UVERBS_WC_COMP_SWAP, IB_WC_FETCH_ADD = IB_UVERBS_WC_FETCH_ADD, diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index efa06f53c7c6..b9d6b3ca5708 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -476,6 +476,7 @@ enum ib_uverbs_wc_opcode { IB_UVERBS_WC_BIND_MW = 5, IB_UVERBS_WC_LOCAL_INV = 6, IB_UVERBS_WC_TSO = 7, + IB_UVERBS_WC_FLUSH = 8, }; struct ib_uverbs_wc { From patchwork Tue Dec 28 08:07:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12700235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4753C433FE for ; Tue, 28 Dec 2021 08:02:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235580AbhL1IC2 (ORCPT ); Tue, 28 Dec 2021 03:02:28 -0500 Received: from mail.cn.fujitsu.com ([183.91.158.132]:47599 "EHLO heian.cn.fujitsu.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235496AbhL1ICY (ORCPT ); Tue, 28 Dec 2021 03:02:24 -0500 IronPort-Data: A9a23:zZyyYKB7kOjqQBVW/3fiw5YqxClBgxIJ4g17XOLfAVOw1GhxhDIPn2FOWGnVOv3fZGv9ctAiPY/n800Ou5SAx9UxeLYW3SszFioV86IpJjg4wn/YZnrUdouaJK5ex512huLocYZkHhcwmj/3auK49CMmhfnRLlbBILWs1h5ZFFYMpBgJ2UoLd94R2uaEsPDha++/kYqaT/73ZDdJ7wVJ3lc8sMpvnv/AUMPa41v0tnRmDRxCUcS3e3M9VPrzLonpR5f0rxU9IwK0ewrD5OnREmLx9BFrBM6nk6rgbwsBRbu60Qqm0yIQAvb9xEMZ4HFaPqUTbZLwbW9TiieJntJwwdNlu4GySBsyI+vHn+F1vxxwSngvY/AZpOWcSZS4mYnJp6HcSFP22/hnFloxO40A9854BGhP8boTLzVlRgKShfCnwujjErFEicEqLc2tN4Qa0llkzDjfAukrR4jORari5cJRwzoxwMtJGJ72Y8sGZDtvZRLPSx1SM0gaCdQ1m+LArn3ydDtwq1+Po6czpW/Jw2RZ2bjkKt3TfvSMW8RZn0/erWXDl0z8CBUdP9y3zySE/nOlwOTImEvTXIMUCa399fNwhlCX7nIcBQdQVlahp/S9zEmkVLp3L00S5zprt6Q3/WS1QdTnGR61uniJulgbQdU4O+815ymfy6fM7kCSDwA5opRpADA9nJZuA2V0iRnSxJW0bQGDeYa9ERq1nop4ZxvoZUD59VM/WBI= IronPort-HdrOrdr: A9a23:9zTfxK6D+bQqeArINwPXwC7XdLJyesId70hD6qhwISY1TiX+rbHWoB17726NtN9/YgBCpTntAsa9qDbnhPpICOoqTNGftWvdyQmVxehZhOOIqVCNJ8S9zJ876U4KSchD4bPLY2SS9fyKhTVQDexQvOWvweS5g/vE1XdxQUVPY6Fk1Q1wDQGWCSRNNXN7LKt8BJyB/dBGujblXXwWa/6wDn4DU/OGiMbMkPvdEGM7Li9i+A+Tlimp9bK/NxCZ2y0VWzRJzaxn0UWtqX2D2pme X-IronPort-AV: E=Sophos;i="5.88,241,1635177600"; d="scan'208";a="119657423" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 28 Dec 2021 16:01:58 +0800 Received: from G08CNEXMBPEKD06.g08.fujitsu.local (unknown [10.167.33.206]) by cn.fujitsu.com (Postfix) with ESMTP id 0CE9D4D15A21; Tue, 28 Dec 2021 16:01:55 +0800 (CST) Received: from G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) by G08CNEXMBPEKD06.g08.fujitsu.local (10.167.33.206) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:53 +0800 Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 28 Dec 2021 16:01:54 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 28 Dec 2021 16:01:52 +0800 From: Li Zhijian To: , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH rdma-next 10/10] RDMA/rxe: Add RD FLUSH service support Date: Tue, 28 Dec 2021 16:07:17 +0800 Message-ID: <20211228080717.10666-11-lizhijian@cn.fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> References: <20211228080717.10666-1-lizhijian@cn.fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 0CE9D4D15A21.AF2B7 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Although the SPEC said FLUSH is supported by RC/RD/XRC services, XRC has not been supported by the rxe. So XRC FLUSH will not be supported until rxe implements XRC service. Signed-off-by: Li Zhijian --- I have not setup a RD environment to test this protocol --- drivers/infiniband/sw/rxe/rxe_opcode.c | 20 ++++++++++++++++++++ include/rdma/ib_pack.h | 1 + 2 files changed, 21 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.c b/drivers/infiniband/sw/rxe/rxe_opcode.c index d61c8b354af4..0d5f555f5ec5 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.c +++ b/drivers/infiniband/sw/rxe/rxe_opcode.c @@ -919,6 +919,26 @@ struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = { + RXE_RDETH_BYTES, } }, + [IB_OPCODE_RD_RDMA_FLUSH] = { + .name = "IB_OPCODE_RD_RDMA_FLUSH", + .mask = RXE_RDETH_MASK | RXE_FETH_MASK | RXE_RETH_MASK + | RXE_FLUSH_MASK | RXE_START_MASK + | RXE_END_MASK | RXE_REQ_MASK, + .length = RXE_BTH_BYTES + RXE_FETH_BYTES + RXE_RETH_BYTES, + .offset = { + [RXE_BTH] = 0, + [RXE_RDETH] = RXE_BTH_BYTES, + [RXE_FETH] = RXE_BTH_BYTES + + RXE_RDETH_BYTES, + [RXE_RETH] = RXE_BTH_BYTES + + RXE_RDETH_BYTES + + RXE_FETH_BYTES, + [RXE_PAYLOAD] = RXE_BTH_BYTES + + RXE_RDETH_BYTES + + RXE_FETH_BYTES + + RXE_RETH_BYTES, + } + }, /* UD */ [IB_OPCODE_UD_SEND_ONLY] = { diff --git a/include/rdma/ib_pack.h b/include/rdma/ib_pack.h index d19edb502de6..40568a33ead8 100644 --- a/include/rdma/ib_pack.h +++ b/include/rdma/ib_pack.h @@ -151,6 +151,7 @@ enum { IB_OPCODE(RD, ATOMIC_ACKNOWLEDGE), IB_OPCODE(RD, COMPARE_SWAP), IB_OPCODE(RD, FETCH_ADD), + IB_OPCODE(RD, RDMA_FLUSH), /* UD */ IB_OPCODE(UD, SEND_ONLY),