From patchwork Fri Aug 5 07:46:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E0CEC00140 for ; Fri, 5 Aug 2022 07:40:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240258AbiHEHkJ (ORCPT ); Fri, 5 Aug 2022 03:40:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240227AbiHEHkG (ORCPT ); Fri, 5 Aug 2022 03:40:06 -0400 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6AEB74DE8; Fri, 5 Aug 2022 00:39:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685193; i=@fujitsu.com; bh=zWQzyxasTlVBGOYCkg6w45iiHbCTBaThAhUnvGcr8Tg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tH8ghDi6ZgsHuKN0FmGDlHsl5985rA4NaMApqAtP3VHptTh6nE7zw3NPiFBTI5qCX +v/c/41riHwClx0p6thkb/BYykYAAppKpteu9XGrHnjTStzj6xw0BuSFFlUqoEX0Gw a8+rizHVq8yXWxByI0V0n7F3YsNAetQWEk/xcYrfgZa1QjVC1k9Mb0pT0gBloxxY4p 1fJxeGaTkQr6+ixUbiLwWDNYtjQDxRX7Y2vABSt42h0n8QN7md8HetrmgmSryoTVew Fd4Hz8P3reObrni8Mp09zsiFCas4X/IekfqtmglVS/kaDn0OMT8LAuPzvoSMrCs/Dc SpWI33fjrmafQ== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLKsWRWlGSWpSXmKPExsViZ8ORqOtx8k2 SweaL5hZzv8tZTJ96gdFi5owTjBZTfi1ltri8aw6bxbNDvSwWH6YeYbb4MnUas8WpX6eYLP5e +sdmcf5YP7sDt8fOWXfZPRbvecnksWlVJ5tHb/M7No/LT64wenzeJOex9fNtlgD2KNbMvKT8i gTWjPbNc9gK3slXLH1xjr2B8bR0FyMXh5DAFkaJj+faWCCc5UwS8yf9YYRw9jNKzHlxESjDyc EmoCFxr+UmWEJEoJNR4lH/MTYQh1ngPJPEtE3/2EGqhAW8JO4+vMAKYrMIqEg8+LGUCcTmFXC UmPW5AaxGQkBBYsrD98wgNqeAk8Smb6sYQWwhoJqjLQtZIOoFJU7OfAJmMwtISBx88QKongOo V0liZnc8xJgKiVmz2pggbDWJq+c2MU9gFJyFpHsWku4FjEyrGG2TijLTM0pyEzNzdA0NDHQND U2BtJGuoamxXmKVbqJeaqluXn5RSYauoV5iebFeanGxXnFlbnJOil5easkmRmCspRSnzdzBuG XfL71DjJIcTEqivOeOv0kS4kvKT6nMSCzOiC8qzUktPsQow8GhJMEbdAIoJ1iUmp5akZaZA4x 7mLQEB4+SCO86kFbe4oLE3OLMdIjUKUZFKXFeI5CEAEgiozQPrg2Wai4xykoJ8zIyMDAI8RSk FuVmlqDKv2IU52BUEua9BzKFJzOvBG76K6DFTECLuf6/BllckoiQkmpgypWryX/2dLvM7gKlO yrcBR/dXh+rTN2QXCRYX3Z93sKif62SS2737fzbfuFGZdihs5HbfyhsuCfWkH9S8t1kIWGmaO 1PLRc3ejb2d3avffBfuujP9h8rPtjsatxxKfLB2Yz5Tuf59DWK+pZt5lU+E7rO28WvcN5je5/ yjNZzTQ9mZEnt3O+RpndlrYbM4ce7nxjmNzNuOj373Q+BN7eEfjy5PLEgbtGzZV07H+/a+7Y2 4XP388fG0zbMenD12anSFHfTurtvjEK9KgUmhHRXcB0XjXzpdk/pq0/rNp2lnYtWljIHPdlVY rZmZn/S0jcFaULGtx+2RCsbbfR9sL/ra27ndHnOR8r+l4rnLd6R76vEUpyRaKjFXFScCAAomn RwsAMAAA== X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-6.tower-728.messagelabs.com!1659685192!59229!1 X-Originating-IP: [62.60.8.97] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 6197 invoked from network); 5 Aug 2022 07:39:52 -0000 Received: from unknown (HELO n03ukasimr01.n03.fujitsu.local) (62.60.8.97) by server-6.tower-728.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:39:52 -0000 Received: from n03ukasimr01.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTP id 39FB11001A0; Fri, 5 Aug 2022 08:39:52 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTPS id 2C4B510019D; Fri, 5 Aug 2022 08:39:52 +0100 (BST) Received: from 4084fd6ad2a8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:39:46 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Aharon Landau , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 1/6] RDMA: Allow registering MR with flush access flags Date: Fri, 5 Aug 2022 07:46:14 +0000 Message-ID: <1659685579-2-2-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org It makes device/HCA support new FLUSH attributes/capabilities, and it also makes memory region support new FLUSH access flags. Users can use ibv_reg_mr(3) to register flush access flags. Only the access flags also supported by device's capabilities can be registered successfully. Once registered successfully, it means the MR is flushable. Similarly, A flushable MR should also have one or both of GLOBAL_VISIBILITY and PERSISTENT attributes/capabilities like device/HCA. CC: Dan Williams Signed-off-by: Li Zhijian --- V4: set is_pmem more simply V2: new scheme check is_pmem # Dan --- include/rdma/ib_verbs.h | 17 ++++++++++++++++- include/uapi/rdma/ib_user_ioctl_verbs.h | 2 ++ include/uapi/rdma/ib_user_verbs.h | 2 ++ 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 7c2f76f34f6f..aa174cdcdf5a 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -270,6 +270,10 @@ enum ib_device_cap_flags { /* The device supports padding incoming writes to cacheline. */ IB_DEVICE_PCI_WRITE_END_PADDING = IB_UVERBS_DEVICE_PCI_WRITE_END_PADDING, + /* Placement type attributes */ + IB_DEVICE_PLT_GLOBAL_VISIBILITY = + IB_UVERBS_DEVICE_PLT_GLOBAL_VISIBILITY, + IB_DEVICE_PLT_PERSISTENT = IB_UVERBS_DEVICE_PLT_PERSISTENT, }; enum ib_kernel_cap_flags { @@ -1458,10 +1462,14 @@ enum ib_access_flags { IB_ACCESS_ON_DEMAND = IB_UVERBS_ACCESS_ON_DEMAND, IB_ACCESS_HUGETLB = IB_UVERBS_ACCESS_HUGETLB, IB_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_RELAXED_ORDERING, + IB_ACCESS_FLUSH_GLOBAL_VISIBILITY = IB_UVERBS_ACCESS_FLUSH_GLOBAL_VISIBILITY, + IB_ACCESS_FLUSH_PERSISTENT = IB_UVERBS_ACCESS_FLUSH_PERSISTENT, + IB_ACCESS_FLUSHABLE = IB_ACCESS_FLUSH_GLOBAL_VISIBILITY | + IB_ACCESS_FLUSH_PERSISTENT, IB_ACCESS_OPTIONAL = IB_UVERBS_ACCESS_OPTIONAL_RANGE, IB_ACCESS_SUPPORTED = - ((IB_ACCESS_HUGETLB << 1) - 1) | IB_ACCESS_OPTIONAL, + ((IB_ACCESS_FLUSH_PERSISTENT << 1) - 1) | IB_ACCESS_OPTIONAL, }; /* @@ -4310,6 +4318,7 @@ int ib_dealloc_xrcd_user(struct ib_xrcd *xrcd, struct ib_udata *udata); static inline int ib_check_mr_access(struct ib_device *ib_dev, unsigned int flags) { + u64 device_cap = ib_dev->attrs.device_cap_flags; /* * Local write permission is required if remote write or * remote atomic permission is also requested. @@ -4324,6 +4333,12 @@ static inline int ib_check_mr_access(struct ib_device *ib_dev, if (flags & IB_ACCESS_ON_DEMAND && !(ib_dev->attrs.kernel_cap_flags & IBK_ON_DEMAND_PAGING)) return -EINVAL; + + if ((flags & IB_ACCESS_FLUSH_GLOBAL_VISIBILITY && + !(device_cap & IB_DEVICE_PLT_GLOBAL_VISIBILITY)) || + (flags & IB_ACCESS_FLUSH_PERSISTENT && + !(device_cap & IB_DEVICE_PLT_PERSISTENT))) + return -EINVAL; return 0; } diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h index 7dd56210226f..32d44ca328b9 100644 --- a/include/uapi/rdma/ib_user_ioctl_verbs.h +++ b/include/uapi/rdma/ib_user_ioctl_verbs.h @@ -57,6 +57,8 @@ enum ib_uverbs_access_flags { IB_UVERBS_ACCESS_ZERO_BASED = 1 << 5, IB_UVERBS_ACCESS_ON_DEMAND = 1 << 6, IB_UVERBS_ACCESS_HUGETLB = 1 << 7, + IB_UVERBS_ACCESS_FLUSH_GLOBAL_VISIBILITY = 1 << 8, + IB_UVERBS_ACCESS_FLUSH_PERSISTENT = 1 << 9, IB_UVERBS_ACCESS_RELAXED_ORDERING = IB_UVERBS_ACCESS_OPTIONAL_FIRST, IB_UVERBS_ACCESS_OPTIONAL_RANGE = diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index 7dd903d932e5..a58df0ebcb79 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -1331,6 +1331,8 @@ enum ib_uverbs_device_cap_flags { /* Deprecated. Please use IB_UVERBS_RAW_PACKET_CAP_SCATTER_FCS. */ IB_UVERBS_DEVICE_RAW_SCATTER_FCS = 1ULL << 34, IB_UVERBS_DEVICE_PCI_WRITE_END_PADDING = 1ULL << 36, + IB_UVERBS_DEVICE_PLT_GLOBAL_VISIBILITY = 1ULL << 38, + IB_UVERBS_DEVICE_PLT_PERSISTENT = 1ULL << 39, }; enum ib_uverbs_raw_packet_caps { From patchwork Fri Aug 5 07:46:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEE96C00140 for ; Fri, 5 Aug 2022 07:40:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240281AbiHEHkM (ORCPT ); Fri, 5 Aug 2022 03:40:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240242AbiHEHkH (ORCPT ); Fri, 5 Aug 2022 03:40:07 -0400 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B70B874E26; Fri, 5 Aug 2022 00:40:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685198; i=@fujitsu.com; bh=q+vZkm8yJ9zFKnWMnygIT/3sauEaYG/Tmtem+Zmc1j0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UtcNs0XZ9FtiYTTdTGnw3IuaLa9bQNail6fhbWBAV4WJboj7trmM3bqJDFx4rV+y2 tvw5RS815ASqi80pz6/VO4Lo3OmMxqZIszSWwqSLnOBrCN/82DJYFUrOqUlMXQwOkZ JduBZrcIkV+XqDH+JjCuxAq55XYZn9q+lFDvRqKChicRE3POpQsFyavouEEdkwyVaH R0+YrKo5o0ftl8jrFv+KypVhU+Q0UAclQyNRDC8FG6d5uZeZDAQtW5Km9xMo0m1a6f oB2Nib1HKWPWZ5Fvi3OFj5sUEOotIEU2SaGb9AdhLwSnRO40GRkMXGNEuUYiLa2b/z nnI2QF7Fhi7mg== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOKsWRWlGSWpSXmKPExsViZ8OxWdfv5Js kg6f97BZzv8tZTJ96gdFi5owTjBZTfi1ltri8aw6bxbNDvSwWH6YeYbb4MnUas8WpX6eYLP5e +sdmcf5YP7sDt8fOWXfZPRbvecnksWlVJ5tHb/M7No/LT64wenzeJOex9fNtlgD2KNbMvKT8i gTWjA0/5QuOC1Tc//WbsYHxNm8XIxeHkMBGRonTP98wQziLmSSmvnrN2sXICeTsZ5S4ONUExG YT0JC413KTEaRIRKCTUeJR/zE2EIdZ4DyTxLRN/9hBqoQFQiQ2PN3PCGKzCKhIfG6aDRbnFXC U2LZhPhuILSGgIDHl4XtmEJtTwEli07dVjBDbHCWOtixkgagXlDg58wmYzSwgIXHwxQugeg6g XiWJmd3xEGMqJGbNamOCsNUkrp7bxDyBUXAWku5ZSLoXMDKtYrRNKspMzyjJTczM0TU0MNA1N DQF0ka6hqbGeolVuol6qaW6eflFJRm6hnqJ5cV6qcXFesWVuck5KXp5qSWbGIFxllKcNnMH45 Z9v/QOMUpyMCmJ8p47/iZJiC8pP6UyI7E4I76oNCe1+BCjDAeHkgRv0AmgnGBRanpqRVpmDjD mYdISHDxKIrzrQFp5iwsSc4sz0yFSpxh1Oc7v3L+XWYglLz8vVUqc1wikSACkKKM0D24ELP1c YpSVEuZlZGBgEOIpSC3KzSxBlX/FKM7BqCTMew9kCk9mXgncpldARzABHcH1/zXIESWJCCmpB qYaCYv4El9ZS8mVV6IF7xdPF3+7oWJtsc7y3t471im8Z5fv+v6+reBsc8Iat70vHnkpFrPx1C S/uvb65c32Y+lxyyTcrbJu1Las7FI899r/419FpT8xQkqHbvGa/Ayav/ZYv/+M45VfBNp00gX c4rY/vXnFf5q+vD5nC2dtPefMps1N+6JnHvt7eM+EVVl8k4+5Cj84nv844FqGa81RlcjMTRkn 1ZVFtAJ4v62RvdYdyCW5pepA390DRkolVfaiD5i702fOvs7HteTOu6Yb/omqbV2HjtWmbXu9N O0AZz/HEd6+RXIeV/133BDfnbV3k3W177vV/p6O2RoCOxt2vl6z7EZF7NH/8Qvn7Epl/aPEUp yRaKjFXFScCACmjgKuugMAAA== X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-2.tower-745.messagelabs.com!1659685197!148381!1 X-Originating-IP: [62.60.8.179] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 10002 invoked from network); 5 Aug 2022 07:39:58 -0000 Received: from unknown (HELO n03ukasimr04.n03.fujitsu.local) (62.60.8.179) by server-2.tower-745.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:39:58 -0000 Received: from n03ukasimr04.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr04.n03.fujitsu.local (Postfix) with ESMTP id 7035E153; Fri, 5 Aug 2022 08:39:57 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr04.n03.fujitsu.local (Postfix) with ESMTPS id 627DF75; Fri, 5 Aug 2022 08:39:57 +0100 (BST) Received: from 4084fd6ad2a8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:39:52 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Aharon Landau , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 2/6] RDMA/rxe: Allow registering persistent flag for pmem MR only Date: Fri, 5 Aug 2022 07:46:15 +0000 Message-ID: <1659685579-2-3-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Memory region could at most support 2 access flags: IB_ACCESS_FLUSH_PERSISTENT and IB_ACCESS_FLUSH_GLOBAL_VISIBILITY But we only allow user to register persistent flush flags to the pmem MR that supports the ability of persisting data across power cycles. So register a persistent access flag to a non-pmem MR will be rejected by kernel. Signed-off-by: Li Zhijian --- v2: update commit message, get rid of confusing ib_check_flush_access_flags() # Tom --- drivers/infiniband/sw/rxe/rxe_mr.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 9e3e1a18f2dd..24ca014cdecd 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -113,6 +113,13 @@ void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr) mr->type = IB_MR_TYPE_DMA; } +static bool vaddr_in_pmem(char *vaddr) +{ + return REGION_INTERSECTS == + region_intersects(virt_to_phys(vaddr), 1, IORESOURCE_MEM, + IORES_DESC_PERSISTENT_MEMORY); +} + int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int access, struct rxe_mr *mr) { @@ -123,6 +130,7 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int num_buf; void *vaddr; int err; + bool first = true, is_pmem = false; int i; umem = ib_umem_get(pd->ibpd.device, start, length, access); @@ -167,6 +175,11 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, goto err_cleanup_map; } + if (first) { + first = false; + is_pmem = vaddr_in_pmem(vaddr); + } + buf->addr = (uintptr_t)vaddr; buf->size = PAGE_SIZE; num_buf++; @@ -175,6 +188,12 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, } } + if (!is_pmem && access & IB_ACCESS_FLUSH_PERSISTENT) { + pr_warn("Cannot register IB_ACCESS_FLUSH_PERSISTENT for non-pmem memory\n"); + err = -EINVAL; + goto err_release_umem; + } + mr->ibmr.pd = &pd->ibpd; mr->umem = umem; mr->access = access; From patchwork Fri Aug 5 07:46:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6865DC00140 for ; Fri, 5 Aug 2022 07:40:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240120AbiHEHkZ (ORCPT ); Fri, 5 Aug 2022 03:40:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240237AbiHEHkL (ORCPT ); Fri, 5 Aug 2022 03:40:11 -0400 Received: from mail1.bemta32.messagelabs.com (mail1.bemta32.messagelabs.com [195.245.230.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C513D75393; Fri, 5 Aug 2022 00:40:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685204; i=@fujitsu.com; bh=0/T7xN6P2RLjUJ/goz9fWNYkCJxhMqfVYxGHi4a2fQ0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Xa9hr7HKirmOr/IzQYPhVicCV4/XRbORWX+3wX5kfw9d+kdV/v//qE4RMlLwZb8U2 OAPWhic5AyJ8DvuNc2xQaYWygyWfM5bUzsxKkEEw3++9UcsP0pRvWGJiwvKP0wsWZI LhplUb3m981Rbz5sGrAJnkXoueQd/MZrXjzAsd0Ddiauj4MT/e3UIaMNTo3JJNb2z3 T89KQlMPwx33WJTk9GeWTC1wxbMlk4pKbrI94dU8QK0Jtc8ojF876icRfQ1IObTI7K qqEZKchKrbP4OE42EXxTR0jN3vxkVrPfDsYT0Er6oPQITXkWLZv93lI+IMhBnZ2Jpg ichfYRv9rFMCw== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGKsWRWlGSWpSXmKPExsViZ8ORpBt88k2 Swbb/QhZzv8tZTJ96gdFi5owTjBZTfi1ltri8aw6bxbNDvSwWH6YeYbb4MnUas8WpX6eYLP5e +sdmcf5YP7sDt8fOWXfZPRbvecnksWlVJ5tHb/M7No/LT64wenzeJOex9fNtlgD2KNbMvKT8i gTWjE039jAXHLSv+Ncwn7WB8bppFyMXh5DABkaJoz+nMEM4S5gk3l55xATh7GeU+LB0OmMXIy cHm4CGxL2Wm4wgCRGBTkaJR/3H2EAcZoHzTBLTNv1jB6kSFgiWOL51LTOIzSKgIjHl/DpWEJt XwFHi167DbCC2hICCxJSH78FqOAWcJDZ9WwW2QQio5mjLQhaIekGJkzOfgNnMAhISB1+8AKrn AOpVkpjZHQ8xpkJi1qw2JghbTeLquU3MExgFZyHpnoWkewEj0ypGi6SizPSMktzEzBxdQwMDX UNDU11DXSO9xCrdRL3UUt3y1OISXUO9xPJivdTiYr3iytzknBS9vNSSTYzACEspZmTawbis76 feIUZJDiYlUd5zx98kCfEl5adUZiQWZ8QXleakFh9ilOHgUJLgDToBlBMsSk1PrUjLzAFGO0x agoNHSYR3HUgrb3FBYm5xZjpE6hSjPcf5nfv3MnNMnf1vPzPHcjA582vbAWYhlrz8vFQpcV4j kDYBkLaM0jy4obDkdIlRVkqYl5GBgUGIpyC1KDezBFX+FaM4B6OSMO89kCk8mXklcLtfAZ3FB HQW1//XIGeVJCKkpBqY1A0ClWdM1Hup3LkoR8fIPyu40XHD9Arl3HsOywr3JCT/NZ189Jr42p 0a3sfq9Oazrzv09U9vz5ITWkGrbTLV13zTMlh/2CjoutFqGcVX/w7wv5b2VDp8Skm2KXENdyT rV9cbxj99PZ9NPCJwvuK00uumjIY7WsHXTB76b9ObYNdj0qVyLOHw1PVRAglOAfvdFZPZJNZ+ n/lppl/QFY57UxZ4X043qVDk3DuP88lq+SNrbi53XM0pqxoS8L52TaLhl6aHx3tOCdokf7a5e qPXYYd02+Frs6NaVRlcZz2+5ecaPW3N4/6VvrXrnsjZqsRe+V+8ZeLayvhlWiuPr3nTmn+cN8 f/S3Pi5M4g7SX/lFiKMxINtZiLihMBAt5FlckDAAA= X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-21.tower-591.messagelabs.com!1659685203!311870!1 X-Originating-IP: [62.60.8.98] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 11292 invoked from network); 5 Aug 2022 07:40:03 -0000 Received: from unknown (HELO n03ukasimr03.n03.fujitsu.local) (62.60.8.98) by server-21.tower-591.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:40:03 -0000 Received: from n03ukasimr03.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTP id CDA797C; Fri, 5 Aug 2022 08:40:02 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTPS id C198F1AE; Fri, 5 Aug 2022 08:40:02 +0100 (BST) Received: from 4084fd6ad2a8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:39:57 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Aharon Landau , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 3/6] RDMA/rxe: Implement RC RDMA FLUSH service in requester side Date: Fri, 5 Aug 2022 07:46:16 +0000 Message-ID: <1659685579-2-4-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org We implement new packet for FLUSH request. Signed-off-by: Li Zhijian --- V4: Remove flush union for legecy API, add WR_FLUSH_MASK V3: Fix sparse: incorrect type in assignment; Reported-by: kernel test robot V2: extend flush to include length field. --- drivers/infiniband/sw/rxe/rxe_hdr.h | 20 ++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_opcode.c | 21 +++++++++++++++++++++ drivers/infiniband/sw/rxe/rxe_opcode.h | 4 ++++ drivers/infiniband/sw/rxe/rxe_req.c | 15 ++++++++++++++- include/rdma/ib_pack.h | 2 ++ include/rdma/ib_verbs.h | 1 + include/uapi/rdma/ib_user_verbs.h | 1 + include/uapi/rdma/rdma_user_rxe.h | 7 +++++++ 8 files changed, 70 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_hdr.h b/drivers/infiniband/sw/rxe/rxe_hdr.h index e432f9e37795..8063b5018445 100644 --- a/drivers/infiniband/sw/rxe/rxe_hdr.h +++ b/drivers/infiniband/sw/rxe/rxe_hdr.h @@ -607,6 +607,25 @@ static inline void reth_set_len(struct rxe_pkt_info *pkt, u32 len) rxe_opcode[pkt->opcode].offset[RXE_RETH], len); } +/* + * FLUSH Extended Transport Header + */ +#define FETH_PLT_SHIFT 0UL +#define FETH_SEL_SHIFT 4UL +#define FETH_RESERVED_SHIFT 6UL +#define FETH_PLT_MASK ((1UL << FETH_SEL_SHIFT) - 1UL) +#define FETH_SEL_MASK (~FETH_PLT_MASK & ((1UL << FETH_RESERVED_SHIFT) - 1UL)) + +static inline void feth_init(struct rxe_pkt_info *pkt, u8 type, u8 level) +{ + __be32 *p = (__be32 *) + (pkt->hdr + rxe_opcode[pkt->opcode].offset[RXE_FETH]); + u32 feth = ((level << FETH_SEL_SHIFT) & FETH_SEL_MASK) | + ((type << FETH_PLT_SHIFT) & FETH_PLT_MASK); + + *p = cpu_to_be32(feth); +} + /****************************************************************************** * Atomic Extended Transport Header ******************************************************************************/ @@ -910,6 +929,7 @@ enum rxe_hdr_length { RXE_ATMETH_BYTES = sizeof(struct rxe_atmeth), RXE_IETH_BYTES = sizeof(struct rxe_ieth), RXE_RDETH_BYTES = sizeof(struct rxe_rdeth), + RXE_FETH_BYTES = sizeof(u32), }; static inline size_t header_size(struct rxe_pkt_info *pkt) diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.c b/drivers/infiniband/sw/rxe/rxe_opcode.c index d4ba4d506f17..45dc752bd31a 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.c +++ b/drivers/infiniband/sw/rxe/rxe_opcode.c @@ -101,6 +101,12 @@ struct rxe_wr_opcode_info rxe_wr_opcode_info[] = { [IB_QPT_UC] = WR_LOCAL_OP_MASK, }, }, + [IB_WR_RDMA_FLUSH] = { + .name = "IB_WR_RDMA_FLUSH", + .mask = { + [IB_QPT_RC] = WR_FLUSH_MASK, + }, + }, }; struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = { @@ -314,6 +320,21 @@ struct rxe_opcode_info rxe_opcode[RXE_NUM_OPCODE] = { RXE_AETH_BYTES, } }, + [IB_OPCODE_RC_RDMA_FLUSH] = { + .name = "IB_OPCODE_RC_RDMA_FLUSH", + .mask = RXE_FETH_MASK | RXE_RETH_MASK | RXE_FLUSH_MASK | + RXE_START_MASK | RXE_END_MASK | RXE_REQ_MASK, + .length = RXE_BTH_BYTES + RXE_FETH_BYTES + RXE_RETH_BYTES, + .offset = { + [RXE_BTH] = 0, + [RXE_FETH] = RXE_BTH_BYTES, + [RXE_RETH] = RXE_BTH_BYTES + + RXE_FETH_BYTES, + [RXE_PAYLOAD] = RXE_BTH_BYTES + + RXE_FETH_BYTES + + RXE_RETH_BYTES, + } + }, [IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE] = { .name = "IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE", .mask = RXE_AETH_MASK | RXE_ATMACK_MASK | RXE_ACK_MASK | diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.h b/drivers/infiniband/sw/rxe/rxe_opcode.h index 8f9aaaf260f2..9274c2016339 100644 --- a/drivers/infiniband/sw/rxe/rxe_opcode.h +++ b/drivers/infiniband/sw/rxe/rxe_opcode.h @@ -20,6 +20,7 @@ enum rxe_wr_mask { WR_READ_MASK = BIT(3), WR_WRITE_MASK = BIT(4), WR_LOCAL_OP_MASK = BIT(5), + WR_FLUSH_MASK = BIT(6), WR_READ_OR_WRITE_MASK = WR_READ_MASK | WR_WRITE_MASK, WR_WRITE_OR_SEND_MASK = WR_WRITE_MASK | WR_SEND_MASK, @@ -48,6 +49,7 @@ enum rxe_hdr_type { RXE_DETH, RXE_IMMDT, RXE_PAYLOAD, + RXE_FETH, NUM_HDR_TYPES }; @@ -63,6 +65,7 @@ enum rxe_hdr_mask { RXE_IETH_MASK = BIT(RXE_IETH), RXE_RDETH_MASK = BIT(RXE_RDETH), RXE_DETH_MASK = BIT(RXE_DETH), + RXE_FETH_MASK = BIT(RXE_FETH), RXE_PAYLOAD_MASK = BIT(RXE_PAYLOAD), RXE_REQ_MASK = BIT(NUM_HDR_TYPES + 0), @@ -80,6 +83,7 @@ enum rxe_hdr_mask { RXE_END_MASK = BIT(NUM_HDR_TYPES + 10), RXE_LOOPBACK_MASK = BIT(NUM_HDR_TYPES + 12), + RXE_FLUSH_MASK = BIT(NUM_HDR_TYPES + 13), RXE_READ_OR_ATOMIC_MASK = (RXE_READ_MASK | RXE_ATOMIC_MASK), RXE_WRITE_OR_SEND_MASK = (RXE_WRITE_MASK | RXE_SEND_MASK), diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c index f63771207970..2f0161b90fa7 100644 --- a/drivers/infiniband/sw/rxe/rxe_req.c +++ b/drivers/infiniband/sw/rxe/rxe_req.c @@ -241,6 +241,9 @@ static int next_opcode_rc(struct rxe_qp *qp, u32 opcode, int fits) IB_OPCODE_RC_SEND_ONLY_WITH_IMMEDIATE : IB_OPCODE_RC_SEND_FIRST; + case IB_WR_RDMA_FLUSH: + return IB_OPCODE_RC_RDMA_FLUSH; + case IB_WR_RDMA_READ: return IB_OPCODE_RC_RDMA_READ_REQUEST; @@ -421,11 +424,18 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp, /* init optional headers */ if (pkt->mask & RXE_RETH_MASK) { - reth_set_rkey(pkt, ibwr->wr.rdma.rkey); + if (pkt->mask & RXE_FETH_MASK) + reth_set_rkey(pkt, ibwr->wr.flush.rkey); + else + reth_set_rkey(pkt, ibwr->wr.rdma.rkey); reth_set_va(pkt, wqe->iova); reth_set_len(pkt, wqe->dma.resid); } + /* Fill Flush Extension Transport Header */ + if (pkt->mask & RXE_FETH_MASK) + feth_init(pkt, ibwr->wr.flush.type, ibwr->wr.flush.level); + if (pkt->mask & RXE_IMMDT_MASK) immdt_set_imm(pkt, ibwr->ex.imm_data); @@ -484,6 +494,9 @@ static int finish_packet(struct rxe_qp *qp, struct rxe_av *av, memset(pad, 0, bth_pad(pkt)); } + } else if (pkt->mask & RXE_FLUSH_MASK) { + /* oA19-2: shall have no payload. */ + wqe->dma.resid = 0; } return 0; diff --git a/include/rdma/ib_pack.h b/include/rdma/ib_pack.h index a9162f25beaf..d19edb502de6 100644 --- a/include/rdma/ib_pack.h +++ b/include/rdma/ib_pack.h @@ -84,6 +84,7 @@ enum { /* opcode 0x15 is reserved */ IB_OPCODE_SEND_LAST_WITH_INVALIDATE = 0x16, IB_OPCODE_SEND_ONLY_WITH_INVALIDATE = 0x17, + IB_OPCODE_RDMA_FLUSH = 0x1C, /* real constants follow -- see comment about above IB_OPCODE() macro for more details */ @@ -112,6 +113,7 @@ enum { IB_OPCODE(RC, FETCH_ADD), IB_OPCODE(RC, SEND_LAST_WITH_INVALIDATE), IB_OPCODE(RC, SEND_ONLY_WITH_INVALIDATE), + IB_OPCODE(RC, RDMA_FLUSH), /* UC */ IB_OPCODE(UC, SEND_FIRST), diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index aa174cdcdf5a..16db9eb3467a 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -1314,6 +1314,7 @@ struct ib_qp_attr { enum ib_wr_opcode { /* These are shared with userspace */ IB_WR_RDMA_WRITE = IB_UVERBS_WR_RDMA_WRITE, + IB_WR_RDMA_FLUSH = IB_UVERBS_WR_RDMA_FLUSH, IB_WR_RDMA_WRITE_WITH_IMM = IB_UVERBS_WR_RDMA_WRITE_WITH_IMM, IB_WR_SEND = IB_UVERBS_WR_SEND, IB_WR_SEND_WITH_IMM = IB_UVERBS_WR_SEND_WITH_IMM, diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index a58df0ebcb79..808cf7a39498 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -784,6 +784,7 @@ enum ib_uverbs_wr_opcode { IB_UVERBS_WR_RDMA_READ_WITH_INV = 11, IB_UVERBS_WR_MASKED_ATOMIC_CMP_AND_SWP = 12, IB_UVERBS_WR_MASKED_ATOMIC_FETCH_AND_ADD = 13, + IB_UVERBS_WR_RDMA_FLUSH = 14, /* Review enum ib_wr_opcode before modifying this */ }; diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index f09c5c9e3dd5..3de56ed5c24f 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -82,6 +82,13 @@ struct rxe_send_wr { __u32 invalidate_rkey; } ex; union { + struct { + __aligned_u64 remote_addr; + __u32 length; + __u32 rkey; + __u8 type; + __u8 level; + } flush; struct { __aligned_u64 remote_addr; __u32 rkey; From patchwork Fri Aug 5 07:46:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2E41C25B08 for ; Fri, 5 Aug 2022 07:40:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240397AbiHEHk1 (ORCPT ); Fri, 5 Aug 2022 03:40:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240219AbiHEHkO (ORCPT ); Fri, 5 Aug 2022 03:40:14 -0400 Received: from mail1.bemta34.messagelabs.com (mail1.bemta34.messagelabs.com [195.245.231.3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A48D522F; Fri, 5 Aug 2022 00:40:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685209; i=@fujitsu.com; bh=IZ98BgtTmjYOfpTuaMf9/yrmDSoKEaBVEWxvduNhSFk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OQn/NObv0U+ag6azMY1/znyFKJZihZ4uNiBlVR7iGHYPjZBtOJtigSVoydyVkaqls ZMz8rsp21BQN8q7ETCNqqLlguaCQ74sR3gmFOTS0mpBLEIWKyoWUxKTh+n74vHXKmf 7FF77MaakkZR5ZxcteKAE5b7D/TBNTX+XsApqwCTsQVahZtDc33ZNmtto5hR2WfvLJ nFX4uA+81vJrEeqa953sWWXJKwZpDf76UjSN/c3DYUYCha+n1ZcM0tlLuwJ/IKYBK1 kJDXuFPKTFrR/yIuQBMKezVE+SF8tubiLSV1ZCNiipoFsk+6Vp2Umr4q7G7k+pBdrp JoqGz2qScyXFQ== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKKsWRWlGSWpSXmKPExsViZ8MxSTfi5Js kg3u3xC3mfpezmD71AqPFzBknGC2m/FrKbHF51xw2i2eHelksPkw9wmzxZeo0ZotTv04xWfy9 9I/N4vyxfnYHbo+ds+6yeyze85LJY9OqTjaP3uZ3bB6Xn1xh9Pi8Sc5j6+fbLAHsUayZeUn5F QmsGafPPmYteJBdseXvDbYGxoORXYxcHEICWxglLv3ZzNLFyAnkLGeSuH49BSKxn1Hi6++z7C AJNgENiXstNxlBEiICnYwSj/qPsYE4zALnmSSmbfoHViUs4Cuxdu43VhCbRUBFYt2Ku4wgNq+ Ao8SWI2vYQGwJAQWJKQ/fM4PYnAJOEpu+rWKEWO0ocbRlIQtEvaDEyZlPwGxmAQmJgy9eANVz APUqSczsjocYUyExa1YbE4StJnH13CbmCYyCs5B0z0LSvYCRaRWjVVJRZnpGSW5iZo6uoYGBr qGhqa6xpa6FkV5ilW6iXmqpbnlqcYkukFterJdaXKxXXJmbnJOil5dasokRGGMpxepBOxi/rf ipd4hRkoNJSZT33PE3SUJ8SfkplRmJxRnxRaU5qcWHGGU4OJQkeINOAOUEi1LTUyvSMnOA8Q6 TluDgURLhXQfSyltckJhbnJkOkTrFqMsxdfa//cxCLHn5ealS4rxGIEUCIEUZpXlwI2Cp5xKj rJQwLyMDA4MQT0FqUW5mCar8K0ZxDkYlYd57IFN4MvNK4Da9AjqCCegIrv+vQY4oSURISTUwF SQVnm59q/nNUy79xX6FJtaeaQvdNs/9rZ/OfGlL4YsFl65tTVjUb+2a8euSlajjfU+7zOhpQT xxIgWbYuSrw7TvnytS0H3aqf+62C9Iut7Zau37bx997tvLak401i+U5nhQnLOA4dA0sYUKlw8 d0f0dYRwZtzTo2bTU+oMRLwQqtK/GCae8mNunfszGNsV9y8/Qxs8Juy4xain8fpXUKV2fWB7M xh/fP3GewgzNpftqal6Vdhuttvgb1Pg8OSt62UzBqx4nBdWWL/m+feKDEqmI99ka5pOSp506E MmR9699+TYu4+8iv1k5sk9tVjpmt8ZnW3TV8aD4IweqXbqu7t8zc+uk8MAjVvvfPnFSYinOSD TUYi4qTgQAenHkuLgDAAA= X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-2.tower-571.messagelabs.com!1659685208!174122!1 X-Originating-IP: [62.60.8.146] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 6637 invoked from network); 5 Aug 2022 07:40:08 -0000 Received: from unknown (HELO n03ukasimr02.n03.fujitsu.local) (62.60.8.146) by server-2.tower-571.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:40:08 -0000 Received: from n03ukasimr02.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr02.n03.fujitsu.local (Postfix) with ESMTP id EFBE71000C2; Fri, 5 Aug 2022 08:40:07 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr02.n03.fujitsu.local (Postfix) with ESMTPS id E17EB100077; Fri, 5 Aug 2022 08:40:07 +0100 (BST) Received: from 4084fd6ad2a8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:40:02 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Aharon Landau , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 4/6] RDMA/rxe: Implement flush execution in responder side Date: Fri, 5 Aug 2022 07:46:17 +0000 Message-ID: <1659685579-2-5-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org In contrast to other opcodes, after a series of sanity checking, FLUSH opcode will do a Placement Type checking before it really do the FLUSH operation. Only the requesting placement types that also registered in the destination memory region are acceptable. Otherwise, responder will also reply NAK "Remote Access Error" if it found a placement type violation. We will persist data via arch_wb_cache_pmem(), which could be architecture specific. Signed-off-by: Li Zhijian --- v4: add send_read_response_ack and flush resource --- drivers/infiniband/sw/rxe/rxe_hdr.h | 28 ++++ drivers/infiniband/sw/rxe/rxe_loc.h | 2 + drivers/infiniband/sw/rxe/rxe_mr.c | 4 +- drivers/infiniband/sw/rxe/rxe_resp.c | 187 +++++++++++++++++++++++++- drivers/infiniband/sw/rxe/rxe_verbs.h | 6 + include/uapi/rdma/ib_user_verbs.h | 10 ++ 6 files changed, 231 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_hdr.h b/drivers/infiniband/sw/rxe/rxe_hdr.h index 8063b5018445..2fe98146130e 100644 --- a/drivers/infiniband/sw/rxe/rxe_hdr.h +++ b/drivers/infiniband/sw/rxe/rxe_hdr.h @@ -626,6 +626,34 @@ static inline void feth_init(struct rxe_pkt_info *pkt, u8 type, u8 level) *p = cpu_to_be32(feth); } +static inline u32 __feth_plt(void *arg) +{ + __be32 *fethp = arg; + u32 feth = be32_to_cpu(*fethp); + + return (feth & FETH_PLT_MASK) >> FETH_PLT_SHIFT; +} + +static inline u32 __feth_sel(void *arg) +{ + __be32 *fethp = arg; + u32 feth = be32_to_cpu(*fethp); + + return (feth & FETH_SEL_MASK) >> FETH_SEL_SHIFT; +} + +static inline u32 feth_plt(struct rxe_pkt_info *pkt) +{ + return __feth_plt(pkt->hdr + + rxe_opcode[pkt->opcode].offset[RXE_FETH]); +} + +static inline u32 feth_sel(struct rxe_pkt_info *pkt) +{ + return __feth_sel(pkt->hdr + + rxe_opcode[pkt->opcode].offset[RXE_FETH]); +} + /****************************************************************************** * Atomic Extended Transport Header ******************************************************************************/ diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h index 22f6cc31d1d6..a77266cdc066 100644 --- a/drivers/infiniband/sw/rxe/rxe_loc.h +++ b/drivers/infiniband/sw/rxe/rxe_loc.h @@ -72,6 +72,8 @@ int rxe_mr_copy(struct rxe_mr *mr, u64 iova, void *addr, int length, enum rxe_mr_copy_dir dir); int copy_data(struct rxe_pd *pd, int access, struct rxe_dma_info *dma, void *addr, int length, enum rxe_mr_copy_dir dir); +void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, + size_t *offset_out); void *iova_to_vaddr(struct rxe_mr *mr, u64 iova, int length); struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key, enum rxe_mr_lookup_type type); diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 24ca014cdecd..98460fde7332 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -238,8 +238,8 @@ int rxe_mr_init_fast(struct rxe_pd *pd, int max_pages, struct rxe_mr *mr) return err; } -static void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, - size_t *offset_out) +void lookup_iova(struct rxe_mr *mr, u64 iova, int *m_out, int *n_out, + size_t *offset_out) { size_t offset = iova - mr->iova + mr->offset; int map_index; diff --git a/drivers/infiniband/sw/rxe/rxe_resp.c b/drivers/infiniband/sw/rxe/rxe_resp.c index 4c398fa220fa..4602cfbab78f 100644 --- a/drivers/infiniband/sw/rxe/rxe_resp.c +++ b/drivers/infiniband/sw/rxe/rxe_resp.c @@ -5,6 +5,7 @@ */ #include +#include #include "rxe.h" #include "rxe_loc.h" @@ -19,9 +20,11 @@ enum resp_states { RESPST_CHK_RESOURCE, RESPST_CHK_LENGTH, RESPST_CHK_RKEY, + RESPST_CHK_PLT, RESPST_EXECUTE, RESPST_READ_REPLY, RESPST_ATOMIC_REPLY, + RESPST_PROCESS_FLUSH, RESPST_COMPLETE, RESPST_ACKNOWLEDGE, RESPST_CLEANUP, @@ -36,6 +39,7 @@ enum resp_states { RESPST_ERR_TOO_MANY_RDMA_ATM_REQ, RESPST_ERR_RNR, RESPST_ERR_RKEY_VIOLATION, + RESPST_ERR_PLT_VIOLATION, RESPST_ERR_INVALIDATE_RKEY, RESPST_ERR_LENGTH, RESPST_ERR_CQ_OVERFLOW, @@ -54,9 +58,11 @@ static char *resp_state_name[] = { [RESPST_CHK_RESOURCE] = "CHK_RESOURCE", [RESPST_CHK_LENGTH] = "CHK_LENGTH", [RESPST_CHK_RKEY] = "CHK_RKEY", + [RESPST_CHK_PLT] = "CHK_PLACEMENT_TYPE", [RESPST_EXECUTE] = "EXECUTE", [RESPST_READ_REPLY] = "READ_REPLY", [RESPST_ATOMIC_REPLY] = "ATOMIC_REPLY", + [RESPST_PROCESS_FLUSH] = "PROCESS_FLUSH", [RESPST_COMPLETE] = "COMPLETE", [RESPST_ACKNOWLEDGE] = "ACKNOWLEDGE", [RESPST_CLEANUP] = "CLEANUP", @@ -71,6 +77,7 @@ static char *resp_state_name[] = { [RESPST_ERR_TOO_MANY_RDMA_ATM_REQ] = "ERR_TOO_MANY_RDMA_ATM_REQ", [RESPST_ERR_RNR] = "ERR_RNR", [RESPST_ERR_RKEY_VIOLATION] = "ERR_RKEY_VIOLATION", + [RESPST_ERR_PLT_VIOLATION] = "ERR_PLACEMENT_TYPE_VIOLATION", [RESPST_ERR_INVALIDATE_RKEY] = "ERR_INVALIDATE_RKEY_VIOLATION", [RESPST_ERR_LENGTH] = "ERR_LENGTH", [RESPST_ERR_CQ_OVERFLOW] = "ERR_CQ_OVERFLOW", @@ -402,6 +409,24 @@ static enum resp_states check_length(struct rxe_qp *qp, } } +static enum resp_states check_placement_type(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + struct rxe_mr *mr = qp->resp.mr; + u32 plt = feth_plt(pkt); + + if ((plt & IB_EXT_PLT_GLB_VIS && + !(mr->access & IB_ACCESS_FLUSH_GLOBAL_VISIBILITY)) || + (plt & IB_EXT_PLT_PERSIST && + !(mr->access & IB_ACCESS_FLUSH_PERSISTENT))) { + pr_info("Target MR didn't support this placement type, registered flag: %x, requested flag: %x\n", + (mr->access & IB_ACCESS_FLUSHABLE) >> 8, plt); + return RESPST_ERR_PLT_VIOLATION; + } + + return RESPST_EXECUTE; +} + static enum resp_states check_rkey(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { @@ -415,7 +440,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, enum resp_states state; int access; - if (pkt->mask & RXE_READ_OR_WRITE_MASK) { + if (pkt->mask & (RXE_READ_OR_WRITE_MASK | RXE_FLUSH_MASK)) { if (pkt->mask & RXE_RETH_MASK) { qp->resp.va = reth_va(pkt); qp->resp.offset = 0; @@ -423,8 +448,12 @@ static enum resp_states check_rkey(struct rxe_qp *qp, qp->resp.resid = reth_len(pkt); qp->resp.length = reth_len(pkt); } - access = (pkt->mask & RXE_READ_MASK) ? IB_ACCESS_REMOTE_READ - : IB_ACCESS_REMOTE_WRITE; + if (pkt->mask & RXE_FLUSH_MASK) + access = IB_ACCESS_FLUSHABLE; + else if (pkt->mask & RXE_READ_MASK) + access = IB_ACCESS_REMOTE_READ; + else + access = IB_ACCESS_REMOTE_WRITE; } else if (pkt->mask & RXE_ATOMIC_MASK) { qp->resp.va = atmeth_va(pkt); qp->resp.offset = 0; @@ -436,8 +465,10 @@ static enum resp_states check_rkey(struct rxe_qp *qp, } /* A zero-byte op is not required to set an addr or rkey. */ + /* RXE_FETH_MASK carraies zero-byte payload */ if ((pkt->mask & RXE_READ_OR_WRITE_MASK) && (pkt->mask & RXE_RETH_MASK) && + !(pkt->mask & RXE_FETH_MASK) && reth_len(pkt) == 0) { return RESPST_EXECUTE; } @@ -507,7 +538,7 @@ static enum resp_states check_rkey(struct rxe_qp *qp, WARN_ON_ONCE(qp->resp.mr); qp->resp.mr = mr; - return RESPST_EXECUTE; + return pkt->mask & RXE_FETH_MASK ? RESPST_CHK_PLT : RESPST_EXECUTE; err: if (mr) @@ -553,6 +584,64 @@ static enum resp_states write_data_in(struct rxe_qp *qp, return rc; } +static int nvdimm_flush_iova(struct rxe_mr *mr, u64 iova, int length) +{ + int err; + int bytes; + u8 *va; + struct rxe_map **map; + struct rxe_phys_buf *buf; + int m; + int i; + size_t offset; + + if (length == 0) + return 0; + + if (mr->type == IB_MR_TYPE_DMA) { + err = -EFAULT; + goto err1; + } + + err = mr_check_range(mr, iova, length); + if (err) { + err = -EFAULT; + goto err1; + } + + lookup_iova(mr, iova, &m, &i, &offset); + + map = mr->map + m; + buf = map[0]->buf + i; + + while (length > 0) { + va = (u8 *)(uintptr_t)buf->addr + offset; + bytes = buf->size - offset; + + if (bytes > length) + bytes = length; + + arch_wb_cache_pmem(va, bytes); + + length -= bytes; + + offset = 0; + buf++; + i++; + + if (i == RXE_BUF_PER_MAP) { + i = 0; + map++; + buf = map[0]->buf; + } + } + + return 0; + +err1: + return err; +} + static struct resp_res *rxe_prepare_res(struct rxe_qp *qp, struct rxe_pkt_info *pkt, int type) @@ -587,11 +676,60 @@ static struct resp_res *rxe_prepare_res(struct rxe_qp *qp, res->last_psn = pkt->psn; res->cur_psn = pkt->psn; break; + case RXE_FLUSH_MASK: + res->flush.va = qp->resp.va + qp->resp.offset; + res->flush.length = qp->resp.length; + res->flush.type = feth_plt(pkt); + res->flush.level = feth_sel(pkt); } return res; } +static enum resp_states process_flush(struct rxe_qp *qp, + struct rxe_pkt_info *pkt) +{ + u64 length, start; + struct rxe_mr *mr = qp->resp.mr; + struct resp_res *res = qp->resp.res; + + /* oA19-14, oA19-15 */ + if (res && res->replay) + return RESPST_ACKNOWLEDGE; + else if (!res) { + res = rxe_prepare_res(qp, pkt, RXE_FLUSH_MASK); + qp->resp.res = res; + } + + if (res->flush.level == IB_EXT_SEL_MR_RANGE) { + start = res->flush.va; + length = res->flush.length; + } else { /* level == IB_EXT_SEL_MR_WHOLE */ + start = mr->iova; + length = mr->length; + } + + if (res->flush.type & IB_EXT_PLT_PERSIST) { + if (nvdimm_flush_iova(mr, start, length)) + return RESPST_ERR_RKEY_VIOLATION; + /* Make data persistent. */ + wmb(); + } else if (res->flush.type & IB_EXT_PLT_GLB_VIS) + /* Make data global visibility. */ + wmb(); + + qp->resp.msn++; + + /* next expected psn, read handles this separately */ + qp->resp.psn = (pkt->psn + 1) & BTH_PSN_MASK; + qp->resp.ack_psn = qp->resp.psn; + + qp->resp.opcode = pkt->opcode; + qp->resp.status = IB_WC_SUCCESS; + + return RESPST_ACKNOWLEDGE; +} + /* Guarantee atomicity of atomic operations at the machine level. */ static DEFINE_SPINLOCK(atomic_ops_lock); @@ -892,6 +1030,8 @@ static enum resp_states execute(struct rxe_qp *qp, struct rxe_pkt_info *pkt) return RESPST_READ_REPLY; } else if (pkt->mask & RXE_ATOMIC_MASK) { return RESPST_ATOMIC_REPLY; + } else if (pkt->mask & RXE_FLUSH_MASK) { + return RESPST_PROCESS_FLUSH; } else { /* Unreachable */ WARN_ON_ONCE(1); @@ -1065,6 +1205,19 @@ static int send_atomic_ack(struct rxe_qp *qp, u8 syndrome, u32 psn) return ret; } +static int send_read_response_ack(struct rxe_qp *qp, u8 syndrome, u32 psn) +{ + int ret = send_common_ack(qp, syndrome, psn, + IB_OPCODE_RC_RDMA_READ_RESPONSE_ONLY, + "RDMA READ response of length zero ACK"); + + /* have to clear this since it is used to trigger + * long read replies + */ + qp->resp.res = NULL; + return ret; +} + static enum resp_states acknowledge(struct rxe_qp *qp, struct rxe_pkt_info *pkt) { @@ -1075,6 +1228,8 @@ static enum resp_states acknowledge(struct rxe_qp *qp, send_ack(qp, qp->resp.aeth_syndrome, pkt->psn); else if (pkt->mask & RXE_ATOMIC_MASK) send_atomic_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); + else if (pkt->mask & RXE_FLUSH_MASK) + send_read_response_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); else if (bth_ack(pkt)) send_ack(qp, AETH_ACK_UNLIMITED, pkt->psn); @@ -1131,6 +1286,22 @@ static enum resp_states duplicate_request(struct rxe_qp *qp, /* SEND. Ack again and cleanup. C9-105. */ send_ack(qp, AETH_ACK_UNLIMITED, prev_psn); return RESPST_CLEANUP; + } else if (pkt->mask & RXE_FLUSH_MASK) { + struct resp_res *res; + + /* Find the operation in our list of responder resources. */ + res = find_resource(qp, pkt->psn); + if (res) { + res->replay = 1; + res->cur_psn = pkt->psn; + qp->resp.res = res; + rc = RESPST_PROCESS_FLUSH; + goto out; + } + + /* Resource not found. Class D error. Drop the request. */ + rc = RESPST_CLEANUP; + goto out; } else if (pkt->mask & RXE_READ_MASK) { struct resp_res *res; @@ -1312,6 +1483,9 @@ int rxe_responder(void *arg) case RESPST_CHK_RKEY: state = check_rkey(qp, pkt); break; + case RESPST_CHK_PLT: + state = check_placement_type(qp, pkt); + break; case RESPST_EXECUTE: state = execute(qp, pkt); break; @@ -1324,6 +1498,9 @@ int rxe_responder(void *arg) case RESPST_ATOMIC_REPLY: state = atomic_reply(qp, pkt); break; + case RESPST_PROCESS_FLUSH: + state = process_flush(qp, pkt); + break; case RESPST_ACKNOWLEDGE: state = acknowledge(qp, pkt); break; @@ -1369,6 +1546,8 @@ int rxe_responder(void *arg) break; case RESPST_ERR_RKEY_VIOLATION: + /* oA19-13 8 */ + case RESPST_ERR_PLT_VIOLATION: if (qp_type(qp) == IB_QPT_RC) { /* Class C */ do_class_ac_error(qp, AETH_NAK_REM_ACC_ERR, diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h index 96af3e054f4d..ac04cd275400 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.h +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h @@ -165,6 +165,12 @@ struct resp_res { u64 va; u32 resid; } read; + struct { + u32 length; + u64 va; + u8 type; + u8 level; + } flush; }; }; diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index 808cf7a39498..4efa3d76d71d 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -105,6 +105,16 @@ enum { IB_USER_VERBS_EX_CMD_MODIFY_CQ }; +enum ib_ext_placement_type { + IB_EXT_PLT_GLB_VIS = 1 << 0, + IB_EXT_PLT_PERSIST = 1 << 1, +}; + +enum ib_ext_selectivity_level { + IB_EXT_SEL_MR_RANGE = 0, /* select a MR range */ + IB_EXT_SEL_MR_WHOLE, /* select the whole MR */ +}; + /* * Make sure that all structs defined in this file remain laid out so * that they pack the same way on 32-bit and 64-bit architectures (to From patchwork Fri Aug 5 07:55:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937038 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4EA8C00140 for ; Fri, 5 Aug 2022 07:48:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240047AbiHEHsv (ORCPT ); Fri, 5 Aug 2022 03:48:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48732 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237757AbiHEHsu (ORCPT ); Fri, 5 Aug 2022 03:48:50 -0400 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.112]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 744B5DA1; Fri, 5 Aug 2022 00:48:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685725; i=@fujitsu.com; bh=TCpF577v1pVIMuLfmc3qaPkxMJJHrE4qQTpf21EjPRs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Kq+MI8lgyKFPPGXn9mT3JdAKG++MYaO65q2bvAkKz296ydqcXXtRSWC0VfMoG/F88 wi+VrPUxhvvS7YFP4qLVM3w91RCxlDxJ17hvXJ99KC5OW5am4uTOwukZXOlYA7lvE7 uD1wTYK5TwW8dGtALyOwewcSDYZE2Ua/SdwaUub/ppy8zto1Wr0uH+JHz1hq4S5W87 cSkZByf+OfCsvjNthFx2QnFp0YbUnCm0tFzgoKiWkNFeOsWuD2h2PZIYnTy2FtsTrb 3wOJ3LXFHAyGkpfsTka0uwNrqp9Q7hGziUTsw5Hpbcx4IyxrrYIaDryma0VqfcwCVe fOGSnuZr+oplg== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKKsWRWlGSWpSXmKPExsViZ8ORpBt7+k2 SwYdtAhbTp15gtJg54wSjxZRfS5ktLu+aw2bx7FAvi8WHqUeYLb5MncZscerXKSaLv5f+sVmc P9bP7sDlsXPWXXaPxXteMnlsWtXJ5tHb/I7N4/KTK4wenzfJeWz9fJslgD2KNTMvKb8igTXjz r317AXfBCp+93YwNjCe5eti5OIQEtjIKHHi721mCGcxk0TPxnVsEM5+Rok3e5+xdzFycrAJaE jca7nJCJIQEehklHjUfwysillgNZNEX89rFpAqYQEbidWrFoF1sAioSHT/ucwGYvMKOEp8ub2 HEcSWEFCQmPLwPTOIzSngJLHp2yqwuBBQzdGWhSwQ9YISJ2c+AbOZBSQkDr54AVTPAdSrJDGz Ox5iTIXErFltTBC2msTVc5uYJzAKzkLSPQtJ9wJGplWMtklFmekZJbmJmTm6hgYGuoaGpkDaW NfI0EwvsUo3US+1VDcvv6gkQ9dQL7G8WC+1uFivuDI3OSdFLy+1ZBMjMMZSitNv7GDcte+X3i FGSQ4mJVHec8ffJAnxJeWnVGYkFmfEF5XmpBYfYpTh4FCS4A06AZQTLEpNT61Iy8wBxjtMWoK DR0mE9/RJoDRvcUFibnFmOkTqFKMux9TZ//YzC7Hk5eelSonzlpwCKhIAKcoozYMbAUs9lxhl pYR5GRkYGIR4ClKLcjNLUOVfMYpzMCoJ81qBTOHJzCuB2/QK6AgmoCO4/r8GOaIkESEl1cBkm 6lgkPW3+nDlPNZ//6wcT0zpcfhlcN3lefF2ZQc+WW3r47u9eZfmbJK4unGhw8+vfXrdJcLtm+ /91LyX8On3i3t/PC+cNtjPuF929c1LeemxT+T2fS9skGF5fH9RhIzg/rnctrdMzSJfTPKw9Nr bd2PlTGcj5quC/E8vvP5ty3ig8f+ytsRPISeW8vjP/hjIe3xqvHzSRas7ov+nSZXIzq1b1uYt vcv10HQFBxarnmrrD0uf7zrMWe3p4NfyuHZz454HQWYVZnPmai388aS2dt7E5/OW3b25YM0Dt 9xN5y9GftRxXbAkemVc/93AlSfuh1V8EetfN5012CL07dYMdVFj/Xe8URziH6ZE9XnbKbEUZy QaajEXFScCAOCA7AS4AwAA X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-3.tower-732.messagelabs.com!1659685724!304718!1 X-Originating-IP: [62.60.8.98] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 10325 invoked from network); 5 Aug 2022 07:48:45 -0000 Received: from unknown (HELO n03ukasimr03.n03.fujitsu.local) (62.60.8.98) by server-3.tower-732.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:48:45 -0000 Received: from n03ukasimr03.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTP id 95FF81AC; Fri, 5 Aug 2022 08:48:44 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTPS id 8A8237C; Fri, 5 Aug 2022 08:48:44 +0100 (BST) Received: from 4527b00272f8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:48:39 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 5/6] RDMA/rxe: Implement flush completion Date: Fri, 5 Aug 2022 07:55:32 +0000 Message-ID: <1659686133-2-1-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce a new IB_UVERBS_WC_FLUSH code to tell userspace a FLUSH completion. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_comp.c | 4 +++- include/rdma/ib_verbs.h | 1 + include/uapi/rdma/ib_user_verbs.h | 1 + 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c index fb0c008af78c..137ef9945da5 100644 --- a/drivers/infiniband/sw/rxe/rxe_comp.c +++ b/drivers/infiniband/sw/rxe/rxe_comp.c @@ -104,6 +104,7 @@ static enum ib_wc_opcode wr_to_wc_opcode(enum ib_wr_opcode opcode) case IB_WR_LOCAL_INV: return IB_WC_LOCAL_INV; case IB_WR_REG_MR: return IB_WC_REG_MR; case IB_WR_BIND_MW: return IB_WC_BIND_MW; + case IB_WR_RDMA_FLUSH: return IB_WC_RDMA_FLUSH; default: return 0xff; @@ -263,7 +264,8 @@ static inline enum comp_state check_ack(struct rxe_qp *qp, */ case IB_OPCODE_RC_RDMA_READ_RESPONSE_MIDDLE: if (wqe->wr.opcode != IB_WR_RDMA_READ && - wqe->wr.opcode != IB_WR_RDMA_READ_WITH_INV) { + wqe->wr.opcode != IB_WR_RDMA_READ_WITH_INV && + wqe->wr.opcode != IB_WR_RDMA_FLUSH) { wqe->status = IB_WC_FATAL_ERR; return COMPST_ERROR; } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 16db9eb3467a..832ea8113221 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -980,6 +980,7 @@ const char *__attribute_const__ ib_wc_status_msg(enum ib_wc_status status); enum ib_wc_opcode { IB_WC_SEND = IB_UVERBS_WC_SEND, IB_WC_RDMA_WRITE = IB_UVERBS_WC_RDMA_WRITE, + IB_WC_RDMA_FLUSH = IB_UVERBS_WC_FLUSH, IB_WC_RDMA_READ = IB_UVERBS_WC_RDMA_READ, IB_WC_COMP_SWAP = IB_UVERBS_WC_COMP_SWAP, IB_WC_FETCH_ADD = IB_UVERBS_WC_FETCH_ADD, diff --git a/include/uapi/rdma/ib_user_verbs.h b/include/uapi/rdma/ib_user_verbs.h index 4efa3d76d71d..d426609ad453 100644 --- a/include/uapi/rdma/ib_user_verbs.h +++ b/include/uapi/rdma/ib_user_verbs.h @@ -476,6 +476,7 @@ enum ib_uverbs_wc_opcode { IB_UVERBS_WC_BIND_MW = 5, IB_UVERBS_WC_LOCAL_INV = 6, IB_UVERBS_WC_TSO = 7, + IB_UVERBS_WC_FLUSH = 8, }; struct ib_uverbs_wc { From patchwork Fri Aug 5 07:55:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Zhijian X-Patchwork-Id: 12937039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FC92C00140 for ; Fri, 5 Aug 2022 07:48:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237624AbiHEHs4 (ORCPT ); Fri, 5 Aug 2022 03:48:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48754 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240211AbiHEHsy (ORCPT ); Fri, 5 Aug 2022 03:48:54 -0400 Received: from mail1.bemta32.messagelabs.com (mail1.bemta32.messagelabs.com [195.245.230.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3083BDA1; Fri, 5 Aug 2022 00:48:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1659685730; i=@fujitsu.com; bh=AiBJl2291nXEWiXxn1/wC4wkcqyLBRlWNI6lthScviw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IOG5jBiimkh1iGRp78ZD5HBjTWbHYT5u+KDLdmlDZbOHvU5JUCFc1ZywavnEC6K0G Q4sZnS1GlpBNnBfXn+jXsQ5ZScF6z4vyeQV94S4XlK5HnOJrHz5E6J/fpomtAXmRgZ OHpKF2ukuyIcj/OcqpT8MoXS1W5hDRWx4IrUYgk4CPbqN6DFZB4wstYxX3cAt/AXah 11vf45m6u58MMlovH/a4Kjucp0tfE2Kh0yvLYjDwKEdjutuNqxZQIhNav6tsU1eErN g4LfzGN29DZOwpWalFcqpenseTcsHvheK4oJbQLRYPh4irSpC9QPrVnBhxxTPUOIAz yJ4yKlX9r9znQ== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprEKsWRWlGSWpSXmKPExsViZ8ORqJt0+k2 Swdo7LBbTp15gtJg54wSjxZRfS5ktLu+aw2bx7FAvi8WHqUeYLb5MncZscerXKSaLv5f+sVmc P9bP7sDlsXPWXXaPxXteMnlsWtXJ5tHb/I7N4/KTK4wenzfJeWz9fJslgD2KNTMvKb8igTXj6 /v4ggnsFRtvsTUwNrN1MXJxCAlsYZS423oPylnOJPHm+F1WCGc/o8SWg4/Yuxg5OdgENCTutd xkBEmICHQySjzqPwbWwiywmkmir+c1C0iVsICvxIGDE8FsFgEVibO9L5lAbF4BR4mGG49ZQWw JAQWJKQ/fM3cxcnBwCjhJHL0KViIkUCHxbf4hFohyQYmTM5+A2cwCEhIHX7wAK5cQUJKY2R0P MaVCYtasNiYIW03i6rlNzBMYBWch6Z6FpHsBI9MqRsukosz0jJLcxMwcXUMDA11DQ1NdY10LM 73EKt1EvdRS3fLU4hJdQ73E8mK91OJiveLK3OScFL281JJNjMCoSilmDtvB+Kf3p94hRkkOJi VR3nPH3yQJ8SXlp1RmJBZnxBeV5qQWH2KU4eBQkuANOgGUEyxKTU+tSMvMAUY4TFqCg0dJhPf 0SaA0b3FBYm5xZjpE6hSjopQ4b8kpoIQASCKjNA+uDZZULjHKSgnzMjIwMAjxFKQW5WaWoMq/ YhTnYFQS5rUCmcKTmVcCN/0V0GImoMVc/1+DLC5JREhJNTDFO1aIG2VV3V6YktOwm9HbPc/dR ceuoHh/5Df2qIyOSLdr/9oYfnfXr/x0d2+aiS7T16aPZQ/SLyuy1L5g7Gd8V9c+LUEhNe1Pu5 DCRQkBy51Tbuw9e/q0fef3WwEvrf+Jp7yslBVaeOuyzPMvT9uKfixoSGh8mvRUY6pMkOL9z7M /aP6bYDr5q/ZM56nXDk6605orw8eb1Jri923fup0Rm3lfnsp2kgu7wRb2JijyH095yDOBgyrr Ln1gMXD8frcm/l/mK8G+hbIBj5a9OmF7c86BBfOUxURl+1YnOW1rkWnUuHTw3tncBbvePeHMX 79upVFnz9Tw9fPOzJKanuGyg1e0hvNWhaq8gxprcb0SS3FGoqEWc1FxIgCWv5DYpQMAAA== X-Env-Sender: lizhijian@fujitsu.com X-Msg-Ref: server-3.tower-587.messagelabs.com!1659685729!166593!1 X-Originating-IP: [62.60.8.97] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.87.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 19536 invoked from network); 5 Aug 2022 07:48:50 -0000 Received: from unknown (HELO n03ukasimr01.n03.fujitsu.local) (62.60.8.97) by server-3.tower-587.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 5 Aug 2022 07:48:50 -0000 Received: from n03ukasimr01.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTP id 99C751001A0; Fri, 5 Aug 2022 08:48:49 +0100 (BST) Received: from R01UKEXCASM126.r01.fujitsu.local (R01UKEXCASM126 [10.183.43.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr01.n03.fujitsu.local (Postfix) with ESMTPS id 8C81910019F; Fri, 5 Aug 2022 08:48:49 +0100 (BST) Received: from 4527b00272f8.localdomain (10.167.225.141) by R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) with Microsoft SMTP Server (TLS) id 15.0.1497.32; Fri, 5 Aug 2022 08:48:44 +0100 From: Li Zhijian To: Jason Gunthorpe , Zhu Yanjun , "Leon Romanovsky" , CC: Xiao Yang , , Bob Pearson , Mark Bloch , Tom Talpey , , Dan Williams , , Li Zhijian Subject: [PATCH v4 6/6] RDMA/rxe: Enable RDMA FLUSH capability for rxe device Date: Fri, 5 Aug 2022 07:55:33 +0000 Message-ID: <1659686133-2-2-git-send-email-lizhijian@fujitsu.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1659686133-2-1-git-send-email-lizhijian@fujitsu.com> References: <1659685579-2-1-git-send-email-lizhijian@fujitsu.com> <1659686133-2-1-git-send-email-lizhijian@fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM126.r01.fujitsu.local (10.183.43.178) X-Virus-Scanned: ClamAV using ClamSMTP Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Now we are ready to enable RDMA FLUSH capability for RXE. It can support Global Visibility and Persistence placement types. Signed-off-by: Li Zhijian --- drivers/infiniband/sw/rxe/rxe_param.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h index 86c7a8bf3cbb..3efb515ba735 100644 --- a/drivers/infiniband/sw/rxe/rxe_param.h +++ b/drivers/infiniband/sw/rxe/rxe_param.h @@ -51,7 +51,9 @@ enum rxe_device_param { | IB_DEVICE_SRQ_RESIZE | IB_DEVICE_MEM_MGT_EXTENSIONS | IB_DEVICE_MEM_WINDOW - | IB_DEVICE_MEM_WINDOW_TYPE_2B, + | IB_DEVICE_MEM_WINDOW_TYPE_2B + | IB_DEVICE_PLT_GLOBAL_VISIBILITY + | IB_DEVICE_PLT_PERSISTENT, RXE_MAX_SGE = 32, RXE_MAX_WQE_SIZE = sizeof(struct rxe_send_wqe) + sizeof(struct ib_sge) * RXE_MAX_SGE,