From patchwork Tue Mar 15 10:18:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhijian Li (Fujitsu)" X-Patchwork-Id: 12781254 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE722C433EF for ; Tue, 15 Mar 2022 10:13:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234652AbiCOKOS (ORCPT ); Tue, 15 Mar 2022 06:14:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239017AbiCOKOR (ORCPT ); Tue, 15 Mar 2022 06:14:17 -0400 Received: from heian.cn.fujitsu.com (mail.cn.fujitsu.com [183.91.158.132]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 79F946173; Tue, 15 Mar 2022 03:13:04 -0700 (PDT) IronPort-Data: A9a23:tRSjFqkBDuL/XMJ+DwpEVxjo5gyOJ0RdPkR7XQ2eYbTBsI5bpzdUzTQbD2uEOvyOZTOkLd5+Ptvk90IB6pDUm9QyTQBr+CA2RRqmi+KfW43BcR2Y0wB+jyH7ZBs+qZ1YM7EsFehsJpPnjkrrYuiJQUVUj/nSHOKmULecY0ideCc/IMsfoUM68wIGqt4w6TSJK1vlVeLa+6UzCnf8s9JHGj58B5a4lf9alK+aVAX0EbAJTasjUFf2zxH5BX+ETE27ByOQroJ8RoZWSwtfpYxV8F81/z91Yj+kur39NEMXQL/OJhXIgX1TM0SgqkEa4HVsjeBgb7xBAatUo2zhc9RZzNRftZ2ySA4vFqPRmuUBSAQeGCZ7VUFD0Oadeybj4ZXLkyUqdFOpmZ2CFnoeJ5UV8/xsBmdO7fEwJzUEbxTFjOWzqJqpW+t+l8Z5dJGzFIwas3BkizreCJ4ORZHKRarV6NlA0TE/rsBTFOnTZowSbj8HRBjJZVtNfEgWDJY/leKzrnj5bzBc7lmSoMIf/2/WxRdjlrf3N9/cds6JRO1UmFqVoiTN+GGRKhUXM9q3yjef9H+owOjVkkvTUYIbDrq+8tZsnlyfx2VVAxoTPXO+q/2+gU6WXcxeJ00dvCEpqMAa6EuuZsX0WwW1sTiPuRt0c95RFfAqrQKA0KzZ5y6HCWUeCD1MctorsIkxXzNC/luImc75QCZjtbS9V32Q7PGXoCm0NCxTKnUNDQcGQgQt8djuuIx1hRunczrJOMZZlfWsQXepnW/M93N42t0uYQcw//3T1Tj6b/iE//AlljII2zg= IronPort-HdrOrdr: A9a23:d/6iVq/MPezAO+fXgz5uk+C2I+orL9Y04lQ7vn2ZKCY1TiX2rayTdZggviMc6wx+ZJhDo7+90cC7KBvhHPVOjLX5U43JYDXb X-IronPort-AV: E=Sophos;i="5.88,333,1635177600"; d="scan'208";a="122648104" Received: from unknown (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 15 Mar 2022 18:13:03 +0800 Received: from G08CNEXMBPEKD05.g08.fujitsu.local (unknown [10.167.33.204]) by cn.fujitsu.com (Postfix) with ESMTP id 6930B4D16FD8; Tue, 15 Mar 2022 18:13:02 +0800 (CST) Received: from G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) by G08CNEXMBPEKD05.g08.fujitsu.local (10.167.33.204) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 15 Mar 2022 18:13:02 +0800 Received: from G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.85) by G08CNEXJMPEKD02.g08.fujitsu.local (10.167.33.202) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 15 Mar 2022 18:13:04 +0800 Received: from localhost.localdomain (10.167.225.141) by G08CNEXCHPEKD09.g08.fujitsu.local (10.167.33.209) with Microsoft SMTP Server id 15.0.1497.23 via Frontend Transport; Tue, 15 Mar 2022 18:13:00 +0800 From: Li Zhijian To: , , , , , , CC: , , , , , , , Li Zhijian Subject: [RFC PATCH v3 2/7] RDMA/rxe: Allow registering persistent flag for pmem MR only Date: Tue, 15 Mar 2022 18:18:40 +0800 Message-ID: <20220315101845.4166983-3-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220315101845.4166983-1-lizhijian@fujitsu.com> References: <20220315101845.4166983-1-lizhijian@fujitsu.com> MIME-Version: 1.0 X-yoursite-MailScanner-ID: 6930B4D16FD8.AFDAF X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: lizhijian@fujitsu.com Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Memory region could at most support 2 placement types: IB_ACCESS_FLUSH_PERSISTENT and IB_ACCESS_FLUSH_GLOBAL_VISIBILITY But we only allow user to register persistence flush to non-pmem MR. Signed-off-by: Li Zhijian --- V3: combine [RFC PATCH v2 1/9] RDMA: mr: Introduce is_pmem V2: update commit message, get rid of confusing ib_check_flush_access_flags() # Tom --- drivers/infiniband/sw/rxe/rxe_mr.c | 32 ++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c index 453ef3c9d535..4f5c4af19fe0 100644 --- a/drivers/infiniband/sw/rxe/rxe_mr.c +++ b/drivers/infiniband/sw/rxe/rxe_mr.c @@ -161,6 +161,28 @@ void rxe_mr_init_dma(struct rxe_pd *pd, int access, struct rxe_mr *mr) mr->type = IB_MR_TYPE_DMA; } +static bool iova_in_pmem(struct rxe_mr *mr, u64 iova, int length) +{ + char *vaddr; + int is_pmem; + + /* XXX: Shall me allow length == 0 */ + if (length == 0) { + return false; + } + /* check the 1st byte only to avoid crossing page boundary */ + vaddr = iova_to_vaddr(mr, iova, 1); + if (!vaddr) { + pr_warn("not a valid iova 0x%llx\n", iova); + return false; + } + + is_pmem = region_intersects(virt_to_phys(vaddr), 1, IORESOURCE_MEM, + IORES_DESC_PERSISTENT_MEMORY); + + return is_pmem == REGION_INTERSECTS; +} + int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, int access, struct rxe_mr *mr) { @@ -235,6 +257,16 @@ int rxe_mr_init_user(struct rxe_pd *pd, u64 start, u64 length, u64 iova, set->va = start; set->offset = ib_umem_offset(umem); + // iova_in_pmem() must be called after set is updated + if (!iova_in_pmem(mr, iova, length) && + access & IB_ACCESS_FLUSH_PERSISTENT) { + pr_warn("Cannot register IB_ACCESS_FLUSH_PERSISTENT for non-pmem memory\n"); + mr->state = RXE_MR_STATE_INVALID; + mr->umem = NULL; + err = -EINVAL; + goto err_release_umem; + } + return 0; err_release_umem: