From patchwork Mon Nov 5 14:27:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10668341 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22C96181D for ; Mon, 5 Nov 2018 14:28:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C467422A2A for ; Mon, 5 Nov 2018 14:28:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B85D9256E6; Mon, 5 Nov 2018 14:28:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,SUBJ_OBFU_PUNCT_FEW,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48C23205F6 for ; Mon, 5 Nov 2018 14:28:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726173AbeKEXsV (ORCPT ); Mon, 5 Nov 2018 18:48:21 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:55834 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729198AbeKEXsU (ORCPT ); Mon, 5 Nov 2018 18:48:20 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 5 Nov 2018 16:33:42 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id wA5ESIon008308; Mon, 5 Nov 2018 16:28:18 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id wA5ESIHf017222; Mon, 5 Nov 2018 16:28:18 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id wA5ESIgg017221; Mon, 5 Nov 2018 16:28:18 +0200 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, monis@mellanox.com, aviadye@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH RFC rdma-core 0/3] Introduce ibv_advise_mr verb Date: Mon, 5 Nov 2018 16:27:45 +0200 Message-Id: <1541428068-17056-1-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Overview ======== Today rdma applications can register memory region (MR), deregister and reregister it. This patch set introduce a new ibv_advise_mr() verb that will be used to give advice or directions to the kernel about an address range that belongs to a MR. Motivation ========== Performance - applications that are aware of future access patterns can use this verb in order to leverage this knowledge to improve system or application performance. One example is for pre-fetching a range part of an on-demand paging MR. On-demand-paging (ODP) is a technique to alleviate much of the shortcomings of memory registration. The main idea is that applications no longer pin down the underlying physical pages of the address space and need not track the validity of the mappings. Rather, the HCA requests the latest translations from the OS when pages are not present, and the OS invalidates translations which are no longer valid due to mapping changes. Pre-fetching ODP address ranges, therefore make them present before the actual IO is conducted. This would provide a way to reduce latency by overlapping paging-in and either compute time or IO to other ranges. This RFC patch set includes the following: - Detailed man page for the new ibv_advise_mr verb. - The initial code implementation for the new verb in libibverbs to demonstrate the expected code and usage. Moni, Aviad, Yishai. Moni Shoua (3): Update kernel headers verbs: Add helper for command interface verbs: Introduce ibv_advise_mr verb kernel-headers/rdma/ib_user_ioctl_cmds.h | 8 ++ kernel-headers/rdma/ib_user_ioctl_verbs.h | 9 +++ libibverbs/CMakeLists.txt | 1 + libibverbs/cmd_ioctl.h | 3 + libibverbs/cmd_mr.c | 57 ++++++++++++++ libibverbs/driver.h | 10 +++ libibverbs/dummy_ops.c | 11 +++ libibverbs/libibverbs.map.in | 1 + libibverbs/man/CMakeLists.txt | 1 + libibverbs/man/ibv_advise_mr.3.md | 125 ++++++++++++++++++++++++++++++ libibverbs/verbs.h | 28 +++++++ libibverbs/verbs_api.h | 6 ++ 12 files changed, 260 insertions(+) create mode 100644 libibverbs/cmd_mr.c create mode 100644 libibverbs/man/ibv_advise_mr.3.md