From patchwork Tue Dec 8 15:15:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 7798591 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id D1C6A9F387 for ; Tue, 8 Dec 2015 15:16:48 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id DC98220523 for ; Tue, 8 Dec 2015 15:16:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB45A20396 for ; Tue, 8 Dec 2015 15:16:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932870AbbLHPQm (ORCPT ); Tue, 8 Dec 2015 10:16:42 -0500 Received: from [193.47.165.129] ([193.47.165.129]:39247 "EHLO mellanox.co.il" rhost-flags-FAIL-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S932971AbbLHPQg (ORCPT ); Tue, 8 Dec 2015 10:16:36 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 8 Dec 2015 17:16:10 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id tB8FGArD026671; Tue, 8 Dec 2015 17:16:10 +0200 Received: from vnc17.mtl.labs.mlnx (localhost.localdomain [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id tB8FG9M9026226; Tue, 8 Dec 2015 17:16:09 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id tB8FG9L9026225; Tue, 8 Dec 2015 17:16:09 +0200 From: Yishai Hadas To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, yishaih@mellanox.com, ogerlitz@mellanox.com, talal@mellanox.com Subject: [RFC contig pages support 2/2] IB/mlx5: Exporting to user space the contiguous allocation capability Date: Tue, 8 Dec 2015 17:15:07 +0200 Message-Id: <1449587707-24214-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.7.11.3 In-Reply-To: <1449587707-24214-1-git-send-email-yishaih@mellanox.com> References: <1449587707-24214-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Extend mlx5_ib_mmap to recognize 'allocate contiguous' command. Offset field will be encoded to hold both command and its data. Last 8 bits will hold the command, this will enable future extension for further commands. Signed-off-by: Yishai Hadas --- drivers/infiniband/hw/mlx5/main.c | 35 ++++++++++++++++++++++++++++++++++- drivers/infiniband/hw/mlx5/mlx5_ib.h | 4 +++- 2 files changed, 37 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 7e97cb5..46de426 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -43,6 +43,7 @@ #include #include #include +#include #include "user.h" #include "mlx5_ib.h" @@ -751,6 +752,11 @@ static int get_index(unsigned long offset) return get_arg(offset); } +static int get_pg_order(unsigned long offset) +{ + return get_arg(offset); +} + static int mlx5_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma) { struct mlx5_ib_ucontext *context = to_mucontext(ibcontext); @@ -759,6 +765,11 @@ static int mlx5_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vm unsigned long command; unsigned long idx; phys_addr_t pfn; + unsigned long total_size; + unsigned long order; + struct ib_cmem *ib_cmem; + int err; + int local_numa_node; command = get_command(vma->vm_pgoff); switch (command) { @@ -784,8 +795,30 @@ static int mlx5_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vm (unsigned long long)pfn << PAGE_SHIFT); break; + case MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_CPU_NUMA: + case MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_DEV_NUMA: case MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES: - return -ENOSYS; + if (command == MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_CPU_NUMA) + local_numa_node = numa_node_id(); + else if (command == MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_DEV_NUMA) + local_numa_node = dev_to_node(&dev->mdev->pdev->dev); + else + local_numa_node = -1; + + total_size = vma->vm_end - vma->vm_start; + order = get_pg_order(vma->vm_pgoff); + + ib_cmem = ib_cmem_alloc_contiguous_pages(ibcontext, total_size, + order, local_numa_node); + if (IS_ERR(ib_cmem)) + return PTR_ERR(ib_cmem); + + err = ib_cmem_map_contiguous_pages_to_vma(ib_cmem, vma); + if (err) { + ib_cmem_release_contiguous_pages(ib_cmem); + return err; + } + break; default: return -EINVAL; diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 6333472..1e2c57e 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -62,7 +62,9 @@ enum { enum mlx5_ib_mmap_cmd { MLX5_IB_MMAP_REGULAR_PAGE = 0, - MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, /* always last */ + MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES = 1, + MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_CPU_NUMA = 0xFC, + MLX5_IB_MMAP_GET_CONTIGUOUS_PAGES_DEV_NUMA = 0xFD, }; enum {