From patchwork Thu Aug 13 19:20:53 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 7010351 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 304C8C05AC for ; Thu, 13 Aug 2015 19:21:59 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5DA03207AF for ; Thu, 13 Aug 2015 19:21:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 788B22079F for ; Thu, 13 Aug 2015 19:21:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753504AbbHMTV2 (ORCPT ); Thu, 13 Aug 2015 15:21:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43051 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932231AbbHMTVR (ORCPT ); Thu, 13 Aug 2015 15:21:17 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 640E637258C; Thu, 13 Aug 2015 19:21:17 +0000 (UTC) Received: from dhcp-10-19-62-215.boston.devel.redhat.com (dhcp40-164.desklab.eng.bos.redhat.com [10.19.40.164] (may be forged)) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t7DJL27Q005617; Thu, 13 Aug 2015 15:21:15 -0400 From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= To: , Cc: Christophe Harle , Duncan Poole , Sherry Cheung , Subhash Gutti , John Hubbard , Mark Hairgrove , Lucien Dunning , Cameron Buschardt , Arvind Gopalakrishnan , Haggai Eran , Shachar Raindel , Liran Liss , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= Subject: [RFC PATCH 8/8 v2] IB/mlx5/hmm: enable ODP using HMM v2. Date: Thu, 13 Aug 2015 15:20:53 -0400 Message-Id: <1439493653-1191-9-git-send-email-jglisse@redhat.com> In-Reply-To: <1439493653-1191-1-git-send-email-jglisse@redhat.com> References: <1439493653-1191-1-git-send-email-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP All pieces are in place for ODP (on demand paging) to work using HMM. Add kernel option and final code to enable it. Changed since v1: - Added kernel option in this last patch of the serie. Signed-off-by: Jérôme Glisse --- drivers/infiniband/Kconfig | 10 ++++++++++ drivers/infiniband/core/uverbs_cmd.c | 3 --- drivers/infiniband/hw/mlx5/main.c | 4 ++++ 3 files changed, 14 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig index b899531..764f524 100644 --- a/drivers/infiniband/Kconfig +++ b/drivers/infiniband/Kconfig @@ -49,6 +49,16 @@ config INFINIBAND_ON_DEMAND_PAGING memory regions without pinning their pages, fetching the pages on demand instead. +config INFINIBAND_ON_DEMAND_PAGING_HMM + bool "InfiniBand on-demand paging support using HMM." + depends on HMM + depends on INFINIBAND_ON_DEMAND_PAGING + default n + ---help--- + Use HMM (heterogeneous memory management) kernel API for + on demand paging. No userspace difference, this is just + an alternative implementation of the feature. + config INFINIBAND_ADDR_TRANS bool depends on INFINIBAND diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index 1db6a17..c3e14a8 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -3445,8 +3445,6 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file, goto end; #if IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING) -#if IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING_HMM) -#else /* CONFIG_INFINIBAND_ON_DEMAND_PAGING_HMM */ resp.odp_caps.general_caps = attr.odp_caps.general_caps; resp.odp_caps.per_transport_caps.rc_odp_caps = attr.odp_caps.per_transport_caps.rc_odp_caps; @@ -3455,7 +3453,6 @@ int ib_uverbs_ex_query_device(struct ib_uverbs_file *file, resp.odp_caps.per_transport_caps.ud_odp_caps = attr.odp_caps.per_transport_caps.ud_odp_caps; resp.odp_caps.reserved = 0; -#endif /* CONFIG_INFINIBAND_ON_DEMAND_PAGING_HMM */ #else /* CONFIG_INFINIBAND_ON_DEMAND_PAGING */ memset(&resp.odp_caps, 0, sizeof(resp.odp_caps)); #endif /* CONFIG_INFINIBAND_ON_DEMAND_PAGING */ diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c index 32ed2f1..c340c3a 100644 --- a/drivers/infiniband/hw/mlx5/main.c +++ b/drivers/infiniband/hw/mlx5/main.c @@ -295,6 +295,10 @@ static int mlx5_ib_query_device(struct ib_device *ibdev, #if IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING) #if IS_ENABLED(CONFIG_INFINIBAND_ON_DEMAND_PAGING_HMM) + if (MLX5_CAP_GEN(mdev, pg) && ibdev->hmm_ready) { + props->device_cap_flags |= IB_DEVICE_ON_DEMAND_PAGING; + props->odp_caps = dev->odp_caps; + } #else /* CONFIG_INFINIBAND_ON_DEMAND_PAGING_HMM */ if (MLX5_CAP_GEN(mdev, pg)) props->device_cap_flags |= IB_DEVICE_ON_DEMAND_PAGING;