From patchwork Mon Aug 16 22:16:27 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vu Pham X-Patchwork-Id: 119814 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.4/8.14.3) with ESMTP id o7GMOoIL011875 for ; Mon, 16 Aug 2010 22:24:50 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756751Ab0HPWYP (ORCPT ); Mon, 16 Aug 2010 18:24:15 -0400 Received: from p02c12o144.mxlogic.net ([208.65.145.77]:40190 "EHLO p02c12o144.mxlogic.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756457Ab0HPWYJ (ORCPT ); Mon, 16 Aug 2010 18:24:09 -0400 Received: from unknown [63.251.237.3] (EHLO p02c12o144.mxlogic.net) by p02c12o144.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id 98ab96c4.6819b940.45944.00-580.109918.p02c12o144.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:24:09 -0600 (MDT) X-MXL-Hash: 4c69ba890506fca5-487807b780f33355637d7a83913d0a2f652d1a31 Received: from unknown [63.251.237.3] (EHLO mtiexch01.mti.com) by p02c12o144.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id 9b8b96c4.0.44531.00-319.105993.p02c12o144.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:16:27 -0600 (MDT) X-MXL-Hash: 4c69b8bb7d14e1ca-34ad64d2846e53038fd8b240af4284e576ae0ad0 Received: from vu-lt.mti.mtl.com ([10.2.1.17]) by mtiexch01.mti.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 16 Aug 2010 15:20:51 -0700 Message-ID: <4C69B8BB.4000407@mellanox.com> Date: Mon, 16 Aug 2010 15:16:27 -0700 From: Vu Pham User-Agent: Thunderbird 2.0.0.23 (X11/20090812) MIME-Version: 1.0 To: Roland Dreier CC: OpenFabrics EWG , Linux RDMA , Oren Duer Subject: [PATCH v1 04/10] mlx4_core: Add interface to allocate fmr with pre-reserved MPTs X-OriginalArrivalTime: 16 Aug 2010 22:20:51.0046 (UTC) FILETIME=[48C2A860:01CB3D91] X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010073001)] X-MAIL-FROM: X-SOURCE-IP: [63.251.237.3] X-AnalysisOut: [v=1.0 c=1 a=0h-5IWwh-xUA:10 a=VphdPIyG4kEA:10 a=xupnbh4h0Y] X-AnalysisOut: [LOHZnncC45HQ==:17 a=37r9H06TAAAA:8 a=CbDCq_QkAAAA:8 a=atry] X-AnalysisOut: [Ysc7rmryh-OPUYMA:9 a=JNGGOthy03lzOL7N9hgA:7 a=P_uyKwu5Cnmm] X-AnalysisOut: [B8rpt0yf6ieLse4A:4 a=E3yz0KKPV6YA:10] Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter.kernel.org [140.211.167.41]); Mon, 16 Aug 2010 22:24:51 +0000 (UTC) From 9652f29170c7daabf2e9a62acb848a05dc71db9a Mon Sep 17 00:00:00 2001 From: Vu Pham Date: Tue, 10 Aug 2010 14:16:50 -0700 Subject: [PATCH 04/10] mlx4_core: Add interface to allocate fmr with pre-reserved MPTs As we did with MRs, the fmr_alloc() will call mr_alloc() to allocate bitmap and create MPTs entry. fmr_alloc_reserver will call mr_alloc_reserve() to create MPTs entry with pre-reserved range Signed-off-by: Oren Duer Signed-off-by: Vu Pham --- drivers/net/mlx4/mr.c | 55 +++++++++++++++++++++++++++++++++++++++++++ include/linux/mlx4/device.h | 4 +++ 2 files changed, 59 insertions(+), 0 deletions(-) diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index ba0514d..67d858f 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -655,6 +655,49 @@ err_free: } EXPORT_SYMBOL_GPL(mlx4_fmr_alloc); +int mlx4_fmr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, + u32 pd, u32 access, int max_pages, + int max_maps, u8 page_shift, struct mlx4_fmr *fmr) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + u64 mtt_seg; + int err = -ENOMEM; + + if (page_shift < (ffs(dev->caps.page_size_cap) - 1) || page_shift >= 32) + return -EINVAL; + + /* All MTTs must fit in the same page */ + if (max_pages * sizeof *fmr->mtts > PAGE_SIZE) + return -EINVAL; + + fmr->page_shift = page_shift; + fmr->max_pages = max_pages; + fmr->max_maps = max_maps; + fmr->maps = 0; + + err = mlx4_mr_alloc_reserved(dev, mridx, pd, 0, 0, access, max_pages, + page_shift, &fmr->mr); + if (err) + return err; + + mtt_seg = fmr->mr.mtt.first_seg * dev->caps.mtt_entry_sz; + + fmr->mtts = mlx4_table_find(&priv->mr_table.mtt_table, + fmr->mr.mtt.first_seg, + &fmr->dma_handle); + if (!fmr->mtts) { + err = -ENOMEM; + goto err_free; + } + + return 0; + +err_free: + mlx4_mr_free_reserved(dev, &fmr->mr); + return err; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_alloc_reserved); + int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr) { struct mlx4_priv *priv = mlx4_priv(dev); @@ -697,6 +740,18 @@ int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr) } EXPORT_SYMBOL_GPL(mlx4_fmr_free); +int mlx4_fmr_free_reserved(struct mlx4_dev *dev, struct mlx4_fmr *fmr) +{ + if (fmr->maps) + return -EBUSY; + + fmr->mr.enabled = 0; + mlx4_mr_free_reserved(dev, &fmr->mr); + + return 0; +} +EXPORT_SYMBOL_GPL(mlx4_fmr_free_reserved); + int mlx4_SYNC_TPT(struct mlx4_dev *dev) { return mlx4_cmd(dev, 0, 0, 0, MLX4_CMD_SYNC_TPT, 1000); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index da8ab85..3960033 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -474,11 +474,15 @@ void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index); int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, int npages, u64 iova, u32 *lkey, u32 *rkey); +int mlx4_fmr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u32 access, int max_pages, int max_maps, + u8 page_shift, struct mlx4_fmr *fmr); int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, int max_maps, u8 page_shift, struct mlx4_fmr *fmr); int mlx4_fmr_enable(struct mlx4_dev *dev, struct mlx4_fmr *fmr); void mlx4_fmr_unmap(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u32 *lkey, u32 *rkey); +int mlx4_fmr_free_reserved(struct mlx4_dev *dev, struct mlx4_fmr *fmr); int mlx4_fmr_free(struct mlx4_dev *dev, struct mlx4_fmr *fmr); int mlx4_SYNC_TPT(struct mlx4_dev *dev); -- 1.6.3.3