From patchwork Mon Aug 16 22:16:31 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vu Pham X-Patchwork-Id: 119813 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.4/8.14.3) with ESMTP id o7GMOoIK011875 for ; Mon, 16 Aug 2010 22:24:50 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756670Ab0HPWYH (ORCPT ); Mon, 16 Aug 2010 18:24:07 -0400 Received: from p02c12o144.mxlogic.net ([208.65.145.77]:40168 "EHLO p02c12o144.mxlogic.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756515Ab0HPWYH (ORCPT ); Mon, 16 Aug 2010 18:24:07 -0400 Received: from unknown [63.251.237.3] (EHLO p02c12o144.mxlogic.net) by p02c12o144.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id 68ab96c4.735ad940.45928.00-582.109873.p02c12o144.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:24:06 -0600 (MDT) X-MXL-Hash: 4c69ba863c25b66e-feaebd0de3e513d1b0c72db86ad9c75b2a5ea4a7 Received: from unknown [63.251.237.3] (EHLO mtiexch01.mti.com) by p02c12o144.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id eb8b96c4.0.44541.00-395.106024.p02c12o144.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:16:31 -0600 (MDT) X-MXL-Hash: 4c69b8bf234a982c-dfde452ff3e987a84855a125730dc53aaf931c1f Received: from vu-lt.mti.mtl.com ([10.2.1.17]) by mtiexch01.mti.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 16 Aug 2010 15:20:55 -0700 Message-ID: <4C69B8BF.8020400@mellanox.com> Date: Mon, 16 Aug 2010 15:16:31 -0700 From: Vu Pham User-Agent: Thunderbird 2.0.0.23 (X11/20090812) MIME-Version: 1.0 To: Roland Dreier CC: OpenFabrics EWG , Linux RDMA , Oren Duer Subject: [PATCH v1 05/10] mlx4_core: Map phys_fmr with first byte offset enabled X-OriginalArrivalTime: 16 Aug 2010 22:20:55.0250 (UTC) FILETIME=[4B442320:01CB3D91] X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010073001)] X-MAIL-FROM: X-SOURCE-IP: [63.251.237.3] X-AnalysisOut: [v=1.0 c=1 a=PnGX4CQX4h8A:10 a=VphdPIyG4kEA:10 a=xupnbh4h0Y] X-AnalysisOut: [LOHZnncC45HQ==:17 a=37r9H06TAAAA:8 a=CbDCq_QkAAAA:8 a=6vAo] X-AnalysisOut: [BKL4FSQwcemzUPEA:9 a=Tsaz87t9c2NR-lgyFlwA:7 a=xFztDC6iIEHj] X-AnalysisOut: [SNfuQXKmnWVOwPUA:4 a=E3yz0KKPV6YA:10 a=71aK_Kh9bfNoKluU:21] X-AnalysisOut: [ a=wXT0F3NFJVS4lOLh:21] Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter.kernel.org [140.211.167.41]); Mon, 16 Aug 2010 22:24:50 +0000 (UTC) From 1afc1f9cfe33444c26dc027556035c7c85a9a565 Mon Sep 17 00:00:00 2001 From: Vu Pham Date: Tue, 10 Aug 2010 14:27:23 -0700 Subject: [PATCH 05/10] mlx4_core: Map phys_fmr with first byte offset enabled map_phys_fmr_fbo() is very much like the original map_phys_fmr(): . allows setting an FBO (First Byte Offset) for the MPT . allows setting the data length for the MPT . does not increase the higher bits of the key after every map. Signed-off-by: Oren Duer Signed-off-by: Vu Pham --- drivers/net/mlx4/mr.c | 28 +++++++++++++++++++++++----- include/linux/mlx4/device.h | 3 +++ 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 67d858f..42527b5 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -52,7 +52,9 @@ struct mlx4_mpt_entry { __be64 length; __be32 lkey; __be32 win_cnt; - u8 reserved1[3]; + u8 reserved1; + u8 flags2; + u8 reserved2; u8 mtt_rep; __be64 mtt_seg; __be32 mtt_sz; @@ -71,6 +73,8 @@ struct mlx4_mpt_entry { #define MLX4_MPT_PD_FLAG_RAE (1 << 28) #define MLX4_MPT_PD_FLAG_EN_INV (3 << 24) +#define MLX4_MPT_FLAG2_FBO_EN (1 << 7) + #define MLX4_MPT_STATUS_SW 0xF0 #define MLX4_MPT_STATUS_HW 0x00 @@ -566,8 +570,9 @@ static inline int mlx4_check_fmr(struct mlx4_fmr *fmr, u64 *page_list, return 0; } -int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, - int npages, u64 iova, u32 *lkey, u32 *rkey) +int mlx4_map_phys_fmr_fbo(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u64 *page_list, int npages, u64 iova, u32 fbo, + u32 len, u32 *lkey, u32 *rkey, int same_key) { u32 key; int i, err; @@ -579,7 +584,8 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list ++fmr->maps; key = key_to_hw_index(fmr->mr.key); - key += dev->caps.num_mpts; + if (!same_key) + key += dev->caps.num_mpts; *lkey = *rkey = fmr->mr.key = hw_index_to_key(key); *(u8 *) fmr->mpt = MLX4_MPT_STATUS_SW; @@ -598,8 +604,10 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list fmr->mpt->key = cpu_to_be32(key); fmr->mpt->lkey = cpu_to_be32(key); - fmr->mpt->length = cpu_to_be64(npages * (1ull << fmr->page_shift)); + fmr->mpt->length = cpu_to_be64(len); fmr->mpt->start = cpu_to_be64(iova); + fmr->mpt->first_byte_offset = cpu_to_be32(fbo & 0x001fffff); + fmr->mpt->flags2 = (fbo ? MLX4_MPT_FLAG2_FBO_EN : 0); /* Make MTT entries are visible before setting MPT status */ wmb(); @@ -611,6 +619,16 @@ int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list return 0; } +EXPORT_SYMBOL_GPL(mlx4_map_phys_fmr_fbo); + +int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, + int npages, u64 iova, u32 *lkey, u32 *rkey) +{ + u32 len = npages * (1ull << fmr->page_shift); + + return mlx4_map_phys_fmr_fbo(dev, fmr, page_list, npages, iova, 0, + len, lkey, rkey, 0); +} EXPORT_SYMBOL_GPL(mlx4_map_phys_fmr); int mlx4_fmr_alloc(struct mlx4_dev *dev, u32 pd, u32 access, int max_pages, diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 3960033..ae09787 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -472,6 +472,9 @@ void mlx4_unregister_mac(struct mlx4_dev *dev, u8 port, int index); int mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index); void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index); +int mlx4_map_phys_fmr_fbo(struct mlx4_dev *dev, struct mlx4_fmr *fmr, + u64 *page_list, int npages, u64 iova, u32 fbo, + u32 len, u32 *lkey, u32 *rkey, int same_key); int mlx4_map_phys_fmr(struct mlx4_dev *dev, struct mlx4_fmr *fmr, u64 *page_list, int npages, u64 iova, u32 *lkey, u32 *rkey); int mlx4_fmr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, -- 1.6.3.3