From patchwork Mon Aug 16 22:16:18 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vu Pham X-Patchwork-Id: 119815 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter.kernel.org (8.14.4/8.14.3) with ESMTP id o7GMOoIN011875 for ; Mon, 16 Aug 2010 22:24:51 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756756Ab0HPWYf (ORCPT ); Mon, 16 Aug 2010 18:24:35 -0400 Received: from p02c12o149.mxlogic.net ([208.65.145.82]:49046 "EHLO p02c12o149.mxlogic.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756646Ab0HPWYf (ORCPT ); Mon, 16 Aug 2010 18:24:35 -0400 Received: from unknown [63.251.237.3] (EHLO p02c12o149.mxlogic.net) by p02c12o149.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id 3aab96c4.5efe3940.35264.00-575.79318.p02c12o149.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:24:35 -0600 (MDT) X-MXL-Hash: 4c69baa33764d938-c7411b7531211d6f450608a1c6b768164e7a5c18 Received: from unknown [63.251.237.3] (EHLO mtiexch01.mti.com) by p02c12o149.mxlogic.net(mxl_mta-6.7.0-0) with ESMTP id 0b8b96c4.0.33716.00-399.75862.p02c12o149.mxlogic.net (envelope-from ); Mon, 16 Aug 2010 16:16:18 -0600 (MDT) X-MXL-Hash: 4c69b8b238d11b19-2fab510fc220eb41dd9a1d53d493969f25a956c6 Received: from vu-lt.mti.mtl.com ([10.2.1.17]) by mtiexch01.mti.com with Microsoft SMTPSVC(6.0.3790.3959); Mon, 16 Aug 2010 15:20:41 -0700 Message-ID: <4C69B8B2.40805@mellanox.com> Date: Mon, 16 Aug 2010 15:16:18 -0700 From: Vu Pham User-Agent: Thunderbird 2.0.0.23 (X11/20090812) MIME-Version: 1.0 To: Roland Dreier CC: OpenFabrics EWG , Linux RDMA , Oren Duer Subject: [PATCH v1 02/10] mlx4_core: Add mr_alloc_reserved to able create MPTs with pre-reserve range X-OriginalArrivalTime: 16 Aug 2010 22:20:41.0375 (UTC) FILETIME=[42FEFAF0:01CB3D91] X-Spam: [F=0.2000000000; CM=0.500; S=0.200(2010073001)] X-MAIL-FROM: X-SOURCE-IP: [63.251.237.3] X-AnalysisOut: [v=1.0 c=1 a=bnPmFLdnH3MA:10 a=VphdPIyG4kEA:10 a=xupnbh4h0Y] X-AnalysisOut: [LOHZnncC45HQ==:17 a=37r9H06TAAAA:8 a=CbDCq_QkAAAA:8 a=hws7] X-AnalysisOut: [JPDEeeRZMRUwFpkA:9 a=zK3fv6vGx3FxZOyQR9cA:7 a=peazlOdu96Hz] X-AnalysisOut: [QfE7QvjkhpFK4cwA:4 a=E3yz0KKPV6YA:10 a=1jyq7thjEr3XUkaB:21] X-AnalysisOut: [ a=Vae0QIffq_6NaJxG:21] Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter.kernel.org [140.211.167.41]); Mon, 16 Aug 2010 22:24:51 +0000 (UTC) From 7c943349148dcbcb173814a9b6800ac2a2b12626 Mon Sep 17 00:00:00 2001 From: Vu Pham Date: Tue, 10 Aug 2010 14:03:53 -0700 Subject: [PATCH 02/10] mlx4_core: Add mr_alloc_reserved to able create MPTs with pre-reserve range As we did with QPs, range of the MPTs are pre-reserved (the MPTs that are mapped for FEXCHs, 2*64K of them). We need to split the operation of allocating an MPT to two: . The allocation of a bit from the bitmap . The actual creation of the entry (and it's MTT). So, mr_alloc_reserved() is the second part, where you know which MPT number was allocated. mr_alloc() is the one that allocates a number from the bitmap. Normal users keep using the original mr_alloc(). For FEXCH, when we know the pre-reserved MPT entry, we call mr_alloc_reserved() directly. Same with the mr_free() and corresponding mr_free_reserved(). The first will just put back the bit, the later will actually destroy the entry, but will leave the bit set. Signed-off-by: Yevgeny Petrilin Signed-off-by: Oren Duer Signed-off-by: Vu Pham --- drivers/net/mlx4/mr.c | 35 +++++++++++++++++++++++++---------- include/linux/mlx4/device.h | 4 ++++ 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c index 9c188bd..35c0af6 100644 --- a/drivers/net/mlx4/mr.c +++ b/drivers/net/mlx4/mr.c @@ -263,6 +263,21 @@ static int mlx4_HW2SW_MPT(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox !mailbox, MLX4_CMD_HW2SW_MPT, MLX4_CMD_TIME_CLASS_B); } +int mlx4_mr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u64 iova, u64 size, u32 access, int npages, + int page_shift, struct mlx4_mr *mr) +{ + mr->iova = iova; + mr->size = size; + mr->pd = pd; + mr->access = access; + mr->enabled = 0; + mr->key = hw_index_to_key(mridx); + + return mlx4_mtt_init(dev, npages, page_shift, &mr->mtt); +} +EXPORT_SYMBOL_GPL(mlx4_mr_alloc_reserved); + int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, int npages, int page_shift, struct mlx4_mr *mr) { @@ -274,14 +289,8 @@ int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, if (index == -1) return -ENOMEM; - mr->iova = iova; - mr->size = size; - mr->pd = pd; - mr->access = access; - mr->enabled = 0; - mr->key = hw_index_to_key(index); - - err = mlx4_mtt_init(dev, npages, page_shift, &mr->mtt); + err = mlx4_mr_alloc_reserved(dev, index, pd, iova, size, + access, npages, page_shift, mr); if (err) mlx4_bitmap_free(&priv->mr_table.mpt_bitmap, index); @@ -289,9 +298,8 @@ int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, } EXPORT_SYMBOL_GPL(mlx4_mr_alloc); -void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) +void mlx4_mr_free_reserved(struct mlx4_dev *dev, struct mlx4_mr *mr) { - struct mlx4_priv *priv = mlx4_priv(dev); int err; if (mr->enabled) { @@ -303,6 +311,13 @@ void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) } mlx4_mtt_cleanup(dev, &mr->mtt); +} +EXPORT_SYMBOL_GPL(mlx4_mr_free_reserved); + +void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr) +{ + struct mlx4_priv *priv = mlx4_priv(dev); + mlx4_mr_free_reserved(dev, mr); mlx4_bitmap_free(&priv->mr_table.mpt_bitmap, key_to_hw_index(mr->key)); } EXPORT_SYMBOL_GPL(mlx4_mr_free); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index 7a7f9c1..66849cf 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h @@ -424,8 +424,12 @@ int mlx4_mtt_init(struct mlx4_dev *dev, int npages, int page_shift, void mlx4_mtt_cleanup(struct mlx4_dev *dev, struct mlx4_mtt *mtt); u64 mlx4_mtt_addr(struct mlx4_dev *dev, struct mlx4_mtt *mtt); +int mlx4_mr_alloc_reserved(struct mlx4_dev *dev, u32 mridx, u32 pd, + u64 iova, u64 size, u32 access, int npages, + int page_shift, struct mlx4_mr *mr); int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access, int npages, int page_shift, struct mlx4_mr *mr); +void mlx4_mr_free_reserved(struct mlx4_dev *dev, struct mlx4_mr *mr); void mlx4_mr_free(struct mlx4_dev *dev, struct mlx4_mr *mr); int mlx4_mr_enable(struct mlx4_dev *dev, struct mlx4_mr *mr); int mlx4_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt, -- 1.6.3.3