From patchwork Thu Apr 13 15:56:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 9679679 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 39F9C60383 for ; Thu, 13 Apr 2017 15:56:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2CE6128658 for ; Thu, 13 Apr 2017 15:56:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 21BCD2866A; Thu, 13 Apr 2017 15:56:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CEC3928658 for ; Thu, 13 Apr 2017 15:56:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751657AbdDMP41 (ORCPT ); Thu, 13 Apr 2017 11:56:27 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:33372 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751960AbdDMP4Z (ORCPT ); Thu, 13 Apr 2017 11:56:25 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from maxg@mellanox.com) with ESMTPS (AES256-SHA encrypted); 13 Apr 2017 18:56:18 +0300 Received: from r-vnc08.mtr.labs.mlnx (r-vnc08.mtr.labs.mlnx [10.208.0.121]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id v3DFuHox028607; Thu, 13 Apr 2017 18:56:18 +0300 From: Max Gurtovoy To: linux-nvme@lists.infradead.org, keith.busch@intel.com, hch@lst.de, linux-rdma@vger.kernel.org, leon@kernel.org Cc: vladimirk@mellanox.com, maxg@mellanox.com Subject: [PATCH v2 4/4] nvme-rdma: add support for arbitrary sg lists mapping Date: Thu, 13 Apr 2017 18:56:17 +0300 Message-Id: <1492098977-5231-5-git-send-email-maxg@mellanox.com> X-Mailer: git-send-email 1.7.8.2 In-Reply-To: <1492098977-5231-1-git-send-email-maxg@mellanox.com> References: <1492098977-5231-1-git-send-email-maxg@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If the device support arbitrary sg list mapping (device cap IB_DEVICE_SG_GAPS_REG set) we allocate the memory regions with IB_MR_TYPE_SG_GAPS and allow the block layer to send us gappy bio vectors. We use this functionality in iSER driver as well. Signed-off-by: Max Gurtovoy Reviewed-by: Leon Romanovsky --- drivers/nvme/host/rdma.c | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index 47a479f..1c6f974 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -294,8 +294,8 @@ static int nvme_rdma_reinit_request(void *data, struct request *rq) ib_dereg_mr(req->mr); - req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG, - ctrl->max_fr_pages); + req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(dev->dev), + ctrl->max_fr_pages); if (IS_ERR(req->mr)) { ret = PTR_ERR(req->mr); req->mr = NULL; @@ -348,8 +348,8 @@ static int __nvme_rdma_init_request(struct nvme_rdma_ctrl *ctrl, if (ret) return ret; - req->mr = ib_alloc_mr(dev->pd, IB_MR_TYPE_MEM_REG, - ctrl->max_fr_pages); + req->mr = ib_alloc_mr(dev->pd, ib_get_sg_mr_type(ibdev), + ctrl->max_fr_pages); if (IS_ERR(req->mr)) { ret = PTR_ERR(req->mr); goto out_free_qe; @@ -1608,6 +1608,9 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl) ctrl->ctrl.sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap) + 1, ctrl->ctrl.sqsize); + if (ctrl->device->dev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG) + ctrl->ctrl.sg_gaps_support = true; + error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->cap); if (error) goto out_cleanup_queue;