From patchwork Tue Feb 14 18:56:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9572539 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AA230601E7 for ; Tue, 14 Feb 2017 19:03:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8052628442 for ; Tue, 14 Feb 2017 19:03:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73E772845E; Tue, 14 Feb 2017 19:03:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D51D428442 for ; Tue, 14 Feb 2017 19:03:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753116AbdBNTDJ (ORCPT ); Tue, 14 Feb 2017 14:03:09 -0500 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:7954 "EHLO esa2.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753051AbdBNTDI (ORCPT ); Tue, 14 Feb 2017 14:03:08 -0500 X-IronPort-AV: E=Sophos;i="5.35,162,1483977600"; d="scan'208";a="81525959" Received: from unknown (HELO milsmgep14.sandisk.com) ([63.163.107.225]) by ob1.hgst.iphmx.com with ESMTP; 15 Feb 2017 03:07:27 +0800 Received: from MILHUBIP04.sdcorp.global.sandisk.com (Unknown_Domain [10.201.67.162]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 67.9B.18895.8E253A85; Tue, 14 Feb 2017 10:56:43 -0800 (PST) Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.319.2; Tue, 14 Feb 2017 10:56:40 -0800 X-AuditID: 0ac94371-d73ff700000049cf-88-58a352e8f6ca Received: from exp-402881.sdcorp.global.sandisk.com ( [10.177.9.6]) by (Symantec Messaging Gateway) with SMTP id 80.D6.18148.7E253A85; Tue, 14 Feb 2017 10:56:40 -0800 (PST) From: Bart Van Assche To: Doug Ledford CC: , Bart Van Assche , Israel Rukshin , Max Gurtovoy , Leon Romanovsky , Mark Bloch , Yuval Shaia , "# 4 . 7+" Subject: [PATCH v2 1/8] IB/SRP: Avoid using IB_MR_TYPE_SG_GAPS Date: Tue, 14 Feb 2017 10:56:29 -0800 Message-ID: <20170214185636.29250-2-bart.vanassche@sandisk.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170214185636.29250-1-bart.vanassche@sandisk.com> References: <20170214185636.29250-1-bart.vanassche@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrFLMWRmVeSWpSXmKPExsXCddJ5ke7roMURBpNP81m8PP+B1eLG4rUs Fif7/jFbPDvUy2Lx78QNNovTjw6wWyzY+IjRYtaMtSwOHB7Pph9m8vj49BaLx/t9V9k8Pm+S C2CJ4rJJSc3JLEst0rdL4Mp4sew1S8FFsYo/r5YzNTB2CXcxcnJICJhIHDu9grWLkYtDSGAp k8T7TZeYIZwdjBJHFs1ihal6sK6BGcQWEtjIKPH4TyKIzSZgJPHt/UwWEFtEQE1i06tF7CA2 s8AmJonNX61AbGEBO4nni+4ygtgsAqoSl+bsYwOxeQXsJRYfPQ41X15iV9tFMJtTwEFi5reF rBC77CXO9s9hBzlIQmAZq8SlJ5tZIZoFJU7OfMICsUxC4uCLF1DHqUucXDKfaQKj0CwkZbOQ lC1gZFrFKJabmVOcm55aYGiiV5yYl5JZnK2XnJ+7iRESA4U7GF/f9j7EKMDBqMTDe0J2cYQQ a2JZcWXuIUYJDmYlEV4NB6AQb0piZVVqUX58UWlOavEhRmkOFiVx3izZqRFCAumJJanZqakF qUUwWSYOTqkGxuIHBgmhz3hnH9bMu6rhsq4ybdZ3w1t9wnny6xamxk/9a/KdyW/HrsNL7Z/K xZfbLD36Prlv2aoTgq/9rPfOO8S+l9cxea1OUeDJ+cXLfzhGSz/+KR95Lf77vaOMvg88Ft9b u2KLG9N8hdVns+/Wycq9d58bu/fieluF2tylrN4sExn+Bm+XPaXEUpyRaKjFXFScCACxgxJt fQIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNJMWRmVeSWpSXmKPExsXCtZGTTfdF0OIIgx3HWC0O/mxjtHh5/gOr xY3Fa1ksTvb9Y7Z4dqiXxeLfiRtsFqcfHWC3WLDxEaPFrBlrWRw4PZ5NP8zk8fHpLRaP9/uu snlMW3OeyePzJrkA1igum5TUnMyy1CJ9uwSujBfLXrMUXBSr+PNqOVMDY5dwFyMnh4SAicSD dQ3MXYxcHEIC6xklTr5+wwySYBMwkvj2fiYLiC0ioCax6dUidpAiZoFtTBLvlm5gAkkIC9hJ PF90lxHEZhFQlbg0Zx8biM0rYC9x6OprFogN8hK72i6ygticAg4SM78tBLOFgGrO9s9hn8DI vYCRYRWjWG5mTnFuemaBoZFecWJeSmZxtl5yfu4mRkjwRO1gvD7R/BAjEwenVAPjwtXCTW3r nJxOrZguYJHDnvM4fFPz9u0n8n4/Xpzec2JuGoPs7dcHVs8q0NJcdCW854qtbMDbXNcsx2Rv Zvm4Mv/u98sKd91o7K1alleWtqhp5TqhrCO/7a17eV8djSvdu3Hf4/prt9siT2UV5s9boKP1 Z07dM55+sc+SwjIOzF9vC2pInTJXYinOSDTUYi4qTgQArTrZus4BAAA= MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Tests have shown that the following error message is reported when using SG-GAPS registration with an mlx5 adapter: scsi host1: ib_srp: failed RECV status WR flushed (5) for CQE ffff880bd4270eb0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0f007806 2500002a ad9fafd1 scsi host1: ib_srp: reconnect succeeded mlx5_0:dump_cqe:262:(pid 7369): dump error cqe 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0f007806 25000032 00105dd0 scsi host1: ib_srp: failed FAST REG status memory management operation error (6) for CQE ffff880b92860138 Hence avoid using SG-GAPS memory registrations. Additionally, always configure the blk_queue_virt_boundary() to avoid to trigger a mapping failure when using adapters that support SG-GAPS (e.g. mlx5). Fixes: commit ad8e66b4a801 ("IB/srp: fix mr allocation when the device supports sg gaps") Fixes: commit 509c5f33f4f6 ("IB/srp: Prevent mapping failures") Reported-by: Laurence Oberman Signed-off-by: Bart Van Assche Cc: Israel Rukshin Cc: Max Gurtovoy Cc: Leon Romanovsky Cc: Mark Bloch Cc: Yuval Shaia Cc: # 4.7+ --- drivers/infiniband/ulp/srp/ib_srp.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 79bf48477ddb..07e3613e4798 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -371,7 +371,6 @@ static struct srp_fr_pool *srp_create_fr_pool(struct ib_device *device, struct srp_fr_desc *d; struct ib_mr *mr; int i, ret = -EINVAL; - enum ib_mr_type mr_type; if (pool_size <= 0) goto err; @@ -385,13 +384,9 @@ static struct srp_fr_pool *srp_create_fr_pool(struct ib_device *device, spin_lock_init(&pool->lock); INIT_LIST_HEAD(&pool->free_list); - if (device->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG) - mr_type = IB_MR_TYPE_SG_GAPS; - else - mr_type = IB_MR_TYPE_MEM_REG; - for (i = 0, d = &pool->desc[0]; i < pool->size; i++, d++) { - mr = ib_alloc_mr(pd, mr_type, max_page_list_len); + mr = ib_alloc_mr(pd, IB_MR_TYPE_MEM_REG, + max_page_list_len); if (IS_ERR(mr)) { ret = PTR_ERR(mr); if (ret == -ENOMEM) @@ -2666,7 +2661,7 @@ static int srp_slave_alloc(struct scsi_device *sdev) struct srp_device *srp_dev = target->srp_host->srp_dev; struct ib_device *ibdev = srp_dev->dev; - if (!(ibdev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)) + if (true) blk_queue_virt_boundary(sdev->request_queue, ~srp_dev->mr_page_mask);