From patchwork Wed May 11 00:16:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9064161 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 7C2B59F1D3 for ; Wed, 11 May 2016 00:16:26 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 106C720172 for ; Wed, 11 May 2016 00:16:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8D91020160 for ; Wed, 11 May 2016 00:16:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752278AbcEKAQW (ORCPT ); Tue, 10 May 2016 20:16:22 -0400 Received: from mail-bl2on0091.outbound.protection.outlook.com ([65.55.169.91]:43731 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751049AbcEKAQV (ORCPT ); Tue, 10 May 2016 20:16:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandiskcorp.onmicrosoft.com; s=selector1-sandisk-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=nh/j0a/qSAwbdJSP2ZzFVGdlpo2CuFNDxVPyTygur8M=; b=Mjo6QB6LvNFboIRRGtcakXt2t6y+w/jNh0iaZKRUbKpxvrI4t2Xnc5C/oXStxXH+nO0nyrQUJXkPyxCAK8+rWl77sxjpnqNK+iJ5lPEeUZhTIP9xNx2NBPc40lta70+Ajwz+qOEnNTLxlZvYYyLJ0DnwgNVAvNhp+74VOiwqNqY= Received: from BLUPR02CA040.namprd02.prod.outlook.com (10.160.23.158) by CY1PR0201MB1867.namprd02.prod.outlook.com (10.163.56.13) with Microsoft SMTP Server (TLS) id 15.1.492.11; Wed, 11 May 2016 00:16:17 +0000 Received: from BN1BFFO11FD005.protection.gbl (2a01:111:f400:7c10::1:188) by BLUPR02CA040.outlook.office365.com (2a01:111:e400:8ad::30) with Microsoft SMTP Server (TLS) id 15.1.492.11 via Frontend Transport; Wed, 11 May 2016 00:16:18 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.225) smtp.mailfrom=sandisk.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=bestguesspass action=none header.from=sandisk.com; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.225 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.225; helo=milsmgep14.sandisk.com; Received: from milsmgep14.sandisk.com (63.163.107.225) by BN1BFFO11FD005.mail.protection.outlook.com (10.58.144.68) with Microsoft SMTP Server (TLS) id 15.1.492.8 via Frontend Transport; Wed, 11 May 2016 00:16:16 +0000 Received: from MILHUBIP03.sdcorp.global.sandisk.com (Unknown_Domain [172.22.12.162]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id F7.61.04010.43972375; Tue, 10 May 2016 17:13:40 -0700 (PDT) Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.279.2; Tue, 10 May 2016 17:16:15 -0700 X-AuditID: ac160c71-327ff70000000faa-10-573279343e87 Received: from exp-402881.sandisk.com ( [10.177.8.100]) by (Symantec Messaging Gateway) with SMTP id E8.14.03935.FC972375; Tue, 10 May 2016 17:16:15 -0700 (PDT) Subject: [PATCH 6/6] IB/srp: Prevent mapping failures To: Doug Ledford References: <573278D9.4050908@sandisk.com> CC: Christoph Hellwig , Sagi Grimberg , Laurence Oberman , "linux-rdma@vger.kernel.org" From: Bart Van Assche Message-ID: <573279CF.4060006@sandisk.com> Date: Tue, 10 May 2016 17:16:15 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <573278D9.4050908@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrILMWRmVeSWpSXmKPExsWyRoxnka5JpVG4wfpbhhYvz39gtVi5+iiT xbNDvSwW5z+8YrJY9/o9iwOrx/l7G1k8dt9sYPN4v+8qm8fnTXIBLFFcNimpOZllqUX6dglc GROXbmAu+Odf8b11LVMDY5djFyMnh4SAicSuJ9dZuhi5OIQE5jJJvLuxkBXC2cEo8WbGD0aY qlWT5kNVLWWUuPVhFnMXIweHsICpROOxRJAaEQE1iU2vFrGD2EICWhIr2+eDDWIWWM0o8WXZ KyaQBJuAkcS39zNZQGxeoKItNzrBGlgEVCXufZwIFhcViJBYve4aM0SNoMTJmU/A4pwC2hL/ d/1nA9nLLKApsX6XPkiYWUBeYvvbOcwQd25jlVi3ngfiBnWJk0vmM01gFJ6FZNIshO5ZSLoX MDKvYhTLzcwpzk1PLTA00StOzEvJLM7WS87P3cQIiYjCHYyvb3sfYhTgYFTi4Y2oMAoXYk0s K67MPcQowcGsJMKbUwwU4k1JrKxKLcqPLyrNSS0+xCjNwaIkznsx+mOYkEB6YklqdmpqQWoR TJaJg1OqgZH7qmSU7aNTgd4zd/9X/lLCW7mUZfaxa3fXZ0w7ItT0+GU4l21+2Z5p02/bPr/U 53nBKunQa6aprPKrN1zymM12qT1yxxo21Qb5769VtnaKnUhednxR4k7zZ7xHP1xzCDXbcFDW 4onKJOYdMX9+cHEVpXy+pn5mi6WxxnuVlK//nJgv7F4ZxMWgxFKckWioxVxUnAgA0yLfLIQC AAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJJMWRmVeSWpSXmKPExsXCtZEjRfd8pVG4wZ61KhYvz39gtVi5+iiT xbNDvSwW5z+8YrJY9/o9iwOrx/l7G1k8dt9sYPN4v+8qm8fnTXIBLFFcNimpOZllqUX6dglc GROXbmAu+Odf8b11LVMDY5djFyMnh4SAicSqSfNZuhi5OIQEFjNKXP6ynq2LkYNDWMBUovFY IkiNiICaxKZXi9hBbCEBLYmV7fNZQeqZBVYzStzr+8UIkmATMJL49n4mC4jNC1S05UYnWAOL gKrEvY8TweKiAhESq9ddY4aoEZQ4OfMJWJxTQFvi/67/bCA2s4C6xJ95l5ghbHmJ7W/nME9g 5JuFpGUWkrJZSMoWMDKvYhTLzcwpzk3PLDA00itOzEvJLM7WS87P3cQIDkzOqB2M1yeaH2Jk 4uCUamD0vRVRspXxxmlGXaeoaSoy+Wt5s9K9LPc+n737/MNSaS7XF9IZb+tTv/IeVHWbd6x+ /fm+T+9fam7fcjvxgYvBo9oV4e688/8GnPPRkzlkz2blYPVORj8uvaLrVKh+gGw45z7Bf8nn b81ZNeVPOJ8c0wLGaL76Fq+S3b8jPZXFPsy69zDvcZoSS3FGoqEWc1FxIgA/Puzw/AEAAA== X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:63.163.107.225; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(199003)(189002)(9170700003)(87266999)(65816999)(54356999)(33656002)(47776003)(76176999)(50986999)(106466001)(53416004)(23676002)(4326007)(189998001)(4001350100001)(81166006)(86362001)(6806005)(230700001)(110136002)(65806001)(65956001)(92566002)(2950100001)(2906002)(36756003)(59896002)(50466002)(99136001)(8936002)(11100500001)(5008740100001)(19580395003)(229853001)(1220700001)(19580405001)(77096005)(87936001)(5003600100002)(586003)(80316001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0201MB1867; H:milsmgep14.sandisk.com; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD005; 1:p36QErD5i5EQ8Y7pq/cvicHyPQQopUk+hYIuADW2Mca32fewkVmzGX8YuYqNTTOUh6RbsSDBY771DyJGImWb3Kh9T2LuqLzjmck5tOGimmQwJM4Tpwp0enMVS87DfKmf2+ZoihMvzyH5jvWVPhKvJ5ap5MLt5fjZ218m5hkB/2QKRzAVLiNtJmhYE5/SNrbXYKh55pAoi1Z2/Lny6xBHG3RQAawFXPe7vi2p9lJIKxffCUAU8Jgh+88vcgAuAyA8ndE9ujW8KPjLScLOnMq8DcY7Jw9AmLY1oR9/CLm98jOq84gUw6LX9D4Hpu2vvwsFDfKb0vkt6fnhEX2FmbubGU4zOydYWBZcg5rWbdAj0sxRXqDV6rXFuio3YoA5FPi+fQ6PBdKTCM+22x+Rz+CTWRDLiZYHk1le0xsoHzUZGMcVgzMzSzoHMaeDMqtVK5MD X-MS-Office365-Filtering-Correlation-Id: 1679c38c-73b4-4154-9e3d-08d3793178b6 X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1867; 2:hBsm4yqxvYGbtgK0rssDBm8/cSgna7J2ZB5QlC/Y3cOuIUfHxj2vL6I59JUGAmtFX5u+/HSl6MDJfMzeORv4uOQ355Gm79BVnfUHfqI1/zJHwMIIb4n1wzfR9rKHzQAxtxLGmVGQLKgRuKhkhdN6cRKvm4av1nGvvLSLpknkXVMJXoS+yd9R90sQESy4POyF; 3:HeCBBKK11DOi5tKRefARF+tHhtmsJa5d9UavEYeb4rt8svoia3xa+rzylBxpx3CaZoPAvuY2ChXmAFMa0nCZzwluWuQQGj2Malzwwq1MH3yRWJ5k+FjVmGNxO89yKp8XWHJCNEqX9ZuXab7EqiZ6px87+X6qxXhTFPP083OpuGJH1ogARlrz9PfsTHNmieZBJerMPQCMhNVEsbIM9CzfJxuwT9hNPbYqjCC9cdJrosHgT1uc+FkUSBbtPGseS24SJG+Pm6f/px9uvfNGSt40eg== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:CY1PR0201MB1867; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1867; 25:If3ybaWq0mF+UUDX+XxzocGc46+ZJErT2ZOhyC2lBFM7+Yclzve+1JMg3HB7Fa8TOpQ7yDkNaHgwLTwbL0l4iznsr4Hejw02+Vfb0j9d3+NLYgeJ/7opCrgCxNztwHaig3QD9Wh8wrpF5dY/Fd8ZwXFwLx7K7K2RysffLA1MKze2ME1ZKbX+R0TIxQ84RFxcu/VaTLoT6egVJWJ+Ye4NO1Xl7Y+yfQI1U+aFm5vG+gPbKbV/h3SM2oIA+xVPW3SPFGptx89Vabcb7lOptmY/TO9/QzDXmOjJzyQx5/jteYDMBqs5AwxoWpKVcqnoFVfeamHuLWGhI+IWC1Ste8AFJManevLNQIVKDt1F3zuPBoNHPAN3MOLcPPm9zNaetaUQZSEidVDSnfdZewt605iMtymP3KrhStkHvfCL+ICPi6jH2pOG+35VtboSdKYsPBT+bihlNsSoONChv1aoQREI6RSSoOksziJlzeDT/yw4F3ftpSab8TlQou9qA8PiR9oW0aZBbz9zdQY3AKrWk+BM3PnBUu8CQ1TeXa+7PZEC4kNqu8hAK/hQFdt9woY3LJZxh7hfIjdIatcTA3LxJQz0FLmBtmJcSpkdYeqVINmUP4zg+kT5/iVt8t5B0ierLK16ybL6bg2b7aV2pQIM+axSQl/SQ2dwsshdwYAl7ZtWMpKQclEVpmMD9e5btzYJt1AMxli4n1jKJWE2Vk4F2SgzPo0OMllGXqhwe/NAAsRuDgoZPa4EXbdmL4jDdyhoom5T5yS/qM2iofuW3+dk6gvT2WgAc8pebohktnz5ljBaDJA= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1867; 20:wIdxm0cGOoUfAyx4y3TnBFkeU74gbHIN7VQWd2ZBlLuORmQf71s1xmHGmOgNWRiOMz/yEIOH4AgifTWyzyaIR1lqkAOqtOov5aJSviDMjPXvoEpxVDklEq39Jqz7+9hdEtA5J4mYPFVrYzcE7/W4kg+kX0s+ppoYXsl6CN4nG/k6T+nQfwsUVq12Ry741PcnPR7rq/nKS9F4qnaasnm+5y6H6j4gdmTC+r2MgSTb5cZt4GAG7p1cfJ42xMCtMLS3oT3j5m9VrouCjXwMaHABQqDpQrRjgBWEspt1853qaOa10FJARoA9w3l98Sx25i2q9fXdxmI6V0m9/P43X8ntkDVeClzFywBIEwNfjZsTKmtBu2UMkVSg4b4x+z+5S50yw3/aohXr+yMaeXOuXUzb0FzaJA38f91J4dNUcMo3amLt7PuBebCnHKNo1K6A2T8/gdWo/BBZmntuTw+bQozwyk/2cYwXHZFWs5ZsPMaYin08RTOrk+Ye0XZNCyItUaMS; 4:z2/HS6WEZjBvwmN1VvfHrfa72KSJ4omMLfEQav6DcUai/mSk+NX/iXci+iZHMT0Qb1PKC6nU/E9h3okjLVD5sp7T8s31+24U6dlnD/dbNOo5asfoo3PB5JjqO1m2DzvjekJJZN6RbdKWXPQXtCnT44xor19roXxAc5rsICrbROtz/fVlbtnFjOe6C+YfiRjpihcBDQ1ePeg5NN5Percw/2Omk9OzApUPxfqhwGU7pOHHqYvDnDDzNhRS7ir1dHsfZANBplbNvgR4O9PCounN4vlTI5+v7Ga4LfAAPorJVmd8VfBbc+8pYd8W9/9Yjc3xNialIOvIXWJMukUeleAJm1sxjLPjzrFEZ7wXi9/XmpHcL9bVUutQqT4vIj7Cs0Edubp50Y2L/H+Pl5ayo6CI5Ch8dXrLgClqOytU PpDrXM8= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(3002001)(10201501046)(6055026); SRVR:CY1PR0201MB1867; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0201MB1867; X-Forefront-PRVS: 0939529DE2 X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTFQUjAyMDFNQjE4Njc7MjM6T2x2SVREbWpqWVdhSmpSVkk5bi9SdnNF?= =?utf-8?B?RndXQm9pVURBNzRuemJURGhRbDBVbEUzOTc5SzY2a0NIelA3MnBxRHE2elRF?= =?utf-8?B?c3psWmVGc3hnbTQ3Mk5BaDJQZUZzTUFxdUZUcmJPRDRLaTUzNXYyYkczdG1T?= =?utf-8?B?eURHNTVaZnlHTFI5UENicGRRY0JpdGpoa2RaeXkrZ0lhS0pvL256UUx6L0FZ?= =?utf-8?B?TS83djVUK3FJZ25HMmpCa1pUSTlpcWpiZGZPbmpVeXF1V0NVTGZFWXRHQjJ3?= =?utf-8?B?UlZ3OElKd1NDM3JqTVdXNGpKaE4yWVhmUlhMWm1Vb1NvNFdheGFDSkV6Qm84?= =?utf-8?B?akRjVmhLSXNieDhZemdjZ2Q0VGwyZE5HUk9PbUxVTjVGZkN2K0wySVp2WXJG?= =?utf-8?B?UFB1TXlXaDlVY0hpOE5vZXpRNkhZR2hmSnpadkZ3bXF5Qms3Y2JBWnNUZnlm?= =?utf-8?B?c1h3ZVh6YnBEdURaTHhxYlV0VVd1OWd5NWIvdUhSU095RWFZY3BIOFYyU2hY?= =?utf-8?B?VklscVlCdnpYR1h1VnZtb3k1cmIwZjB0VFlTRWdGNWdKZXhyMlZIcXZodi9M?= =?utf-8?B?R0lBRkIwMU90dE9Oa3RuTWhRZVhLdERURHI4TjVOTEZCZVVhd214ZUFFN3Uv?= =?utf-8?B?azBmNnA4SlV0WFVqZzlZQzNCYjVqUGd0TlEzRzltdmRqV0cxcG5wRkZadU9O?= =?utf-8?B?N1ZzbFQxaC81UzV3VE5kRXlOaGV0NHhBNEQwK3VIODI3cURHY3E2KzVrRVlQ?= =?utf-8?B?S2tzUytoczhNd0MwNkVHSnlFeDR2eHRkVDNUWlhmeU1JQ1hEdE12WFZwVnlz?= =?utf-8?B?djdoYy9DajJ1WENBOEthQTlZS3lSNlJmK3VUd0pNN1daWjk5bEpzbkdmaHhD?= =?utf-8?B?TFZ6cUlsOWI1b1p1SHdnT3NDR3ZrZVMrNmNMekRkamhYcDdVU3RsamxGLzJ3?= =?utf-8?B?VE9nM0ZjSlVwSWdoaHp5WFhRUFh4aE5JSU1GM1pEV1RieHl5R2lnOTFBMUFP?= =?utf-8?B?c3haelRuRU9jMjFSNm1VYlFQV3NYd2FFWVEzaXpNaGVKVUp0U3ZiMVQzR2Zw?= =?utf-8?B?VktwYVI4M0w5TGtIektPaVBGeTRjQ0V5MnlqUDQxQkxrakRBcmg2ZHgwQ1li?= =?utf-8?B?YjdYaWRHNXpETkw3SjNlb204Y25pTithSko0NEZYbzByc3o4Z0pkaFRBaHRy?= =?utf-8?B?RHBqWGtqSTZXNE5vTFJQckFKWnJpbmZRTGlJVVQ0VDdZbklOaEllTmZQb1FL?= =?utf-8?B?TlpMQUhuODcxMzVlZk00OVI1aDh6dVFLM2MwTzdWRkpsMGxCeUllaHVqQVl5?= =?utf-8?B?N3Q2ejVrTmRWNjVLSTV3dVIzMmhSaktDOHNDejhHOVdoNmJvMG9tU2xxcW1u?= =?utf-8?B?R3lPSjBkRVVUaU9lLyt3NVVKMGhxQ2ZrekFuN25ZdWQ2ZHRGdG9Vc1o1blhJ?= =?utf-8?B?WEtielhvQlozRklMcXZiM1dhMTZsRnk4NVlORzdySzZlZkFJbUJGQU1EQmFv?= =?utf-8?B?Q2RpSUJOMDRYbXVjbUF2WjFRdkdXWDV2cktUZmViZlVjVENpUmtDRi9jRHNp?= =?utf-8?B?MGc1cllYUFVHQmJaOVN0cU8wczZUdERvUT09?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1867; 5:UNyXqb9oyIf2yrtzdKs+/touyhWcxuu6f8Dm5n6sDZ7sUYYjDlqjIa1Gf3wxKMArzEDlfEph58N671N6QJ7Q/BG4YGMXsm74yu9H4GifpYGx8sASvYNe3lt8AMsr9yaWRtw4jwL+LQUdjNqYcdQaNQ==; 24:0eZv7EJKpexWg/rjuLunuAB6dCOSee7bTJXwa10HlZZ/GkrV0PgpH7yUXrAwYlv3XpojjkwEW/myo84cWGtEqbyEP0uX1fMRsYuTag7ewQY=; 7:82U1xMEsK8xu/JRHlc+NIz/uUF5Lq4MW8Lgpldo8r3QGkpYHkrRPip+jp/yXagzCU7cRP3vJn/aGHMLgFPRhDHerXgpCqvfIJ26LGreOJd9uLBKtmHdMun54vCPpRMRZH7KZxpHLpKBXEF2KbI4TClcLni4cPn3avC/o0n+3wbwAR3x4Jbh3qMauSpcFQg+m; 20:W3v8gCnmLyIHssloKXb10ax/uY9MZDX6Hao3A5ukCv9a6TCngO+2YkN3hNBHP21Qzv9oKd7J7o1GDxHqkbidXfkcpCLDAn8Y20C41p5QZ4YT34OE/n4D41katlK3cwCztM7BRp4vbRi2Ig3x4/0V7h6kTis6y5/n0Lbo9IGPdjsooS1HHUuXM3bQX7qVfTlhFeLWATvNcUvs8+uuP+rzP8NEpKUNsx7RBFrDAibTOw1oXlZN0Wjr0jGAUq6qdPlC SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 May 2016 00:16:16.9036 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.225]; Helo=[milsmgep14.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0201MB1867 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If both max_sectors and the queue_depth are high enough it can happen that the MR pool is depleted temporarily. This causes the SRP initiator to report mapping failures. Although the SRP initiator recovers from such mapping failures, prevent that this can happen by allocating more memory regions. Additionally, only enable memory registration if at least two pages can be registered per memory region. Reported-by: Laurence Oberman Signed-off-by: Bart Van Assche Cc: Christoph Hellwig Cc: Sagi Grimberg --- drivers/infiniband/ulp/srp/ib_srp.c | 102 +++++++++++++++++++++++++++++------- drivers/infiniband/ulp/srp/ib_srp.h | 1 + 2 files changed, 84 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index e530a77..075cd7a 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -468,7 +468,7 @@ static int srp_create_ch_ib(struct srp_rdma_ch *ch) struct ib_qp *qp; struct ib_fmr_pool *fmr_pool = NULL; struct srp_fr_pool *fr_pool = NULL; - const int m = dev->use_fast_reg ? 3 : 1; + const int m = 1 + dev->use_fast_reg * target->mr_per_cmd * 2; int ret; init_attr = kzalloc(sizeof *init_attr, GFP_KERNEL); @@ -849,7 +849,7 @@ static int srp_alloc_req_data(struct srp_rdma_ch *ch) for (i = 0; i < target->req_ring_size; ++i) { req = &ch->req_ring[i]; - mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *), + mr_list = kmalloc(target->mr_per_cmd * sizeof(void *), GFP_KERNEL); if (!mr_list) goto out; @@ -1298,9 +1298,16 @@ static void srp_reg_mr_err_done(struct ib_cq *cq, struct ib_wc *wc) srp_handle_qp_err(cq, wc, "FAST REG"); } +/* + * Map up to sg_nents elements of state->sg where *sg_offset_p is the offset + * where to start in the first element. If sg_offset_p != NULL then + * *sg_offset_p is updated to the offset in state->sg[retval] of the first + * byte that has not yet been mapped. + */ static int srp_map_finish_fr(struct srp_map_state *state, struct srp_request *req, - struct srp_rdma_ch *ch, int sg_nents) + struct srp_rdma_ch *ch, int sg_nents, + unsigned int *sg_offset_p) { struct srp_target_port *target = ch->target; struct srp_device *dev = target->srp_host->srp_dev; @@ -1316,9 +1323,13 @@ static int srp_map_finish_fr(struct srp_map_state *state, WARN_ON_ONCE(!dev->use_fast_reg); if (sg_nents == 1 && target->global_mr) { - srp_map_desc(state, sg_dma_address(state->sg), - sg_dma_len(state->sg), + unsigned int sg_offset = sg_offset_p ? *sg_offset_p : 0; + + srp_map_desc(state, sg_dma_address(state->sg) + sg_offset, + sg_dma_len(state->sg) - sg_offset, target->global_mr->rkey); + if (sg_offset_p) + *sg_offset_p = 0; return 1; } @@ -1329,15 +1340,18 @@ static int srp_map_finish_fr(struct srp_map_state *state, rkey = ib_inc_rkey(desc->mr->rkey); ib_update_fast_reg_key(desc->mr, rkey); - n = ib_map_mr_sg(desc->mr, state->sg, sg_nents, NULL, dev->mr_page_size); + n = ib_map_mr_sg(desc->mr, state->sg, sg_nents, sg_offset_p, + dev->mr_page_size); if (unlikely(n < 0)) { srp_fr_pool_put(ch->fr_pool, &desc, 1); - pr_debug("%s: ib_map_mr_sg(%d) returned %d.\n", + pr_debug("%s: ib_map_mr_sg(%d, %d) returned %d.\n", dev_name(&req->scmnd->device->sdev_gendev), sg_nents, - n); + sg_offset_p ? *sg_offset_p : -1, n); return n; } + WARN_ON_ONCE(desc->mr->length == 0); + req->reg_cqe.done = srp_reg_mr_err_done; wr.wr.next = NULL; @@ -1358,8 +1372,10 @@ static int srp_map_finish_fr(struct srp_map_state *state, desc->mr->length, desc->mr->rkey); err = ib_post_send(ch->qp, &wr.wr, &bad_wr); - if (unlikely(err)) + if (unlikely(err)) { + WARN_ON_ONCE(err == -ENOMEM); return err; + } return n; } @@ -1416,7 +1432,7 @@ static int srp_map_sg_fmr(struct srp_map_state *state, struct srp_rdma_ch *ch, state->pages = req->map_page; state->fmr.next = req->fmr_list; - state->fmr.end = req->fmr_list + ch->target->cmd_sg_cnt; + state->fmr.end = req->fmr_list + ch->target->mr_per_cmd; for_each_sg(scat, sg, count, i) { ret = srp_map_sg_entry(state, ch, sg, i); @@ -1435,8 +1451,10 @@ static int srp_map_sg_fr(struct srp_map_state *state, struct srp_rdma_ch *ch, struct srp_request *req, struct scatterlist *scat, int count) { + unsigned int sg_offset = 0; + state->fr.next = req->fr_list; - state->fr.end = req->fr_list + ch->target->cmd_sg_cnt; + state->fr.end = req->fr_list + ch->target->mr_per_cmd; state->sg = scat; if (count == 0) @@ -1445,7 +1463,7 @@ static int srp_map_sg_fr(struct srp_map_state *state, struct srp_rdma_ch *ch, while (count) { int i, n; - n = srp_map_finish_fr(state, req, ch, count); + n = srp_map_finish_fr(state, req, ch, count, &sg_offset); if (unlikely(n < 0)) return n; @@ -1509,9 +1527,10 @@ static int srp_map_idb(struct srp_rdma_ch *ch, struct srp_request *req, #ifdef CONFIG_NEED_SG_DMA_LENGTH idb_sg->dma_length = idb_sg->length; /* hack^2 */ #endif - ret = srp_map_finish_fr(&state, req, ch, 1); + ret = srp_map_finish_fr(&state, req, ch, 1, NULL); if (ret < 0) return ret; + WARN_ON_ONCE(ret < 1); } else if (dev->use_fmr) { state.pages = idb_pages; state.pages[0] = (req->indirect_dma_addr & @@ -2579,6 +2598,20 @@ static int srp_reset_host(struct scsi_cmnd *scmnd) return srp_reconnect_rport(target->rport) == 0 ? SUCCESS : FAILED; } +static int srp_slave_alloc(struct scsi_device *sdev) +{ + struct Scsi_Host *shost = sdev->host; + struct srp_target_port *target = host_to_target(shost); + struct srp_device *srp_dev = target->srp_host->srp_dev; + struct ib_device *ibdev = srp_dev->dev; + + if (!(ibdev->attrs.device_cap_flags & IB_DEVICE_SG_GAPS_REG)) + blk_queue_virt_boundary(sdev->request_queue, + ~srp_dev->mr_page_mask); + + return 0; +} + static int srp_slave_configure(struct scsi_device *sdev) { struct Scsi_Host *shost = sdev->host; @@ -2770,6 +2803,7 @@ static struct scsi_host_template srp_template = { .module = THIS_MODULE, .name = "InfiniBand SRP initiator", .proc_name = DRV_NAME, + .slave_alloc = srp_slave_alloc, .slave_configure = srp_slave_configure, .info = srp_target_info, .queuecommand = srp_queuecommand, @@ -3176,6 +3210,7 @@ static ssize_t srp_create_target(struct device *dev, struct srp_device *srp_dev = host->srp_dev; struct ib_device *ibdev = srp_dev->dev; int ret, node_idx, node, cpu, i; + unsigned int max_sectors_per_mr, mr_per_cmd = 0; bool multich = false; target_host = scsi_host_alloc(&srp_template, @@ -3232,8 +3267,33 @@ static ssize_t srp_create_target(struct device *dev, target->sg_tablesize = target->cmd_sg_cnt; } + if (srp_dev->use_fast_reg || srp_dev->use_fmr) { + /* + * FR and FMR can only map one HCA page per entry. If the + * start address is not aligned on a HCA page boundary two + * entries will be used for the head and the tail although + * these two entries combined contain at most one HCA page of + * data. Hence the "+ 1" in the calculation below. + * + * The indirect data buffer descriptor is contiguous so the + * memory for that buffer will only be registered if + * register_always is true. Hence add one to mr_per_cmd if + * register_always has been set. + */ + max_sectors_per_mr = srp_dev->max_pages_per_mr << + (ilog2(srp_dev->mr_page_size) - 9); + mr_per_cmd = register_always + + (target->scsi_host->max_sectors + 1 + + max_sectors_per_mr - 1) / max_sectors_per_mr; + pr_debug("max_sectors = %u; max_pages_per_mr = %u; mr_page_size = %u; max_sectors_per_mr = %u; mr_per_cmd = %u\n", + target->scsi_host->max_sectors, + srp_dev->max_pages_per_mr, srp_dev->mr_page_size, + max_sectors_per_mr, mr_per_cmd); + } + target_host->sg_tablesize = target->sg_tablesize; - target->mr_pool_size = target->scsi_host->can_queue; + target->mr_pool_size = target->scsi_host->can_queue * mr_per_cmd; + target->mr_per_cmd = mr_per_cmd; target->indirect_size = target->sg_tablesize * sizeof (struct srp_direct_buf); target->max_iu_len = sizeof (struct srp_cmd) + @@ -3440,6 +3500,9 @@ static void srp_add_one(struct ib_device *device) srp_dev->mr_page_mask = ~((u64) srp_dev->mr_page_size - 1); max_pages_per_mr = device->attrs.max_mr_size; do_div(max_pages_per_mr, srp_dev->mr_page_size); + pr_debug("%s: %llu / %u = %llu <> %u\n", __func__, + device->attrs.max_mr_size, srp_dev->mr_page_size, + max_pages_per_mr, SRP_MAX_PAGES_PER_MR); srp_dev->max_pages_per_mr = min_t(u64, SRP_MAX_PAGES_PER_MR, max_pages_per_mr); @@ -3447,12 +3510,13 @@ static void srp_add_one(struct ib_device *device) device->map_phys_fmr && device->unmap_fmr); srp_dev->has_fr = (device->attrs.device_cap_flags & IB_DEVICE_MEM_MGT_EXTENSIONS); - if (!srp_dev->has_fmr && !srp_dev->has_fr) + if (!srp_dev->has_fmr && !srp_dev->has_fr) { dev_warn(&device->dev, "neither FMR nor FR is supported\n"); - - srp_dev->use_fast_reg = (srp_dev->has_fr && - (!srp_dev->has_fmr || prefer_fr)); - srp_dev->use_fmr = !srp_dev->use_fast_reg && srp_dev->has_fmr; + } else if (device->attrs.max_mr_size >= 2 * srp_dev->mr_page_size) { + srp_dev->use_fast_reg = (srp_dev->has_fr && + (!srp_dev->has_fmr || prefer_fr)); + srp_dev->use_fmr = !srp_dev->use_fast_reg && srp_dev->has_fmr; + } if (srp_dev->use_fast_reg) { srp_dev->max_pages_per_mr = diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index a00914c..26bb9b0 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -203,6 +203,7 @@ struct srp_target_port { unsigned int scsi_id; unsigned int sg_tablesize; int mr_pool_size; + int mr_per_cmd; int queue_size; int req_ring_size; int comp_vector;