From patchwork Thu Jun 23 07:35:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 9194631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 892B36075F for ; Thu, 23 Jun 2016 07:36:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7AB082843C for ; Thu, 23 Jun 2016 07:36:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6DF8E2843E; Thu, 23 Jun 2016 07:36:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B6732843C for ; Thu, 23 Jun 2016 07:36:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751155AbcFWHgK (ORCPT ); Thu, 23 Jun 2016 03:36:10 -0400 Received: from mail-bn1bon0083.outbound.protection.outlook.com ([157.56.111.83]:6496 "EHLO na01-bn1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751063AbcFWHgI (ORCPT ); Thu, 23 Jun 2016 03:36:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandiskcorp.onmicrosoft.com; s=selector1-sandisk-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=k0obIUw/V/o8x4/BGezyWMmTVjQY3kOs33SIErLEgqg=; b=nVoD+Negt+wh2ZW14MkeE+OQuHAR+V6QueFQvY0AeAgDB2aF2L1F11035/x5yi/MYXIg5SHD1JcH8X+LyFmT1vuJB9udXRlHgNJjJG7H3TuwLkaSdZNoiymsjxegM1CxG6HCuKDxeLejUuDvTXt5Eesv3dDfPZUSI2IVAhfqExA= Received: from BY2PR02CA0054.namprd02.prod.outlook.com (10.141.216.44) by CY1PR0201MB1771.namprd02.prod.outlook.com (10.163.55.16) with Microsoft SMTP Server (TLS) id 15.1.523.12; Thu, 23 Jun 2016 07:36:04 +0000 Received: from BN1AFFO11FD046.protection.gbl (2a01:111:f400:7c10::123) by BY2PR02CA0054.outlook.office365.com (2a01:111:e400:2c40::44) with Microsoft SMTP Server (TLS) id 15.1.523.12 via Frontend Transport; Thu, 23 Jun 2016 07:36:04 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.21) smtp.mailfrom=sandisk.com; redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=bestguesspass action=none header.from=sandisk.com; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.21 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.21; helo=milsmgep15.sandisk.com; Received: from milsmgep15.sandisk.com (63.163.107.21) by BN1AFFO11FD046.mail.protection.outlook.com (10.58.53.61) with Microsoft SMTP Server (TLS) id 15.1.517.7 via Frontend Transport; Thu, 23 Jun 2016 07:36:03 +0000 Received: from MILHUBIP03.sdcorp.global.sandisk.com (Unknown_Domain [172.22.12.162]) (using TLS with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 66.CD.03023.0209B675; Thu, 23 Jun 2016 00:30:44 -0700 (PDT) Received: from milsmgip11.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.279.2; Thu, 23 Jun 2016 00:35:59 -0700 X-AuditID: ac160c69-353ff70000000bcf-85-576b902067e5 Received: from exp-402881.sandisk.com.com ( [10.177.8.100]) by (Symantec Messaging Gateway) with SMTP id E2.2E.02607.B519B675; Thu, 23 Jun 2016 00:35:58 -0700 (PDT) From: Bart Van Assche To: Doug Ledford CC: , Laurence Oberman , Christoph Hellwig , Sagi Grimberg Subject: [PATCH for v4.7] IB/srpt: Reduce QP buffer size Date: Thu, 23 Jun 2016 09:35:48 +0200 Message-ID: <20160623073548.9883-1-bart.vanassche@sandisk.com> X-Mailer: git-send-email 2.8.4 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrKLMWRmVeSWpSXmKPExsWyRoxnka7KhOxwg9mnOCxenv/AarFy9VEm i2eHelkszn94xWSx7vV7FgdWj/P3NrJ47L7ZwObxft9VNo/Pm+QCWKK4bFJSczLLUov07RK4 Mt7/e8NWcE6iYvbptSwNjCtFuxg5OSQETCT+zp3G2sXIxSEkMJdJ4s7F+4wQzg5GiZVduxhh qt7P2cUMkVjJKHGmfwsLSIJNwEji2/uZYLaIgJrEpleL2EGKmAV6GCUmnzgMlhAWsJA48Xg3 K4jNIqAqMfNYKzuIzStgJ7FoxxZWiA1yEht/vGYCaZYQeM4icW1/EzNEkaDEyZlPwAYxC0hI HHzxAiwuJKAucXLJfKYJjAKzkJTNQlK2gJFpFaNYbmZOcW56aoGhqV5xYl5KZnG2XnJ+7iZG SOBm7mC8+8T7EKMAB6MSD2/GsaxwIdbEsuLK3EOMEhzMSiK8v7uzw4V4UxIrq1KL8uOLSnNS iw8xSnOwKInzut37EiYkkJ5YkpqdmlqQWgSTZeLglGpgbJCSUprSsVRvQ+Lxhyut5Ll2fN8k +63shnQ110PeOO4iifq8i6zJ+lzpNw+F7J6TY9/lUPGYxySTPV6LP9t+/sLjJZ8KNF2mPj0U /Glp+NQXDC5B4TICEdHh2l6VFu8YJoSssbNyS+8Re1T+gHNzBXeb79alyyctnjLvlp/ukU3a 5QcyGXKVWIozEg21mIuKEwGmcrhWWAIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupgkuLIzCtJLcpLzFFi42Lh2siRohs3MTvcYO0yFouX5z+wWqxcfZTJ 4tmhXhaL8x9eMVmse/2exYHV4/y9jSweu282sHm833eVzePzJrkAligum5TUnMyy1CJ9uwSu jPf/3rAVnJOomH16LUsD40rRLkZODgkBE4n3c3Yxg9hCAssZJeZPiQex2QSMJL69n8kCYosI qElserWIvYuRi4NZoI9R4vD+h4wgCWEBC4kTj3ezgtgsAqoSM4+1AhVxcPAK2El8/qoDMV9O YuOP10wTGDkXMDKsYhTLzcwpzk3PLDA01CtOzEvJLM7WS87P3cQI9itn5A7GpxPNDzEycXBK NTCeNE0xv/H47F/FqGTFSyE34noSOJ1/VKXOvTeHf+qFn3viz7bntN/1u7Qr7P1C+baL3ROk rul0Xa6Iu7Cr7W6ctEWP4od+zYUy+uaRYVZNvK8ddn9/PteCa5lzke/SX1znt9sm5cqe2951 4X+L34GvXc8iHy07tLWdq2oyT1nMqtl7My4dX6SgxFKckWioxVxUnAgAXJT/a5sBAAA= MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:63.163.107.21; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(7916002)(2980300002)(438002)(189002)(199003)(86362001)(575784001)(110136002)(19580395003)(19580405001)(33646002)(106466001)(229853001)(69596002)(50986999)(97736004)(92566002)(48376002)(2270400002)(586003)(50466002)(77096005)(305945005)(1076002)(5003940100001)(189998001)(47776003)(6806005)(356003)(2906002)(50226002)(87936001)(11100500001)(8936002)(36756003)(81166006)(81156014)(8676002)(7696003)(5003600100003)(7846002)(4326007)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0201MB1771; H:milsmgep15.sandisk.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1AFFO11FD046; 1:/rx3c34OmvLnfPedexXjP4bJLP+tscA1Eo3IBm1c4dggzpVjiJe7dzJAFKqINb7YHZNJV5mwPrkuEPQ8dvMvamqhpinRDAj4AyDtCFVM+GXQR2TInTK+70UmCqQC+M24z7L/MhI87Y00mZYtjBzZzRrHmm3GZHkEXBbRrBoWvW8BUdHc6lco2YZ46CnstQIxDtwmPL+3YOcZJFAbmrcJexyZSCcnP5k38IxLH+sVuj336o70gNa4aObolbC1QyQLHPZhVKD4XfIWX430+7DOFHCLF39JmIOWpT+bYQsVD4o508gfJKK1/2OtqPyi/smlh9w8NXkce4v1fUHWOzC53qOQy/pcx9G7LG439xz39iZPfh10wp1mboKiZU4jpetsRL9TZ31XksPfdcxrpnGAYJtkm+H2FaF48jA4WpAlgfCmZsZoaIakYzqv/7e5B9c44j7orE6pW7uGkWM5AKJDEQ== X-MS-Office365-Filtering-Correlation-Id: 2ba2b65a-200b-4bad-abdf-08d39b39080c X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 2:s22n3yhca0zYy5PhmDBZNCG7REPtU+Veyx9kFp8BNKCpGZespNFRHMhs86Sozn5b2orM9HwbwQFhd6mGtJLR+abz2dAQ0OSJJYdR3Ck67PUTgviHzxfjh3USXiAblNCumjP2ekG4rILb58qBV/AT7KtKsjNzwNSWEmu71MEZjXYbCWa03DJ5ywtb/lJTF9/c; 3:mHQ6sTlXVAqq/P8vgLnSmQlt3DtjwQBQ30+BNCMy8o0JrgFF2jFzhR9e7k5GXZnJAOpRWPJS/13lHDRTifOctI2obDY1LL2ZkbQIULBznc39LqiCsi7tOU1T6WVlP/DFYldTu14vVpZmWPScFsMRGkN9Pq5MKe9XjPWbRIEb64iSrN2/WQN5gqfbQqASKAq4fB/Ljp5F4eTtRxN9XaXq/XkXsNfb/agVYEEhsJAlNDAZonaRDoElZjLp9ZffcAE/RlqHF4XZDQrEm9Jdi+svhQ== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501002); SRVR:CY1PR0201MB1771; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 25:6yHztRDXsGRujiGxNp/GVp2nkqZ8pcqAmw+PgfC2z9oZ8UJiFWXtQL78k+9KoOn2SKnnUIoQNU/+aHjyXaK83eoH8dxZi3ykEY1OHfcYHasW1QGwHpswEZujFWSnHzg9d+eOJ694t7u3DPi/KhffvCfKr2//a5yGUQxbHMuYvfl3yX1LHg1wTTyW1HbLv9Vwl+YC3HWWxqn2blxoimcpM2YP9fNLNq1aivZLwi7KJZoNJHLiMbd4rWKgxvPJ7KrbiNluvHXzz/g9xj/dEcxR3imZ3D0OUNluITg6EdKG3/InwiC9E4IOt8Y2577EoAC7HPYxH6eHMuDmnabOi24zv1c6XluAZITuOB2TR64nHFnUz+IWaLwdoGlCVJrfCbzm+ak3FBJP1VxLL4YnDhLTuc3Yzm54WHfai9tJ47tvrXtMi/isgqA2yLD2/JqgOWP4FOxgD+la0JnOvxX3jfRkC/OjxbqbnWmOMzelsdJiaAL8q5HH1Ka6TsBwRskpyWiPmbxq/IICdyuHUanC1x8WnZIaOlUU++3DJW4lXaoIPcWcpcntLsfyQcVUQ/u37/si1g5i1DBxyR11WMe6f9ozSSgOUsAwW2UPoZlhHLNWT/Du3sjr2r2kBm/WAFAlvvJPbCxWqrxcTzAe4aSJneBlxH2oHUXOm8GbgtVFL6IhVNhoETNkLHReiiEFwsObxZKTqaBoLQC2rr3eLcekJhcbVT1P0EoQBKh1pFNuCTnLtJ1UxIquCXruOgDvY3w9ZYAnpNvLfuwL6TesnNq1tgHRig1OojplJatjVbOLaC9PD7g= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 20:xiS6UmF7sNkNn/qVOD3pZoM6dlYzES5+D9OEIWs9ZKtTMPJByOwndE54CyxEcZrWVn+MSeCwbgpsr2UuOxDczq/NmrD1YYk8VY6JKSNEf3rc4sakDkeuvfNE7TLqlCvwDNhkL71fprH7g66qhh2exyNhRB3UfvXU35P3/AJPtsK7xi98SRwYafO7jAQRhinl3Qqxxz2qJcRWJ/7PZ5j1u2pLOzm9rJRwYkUs9uW5dcxxgT6kUBwVHY6AumHj2YY1ShSz5ibhKT4BkMi8DyPZf6cZsPn0Au+M2TDqEYfk590/Y7seP+zRirGkJ+y/qXFkMuHefr45oV4bePK1o+6Nj1zXRAxMymBzmbKoeVwCMiidcFIiCTwyFVfIbiy43vMO1U+7oAMqqWcmTEdXdo89ZICZuCTFUPCspGaik3eGBxoJgQS3zmR/JLFUQr4HaMI9rXJAjOtY6L9vgHTWwamcZ8+7aH5fHSLincit8BQxRz+3cg4cH1JSt6EprHsAxxeG X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(42932892334569); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(13018025)(13016025)(8121501046)(5005006)(10201501046)(3002001)(6055026); SRVR:CY1PR0201MB1771; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0201MB1771; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 4:Lz3ryDdrE5FcggAkIj022F5x9gnGyAone2Tfo/DJYYz7WuLC4GXIMSdSEDXHFodCtbo9Yroe3luBQV/ZHTdE7IjzJHdtYGscI01ZL/r7SH99jCWzb6or90pEQAilLaJvDBctqgc0EZ4J666Lgr9fBMLmtjieI5CMs85WoBexK7duzvOk8qHLNo2yQeH4FBkgHAi83B/rvwfwzSWmRymOvbmrVqbOSqlQBhu0cvG4KkK8AOrs/AmnhNEDsSbjOf+GKQMXZU8zBQ+QvO0ecPOawtIfgI1p7GGyI9URkBCxMpXv7/7tXxIN0VII2+S8LRGKUS4GciLnbN85G5wA0J67m+Y8iQRCNJ0OvhuBVmFIf5xGUnsAYBj8Vl/ViA2zUFLDgYOYBfbXs6OIw81eu1XQMxZaum6n/KwtBHx6YRsIg+aeNPdJII36WmlScznQiWS3HPhyOYyW86YffKGIieMyiQ== X-Forefront-PRVS: 098291215C X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR0201MB1771; 23:UqdBlc3oUjKgx+9OFWOUX9z1YCjrvePszOW55BS?= =?us-ascii?Q?awryBmBd2PYZKwMcfRX2JhooWjLEU/1h9aMtWiq1igaYFxJDo2UERBiNl+er?= =?us-ascii?Q?H6sctuTotCl3hRhoRQ/iZ7gxUrqHfW2NBpVcAYCK3/kLUGOaZ2oj7OFvvksO?= =?us-ascii?Q?BUy0Qnjukufhi2eVtRZ0UwQFRrdJvoesQCSWLQuCBOLpl3oUlua5yWtC3oV1?= =?us-ascii?Q?hZc1aGiUsNdK0XdSvB99Tsnd+oEDas5HDMR3AD+VT20VJX3e4lu1FgeLKvYP?= =?us-ascii?Q?O2YbOs73n7j7BaZMeXu1pEv2lVzYACjW3jxL53vWIwE2kS7C0cNQDXd18d2x?= =?us-ascii?Q?170dxHO7KTyQYatwSHXGjgPkEV1GoRvRaKYr0uQEm2LToTnSSuHAJCN4MOMi?= =?us-ascii?Q?/PwuqaRT20MYsV1TAhAw5bvMvF+4Mxpuup8aCOH+tR7aoD+rwsI5fGEi6dlR?= =?us-ascii?Q?HdRpAsx8y4I8E2nS1nu0Sk4dsSEMj0pldE4Qf6qH0sPGxJznLPQ4ui3AOrRG?= =?us-ascii?Q?22Kh98rKkh4Cn727DmgMwvcjA1q5LsRy8KlG/RAcYVqO3Vq9ldUo1rLHYyLm?= =?us-ascii?Q?3tsQWT5NEP3rBvQUGU46soUG7G/D772cjnxGXaD5A/Cgnh3y18re9PDbmyXR?= =?us-ascii?Q?P12hL1VWqXVqDtuAceKQ2RA2kZ3Z8jFsQhb9lJhzOr47lnoWsRbfbFbXPWqm?= =?us-ascii?Q?2xiawBr0TIsh7VYrDSkO5LN3Ftv//yUQ43q8zvgQLmQlKvPJXe66GYZswt8Y?= =?us-ascii?Q?17LzTd6+J9mYek2pfgCKKc2dATHWal34dkq9qV4j5BAUdmDo/BeA4QJYqqLI?= =?us-ascii?Q?4JQ2jhvu28fPx+fI5xxwuVyXCS+EuqXej0cIY2qeoa9tuWuTKUZlvhzCTA0H?= =?us-ascii?Q?FUDBJJkoDRHGv6M1zRVqT9Bv5WAtH9Juv3fbhwmf7r88GJGtnuV0jn7AVQDW?= =?us-ascii?Q?RAb3rmCbbj9+WIDjUeM77YHAUYcyanKf66IRG7qtANFdZWvHx8p7hNcSQOnn?= =?us-ascii?Q?JOF77sxLZMaF7+GtNiOyP6R4eNwoxebIaqX8m0D8lTVWJ/6RMmV1XPbOh6zt?= =?us-ascii?Q?JK5HBcSRfw1QJ8U2svzh+zw87gRDK?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 6:9W5Y1WmxrSjKsqTx7FxH1dJozrLh38BK3ROUqkyooixdpuYJ1bFGde5gwFclMat4ciRJ2Gbv3CUPUaEjdHqTfxj/TK0msf6oI5z8zhPAoiGEa1AqYVCGKNpEJ54Nk9avea302Eqb5PYcxJYbhKqBaU76FnFo/KGRBWEZCMB2JDp827nZE+q0Wh+X/LOalSBgRl5AgwKXQkKZ5TLlnqFVWOYe6U/ibRUPZq3ARITdHcYgTxIgDOzJUXmeD6JIf97iU/bMggLQ8ojC3d5n5IKDuhL7DxC6UWLp2/375RrhEiBJaUWl4PQvv2OrOHtbkmS1+33Z85vEOZeqo66MXzHzZgh6IiCOizV4EJx9CtOZ/ig=; 5:+5YhYtEKtP6N7c+mAQBrZXejc6sA6FJEIdyV40mP1OjjUddbHDo2R2dME3KYxt5rfY89+6PKM6d5BKJBiCMwcpXq9AK+h9hGkjmfA4TFty9pTDfM1PuV5c65ctZc4nfw26TGPzlxJjKNDsbo4fhBTQ==; 24:msaoIlKXEBl4m0PuURSdId6Ed56TEArd5kSZLSVDtAVJudyalHJjjtgjqhEsDZiaXSEyE+dIEEZBEPOCnfgjHBMM6grEIG2A80MGm5kw+Vg=; 7:6dAGZV9AJF6D8b6XdCe63bifT8yPADqBkd6Ig7+30tzIgABDaNc6AnibxIJAEMWcQohfEGE1mEijgoVTleQMagTFS+G+PrAI50EvhPtmU4MDfm4TuYfmtgvEek8mfH8UJmKEvYaSUS1GUIUkaduKAZlaCMO0rqbe8+SGm/AUCd4iijj/iDy/VZYnoauw01/2djjWIkodFXPlXi6iBQfWNB0V4dJTTI+CIWqpsDOnjTxQXnz7H0tgrcohYI8w29hBov/JMKT0pbwb5u6FHaQV0w== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0201MB1771; 20:kf7rCvcKOfNrTeSMUP7dY4q+wwZDQPHblx+KD/ezctrUyxMQ7RCqrR7BaOS7UeMbxPNadhPChL/Qnk5w8oYa0YXp2wqifV9n68/bIKG+SAr24stzRdy5PxeI2NuitoBzE20u1/2YVdRX3GvcPh4jlUYhBoccZHZX3z8sMdaN0QMc5pVtqm8isMoQN+SVLOZ43Sfl8IK0dX050xcRkertj6UAJoiR9DDvj1kV69wJn7JJ9Oa/m3shQaznCkv2Iieo X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2016 07:36:03.1281 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.21]; Helo=[milsmgep15.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0201MB1771 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The memory needed for the send and receive queues associated with a QP is proportional to the max_sge parameter. The current value of that parameter is such that with an mlx4 HCA the QP buffer size is 8 MB. Since DMA is used for communication between HCA and CPU that buffer either has to be allocated coherently or map_single() must succeed for that buffer. Since large contiguous allocations are fragile and since the maximum segment size for e.g. swiotlb is 256 KB, reduce the max_sge parameter. This patch avoids that the following text appears on the console after SRP logout and relogin on a system equipped with multiple IB HCAs: mlx4_core 0000:05:00.0: swiotlb buffer is full (sz: 8388608 bytes) swiotlb: coherent allocation failed for device 0000:05:00.0 size=8388608 CPU: 11 PID: 148 Comm: kworker/11:1 Not tainted 4.7.0-rc4-dbg+ #1 Call Trace: [] dump_stack+0x67/0x92 [] swiotlb_alloc_coherent+0x141/0x150 [] x86_swiotlb_alloc_coherent+0x3e/0x50 [] mlx4_buf_direct_alloc.isra.5+0x9a/0x120 [mlx4_core] [] mlx4_buf_alloc+0x165/0x1a0 [mlx4_core] [] create_qp_common.isra.29+0x57d/0xff0 [mlx4_ib] [] mlx4_ib_create_qp+0x12a/0x3f0 [mlx4_ib] [] ib_create_qp+0x3a/0x250 [ib_core] [] srpt_cm_handler+0x4bb/0xcad [ib_srpt] [] cm_process_work+0x20/0xf0 [ib_cm] [] cm_work_handler+0x1ac0/0x2059 [ib_cm] [] process_one_work+0x19d/0x490 [] worker_thread+0x49/0x490 [] kthread+0xea/0x100 [] ret_from_fork+0x1f/0x40 Fixes: b99f8e4d7bcd ("IB/srpt: convert to the generic RDMA READ/WRITE API") Signed-off-by: Bart Van Assche Cc: Laurence Oberman Cc: Christoph Hellwig Cc: Sagi Grimberg Reviewed-by: Max Gurtovoy --- drivers/infiniband/ulp/srpt/ib_srpt.c | 3 +-- drivers/infiniband/ulp/srpt/ib_srpt.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index e68b20cb..4a41556 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -1638,8 +1638,7 @@ retry: */ qp_init->cap.max_send_wr = srp_sq_size / 2; qp_init->cap.max_rdma_ctxs = srp_sq_size / 2; - qp_init->cap.max_send_sge = max(sdev->device->attrs.max_sge_rd, - sdev->device->attrs.max_sge); + qp_init->cap.max_send_sge = SRPT_DEF_SG_PER_WQE; qp_init->port_num = ch->sport->port; ch->qp = ib_create_qp(sdev->pd, qp_init); diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h index fee6bfd..3890304 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.h +++ b/drivers/infiniband/ulp/srpt/ib_srpt.h @@ -106,6 +106,7 @@ enum { SRP_LOGIN_RSP_MULTICHAN_MAINTAINED = 0x2, SRPT_DEF_SG_TABLESIZE = 128, + SRPT_DEF_SG_PER_WQE = 16, MIN_SRPT_SQ_SIZE = 16, DEF_SRPT_SQ_SIZE = 4096,