From patchwork Mon Jun 4 14:29:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 10446727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6D42C60284 for ; Mon, 4 Jun 2018 14:29:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5BE7B2239C for ; Mon, 4 Jun 2018 14:29:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 50292228C9; Mon, 4 Jun 2018 14:29:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C00952239C for ; Mon, 4 Jun 2018 14:29:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752484AbeFDO3T (ORCPT ); Mon, 4 Jun 2018 10:29:19 -0400 Received: from mail-he1eur01on0073.outbound.protection.outlook.com ([104.47.0.73]:9680 "EHLO EUR01-HE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752131AbeFDO3S (ORCPT ); Mon, 4 Jun 2018 10:29:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kiwJLgBRAB5boQ7A72pwkGZo4yx7OZ6c0GHoNH/ZdGw=; b=OVbZsRlSQDe+B/9NV7+TLtV7zcMD6rW3v/M2qxau7YhA3vAbW7jzA3uljCiqFWwe9k0sliOEV9RK+j1X7cWJ0ItlDEQtkQjlMLsxs3q/9FH0eYQa6VHK8C3UWp6SB8chqLf/ZZpufxog8hEDD5vNfYaIa5Kkri+9FafvbkKf1HY= Received: from HE1PR05CA0380.eurprd05.prod.outlook.com (2603:10a6:7:94::39) by DB5PR05MB1781.eurprd05.prod.outlook.com (2a01:111:e400:5bc5::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.11; Mon, 4 Jun 2018 14:29:14 +0000 Received: from AM5EUR03FT031.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::206) by HE1PR05CA0380.outlook.office365.com (2603:10a6:7:94::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.820.12 via Frontend Transport; Mon, 4 Jun 2018 14:29:13 +0000 Authentication-Results: spf=pass (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=mellanox.com; Received-SPF: Pass (protection.outlook.com: domain of mellanox.com designates 193.47.165.134 as permitted sender) receiver=protection.outlook.com; client-ip=193.47.165.134; helo=mtlcas13.mtl.com; Received: from mtlcas13.mtl.com (193.47.165.134) by AM5EUR03FT031.mail.protection.outlook.com (10.152.16.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.841.10 via Frontend Transport; Mon, 4 Jun 2018 14:29:13 +0000 Received: from MTLCAS13.mtl.com (10.0.8.78) by mtlcas13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Mon, 4 Jun 2018 17:29:11 +0300 Received: from MTLCAS01.mtl.com (10.0.8.71) by MTLCAS13.mtl.com (10.0.8.78) with Microsoft SMTP Server (TLS) id 15.0.1178.4 via Frontend Transport; Mon, 4 Jun 2018 17:29:11 +0300 Received: from [10.223.3.143] (10.223.3.143) by MTLCAS01.mtl.com (10.0.8.71) with Microsoft SMTP Server (TLS) id 14.3.301.0; Mon, 4 Jun 2018 17:29:09 +0300 Subject: Re: [PATCH v3 1/3] nvme-rdma: correctly check for target keyed sgl support To: Steve Wise , 'Christoph Hellwig' , 'Sagi Grimberg' CC: "'Ruhl, Michael J'" , , "'Busch, Keith'" , , , References: <14063C7AD467DE4B82DEDB5C278E8663B38EE822@FMSMSX108.amr.corp.intel.com> <3acbea43-d777-88a5-0059-10275655d545@opengridcomputing.com> <8da47cd4-44d5-2229-ef82-26d165dfc245@opengridcomputing.com> <20180531170201.GB31715@lst.de> <20180531172554.GA32068@lst.de> <702f1949-02eb-07e4-f101-58c32929b29d@grimberg.me> <010001d3fb68$99d01d80$cd705880$@opengridcomputing.com> <55bd8f56-5261-4f2e-d5a7-5bc50970deb3@grimberg.me> <20180604121136.GB29545@lst.de> <05438bc4-61a0-e4ca-a00d-d918c20a0dcc@mellanox.com> <019101d3fc0f$4d32a4f0$e797eed0$@opengridcomputing.com> From: Max Gurtovoy Message-ID: <1d549a4d-c5ea-0bd2-1acd-66a627812eda@mellanox.com> Date: Mon, 4 Jun 2018 17:29:09 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <019101d3fc0f$4d32a4f0$e797eed0$@opengridcomputing.com> Content-Language: he X-Originating-IP: [10.223.3.143] X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:193.47.165.134; IPV:NLI; CTRY:IL; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(396003)(346002)(39380400002)(376002)(2980300002)(438002)(13464003)(199004)(189003)(53546011)(229853002)(31696002)(86362001)(3846002)(6116002)(356003)(305945005)(50466002)(7736002)(8676002)(81166006)(81156014)(8936002)(93886005)(6246003)(478600001)(4326008)(36756003)(64126003)(336012)(65806001)(65956001)(47776003)(58126008)(54906003)(110136005)(16576012)(106002)(11346002)(446003)(476003)(126002)(2616005)(486006)(106466001)(5660300001)(65826007)(16526019)(186003)(31686004)(67846002)(316002)(26005)(77096007)(59450400001)(76176011)(230700001)(2906002)(3940600001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1781; H:mtlcas13.mtl.com; FPR:; SPF:Pass; LANG:en; PTR:mail13.mellanox.com; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; AM5EUR03FT031; 1:Ef39Fgws+dd+JIIkHGUJz13Rw5GF5BiFRzs+WmiwzdBV6fdgGlqY7+f2e9c/D22//ts4iUWkOD8xARltTUAhXzda4UWQESA7Dp8vhfGTgFBGBpsUQl+pBCLBA9MA3udZ X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4608076)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DB5PR05MB1781; X-Microsoft-Exchange-Diagnostics: 1; DB5PR05MB1781; 3:iFpo9E17WUHMdv4i1SIlPxIFOyUrH5hLmoW+FIugCYE4Yj9l+n2+gNQkSMeclAHNXWpOljeIYdZidaWi/Yd784TGQa5hfqp2o4rND2ncPYYieAoB1BP3vJbuyh9MBKuojeLNAMoPEUUiDBbaOLqGqJMIAo4cmmDfrhw9FsMxiHL59x/0zdyPKZz0X/MoFmTckbmDalTb6KKKQhdth0Cg3xMiyZY03UXCl16olG3ygtOZ/CK17yLxVDw06SRPqWG3d9UtclG+n7knCHAXvpXMliz+CiqxhF6oDx0CPUIBedj8ywk1/6aIvxMaRH4fJJRYBP+ml3wb2mljsEsla4ZJ/b1UuCKrV/7rNhu4kII8ME8=; 25:BIR8BUEZJEp6wJ6XAAe4QAxmDwQcKTAoHdjAxkmBAX2UkwiV7G5XU0dJ3CRROadjRQxHypJt7RBAk46cBgPLQQCCZr42TRHBI8xcZLcHT43szHLQx2SiQs+Dnf3UgrDxH6/HF3syXtq0YW1cb+B98iePyFvliK5JFlHZIKht02CVdeDCr5I9slP89LQmSkMYd4N1fXJ0w2rEDtGmYZx/6np5kyAbQN83H+RupOc7gY68ffMawaP6A5t4hiEhJyYMEtKOThT9n/J7ekPu7kvxVIQE+1tv3vOYL+ZMN613YStcGb9c2u+PVGYIq4qKrU3CDw2GMtwbTyQOazj6imqu7A== X-MS-TrafficTypeDiagnostic: DB5PR05MB1781: X-Microsoft-Exchange-Diagnostics: 1; DB5PR05MB1781; 31:LsPGIsv+/uWf6juhfR/I9uJbK11SWiiLUC34lskjeQx+1HQiQct4eQfygil165LNQthsXO+k960rmAe/7r5/LXzy32Qkks9+SSYqFOHiXmcfxrCgpfcsj6/pxWTAp8b0UUVB5/LKzaBS/eNTkBXShAuMbjpSCMraGkv8R2WBPPABWrVr494BUQfQfV8UwbXJAlkKnaq6qhV/a0JfU0dK/IrJ6nfA8egKtX0LvNg0iVE=; 20:cghgAUyxekIa4aAQ43ZWw9wHJKtxyhES7YfylhvOKIYIS0yMDuRZuyMu7Ijn1dkBc/qDi+CG18Kpv/vMsYMtf7r1DgU9wPWWDviPuh/DKnDVjPucSrv72IdIxwIo2kLnQ1ojbJjRfcMb/W8B0mcofaJe4aBqrL+uJHy9ya5IvGX4fS6ED3+a9H/drMIZPO+BVgX+h118QWusArhomvc0chsYYFZhUVDSJ9o7Azd/pLglho1iKh21+54+qiW95ZoSmbyfJP+woorXeyCekODJ0ZUQryuQjwEXYMpMTr/V4AW76AUNY8mlnR0LzBW38IjNhouFsZ0EoEKHezRgHWJ7Iu4sVIiRW/A59VL2YAGcs7x0LQOhHREbdrl9baYV5m5IqKBmvTen/T/q8JAPxtozDmnOPaOVYzk0nPpUnmhYPifbBDN9qtHGzGIVzFMU2uN9YuCxWsKZXz6hkcPKUFViK914v159PMIknuXn59+34kWjHv+QGcDxBM4EIqV7DfoK X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(9452136761055)(258649278758335)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93004095)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:DB5PR05MB1781; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1781; X-Microsoft-Exchange-Diagnostics: 1; DB5PR05MB1781; 4:dI2GIDwhoLS6ZYMnGbbIMQP1zcxosX7Iv/wpW4SOMdhggZh5Lmy3o1A//IynlVrPBT34geQ03jaDALDGXMztG8KKwuZwdixQFoAoeKtHebWC7Zi5Otwa83MrdGG+bO+3G7bMnEU3ZpPa6SZ33FWbr1H3mBfxQI3iFf9IJTesA9uewa43xag9Et4uYMRxidc+ICybnQUg0r9gn64c3WzLwvZ9HDOxOBpPAIQbpJScwHyZx+KhFQFDtcMQly3ShUkIvxbQ7uU8kEV3vYJQYKh/L+CQ7hfynL1gCP83c2TiJPqrEb2YyqxPrqyORBgcH3FunRj4QRtP5dA1JjSmg1jA/AoCQm8fWBhTxLudT4f8iSV9gE3RdCnZFSdlJejA/KSx X-Forefront-PRVS: 069373DFB6 X-Microsoft-Exchange-Diagnostics: =?windows-1255?Q?1; DB5PR05MB1781; 23:TqgorpsxguWWS5w1akA69Spt428TvYlzcAjNf?= =?windows-1255?Q?snEa1x/gwFwbdDFewYK4bvPy3RsiL03EKHan719lKBmSlYhU/XmcogM+?= =?windows-1255?Q?MWIcUMPr3Y99CsVZYdoj3tMpOVHD44Jt/X2HHbRtWjbz9GSxuzzhpqSX?= =?windows-1255?Q?h4LCFLnqrbOSEIVN0mHNBK9GGQg/m5Ya8XgNGy5QwKg/c9ijS7l1vH2G?= =?windows-1255?Q?roVzG9AFUCdcrvaCJYrVfuarCCrFMIEUUxCEGd1CKUnrrqt6fnpGPk1l?= =?windows-1255?Q?n4eA/OukyNsbw5EfEMIeg5KFhx8tvvKrdghCnE/HaFkGVgDjcgvr7gJJ?= =?windows-1255?Q?nQjqc+4XSg+MkQfNgoDSdijc/2y5HFgurFoOQKyjyeaFJB1/8vdKSgIP?= =?windows-1255?Q?eXz2fA04naSaeb0tie1BhOl8rKFryQH7dGQ1aTjwXoQBddRCvPE/csug?= =?windows-1255?Q?DzaifzdaLgt13wlxfHzTLYIXNvpdmD63U/lT5IqkWCBpQFjP8Tl67gOq?= =?windows-1255?Q?HT5uii5tDD3FtM1gvHrNvBpkdzR5ECnBDpBp01JhCfya2QiXPksuM/10?= =?windows-1255?Q?KysDb/IC6LU/uZxTjtIgJeG4bT1hxUbj4w/mhP+uU6XjrqOm3d9HR3ap?= =?windows-1255?Q?58i4vgSFYO5zM53r31ZCGrB8h+UrGYbGtck5GPwmuLdVMNqnsVKJc17P?= =?windows-1255?Q?xer2UhNX/QhVBpRY2psa+WpWo2dJMiY5+A+ekamKfmx45yGCWzcl4xQU?= =?windows-1255?Q?APM8tfUKd/xqgEoyRa5hjocD7/YMRLYQHG06fKcP1U1Njdzv3uIaOiWw?= =?windows-1255?Q?0VaJl3H4Z/dMY8qfWEvl2LQMQIr8SbdmSKKLdhGxxskRrS4SH1/fDRLt?= =?windows-1255?Q?dhBqAUZ1HYq9SKGG6SUb+mFTx/++9TB+NTsbiderQVc0ukQCOC/0yGjs?= =?windows-1255?Q?tR2fy+XCdKDVWL65eN/voNT3sjfd5kxDqOkjowEs0Hr9+yhem0JLyYzV?= =?windows-1255?Q?iOGjMx8tae+SZSlFHABUM+5hvMyPazikOwr5o7ndLoiSxik29Cp8Eey7?= =?windows-1255?Q?XjG3Tm1Lx9G+J/ESEFLgOb4PXPi1bRqi02YpR3qbCeZqFoblUeaHkOkn?= =?windows-1255?Q?jA4gkBh4Gskv3SiwKF4gRQFWObMgu4+xoDpWmeAp0cL+ezlXkVc9fa70?= =?windows-1255?Q?FdFHRGRsi1bH2bfrk19W6S32+0FjunHWbZPLc2yWR2uwm+wWaQhmhYKy?= =?windows-1255?Q?ggELEYWNjga/58hyfHPtHIAFEwoRObcy0M598wryp+W3B2THQioF1AWB?= =?windows-1255?Q?C7ebX6XEs6+8gSpxAAIf7L4EcngAyKTH2dS1jtDEOocuVKvcxyFKqTDs?= =?windows-1255?Q?Wg74IMk4zzoQid1N5mux6L2JdnLlsu6zEa5F1+QgRaFkZzoA3DSlJbeX?= =?windows-1255?Q?XgQHHN9V1HRkgY/3XTw?= X-Microsoft-Antispam-Message-Info: JPrL4KP3uCdmCRMDMidHH91YbwDcuW0uR8fw+RgGL3Xq7cekDqCDXqvnG7H7hpgT7wg66HZFAAhm7TUp4Ig7Jg6MtkA7HAF0VGGFtpK3KqE2NiZKHEKYYTq6mtcrEGppknZKvB/Xrp6RtBvediCSngENfqgCAedkgYDgpOL8yxRO/ribPT3cwcti+5PVK6VZ X-Microsoft-Exchange-Diagnostics: 1; DB5PR05MB1781; 6:Ho3drDavMkL808DnYVjtgM11ezTYEkhMsMPMe82eT6dBKuryml8sUBhNtZ3m5QoB629+ait2guw/z84UCyrnGs0BfCVO7mEVXlaR32nr4V57JZfm0WbB9+0VTFaICGIBx+cH/xQ9NsL6s0mi9axox/tYfsrHPuWWeQpvM2u26eBoDcWavoOwI6h5YEHcmWtuQaX0zB/j0OzsNuIEq/gDtCSAfXxJrBkxuWWBM1sNXiM4JoThD0CnrqoAglNBvvpKgtXCVcT+kU/yp/u80zqaBIg52Aw8KowZiCmiStaxbB59aKgP+5AYUsQdMmxIvQJ7Mogb+Kpd80mZ9CsoiaVMwh7uSC8OEELxnWV32/Wba11vOdRYlA2tRZYU9/bl7gj8batUJXNEdNHpuuIVJFU7oySf55+nE5r9SgFOZ3lqw63vi/ZZPbIEosaUnOQGt6FLwal7mJpweWvDCiEN6xln6w==; 5:KY5rv+fcYarMmQujTQVwx2Xey3fFlUCGIM6fTOmY7mI+LrFmCFXLD2oXXHvP1pK4GXmTNaufkXJ4PQMkVud3UW6j8IYOAOGs3+87ydEtpdpTjTGtWQDUM6p+kREFlamIan5TI2q5gX8xL/cklMJPG9N1PGuAbBEEPTyxuOoYsew=; 24:p7QT6TSP+BrRlYtZUm1ybZR/RJ6CznxlI2gXAADsR7c2jZzkDmWOp+bxBq3CrIkLsom6ruo9Sj7TMJANTb0kVi0zSj5KIQmHOa/sF5oB/b0= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DB5PR05MB1781; 7:szgRQq5gpxmKtVLu3QvbgO095zg4JayzMevlpMFlwSQWwaE8xaDwfzweiIdGbDSJIeinBy6f9v+AKzgCW3ugEd9OQoAuW+VZTXEFloqEBqUBJR7mdJ7pAkNY1K1cus0+Uh8U1DHmz4JqMjSc1x8mWQ0/qBzWwMw3TwzSV7izCPQHY+7kaDa+BagqdgX92/6rT2ST9eQr5/TgXDMd7i+D0nR0v1Q6JQ3kIGEdaUU5MTb2Mrab/WxOpIVZAP6eyt2q X-MS-Office365-Filtering-Correlation-Id: f559e592-cd3a-40e6-6ced-08d5ca278b51 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2018 14:29:13.0822 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f559e592-cd3a-40e6-6ced-08d5ca278b51 X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b; Ip=[193.47.165.134]; Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1781 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 6/4/2018 5:21 PM, Steve Wise wrote: > > >> -----Original Message----- >> From: Max Gurtovoy >> Sent: Monday, June 4, 2018 8:52 AM >> To: Christoph Hellwig ; Sagi Grimberg >> Cc: Steve Wise ; 'Ruhl, Michael J' >> ; axboe@kernel.dk; 'Busch, Keith' >> ; linux-nvme@lists.infradead.org; >> parav@mellanox.com; linux-rdma@vger.kernel.org >> Subject: Re: [PATCH v3 1/3] nvme-rdma: correctly check for target keyed > sgl >> support >> >> >> >> On 6/4/2018 3:11 PM, Christoph Hellwig wrote: >>> On Mon, Jun 04, 2018 at 03:01:43PM +0300, Sagi Grimberg wrote: >>>> >>>>> He's referring to patch 1 and 2, which are the host side. No page >> allocations. >>>> >>>> I'm good with 1 & 2, >>>> >>>> Christoph, you can add my >>>> >>>> Reviewed-by: Sagi Grimberg >>> >>> We've missed the merge window now, so we can just wait for a proper >>> resend from Steve I think. >>> >> >> There are still issue that I'm trying to help Steve with their debug so >> let's wait with the merge until we figure them out. > > I would like review on my new nvmet-rdma changes to avoid > 0 order page > allocations though. Perhaps I'll resend the series and add the RFC tag (or > WIP?) with verbiage saying don't merge yet. > > you should add to your new code: ret = rdma_create_qp(queue->cm_id, ndev->pd, &qp_attr); I currently see some timeout in the initiator also with 4k inline but it works good with old initiator. -Max. --- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 2b6dc19..5828bf2 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -964,7 +965,7 @@ static int nvmet_rdma_create_queue_ib(struct nvmet_rdma_queue *queue) } else { /* +1 for drain */ qp_attr.cap.max_recv_wr = 1 + queue->recv_queue_size; - qp_attr.cap.max_recv_sge = 2; + qp_attr.cap.max_recv_sge = 1 + ndev->inline_page_count; }