From patchwork Thu Dec 31 08:56:43 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 7935791 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1C6089F32E for ; Thu, 31 Dec 2015 08:56:54 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id F0BE220265 for ; Thu, 31 Dec 2015 08:56:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A68F520263 for ; Thu, 31 Dec 2015 08:56:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752207AbbLaI4v (ORCPT ); Thu, 31 Dec 2015 03:56:51 -0500 Received: from mail-bl2on0097.outbound.protection.outlook.com ([65.55.169.97]:44784 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751952AbbLaI4t (ORCPT ); Thu, 31 Dec 2015 03:56:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandiskcorp.onmicrosoft.com; s=selector1-sandisk-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=zYQXPQEUEb0TmuC3seDreoYZSnXW3kpVBIzdMFA1muQ=; b=lhryq/jbm/fDVx0YO44f6ySV+6OX2q8G+1hHPG0/0WNJT24sbqvIKOelDMb5gV6SPaT2YBayW2A6k5nhvf1OzTGv/b7RUAMCvF3Vo59ZfZpAW+ZxUG/afhoXOQ9lv95eBV7TAOBmGPlmqgrG6Q7rA0B1Joyvy67Gv18AbbqMPMc= Received: from BY2PR02CA0057.namprd02.prod.outlook.com (10.141.216.47) by BL2PR02MB290.namprd02.prod.outlook.com (10.141.90.145) with Microsoft SMTP Server (TLS) id 15.1.361.13; Thu, 31 Dec 2015 08:56:47 +0000 Received: from BL2FFO11FD026.protection.gbl (2a01:111:f400:7c09::171) by BY2PR02CA0057.outlook.office365.com (2a01:111:e400:2c40::47) with Microsoft SMTP Server (TLS) id 15.1.361.13 via Frontend Transport; Thu, 31 Dec 2015 08:56:47 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.173) smtp.mailfrom=sandisk.com; mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=bestguesspass action=none header.from=sandisk.com; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.173 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.173; helo=milsmgep12.sandisk.com; Received: from milsmgep12.sandisk.com (63.163.107.173) by BL2FFO11FD026.mail.protection.outlook.com (10.173.161.105) with Microsoft SMTP Server id 15.1.355.15 via Frontend Transport; Thu, 31 Dec 2015 08:56:46 +0000 Received: from MILHUBIP03.sdcorp.global.sandisk.com ( [172.22.12.162]) by milsmgep12.sandisk.com (Symantec Messaging Gateway) with SMTP id C8.E3.32178.DCDE4865; Thu, 31 Dec 2015 00:56:45 -0800 (PST) Received: from milsmgip11.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.248.2; Thu, 31 Dec 2015 00:56:46 -0800 X-AuditID: ac160a69-f79f76d000007db2-87-5684edcd0679 Received: from [10.50.231.75] ( [10.177.8.100]) by milsmgip11.sandisk.com (Symantec Messaging Gateway) with SMTP id 21.D5.15370.CCDE4865; Thu, 31 Dec 2015 00:56:45 -0800 (PST) Subject: [PATCH 2/3] IB/srpt: Fix the RDMA completion handlers To: Doug Ledford References: <5684ED4B.2010303@sandisk.com> CC: Christoph Hellwig , Sagi Grimberg , "linux-rdma@vger.kernel.org" From: Bart Van Assche Message-ID: <5684EDCB.30606@sandisk.com> Date: Thu, 31 Dec 2015 09:56:43 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: <5684ED4B.2010303@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBLMWRmVeSWpSXmKPExsWyRoxnke7Zty1hBrMOGFq8PP+B1WLl6qNM Fs8O9bJY3Gh7wObA4rH7ZgObx7Pph5k83u+7yubxeZNcAEsUl01Kak5mWWqRvl0CV8apKzdY Cn7xV5z5YdbA+JG3i5GTQ0LARGLe5VnMELaYxIV769m6GLk4hAROMEr0P5jBBOHsYJR4ufYn I0xHy5Q9LBCJzYwSHbffsIEkhAXsJI6sOQBWJCKgJrHp1SJ2EFtIQEti2dnbYGOZBZoZJTZe +wS2j03ASOLb+5ksIDavgIbE8gnPwAaxCKhK7JmzBKiGg0NUIEJi0Y5MiBJBiZMzn4CVcwpo S3yef4kJpIRZQFNi/S59kDCzgLzE9rdzmEFWSQhsY5X4fH4jM8QN6hInl8xnmsAoMgvJqFkI 7bOQtC9gZF7FKJabmVOcm55aYGikV5yYl5JZnK2XnJ+7iREcH1yZOxhXTDI/xCjAwajEw7sg rSVMiDWxrLgy9xCjBAezkgivfTFQiDclsbIqtSg/vqg0J7X4EKM0B4uSOK91i1qYkEB6Yklq dmpqQWoRTJaJg1OqgdGt5veV5nnW//vOqjBIXM/dfPdk5uqDa88I5f4/bL3zcW9U45e33wwn tC/r/t63aIPOnivPiutsF7D86dMuy3sZ09x6aW+R3QLLHTVr7y8qMgh4+nm6Wr7ppnc6sS9m 3J0+w7armfPIu4jsTAnlVZcDuR2PLehLsNlS4iHTnx9Q31+87UKft6QSS3FGoqEWc1FxIgAK L/etiwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPJMWRmVeSWpSXmKPExsXCtZEjRffs25Ywgy+9KhYvz39gtVi5+iiT xbNDvSwWN9oesDmweOy+2cDm8Wz6YSaP9/uusnl83iQXwBLFZZOSmpNZllqkb5fAlXHqyg2W gl/8FWd+mDUwfuTtYuTkkBAwkWiZsocFwhaTuHBvPVsXIxeHkMBGRol53W8YQRLCAnYSR9Yc ALNFBNQkNr1axA5iCwloSSw7exusgVmgmVHi7/GfbCAJNgEjiW/vZ4JN5RXQkFg+4RlYnEVA VWLPnCXMXYwcHKICERKLdmRClAhKnJz5BKycU0Bb4vP8S0wgNrOAusSfeZeYIWx5ie1v5zBP YOSfhaRlFpKyWUjKFjAyr2IUy83MKc5NzywwNNQrTsxLySzO1kvOz93ECA5SzsgdjE8nmh9i ZOLglGpgjFCSO/Tl/P/KUHM1zqPvqk+euCmxtHz9wc7JZW31evOF+z2CPry54tn24sTBqBu6 vesDeQ6pN/Wy5c55KRjVOXXHFacPTAHz5ucc3sexc7VGV+vJue9Ffr1VvVN1jDclYbd8Qbal Zbbd968Hwy/en+yvOP+qdF5QQ6m3dKTD4fvnV1xUsmYLUGIpzkg01GIuKk4EAIOVlF0CAgAA X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD026; 1:ymiYWnCtuXuimuvk+y1C3UFdBete2aB6tjIrkg9L+8pFPg8fBr/E+5f2N4elGM4qMrLsa1DR3F07IuoDh/TaUfSYexrsZ+dA2n7OBDlr0+C1jJOZaW46lq7/FJetBMtXEaBt8vkiXhFXO3B1YAXGhwnOFapvaCKpbr3yIrahylysS2dTIHKue71TZsHEeBGbpHJFw0IvJJtvhDZ3hwXnBTBHgzRIXwvkjL8Ixr4pRHXoxrSE5Piia+wbD/qdbzI3ueXajOXPQ+pdMcgkMQTw6yY2juPHSpH6PkFw3tPJpb/5Mbm15cI7ThXwbPrgrKVJbCvC9fDfDikEwdjF9lUOFg/9e7816MgMEqEfxypJ82tKxtWrleqHckWEQJiJXzgOUIB7UhNOFWL8TnU2QcYxwA== X-Forefront-Antispam-Report: CIP:63.163.107.173; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(199003)(189002)(106466001)(65956001)(33656002)(2950100001)(575784001)(50986999)(47776003)(1220700001)(4001350100001)(54356999)(50466002)(19580395003)(5001960100002)(65806001)(77096005)(69596002)(586003)(110136002)(86362001)(97736004)(83506001)(19580405001)(65816999)(81156007)(76176999)(5008740100001)(1096002)(87936001)(36756003)(23676002)(11100500001)(64126003)(230700001)(189998001)(92566002)(229853001)(4326007); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR02MB290; H:milsmgep12.sandisk.com; FPR:; SPF:Pass; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB290; 2:RO3sFHh8rwkF8A74jYhAoHUbVLc+mbmvlMvZlDqAToDIMwEscSZ7o/DH+hh8XYSFZ8R3IOWZky9nxp9fqYUrZaYLwdCNqgnFPGu6Acaqw8Q9mB1Am6KjcUPLDsEmLhnEI4znm2N2D+Gt4ZujbKmMKg==; 3:3B/Z4RrYisqQSanZbuk0UbFjpUdGbA3vsCU/xl3pyqF3LkRMGRxS8CuIAbo9n8iRMS4Pw+8mSUPGL8lbGgV26ScvEK/2CxpT5wi4oJ/1xH6DBk09PieEzCocLI2ESOXcK3sUxZ3VFHA5yzdKzAwtZJxQQ5tl2/hPFM9fMlgbCPfFp75fEAAbR7Tn1bd7AO/awy01e1EgWapJBSasKQufINxuaSgYTCR05fZB2qoNMI0J/h4ha0o5R00xvVVP4X5W/Zo/JMOcMfI4yFejqPsOnA==; 25:AvA6HxzbL2X6zt4Ni62kerwygvCJEBEXf/lFjllwwgjix8g8b+fBDD9/HQfyfgOe5O4WUjDKc+ANyQFh3D5HA3hoWqCNxxySV613XI5jS354UtgB8eTMRd9l4XrmSEx8G3wOqBsLw3xck7esW6em27B0DJulG/PM9EsQzkigphVVnCDLBIIkPd0Kr1+BCp8liFIcKY8xXzfB+eIyrQ9frqfRk/RXHadsI8S+Udf2znuQxwBpHps/Y3KyNZwszZ+kkXCu1R2UHGA1BTbaJ9un5Q== X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(8251501001); SRVR:BL2PR02MB290; X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB290; 20:OrPyeRsH+WvUGKMI4N5HGln10EfNWLYfcwXSAcATkpdwiaaLAuL/PwfyNs0zGoIYBT1P+i8hDrtQ3TIe/F9g+TmF8gnTmI45A11WEwov7pjg6HifZLb0pffe1EsXnnPzLGZPkRbwUSvclbxCSqs9u2uO0MWrrttl7/Cg7cJTaXBlSYYzF12zpzG6ZgZuW834hearLw75onWkSXYVUAxd8Aq0xSzLKPlo8WtWhlw34LhFe7RtNJVUfA5SB/HFH9ntXj0fJcEFwtcBnObeKRF7J/jskO0ZaXLGXRWfmsbJ3hlcsvxP1DbECm26T0KtpEW9d0mYDMnwXaJ/y+jUeoep2ak59KU96wCqnEMjGDiD9g3vcpPRlqBfZnSd1akWYWJ9WOIUPb2rJTcHNwmvLF0NyCh9gzVKoFDE6lX5p4IEGnOVY6PIrVC+C7SQW135LYRdznt4llQ1WIeoJ8Gb3sW9pg1Wltyiopmumn8sTUp1ZL2B79vGqu6I+6SYyJ7oPca4; 4:hxiqUvd8FBfl4/O0D1jL+1IuB98j1Oddtc2gvNzOosrKZ/F3LuZu/CqyMdPvDhsyrCV12iX1JFIfLna55CCiGVpvy9DEgblqkF5lgldiSL1HsQ4wUDd3Ep5h9oq9Ofs4ZQm4JU4kJFxEOOTDUD5djRcHz+XPY+MwSk8bPVGKpSPrRCkYv0YftzDS/GLkzQiNIp+jOtlAC3KVpiRVO20q1n99/UuObjpBCTXVPLcatbEzpStw2HtMQJZEBIVyRLthwQFcgdKjhJG8safmd/k7VbRMBHgdwqzLPCiXeCW4IgpmhQ/dd9MK26MtNMqD+8fI3nVOTMLdJCHwOthWDxQ2fzNEDJ2WHXqyenAkzISyeXnAmPGMwpwBDtkyKXXGii+1K9uaRZnhgmpmFbQc49q3qqQhKZGfaJjNkuBIdcg HKgl2pSieXj1BL13aUgeYcnsU X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(42932892334569); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(8121501046)(520078)(5005006)(10201501046)(3002001); SRVR:BL2PR02MB290; BCL:0; PCL:0; RULEID:; SRVR:BL2PR02MB290; X-Forefront-PRVS: 08076ABC99 X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTDJQUjAyTUIyOTA7MjM6U1h0ZzA0M2F6KzVOOWVCS3ErU2I4ZEVlMWox?= =?utf-8?B?d2FnQnJ3NXFDbnptazUrbnJZOHlMS2c5MnVuVEZlRW0zKzNhaHdPWk9nenN1?= =?utf-8?B?L09pWWJYZmZITkMyR2lIcHNFbkg1NzJRK3JVV2JUUDFHanZtNXVqazNSd2dU?= =?utf-8?B?THc2ZFlRZFJsUGVhVmNiUTBWYXFVSUh5ZUpWRTBSUjNnRDEydmxFVmJFRXBh?= =?utf-8?B?dXkyTXJFbVJRbTdHamE4cWkxY2dZMFNXOHBJVDEvNHB1SHUrQ0hLcW85a1g4?= =?utf-8?B?VmJIMXRuNEUwcWNuNUhCUkZkRU02OXlQZUc4Ri9uUjZWMlNZK3AwOWJVcUFk?= =?utf-8?B?SEZJZGRFLzJTK0dBa3Bta3JxWVpLeW1uM0xzQXo0UGZ6eFN2OUFtUDdIM0Y3?= =?utf-8?B?aDErOU1JcWF4b294d1BOVkZqVXBXS0JLeVlTdmxMdHdZUk1IQmNjazVQVytG?= =?utf-8?B?TExFR0YxWm8yNWJhTisvR1ZYMVR1Rm03bVIvclo3dC9aMGFZb1M2SG1qdjVB?= =?utf-8?B?SVJCMmlmdVRaRXF4b01FSHNFbDlsWjFEb1dmc0ZPTVFsQUVwV1JqVkQyWlM3?= =?utf-8?B?ZDJWUDZRR0RiVjFabndEOCs1VkRGQTJod3ZRRVZ0R2I2aUNLQ0w5OVNMZGVL?= =?utf-8?B?Q3liQjBkWFVseWpadlFvU1dOcTVLU0lXd1NZeDdCZ3FIM3RqNHNrR29YeWpE?= =?utf-8?B?TS91eGJtTVN0amsxMFNSc1ZUVWtGNXRDNGxVRXJ6VCs2Mmh2UDk1WXo2MUxH?= =?utf-8?B?aUZkcUVpTkFodEV0WGhOYm9sUW82K3E2TVBacFdFY1o1WFpCaDhGQ25YaGVJ?= =?utf-8?B?d2wzcVcvak41dDlKajFTR2JqbmluMWoyY3BsQlhqUExJeWdxUEd3RnBaZUZO?= =?utf-8?B?NUFSS0lWSnZxY2Z2Z09hMTk2ejlPQjM0YkdvTjFiOXExQm5lYUJ4YVpscmNL?= =?utf-8?B?cmtralR0NWYrcXVLVGNYLzVDK003WDR6bkEwdXJ3TmNpQ2pjQTJVamtrZ1NS?= =?utf-8?B?QUFQMnM5OVVuUFI2UU1vT1ZXcFFvSTQ0cUdwNno3emdLNkZ6dzZQNXRXcFBG?= =?utf-8?B?b1pDMTQrcVJpOTZ6cnN3OGVxMWxhOVBOMWhsQTRmSFBOU1poeUFMT2tIeFJj?= =?utf-8?B?M0Z1ZjhZZ2FYVDlaS1JVWnBTYkdCUnl3dEFETFY2TGxyUnJEUHpEbnNLS2Rk?= =?utf-8?B?WTd0VEl1THlMZ2xGY3VQTkY3U2xZRWxwcHA1SXpoVDZZbitTMjJGTW9Sb3Vj?= =?utf-8?B?VnlHMDNXdVg5SE9qemwyTVc5Q0xPNVRKMS9kbVM3VHlORHdab3diUGI2ZlV0?= =?utf-8?B?dmg4Qm5CNktFb3ZBNFhPWU5OUlpJZEQzZ29CVlZlcGJXNEVPeTE4NGd5amVQ?= =?utf-8?B?MjROWm92UFU2NGFldXhWTFFEODkvU2o5RGpIRllBOWdKV2djMzdNYlRQUXFh?= =?utf-8?B?SHVMblFac1JsWXFxVlRmaUNNZkl2VndnWWU2N2IxdTc3ZURNSCtURC9EK3hC?= =?utf-8?B?dS93PT0=?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR02MB290; 5:wg1+8dX0Vcmly9E4lRSfaoQGtxQy5Xu2V4c61Wh4fyxYp/McwGeCbTAYbTSk1K6kdHpq+F5M4jW/5ePCaDs7VFRaTBUr7X4ROM+FfEujpMZjyZU5jcDk3xrBZ0FGzSksqxM8TtjbWNM17wK3lQmK8Q==; 24:t9vTyeRlruT+ADcr5ph6WfqFpeiT4jEyjbk0/plZH6N3B1zuyCZteKU6e98NSE7XTqYhy8mYhKv77YEqdU8aypmfnqu46rnlXJVCQjU+MIQ=; 20:1PXS2k34CBohuaXIWGC5nt/i6OmWEBmxG8dIM8s/+0Q5WutHBA2iQfUJ8pGxrRmbTYpzCOadHjOekyxM24Q/+ulEcnMYiP1NQBA0/sQBnfHC798PBteiDtJFjY1YBMZKxKOq8QaR7xlhWQCDt4rgA29b5z1AtdBH2ywEWyDbXR7OSa8gvnhKOE7v4NiUbVHOSg5pCxaDobVnUdOgTPWNqOKIbANNrRR1RYuFWdOUOKD1xDaGvv9dW5LfMVEWtMQM SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Dec 2015 08:56:46.0460 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.173]; Helo=[milsmgep12.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR02MB290 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Avoid that the following kernel crash is triggered when processing an RDMA completion: BUG: unable to handle kernel paging request at 0000000100000198 IP: [] __lock_acquire+0xa2/0x560 Call Trace: [] lock_acquire+0x62/0x80 [] _raw_spin_lock_irqsave+0x43/0x60 [] srpt_rdma_read_done+0x57/0x120 [ib_srpt] [] __ib_process_cq+0x43/0xc0 [ib_core] [] ib_cq_poll_work+0x25/0x70 [ib_core] [] process_one_work+0x1bd/0x460 [] worker_thread+0x118/0x420 [] kthread+0xe4/0x100 [] ret_from_fork+0x3f/0x70 Fixes: commit 59fae4deaad3 ("IB/srpt: chain RDMA READ/WRITE requests"). Signed-off-by: Bart Van Assche Reviewed-by: Christoph Hellwig Reviewed-by: Sagi Grimberg --- drivers/infiniband/ulp/srpt/ib_srpt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 105512d..b42cf0d 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -1395,7 +1395,7 @@ static void srpt_rdma_read_done(struct ib_cq *cq, struct ib_wc *wc) { struct srpt_rdma_ch *ch = cq->cq_context; struct srpt_send_ioctx *ioctx = - container_of(wc->wr_cqe, struct srpt_send_ioctx, ioctx.cqe); + container_of(wc->wr_cqe, struct srpt_send_ioctx, rdma_cqe); WARN_ON(ioctx->n_rdma <= 0); atomic_add(ioctx->n_rdma, &ch->sq_wr_avail); @@ -1418,7 +1418,7 @@ static void srpt_rdma_read_done(struct ib_cq *cq, struct ib_wc *wc) static void srpt_rdma_write_done(struct ib_cq *cq, struct ib_wc *wc) { struct srpt_send_ioctx *ioctx = - container_of(wc->wr_cqe, struct srpt_send_ioctx, ioctx.cqe); + container_of(wc->wr_cqe, struct srpt_send_ioctx, rdma_cqe); if (unlikely(wc->status != IB_WC_SUCCESS)) { pr_info("RDMA_WRITE for ioctx 0x%p failed with status %d\n",