From patchwork Tue Aug 11 00:06:29 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 6987051 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 5B24EC05AC for ; Tue, 11 Aug 2015 00:17:02 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 357D0205D1 for ; Tue, 11 Aug 2015 00:17:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F3B4F205CA for ; Tue, 11 Aug 2015 00:16:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933411AbbHKAQ5 (ORCPT ); Mon, 10 Aug 2015 20:16:57 -0400 Received: from mail-bl2on0083.outbound.protection.outlook.com ([65.55.169.83]:18106 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933339AbbHKAQ4 (ORCPT ); Mon, 10 Aug 2015 20:16:56 -0400 Received: from BLUPR02CA039.namprd02.prod.outlook.com (10.160.23.157) by BLUPR02MB1252.namprd02.prod.outlook.com (10.163.80.157) with Microsoft SMTP Server (TLS) id 15.1.225.19; Tue, 11 Aug 2015 00:06:31 +0000 Received: from BN1BFFO11FD040.protection.gbl (2a01:111:f400:7c10::1:184) by BLUPR02CA039.outlook.office365.com (2a01:111:e400:8ad::29) with Microsoft SMTP Server (TLS) id 15.1.225.19 via Frontend Transport; Tue, 11 Aug 2015 00:06:31 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.173) smtp.mailfrom=sandisk.com; vger.kernel.org; dkim=none (message not signed) header.d=none; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.173 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.173; helo=milsmgep12.sandisk.com; Received: from milsmgep12.sandisk.com (63.163.107.173) by BN1BFFO11FD040.mail.protection.outlook.com (10.58.144.103) with Microsoft SMTP Server id 15.1.243.9 via Frontend Transport; Tue, 11 Aug 2015 00:06:30 +0000 Received: from MILHUBIP04.sdcorp.global.sandisk.com ( [172.22.12.162]) by milsmgep12.sandisk.com (Symantec Messaging Gateway) with SMTP id AE.A4.06265.58C39C55; Mon, 10 Aug 2015 17:06:29 -0700 (PDT) Received: from milsmgip11.sandisk.com (10.177.8.100) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.224.2; Mon, 10 Aug 2015 17:05:42 -0700 X-AuditID: ac160a69-f790f6d000001879-56-55c93c857512 Received: from [10.60.52.33] ( [10.177.8.100]) by milsmgip11.sandisk.com (Symantec Messaging Gateway) with SMTP id 09.5B.03643.58C39C55; Mon, 10 Aug 2015 17:06:29 -0700 (PDT) Subject: [PATCH 1/8] IB/srp: Re-enable FMR for non-page aligned buffers To: Jason Gunthorpe References: <55BBF4B8.2050700@sandisk.com> <20150803152420.GA24193@infradead.org> <55BFB40F.8000500@sandisk.com> <20150804180933.GB5038@obsidianresearch.com> <1438756876.5698.2.camel@haswell.thedillows.org> <20150805195122.GA31595@obsidianresearch.com> <55C2840C.5050301@sandisk.com> <55C2912A.50709@sandisk.com> <20150806001006.GD2483@obsidianresearch.com> <55C2A7FE.7020904@sandisk.com> <20150806043642.GA14153@obsidianresearch.com> <55C93C61.9010508@sandisk.com> CC: "linux-rdma@vger.kernel.org" From: Bart Van Assche Message-ID: <55C93C85.6090003@sandisk.com> Date: Mon, 10 Aug 2015 17:06:29 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <55C93C61.9010508@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWyRoxnkW6rzclQgw0n9C2+3zC1eHaol8WB yeP7jl5Gj8+b5AKYorhsUlJzMstSi/TtErgyFr0NKbjOX/FlVTt7A+Nbni5GDg4JAROJBf99 uhg5gUwxiQv31rN1MXJxCAmcYJR4feQHO4Szg1Hiy5YzjBBVJhIbupdAJTYxStxrWMYCkhAW cJd4+ekCG4gtImAqcfz4HSaIovvMEntmtDCDJJgFbCX+fvkM1sAmYCTx7f1MMJtXQEvi4oV+ NpCTWARUJa40uYKERQUiJCa87GKFKBGUODnzCQtICaeAtkTjVFkQk1lAU2L9Ln2I4fIS29/O YQbZKiEwjVVi07GtYNOFBNQlTi6ZzzSBUWQWkkmzENpnIWlfwMi8ilEsNzOnODc9tcDQSK84 MS8lszhbLzk/dxMjOOC5MncwrphkfohRgINRiYd3xubjoUKsiWXFlbmHGCU4mJVEeJVfnggV 4k1JrKxKLcqPLyrNSS0+xCjNwaIkztubqxMqJJCeWJKanZpakFoEk2Xi4JRqYBSZU2ktej4i 66TQqsaIzFznNkX1TRn8R7YUM9wDBt2hx6pXirccO6edv77+ohbX0r4zH9Qrva3+FW+84Xhd 6ZRaen+bopR6J//aiaJud27tWvrz1ppY480ntj2YInIqMCXHXKosYut1cT8vnYMshy8z+crn lvFdPN/ep6HElNPvffjbUgZ1JZbijERDLeai4kQA+qEisHQCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrJJMWRmVeSWpSXmKPExsXCtZEjRbfV5mSowbx9yhbfb5haPDvUy+LA 5PF9Ry+jx+dNcgFMUVw2Kak5mWWpRfp2CVwZi96GFFznr/iyqp29gfEtTxcjJ4eEgInEhu4l 7BC2mMSFe+vZuhi5OIQENjBKrLp2lRkkISzgLvHy0wU2EFtEwFTi+PE7TBBF95kl9sxoASti FrCV2LK5nRXEZhMwkvj2fiYLiM0roCVx8UI/UDMHB4uAqsSVJleQsKhAhMSEl12sECWCEidn PmEBKeEU0JZonCoLMVFd4s+8S1DT5SW2v53DPIGRfxaSjllIymYhKVvAyLyKUSw3M6c4Nz2z wNBQrzgxLyWzOFsvOT93EyM48DgjdzA+nWh+iJGJg1OqgdG9k63ffU6PytyKJSL6nv/Xf9y3 Yqn+5CplXe4uGY/vt7dXK8iGKISVfjzDZ9jQ/0l4eeTFMGajk05rn2xUK0iuPPxUrFLoxfz7 62NO8L52SIqZdFNf7EypvauHXv5LxgebTjM1mHicdtwTb3vAuGZt2LWg27uW/2H6rvo3/+fB RZ+OiM4P8lViKc5INNRiLipOBABB7+387AEAAA== X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11FD040; 1:EGxDxo0TpDc6L8QHNx2iWo7A6HCtlWbWKSU4/DS/2m/W+VCU3n8rfuoYlGrolEB42cQU7cvSfE4/Js5YpWELO8kpkLIm2S9aWnunL5WOBspU6F+ITQoitYTv+CUD+mfEHQ5QQbI9m4rbmr2hiy58h+xEUmRwuew/2YooeI6exm3RaELYElUb/zhT3t2BH3x0sqRJKsCqDI/e7TjWTnYehx9398NcmD66heV56ZlSEgu0ravPHc4rKUoPJN5+GezNjuPSybTXHxygOoEoXSNmsZ0tKIKT0rqHt9FQ/+MnPymnwydvZKYu9UW9q/nx+BbslrlrREUTFjYWMDRNWm7mCI8aNNL3yzaBZb0Is1cKtcCNy0N3ufbUJ3ENFG66Hq1v X-Forefront-Antispam-Report: CIP:63.163.107.173; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(3050300001)(189002)(199003)(81156007)(33656002)(50466002)(36756003)(54356999)(189998001)(19580395003)(62966003)(229853001)(65956001)(83506001)(87266999)(69596002)(46102003)(65816999)(93886004)(77156002)(76176999)(64126003)(50986999)(87936001)(110136002)(77096005)(86362001)(59896002)(68736005)(80316001)(106466001)(5001860100001)(15975445007)(19580405001)(64706001)(92566002)(47776003)(5001830100001)(2950100001)(5001960100002)(4001540100001)(4001350100001)(97736004)(65806001)(23676002); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR02MB1252; H:milsmgep12.sandisk.com; FPR:; SPF:Pass; PTR:ErrorRetry; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BLUPR02MB1252; 2:EnWCwdsLCwA842weKZMYJ9EEJODric8d5+voNMSiG6q8sN8DmaInmwpVGNWuLPeuhYYB0seWu99uvpBi4sO+wj1IROvYyQPt569ZXDDM6M+GtzA49BwEIxp8WtVr4EDSUaaCtzEkb9xQ59szwi/8iPUzbMpnNVOifm+Kvxg4Ozw=; 3:oXHJ5NXA5/6sPXi3gCSi/q/CrAU+WTvtUYfmqlXHBAvtTmrdOLVvwZAyBkjTGP8g/KMBRr1As70ax1QqZFhm3oHJujcERfUcnX+6fJWGCsKSySFjtAXldgQzseDlZq3+9G3Xg1PYhr4S053NalSjbGHr0rwmPMSiXQ103DvUj2AEjnmVmqUarVduHZ7awioew2pTIontvakPoZxiVPuG06juOraHUytoHFDI/1Aks2FDUeEuAvWV2zDOGC858MtJ; 25:g7bDWPg1yg5IfpyLchM7Pmwd6qPZbMG3tzqNDFDatiHyKLa/y3+Y94M8hQ/i85tn00BogI1LfutCI1kiTAmoS4rz8hixR3rmUXUTGMTD7jmB9yTCT0AzzCb60M/5SwGUJPPwznljSq+hjETCaeVB04+wM0qkvLom3MBtGyyzx+6LuebGCBtn/JNPw053ZxJFctNMzWtOuCfVaAPeyZp3hhfNGfbGkwnPtnY97+aomESFQir+ww9sGAMvB50oDzizmk7naRVigQbDoAa3g8wzNw== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR02MB1252; X-Microsoft-Exchange-Diagnostics: 1; BLUPR02MB1252; 20:OrX8mTYH4d8DOZavCsEc3ktqOlK+TC0xPNoTogzLiwNcnn+4j2bUjk+E9XoqUmsmKY88JGSvInlN01ea5PNP5ZZofHvQKLxegU4SWaQpxNPPTyqGTgIx3pb76CT/G4uGCK706qNfppxrxqFpWbi2qzO5hg/77FJiARTq/bFikgnAyZ1c8UjvP5ehapntHyDuF1ebIrftXzYbvj/MYB+mIPiM8frpet/7VrGrQxwrfvqv3YGlOj6BkQu7cSLHN2hDqNC0iaxFUALgl60FynPl3jLK6KiKpob4M3cpsSugrh3tcGNoxyeroZJXe0EJV4lErre4kPUGDU+NUubqs3/aZTv/qmDwv75EGHR5fAXgf6rKpWEVmM63uZYf3JMVW7DP9m0viIejVA1u+jL0/p9tUmGDhxTZqlMO63Svm8ksC8645cutS7hp1W0mInqSnjvnKsr58XDi83CAswgEbS+UTBfNV+CWckAGVWdw6k7Kupe6juUmbjpsZwfSRZUUzVLN; 4:MbDJkjGwpuZ6ILIiktcmNp4jMV2fs3o0dLG2Zcdgrzj13zDdGLKTU8UV5v6XRnEtSaTFUV+jYG30cOtt5YZ3X5AFKLKAoeBAJ0oRmf+/e8CsczKjTG/uf9cegQ9Cx7251re1cD1KC2rBvWA28ZwoqxKKMQr/FIcwCQ+zbtdI7iUJlUIuQKvs2jX9nXtpHC9Z9MG9GNro1Pimw3RtEP8IT7U59P17jh6x/x0FQZmhyH7PyninBbAn51wU9sGNf6nyDfqK4L3D/R3+FAOrwP6Z3OVfMYBJPZCn1r9BXAe6eRI= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(3002001); SRVR:BLUPR02MB1252; BCL:0; PCL:0; RULEID:; SRVR:BLUPR02MB1252; X-Forefront-PRVS: 066517B35B X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTFVQUjAyTUIxMjUyOzIzOi8zSU9tWE9naGkxa0VQSmpLL1dQVmhuVEJr?= =?utf-8?B?WDkzbFY4azR0UEh5SDJjRFZna29pdUVtVXBqZi83STJpaFJjSW1xMCthQnly?= =?utf-8?B?NjhNQk8rRnhEVGJkY0tSV3NJYzN3T052bmVaKzR2UDFVWHBZd1pYUWg0N0xm?= =?utf-8?B?Sk1ybTNGVFRRLzNSNG45WExhcVNudmtrKzNNV0thdDRZK3JwZWNwd2Q1R1pD?= =?utf-8?B?WXpXRFdQQVRoZDcwUzZzaklTZmtNQnJnK0loMDNoWmNZQ2Z2cHJpSC9Uek9G?= =?utf-8?B?MUtwcjNibUR5Sk5hdlRGK1JjSlZDcWUxcy9TMUhOWkVFaXdFWXpDYSthOXRO?= =?utf-8?B?SGhZWjEzR01VMzNsc2Z1RDhwNUdvSHR4dE1aZjBhN1BoZ2c1NVdwSExWRjN4?= =?utf-8?B?NVFKL1FIaDhIQVkzNjZtVWhKZWRFMDNSdUEzN2NvOGNtd2wwMTUzdlNPWnVw?= =?utf-8?B?eDVCWE5aaS9iS3diZ0tLWGFPWGFXZkJnaWJjUzAreExtbFBTOEdWbUFVMWlJ?= =?utf-8?B?N1V0RFhVWGtoRWR4dm9DbUdHTTBOSUZwbVZORXBpWDNJWFQ3MDAyN2NvNGJo?= =?utf-8?B?TlljcmRwMjZ0RmxIQ2Jpem9TVElDQUxiVm8vYmNCRkJyZ3lFYjljUmFIVCsx?= =?utf-8?B?TlRsZ2gzd2ZzQlhXYjVudzA5dnY3anJ2eUtUaEZxL0s4eFBwN2c5VHJhNndo?= =?utf-8?B?QW13UkR3VlhuK0FQN0NqSkpDT0kvczZ3UWNWckVicURNVVRWR3Vza0dxSEkr?= =?utf-8?B?SlZNK3NWaWRNYVNUZVo2KzFGOXpXemJsSEErWWNkSkxVVG5KNmVtTVY5V3hQ?= =?utf-8?B?T1g1VVhndjFrWElqc1c0a25ETTg5L2plN3pnSnAyY0tMMGVOOXB3S2c1R1c3?= =?utf-8?B?TmVrTTdFaVNBSFRhTFVxQzJrMjVnUjdzOW1nZ2U0ZExGeUlqNUg0K0Zvam5h?= =?utf-8?B?c3hoWFB6Y1BSb2dVV2l4cGgybWx0Q2t1dHhzUHJTQWpvZzFwdVBIcTV4ZHI4?= =?utf-8?B?TmE0QnFmNzlSb0dIOHkwaC9KSXBwaXNEbWxYdzRkZFJ4MHNwc09iRnNnSEEy?= =?utf-8?B?VXNKUFFpNHl2SndCbkdWUTVJendlUHBybUo4UnNnTEJZeDVHWVBIVXB1clNG?= =?utf-8?B?V1A4bGtCeElRR2Q1QVY5NVVJc0d1aWg0NnBMZ2FkajVTaEV1aG5hV05neWRY?= =?utf-8?B?OXp6NnhnbkZHekYzaEVHY3VkWk40cnZqa213NDh2MXhvVGFFalZEbHNzUFox?= =?utf-8?B?RVVJZDNaZlVmblZvYWt1ZHlLQjJ6ZEdJeVg2TUlFRk1icG9qNUI4aUJFZk5z?= =?utf-8?B?M1hVdGY3VkxIaEwwQXpYb3hkT28rUWVuMVROVU1kTlVZZ2Q3QVhYZXhoMWJl?= =?utf-8?B?TXpxWVBDUkREYzRxR2RJS0cyc3dZRHB2OVhJWW56b1lybGVCOWNLZjY3aDF4?= =?utf-8?B?SUFROS9WSFJ0bkYwakgyQ0pXZndoQlJiWFBwdW9NZVlyaUEvWDBwSVpKZWtu?= =?utf-8?B?NzZYRmRpcDRGaWdNVFJFOGI4VHpjZlIrcTJmNHBSZ2RPNjJCUzBPc1ZjZG1r?= =?utf-8?B?a3NHK2EreVIwT3NDOHFocFZMOGtPakY0Z3hXLzl0RmczWDI5YWd6elBoZkdh?= =?utf-8?B?V3RxYnk4andVU1FpMWRVVFlCZnlaejgvenlMb0ZRaktBRWFFaHV3cTQ1RTdq?= =?utf-8?Q?v854b29kUv5e2e8dl0=3D?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR02MB1252; 5:xi1Ufcstb+amCLLoCQ9kImmdJnia8a0z8ogi1OowOVJbqyZIHLNAOGR8wOv91qH0aCqmEu3QGNiVrNVFnVKjLrZlPjd3TUdmbGN54oV6YpElDtWGwI0uXepYN0n4eR2DsqfvcdTX4QbVcOe6OLajeg==; 24:qaGXVpClUDGX8mOQK+lLFmc2hEfivV+SNixqTc+ufCC5BpYuao1pP72bEahGmHGce9fL3QnAAdHq2hVGFMbfLLKky5f5nuFeCH7YXXD/3GU=; 20:euEXGE6IhETZA+RaeULaRsedlBlft7vJXDFFupFENOkQOySkcMHLcE/iXfNG56K8bl4jYsbFRleAidKnG8fGEYPpphLlWmVzeJ2QPC37SXypc3/kasvffpO8zKCqX2yHz2deyzp9mYmps4Q6xY/r+n+RW1mNvIqjNoMWO68ZGGQ= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2015 00:06:30.0634 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.173]; Helo=[milsmgep12.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR02MB1252 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP During a discussion in 2011 nobody recalled why FMR was not used for non-page aligned buffers (see also http://thread.gmane.org/gmane.linux.drivers.rdma/7149). Re-enable FMR for such buffers. For the reason why the srp_map_fmr() function needs to be modified, see also patch "IB/srp: rework mapping engine to use multiple FMR entries" (commit ID 8f26c9ff9cd0; January 2011). Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 2968b7b..887e8ca 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -1272,6 +1272,8 @@ static void srp_map_desc(struct srp_map_state *state, dma_addr_t dma_addr, static int srp_map_finish_fmr(struct srp_map_state *state, struct srp_rdma_ch *ch) { + struct srp_target_port *target = ch->target; + struct srp_device *dev = target->srp_host->srp_dev; struct ib_pool_fmr *fmr; u64 io_addr = 0; @@ -1283,7 +1285,8 @@ static int srp_map_finish_fmr(struct srp_map_state *state, *state->next_fmr++ = fmr; state->nmdesc++; - srp_map_desc(state, 0, state->dma_len, fmr->fmr->rkey); + srp_map_desc(state, state->base_dma_addr & ~dev->mr_page_mask, + state->dma_len, fmr->fmr->rkey); return 0; } @@ -1390,14 +1393,7 @@ static int srp_map_sg_entry(struct srp_map_state *state, return 0; } - /* - * Since not all RDMA HW drivers support non-zero page offsets for - * FMR, if we start at an offset into a page, don't merge into the - * current FMR mapping. Finish it out, and use the kernel's MR for - * this sg entry. - */ - if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) || - dma_len > dev->mr_max_size) { + if (dma_len > dev->mr_max_size) { ret = srp_finish_mapping(state, ch); if (ret) return ret;