From patchwork Tue Aug 11 00:09:36 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 6987091 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 019C89F358 for ; Tue, 11 Aug 2015 00:28:17 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 90FFE2037C for ; Tue, 11 Aug 2015 00:28:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 57BB0202F0 for ; Tue, 11 Aug 2015 00:28:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932862AbbHKA2N (ORCPT ); Mon, 10 Aug 2015 20:28:13 -0400 Received: from mail-by2lp0237.outbound.protection.outlook.com ([207.46.163.237]:53494 "EHLO na01-by2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932678AbbHKA2M (ORCPT ); Mon, 10 Aug 2015 20:28:12 -0400 X-Greylist: delayed 883 seconds by postgrey-1.27 at vger.kernel.org; Mon, 10 Aug 2015 20:28:11 EDT Received: from BLUPR0201CA0001.namprd02.prod.outlook.com (10.163.116.11) by BLUPR0201MB1505.namprd02.prod.outlook.com (10.163.119.16) with Microsoft SMTP Server (TLS) id 15.1.225.19; Tue, 11 Aug 2015 00:11:26 +0000 Received: from BL2FFO11FD032.protection.gbl (2a01:111:f400:7c09::191) by BLUPR0201CA0001.outlook.office365.com (2a01:111:e400:52e7::11) with Microsoft SMTP Server (TLS) id 15.1.225.19 via Frontend Transport; Tue, 11 Aug 2015 00:10:14 +0000 Authentication-Results: spf=pass (sender IP is 63.163.107.172) smtp.mailfrom=sandisk.com; obsidianresearch.com; dkim=none (message not signed) header.d=none; Received-SPF: Pass (protection.outlook.com: domain of sandisk.com designates 63.163.107.172 as permitted sender) receiver=protection.outlook.com; client-ip=63.163.107.172; helo=milsmgep11.sandisk.com; Received: from milsmgep11.sandisk.com (63.163.107.172) by BL2FFO11FD032.mail.protection.outlook.com (10.173.160.73) with Microsoft SMTP Server id 15.1.243.9 via Frontend Transport; Tue, 11 Aug 2015 00:10:12 +0000 Received: from MILHUBIP04.sdcorp.global.sandisk.com ( [172.22.12.162]) by milsmgep11.sandisk.com (Symantec Messaging Gateway) with SMTP id 9F.EB.04667.04D39C55; Mon, 10 Aug 2015 17:09:36 -0700 (PDT) Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP04.sdcorp.global.sandisk.com (10.177.9.97) with Microsoft SMTP Server id 14.3.224.2; Mon, 10 Aug 2015 17:08:49 -0700 X-AuditID: ac160a68-f790b6d00000123b-0b-55c93d408182 Received: from [10.60.52.33] ( [10.177.8.100]) by milsmgip12.sandisk.com (Symantec Messaging Gateway) with SMTP id 75.43.04661.04D39C55; Mon, 10 Aug 2015 17:09:36 -0700 (PDT) Subject: [PATCH 8/8] IB/srp: Create an insecure all physical rkey only if needed To: Jason Gunthorpe References: <55BBF4B8.2050700@sandisk.com> <20150803152420.GA24193@infradead.org> <55BFB40F.8000500@sandisk.com> <20150804180933.GB5038@obsidianresearch.com> <1438756876.5698.2.camel@haswell.thedillows.org> <20150805195122.GA31595@obsidianresearch.com> <55C2840C.5050301@sandisk.com> <55C2912A.50709@sandisk.com> <20150806001006.GD2483@obsidianresearch.com> <55C2A7FE.7020904@sandisk.com> <20150806043642.GA14153@obsidianresearch.com> <55C93C61.9010508@sandisk.com> CC: "linux-rdma@vger.kernel.org" From: Bart Van Assche Message-ID: <55C93D40.2020608@sandisk.com> Date: Mon, 10 Aug 2015 17:09:36 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <55C93C61.9010508@sandisk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrELMWRmVeSWpSXmKPExsWyRoxnka6D7clQgz2/zC2+3zC1eHaol8WB yeP7jl5Gj8+b5AKYorhsUlJzMstSi/TtErgyfjasYy64p1Oxf+oRxgbGGSpdjJwcEgImEt/v bGCFsMUkLtxbzwZiCwmcYJRYMSeni5ELyN7BKNHV8Z4JpuHnsdNsEIlNjBLX/8wC6xYWCJJY sXQFWLeIgKnE8eN3mCCK7jNL7JnRwgySYBawlfj75TMLiM0mYCTx7f1MMJtXQEti8pup7F2M HBwsAqoSE6erg4RFBSIkJrzsYoUoEZQ4OfMJC0gJp4C2RONUWRCTWUBTYv0ufYjh8hLb385h BtkqITCNVWLR/l1MEM+oS5xcMp9pAqPILCSTZiG0z0LSvoCReRWjWG5mTnFuemqBoaFecWJe SmZxtl5yfu4mRnDIc2XsYNw6yfwQowAHoxIP74zNx0OFWBPLiitzDzFKcDArifAqvzwRKsSb klhZlVqUH19UmpNafIhRmoNFSZy3N1cnVEggPbEkNTs1tSC1CCbLxMEp1cBYy650Zidz2/Sf zL1BSrmPLu0yn3UnJlHl76OwX8wtjWoaU1Qy7Vf++cCblfS1yUXyq1u3/ZuWBW67JyTkG1xn yvD873X5xyI5cRcXhqtrvDSkb7zLknNsllKKCN/26SXrjNm3dn79pFJyJHpF5hmbwpPP5izz EDc9seyj2JqjVytaC2ev3vhbiaU4I9FQi7moOBEARba58nUCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrFJMWRmVeSWpSXmKPExsXCtZEjRdfB9mSowZqZOhbfb5haPDvUy+LA 5PF9Ry+jx+dNcgFMUVw2Kak5mWWpRfp2CVwZPxvWMRfc06nYP/UIYwPjDJUuRk4OCQETiZ/H TrNB2GISF+6tB7K5OIQENjBKTLpxmAkkISwQJLFi6QqwIhEBU4njx+8wQRTdZ5bYM6OFGSTB LGArsWVzOyuIzSZgJPHt/UwWEJtXQEti8pup7F2MHBwsAqoSE6erg4RFBSIkJrzsYoUoEZQ4 OfMJC0gJp4C2RONUWYiJ6hJ/5l2Cmi4vsf3tHOYJjPyzkHTMQlI2C0nZAkbmVYxiuZk5xbnp mQWGRnrFiXkpmcXZesn5uZsYwaHHGbWD8fpE80OMTBycUg2MRj3nfGzu8qUHcjnGvn3k3rv/ qrjk3xccrayPo591x0i+TdeIU+DqWCsbFcG63rDqzCSu0KUyCydnMX5KW/TV4nT0KqaPDbkc /1uWLb3JtWC6NfeBpb8ZYg4FSOg/4no6T+pHRe5iy5wvbMm693cfjH1Y6i+5e4nX2yrdn7s1 a3xO5r+Rnn5GiaU4I9FQi7moOBEAVyHRxO0BAAA= X-EOPAttributedMessage: 0 X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD032; 1:q8qJCs1FqcqpDs4UOq4Zz6Wni/K4mYdiI4081R19wY86tVCxNWzsS2V3iaWP07E6N0gHXdU6ojmglibslN/jDkPuooYi9Hq+U1bXi3FvI+E4RD2nrhEYgyRugNFxVfqzjuOc7Jef/HzZv4yY9L4vf7r3rtzPOb/uo/16mAibDH3th0S/3NuRnL1PPT6zOtC2DJp0Jx+tkvccg55+vGiBf01CI3X9iVIlgqNv6k9EH9HuVOybGhssktOxw1yvNKiMgTgfWB2pwB9BGon5QyvPzz1q70boU4TfZpR6ESfqivVOmyfooHQJsFeE8BubZiDKE4gyQbik6iB7XVBYvIVGEg== X-Forefront-Antispam-Report: CIP:63.163.107.172; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(2980300002)(438002)(3050300001)(199003)(189002)(68736005)(92566002)(2950100001)(5001830100001)(110136002)(33656002)(97736004)(65806001)(65956001)(64706001)(80316001)(65816999)(69596002)(19580405001)(87266999)(54356999)(76176999)(77096005)(47776003)(83506001)(36756003)(50466002)(4001540100001)(5001960100002)(5001860100001)(4001350100001)(189998001)(81156007)(106466001)(64126003)(46102003)(19580395003)(93886004)(62966003)(77156002)(229853001)(86362001)(59896002)(50986999)(575784001)(87936001)(23676002); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0201MB1505; H:milsmgep11.sandisk.com; FPR:; SPF:Pass; PTR:ErrorRetry; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0201MB1505; 2:HdDLe//GpC3yfKRMRWttEuTrnCjqZEhh4SYbEv/14meuQxtmwZPMx9yKzaLPs8ZHOUg29MBLnMP+RQ+9l8TxH0+5vgTUGL9oyWOJ2thGI6fBGvLAvW1VU4SKcK7ydsq0a1O4rRASWK/aOkAd8ZF/X4/2GYFF6HRQ5L4wNdiK7u0=; 3:jiLl0/FtAxrw6PAU0kmxYfpRgoOPoWuE/AodG6MsJ9nyoHYNSRgJkaBBlpM3cmWO8o7UvLN04xwZszEgeUYbi/3E5vB/SPsxnyeFl1oFU5gzH9+teLOd99gAmIZdC/Ae8CaTJax1vvnh1zxpIAb8OWzVUaVc9WQOmJuDiLOGUoY6fAv2w3HfO06HcwbFbeUUUYTlmnhD9FM8pe7L5Mi5t4yjkNfzaEqGUhUjEonQbaBVkm5MjQyoCiDudrlCgbZW; 25:F2rGRjJkrVsI1xPgoCZZHfCHZll9oSGHO2g94VhLOU0Al1LvKLNPBt1YjldKE4nqNr+6ZcofOpP9GAyftwKyOQ3RIgX6lrFO2hrxAX9ctpt+swySEVE3m4VLciLiiJHfc1rvUTLGmuIFhVWnzeMlGUyDOmBOEdrn98yCgilgUYfw1e9Vvd/cKoD0pKIi/lNGIGPloEpGlK9/UffvBvNkErMQRAEXQoohng2PHZoHoMeIYDVCInlCQmN1YJI5sg5fD6q+KQ8KRMVmC4uPQf83Rw== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0201MB1505; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0201MB1505; 20:4K+IvCtE163PMk9TlQJ7i7mOBkRcq8zknkxz4cw2Wjn8qUTbehkfm0J6XubetSvZmrwq8uOCZRbntIy09LckJVOsqII3TsEmBIqIsntaTHRkgOG78NaSSf/yCUUV5S+sEP2xLPSiTE7ZwHTIbAc0EsoCmtS6jR72NAPOi62g//6ZE6rABoRuvB4Qn21WTSI3eInxxQgQj0fXbMmrecAGz1IFH3YVMz7RH5ZQNPGT7vQpQ7hYhdPnei+EiVrb9wqcMh6ogzJPLLk7PVZMq0dnM39TPisTTkzq9AueL+yqegRi2/kabKDBsBgNQ+yEmbey9WEwQXT4U34aK21HDMCv6NFzmF4TgieXlDy9z76DcTuXz79cWNP+yfwXVU7iZbwE497fYEbuNCCUfiNgX8JZosj61nhxcYeI3h7VfkO/2Hnu/xw6QFX/ZYDIxTS0D7TaBFcfKv389yHPgBuM2+vp3KuBvzhV2OW3CH8LKcFlW1nxYvhvyb7dS18Hn3Tq81JB; 4:BiZSmpPogYm068Bi2Lv7y3qeo7X2MdSg2zX/ml+yQAe2Bo5yziz5aCKrvOckdSlCNEEyaKmnL2iZqNmzCutrV/AvH3aKhv7ZPL3vm1ztssQlfbuUMr3830E348ziukEDU1I6HDTGxiW65lDCoTTiIexmNCzyRYxZm5Wj6DqzesJrCufOrW4khq7R1oZXXqbEwhZZI8NGt44cIT1QRQRfD3yV91FIjc/U6toKpBhUQAOe+USpIp7l3DEZPcVNeMdml3Imt7Ko6bKX1msanyw2HaYqJbfo5ea+ZvpQnA/1w3c= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006)(3002001); SRVR:BLUPR0201MB1505; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0201MB1505; X-Forefront-PRVS: 066517B35B X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTFVQUjAyMDFNQjE1MDU7MjM6UklKNU96NDlCcEVucGhnSDg0eEs5dWdI?= =?utf-8?B?T0RqeTR2cU9FK3pBTVVNWkMzL1ZuaHl6WTZkRkhGYnRXNjRIZ1BzVkw4dHlm?= =?utf-8?B?OHVpODVzUjNaOG1YWEdCelhQS0pHT1ArTWp6eXNwK2F5RWIvRWdMeTNmMEIw?= =?utf-8?B?NzRlS21zbUJwczkzSlZIS2xwYXZmNDY4LzFJTVUwRzZjczRwaHN3aFpTbWFz?= =?utf-8?B?dG0xQ1BiK1JhYktEZnBlQ1VCcEpQYXdZVldQMzFMcWhQSFNpMTRyWTBCbkNB?= =?utf-8?B?ejRsMDBvZkthQ1BqZURHSWcxWURQL09ucDlUV0EzQlZWR0tLZUR2NFFISE9L?= =?utf-8?B?ZmxmVE9IVHY3dnFoczBiSVBSN1I1SjY5K3Vac1VmNlRpbjJKOERMcUhIUXUy?= =?utf-8?B?YVFMMUJQL2ZPNk5ZQm5SSmNScFNwS1lPRDhMRlM1cFVzWDlSOWU1VmlSRVJM?= =?utf-8?B?djFGR2hodDNwZG1wS3VDckRHTnpWejV5NHNkdU1qaUd2T1VLbHk5eFdmYklF?= =?utf-8?B?SjNMT200T040aEVSRkU2WW9ldTR2Qkd2SFhDVHBUVTdSbCtsWnZIellwQ1F4?= =?utf-8?B?ZWt6RS81NWpPUmxMZHFLTDh5Z3VpTDFHd01CbGVCNXVmTHFMMHNXb2dWU1VB?= =?utf-8?B?NDVyVGJhd3piZkRTNGVpZWFVR2NNNXM1SVNjd2hrK3p1RjYvYUdyQTVzVGJO?= =?utf-8?B?UjBwR2xTSGdHV0xmVTBIS3F1aDR5T0hkcFN2ZmJ2S2ptcDR4aHdjTUp2dXdX?= =?utf-8?B?SEI5cTJJNC9LcmhvSjlJSGl2dzVFZzNMVm1FYWVYSGF2TU0wTnpqRjhuKy9p?= =?utf-8?B?VWN4U0ZGQ3FNSHFmbUROcVR0aVpWNmh6cnB0VnVZSmw3MC9MTGtmZXdXU0hs?= =?utf-8?B?Rjl4MENMam4zT1E4eHlJQjZxWVFqS1M4WEhmUkZ4UzJyb3NRZTI2MWZ5WERP?= =?utf-8?B?MVVvMWQwbWs4L05yRGhhRU1OR3F6M1c2amNNZjAzeWZhZmkwL0U0K2x4aE5y?= =?utf-8?B?RHd5ZGk0T3ljdk9YeG1PeENDb2d1dkpNTmU2bGZYOWlCc2lBclVESVpoaVZR?= =?utf-8?B?WVVVajlXN1BTMmducDFkZy80b3B5b2VjZzVGcFJ0VUpKWERiVXFPcWp5N3NF?= =?utf-8?B?NFpRNFl5MjFFamlIWTNkVzE2N044SjVhSkpHakczNC8xMW9NdURQeTl5cksz?= =?utf-8?B?ejlWQVM4dEpLbnpiSWU4Vk03NmNWSGcyZUVNSUJKSHNkTlBhMUYrYlZkb3B0?= =?utf-8?B?YmdyTWh4aHVDeWYvaUlyRklxeG0xa3gvL1VpbjdlOTRidi8vS1Jsb1c0eGZU?= =?utf-8?B?Nk5yQ2RxNDlYMCt3cXR3QXhTVDM5WWF4emxlUmwvMTJobHhKNHQvLzdyTExQ?= =?utf-8?B?Mi9XR0hHNDZHVlpxT2srWVJMdXdqN2NxMDNvT3NjcERPOHhyRkpqY0hJZnNs?= =?utf-8?B?czFuUVRNYThkeFFHTEVmN3MwelBDSCtsejZXbXdTRzc1MFBBeXdXY3hZYkYv?= =?utf-8?B?MnR1TC9MRjU3b3VmNG0zaERmOHBZQms2a3lxN0dJT0E2a25JcWp2N2E3TXI0?= =?utf-8?B?anR2M1p6SHhuWlZNaU1keXhSNjgwczVsMGpWMnBMZ0pBczFLS2oyekc2YVdx?= =?utf-8?B?ODhRZ1NzRkY1OXFqZ1ZEL2NQeldsOHdkRHpENkpoSWpCK2ZxSkNRMCtKK2pP?= =?utf-8?Q?11+39OH2M339XURFRxao=3D?= X-Microsoft-Exchange-Diagnostics: 1; BLUPR0201MB1505; 5:W+XOqWNirmRzLYI9OyPz4aENB4WjmG+Kksv5c4m5JoiPddIonwQxy3kgmB4ArQWwMuEkOwcH5tnM+Iuo5QyyLLjhSRGk3eEIChZFoCnyc0DfQW7QvxEAeJYbZGxsq/2lvVOgb29mI8KfFRShPYdu3Q==; 20:zmI81U8JVJfoYW+qoT9JKoq46oSw4M+xSXGkHmyFsPs99aMwoOxj55C2nFp4wlTru3fAeq//nyL0tZVtPsWwOfjBo/oFNVAvQBbnL0dvgjuU576Z4u+OQL5CH8JyMPAOn0MpESEz5V0+EuDPXM81TT8RfbgeNEq8ZCqIIWeWp5w= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: sandisk.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2015 00:10:12.4350 (UTC) X-MS-Exchange-CrossTenant-Id: fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fcd9ea9c-ae8c-460c-ab3c-3db42d7ac64d; Ip=[63.163.107.172]; Helo=[milsmgep11.sandisk.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0201MB1505 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The SRP initiator only needs this if the insecure register_always=N performance optimization is enabled, or if FRWR/FMR is not supported in the driver. Do not create an all physical MR unless it is needed to support either of those modes. Default register_always to true so the out of the box configuration does not create an insecure all physical MR. Signed-off-by: Jason Gunthorpe [bvanassche: reworked and rebased this patch] Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srp/ib_srp.c | 40 +++++++++++++++++++++---------------- drivers/infiniband/ulp/srp/ib_srp.h | 4 ++-- 2 files changed, 25 insertions(+), 19 deletions(-) diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 2a16ab4..192f737 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -68,8 +68,8 @@ static unsigned int srp_sg_tablesize; static unsigned int cmd_sg_entries; static unsigned int indirect_sg_entries; static bool allow_ext_sg; -static bool prefer_fr; -static bool register_always; +static bool prefer_fr = true; +static bool register_always = true; static int topspin_workarounds = 1; module_param(srp_sg_tablesize, uint, 0444); @@ -1353,9 +1353,9 @@ static int srp_finish_mapping(struct srp_map_state *state, if (state->npages == 0) return 0; - if (state->npages == 1 && !register_always) + if (state->npages == 1 && target->global_mr) srp_map_desc(state, state->base_dma_addr, state->dma_len, - target->rkey); + target->global_mr->rkey); else ret = dev->use_fast_reg ? srp_map_finish_fr(state, ch) : srp_map_finish_fmr(state, ch); @@ -1442,7 +1442,8 @@ static int srp_map_sg(struct srp_map_state *state, struct srp_rdma_ch *ch, } else { for_each_sg(scat, sg, count, i) { srp_map_desc(state, ib_sg_dma_address(dev->dev, sg), - ib_sg_dma_len(dev->dev, sg), target->rkey); + ib_sg_dma_len(dev->dev, sg), + target->global_mr->rkey); } } @@ -1531,7 +1532,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch, fmt = SRP_DATA_DESC_DIRECT; len = sizeof (struct srp_cmd) + sizeof (struct srp_direct_buf); - if (count == 1 && !register_always) { + if (count == 1 && target->global_mr) { /* * The midlayer only generated a single gather/scatter * entry, or DMA mapping coalesced everything to a @@ -1541,7 +1542,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch, struct srp_direct_buf *buf = (void *) cmd->add_data; buf->va = cpu_to_be64(ib_sg_dma_address(ibdev, scat)); - buf->key = cpu_to_be32(target->rkey); + buf->key = cpu_to_be32(target->global_mr->rkey); buf->len = cpu_to_be32(ib_sg_dma_len(ibdev, scat)); req->nmdesc = 0; @@ -1595,14 +1596,14 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch, memcpy(indirect_hdr->desc_list, req->indirect_desc, count * sizeof (struct srp_direct_buf)); - if (register_always && (dev->use_fast_reg || dev->use_fmr)) { + if (!target->global_mr) { ret = srp_map_idb(ch, req, state.gen.next, state.gen.end, idb_len, &idb_rkey); if (ret < 0) return ret; req->nmdesc++; } else { - idb_rkey = target->rkey; + idb_rkey = target->global_mr->rkey; } indirect_hdr->table_desc.va = cpu_to_be64(req->indirect_dma_addr); @@ -3150,7 +3151,7 @@ static ssize_t srp_create_target(struct device *dev, target->scsi_host = target_host; target->srp_host = host; target->lkey = host->srp_dev->pd->local_dma_lkey; - target->rkey = host->srp_dev->mr->rkey; + target->global_mr = host->srp_dev->global_mr; target->cmd_sg_cnt = cmd_sg_entries; target->sg_tablesize = indirect_sg_entries ? : cmd_sg_entries; target->allow_ext_sg = allow_ext_sg; @@ -3438,12 +3439,16 @@ static void srp_add_one(struct ib_device *device) if (IS_ERR(srp_dev->pd)) goto free_dev; - srp_dev->mr = ib_get_dma_mr(srp_dev->pd, - IB_ACCESS_LOCAL_WRITE | - IB_ACCESS_REMOTE_READ | - IB_ACCESS_REMOTE_WRITE); - if (IS_ERR(srp_dev->mr)) - goto err_pd; + if (!register_always || (!srp_dev->has_fmr && !srp_dev->has_fr)) { + srp_dev->global_mr = ib_get_dma_mr(srp_dev->pd, + IB_ACCESS_LOCAL_WRITE | + IB_ACCESS_REMOTE_READ | + IB_ACCESS_REMOTE_WRITE); + if (IS_ERR(srp_dev->global_mr)) + goto err_pd; + } else { + srp_dev->global_mr = NULL; + } for (p = rdma_start_port(device); p <= rdma_end_port(device); ++p) { host = srp_add_port(srp_dev, p); @@ -3500,7 +3505,8 @@ static void srp_remove_one(struct ib_device *device) kfree(host); } - ib_dereg_mr(srp_dev->mr); + if (srp_dev->global_mr) + ib_dereg_mr(srp_dev->global_mr); ib_dealloc_pd(srp_dev->pd); kfree(srp_dev); diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h index 255b0e5..3608f2e 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.h +++ b/drivers/infiniband/ulp/srp/ib_srp.h @@ -95,7 +95,7 @@ struct srp_device { struct list_head dev_list; struct ib_device *dev; struct ib_pd *pd; - struct ib_mr *mr; + struct ib_mr *global_mr; u64 mr_page_mask; int mr_page_size; int mr_max_size; @@ -183,10 +183,10 @@ struct srp_target_port { spinlock_t lock; /* read only in the hot path */ + struct ib_mr *global_mr; struct srp_rdma_ch *ch; u32 ch_count; u32 lkey; - u32 rkey; enum srp_target_state state; unsigned int max_iu_len; unsigned int cmd_sg_cnt;