From patchwork Mon Mar 5 09:50:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 10258515 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D4E4360134 for ; Mon, 5 Mar 2018 09:50:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C07F5289BC for ; Mon, 5 Mar 2018 09:50:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B5113289B8; Mon, 5 Mar 2018 09:50:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 933D2289C4 for ; Mon, 5 Mar 2018 09:50:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933190AbeCEJuh (ORCPT ); Mon, 5 Mar 2018 04:50:37 -0500 Received: from mail-bl2nam02on0047.outbound.protection.outlook.com ([104.47.38.47]:21152 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S933043AbeCEJuf (ORCPT ); Mon, 5 Mar 2018 04:50:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=bYCMryv6rbz1ldDpCV8B+LBiRCIi8iKI2IeUblWS2tA=; b=mLtipTCLjgUse8Mrw2ARr7VwrfUeF/3HY0c3tBnAgaistjPqDJkAfw3NCEAv5S+q+dL1F0QuxgR2cTdsWl2vBj2a5gPGir0x0vc1Tue/9dyoED0v+KG0qMerLwMLs0QXEZB3sZ23rFOij6uZGVOyhUbUtEAPyEM8g63sFxXNqas= Received: from CY1PR07CA0042.namprd07.prod.outlook.com (2a01:111:e400:c60a::52) by BLUPR0701MB2081.namprd07.prod.outlook.com (2a01:111:e400:52f4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.548.13; Mon, 5 Mar 2018 09:50:34 +0000 Received: from BN1BFFO11OLC004.protection.gbl (2a01:111:f400:7c10::1:168) by CY1PR07CA0042.outlook.office365.com (2a01:111:e400:c60a::52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.548.13 via Frontend Transport; Mon, 5 Mar 2018 09:50:33 +0000 Authentication-Results: spf=fail (sender IP is 50.232.66.26) smtp.mailfrom=cavium.com; mellanox.com; dkim=none (message not signed) header.d=none; mellanox.com; dmarc=none action=none header.from=cavium.com; Received-SPF: Fail (protection.outlook.com: domain of cavium.com does not designate 50.232.66.26 as permitted sender) receiver=protection.outlook.com; client-ip=50.232.66.26; helo=CAEXCH02.caveonetworks.com; Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BN1BFFO11OLC004.mail.protection.outlook.com (10.58.145.15) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.527.18 via Frontend Transport; Mon, 5 Mar 2018 09:50:33 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Mon, 5 Mar 2018 01:50:28 -0800 From: Michal Kalderon To: , CC: , , , , Michal Kalderon , Ariel Elior Subject: [PATCH V2 net] qed: Free RoCE ILT Memory on rmmod qedr Date: Mon, 5 Mar 2018 11:50:17 +0200 Message-ID: <1520243417-12507-1-git-send-email-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 1.9.3 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39380400002)(376002)(346002)(39850400004)(396003)(2980300002)(1109001)(1110001)(339900001)(189003)(199004)(106466001)(86362001)(107886003)(51416003)(478600001)(104016004)(72206003)(85426001)(105606002)(316002)(36756003)(305945005)(5660300001)(6666003)(110136005)(54906003)(47776003)(16586007)(81156014)(50466002)(81166006)(26005)(69596002)(356003)(2906002)(8676002)(186003)(77096007)(59450400001)(50226002)(8936002)(48376002)(336012)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:BLUPR0701MB2081; H:CAEXCH02.caveonetworks.com; FPR:; SPF:Fail; PTR:50-232-66-26-static.hfc.comcastbusiness.net; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN1BFFO11OLC004; 1:U2E8YA1gakiu/n0dP2ISMT7rNM9wD1Rtd60JLeaYyckRbElgR0zE4IW/10vUCEkVPEgYCrunHC2MMtTAvKH+RdRGGCE8+lk8S0ZbJRK9IpReJYYbc1VxGpPBuFuAQtBv X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3223bc65-ed1b-45e0-939d-08d5827e89de X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060); SRVR:BLUPR0701MB2081; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB2081; 3:ecp3dpuNdGvCOV7/pBWbxwcW1uTVgkPcdg4yqQo859TYqEucdoRNOfvXWweYiGkQIudByJoT2yIDFevu37PxM9Tu/+7dT/zn5JPBuq50ySPowfbHmKP9xTxgYOjR9ciiecFP8PO2CouTDDSm7Kak/0yikUGQBcI9rNSD3lVV6Uqaag8atWACtT//8Q+9wM2ikcUWZGnMLdZOB7RbM6SAQSK3e2PAE87epjChjXqIMcmTw2eU6gCg3YmSQWwnRkL6h+JNfcitmU18ErKn1FhOqrZJHbSrL5Dk/Lze/RsdkLQ873quhfAu93d4ytWgeLjSnZGFFc5MQSAwsCg6+uIRkIRkuEcitypOn0DmhkBs0KI=; 25:fDIjyFgy1jE9qYTxUop5Qb1w5jzMDOp2pVnc1dQKFq4DZoRAuqp5WR/AMM+wLk3Kpfm0HnEVPfL7MPfXG3+efy4Ig727jlD2l58nmErC9wrw+0Y5MKrYc/OrbBmiBJQO3iXCWHAuy2a32UA39/U99WxtU+QxmQiP+VlVNa/OnVsUcQc4r0uSNbZTho5pI+WGBDDwS7tIwF4ZUfkwDj59TvoCxz8GU33EgLhJNnB3Y3RUl1iykvFnGxZNuO/O5sjoi310FyKDeBEqniy2XekxMqzu76HZgMobtabYGnetYx5UGeILYhQP+/GvViCQD0v1fRLBpd9kw44XRy6lIIsqvg== X-MS-TrafficTypeDiagnostic: BLUPR0701MB2081: X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB2081; 31:DflFNmr1TwJrlTUGHGhKNZxFqDE3knDkB0J8nk3Xb+pf8kkuT+HeWQY2vVZ3DjIGUOJfVNJyfTk4vnKd71ZYYs0bGjt9m5BbrcbjiugaWSjqqyKZfkdpZ0BpywiSnAqPH3Am4LtQrIJLXgIqN7KZxn1W5GzWpGAe3ch4kJUz+fnmWIJWlZpIBY0x0TT5XD8yKODselGGWiPCwOZDYtKQs3XJRKRB3v/9YNK5JkpfjLY=; 20:OvQ826w3bBrn8URDaM/02q0TFWg0JUJPX59LVH6W+GqFwD6l0IChg0N+/vIz/1japh8Ize5IJCFEd0iZSzhCYpLDjYHdzRDC1lb55dhKczIFpWS2KyMo2FnlnloakavOnfPM7qok0GtjZBnYnqWyle60/mPJw/3bx1wS0du+Vpv68o63OMOANFcqHT43LCcTGV5MVBDUHQCJsl2XAQGiX3pCUyjptwG5GLP17Qlw9naZU71uzlMEVmM6JyrJfZptS4n6TP3efaVfH6Sm5PZjgGDmNzoi8YP2wpBDqxWXkKzpq9m3M5DBj9taO2y3A4He8iBkF5fq7VmVgUH7Lw8arePvfmJulDItES4cO3Ndk4E2D9Jl0ZZorVp8EzuFTATh6/xcBijL0Wj1NS3eLXjGIXxilMcuwEctdWnU1GI6EjjXWJsQIGP/QheCwDARN4bXb96/4bKEHdphmEQ7HkQy4i39ikMeobfBXUgeKpXoUz1OlXrupItP2XGiQW5WFfk1 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040501)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(10201501046)(3231220)(944501244)(52105095)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(6072148)(201708071742011); SRVR:BLUPR0701MB2081; BCL:0; PCL:0; RULEID:; SRVR:BLUPR0701MB2081; X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB2081; 4:0XngWdPE/WWguNvngOvYfuxpthIQP8K1NPbtJD9dGEXqHAc5xT7Zfc+jiNKO70HHsvfUbGQDH2QjXJk03DNXIE4AP6Gwu/4lMYG0ZKXW/APAbaNliczxRkgiqrlS8+K5aSeZzKC1VYSfxzHTimZGzIX0gKFWKgPqeCdVD5THOC72sgp5k9tj+erYa7XPb66n8gZMd/vdxQ0GcJYdXcnFStjihQejlP3oTnpgv4lV0KqNI/efI+gElFIy49r+9fXvTHoM4EC0nVSsbG9FjDfITg== X-Forefront-PRVS: 06022AA85F X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BLUPR0701MB2081; 23:k+KEZFq858HANVA8gemw5rlQ1UODPxAYjidH9AJ?= =?us-ascii?Q?HJC/0lJ3qcLTb8pujxgiZTldPu9yXKklhZifObC/+qBp1+8GbVV1QuZiBnrK?= =?us-ascii?Q?M/BzPXefgv0AA5K10kJirhODEHh2CDO0FBmRhL5RR4l0D/ZKgGOdNcHR0Tet?= =?us-ascii?Q?VDGozBL/bqkhwGERWimh0VSi35SYSncfffZrZ9GPyncbfO/HUWKi4xEHBkxs?= =?us-ascii?Q?8tA6CUz9Axj0ED6ID2FASqBhuXqwztQUgWeEt/ZEebxFmud+p8kd5vYqvW2a?= =?us-ascii?Q?yp5ksznG303hp5tr9uPO8usuzDOKgPmGobxRaAcF3TVtj5Ubpkw8/7tQHEC3?= =?us-ascii?Q?QqL6xzvtfONnJ7tbgzuN8rJMx7VqGwi9CuXh4bL/iK+xhIx8x/W7zOvWTpG6?= =?us-ascii?Q?xzSw7mqEOfa+HdRmGJdanLX8tY/iIOLcmkf8xIgVkVQ4iAMRMA7Lc0AiFOn6?= =?us-ascii?Q?lEDSRd/Pcu53U+hJSsxYKjxxppEpE+hGCZLxfACldRW3Jnpve5XMYDtaecX4?= =?us-ascii?Q?5dXXzUklzgjxk0GRiilPcHhIDaAo4vnc4zZnZwUgEG3sup6eiJbkLklyjOPx?= =?us-ascii?Q?kDb90mtp2LkpB/S1LoiEYcKvxM48RNnPydubZmIn1Mj8IB2Lih1kkr673xkA?= =?us-ascii?Q?AeazZz00ONqxvs7fAsTNGfULC5gLCF3EnsCnBefEv5MQLHc2cXj8EB6c1K6E?= =?us-ascii?Q?so414BJKpEBdjCul+q1KPvSuK6V8cmk7R1CDCi26HiqKoqFBDOPSSRH2hwvX?= =?us-ascii?Q?/mp/ROWh7UozBp7UuUjO17kK+CQo7kVQJPLzWy7n4odt1dOLZBl3hvir+xRb?= =?us-ascii?Q?3Om3MC3D7sCJfsFpYW+wDbrE/gw/Tz7eYG4Sh5KmGL5pM48AMydkCR28o1wg?= =?us-ascii?Q?7fl3dpBzVVu3krW7FKlcJPFQn02m+2Yv7AdbRgM94zzxkov2i5GkcI9Ql1M8?= =?us-ascii?Q?3JkYgHPbO/cZym+0xwGQllsY1/Ub0J427qw2lq3k/6R4M3tnpQDtz8Fi3BxE?= =?us-ascii?Q?r1glyuugV3ztBtrLVporTOWy1hhyIiiucbjpfsmG5BQCkmu+w37nFUhoGfeN?= =?us-ascii?Q?3dRikIBI=3D?= X-Microsoft-Antispam-Message-Info: +yJIVYbUT+zmsHTawptN5mcfFfl0pjI6qVRWzbmtWDnVKACWNRQlQK23JRJIN1zpTNBJiBzbhotyFWoEwoXgwsOZcuZK2PXLt5Am89pMskFqVuSHiMFGBdmg/aAqf6Vr5KD5VyR4eY53kZyN9kdvD+tgwKZqtWygRn8VGqv3XWabep7yHAtSN+TLEXFeL/H1 X-Microsoft-Exchange-Diagnostics: 1; BLUPR0701MB2081; 6:DXX8RgjTrHHrwbGLWN9VZ9cO3QZO+4RveDMvWosY7X1V+srUY63l6jCfWpqVfPbtdwHOp+GEc8qmawnfkmdeQ6XsvZlYdW2mpcOoWDBFg9nTpM0dJLAqFivfbX9psbSdpqRIcYu1Gb56SSG7blLK1Fhgx9FQUjK4ZGj2IZc4peosInLKQTtL9kY89pP1n+1tpFDBWLjslTZG6X4YqV4AWwoWyvfNXuV5m4pW1yngNTcSI3W1AyeI+r2dp0MBH/E3tVLuYuakW7H6Sb99Gd1PnENxIFZeTsRm2azlKhHarXAlU7eM8n1v1UqdP4OxJC6fGGVoC+v9tw/9D6D73ZJgYM9a9dbjvuCGx7OqtHxtu4w=; 5:jTysZD0B2sXQE1gfS9jwNUGd3bZG2xplkMJHvuh0oYF/gpqikNA6iCwpE0mJCqH2gypw7eSBj9e7TDTFQSuW8BHmQNtw5uD0xUynOnnDxcSJrkF0G580MDZ98XV4N3VuGnExHivPPBmX7LiiK0PdhJyrCA3wo/4LEUoXv13ESFs=; 24:w2CZVaFQi7AJAcacW0D14LWZDcyqefyODbCVK6TP1x6SmP0i9CH3HILJS6YvVgjkuCHiH4jFdBnWZ76asRHVAizIKyykXTgt/p2Q12qTG7M=; 7:Cn+Pvv2pnPFw+/2dOFZUCQJQ5ZeWjGJrRXkVvZgSuvW2i/b0bi6pXi3zjtk5exWq5dCGNQsF7crIJkWOCT5E5utem1pBfL/eHfHKziYsv0vg382HHPjy5T4l0eJGLeZQ5pDvJ8WhFeMGz3ce2x5zhEeh7tnU8LBwLUvG7UWDsybU+8/HMQiG0BH93zGfZb9E14snnox7VtcKsgVmwnMmr3Fdu40g8mj+xh25Gu4RizQO1XAfVz3Zo7KuPwyPKwvk SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Mar 2018 09:50:33.0072 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3223bc65-ed1b-45e0-939d-08d5827e89de X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB2081 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rdma requires ILT Memory to be allocated for it's QPs. Each ILT entry points to a page used by several Rdma QPs. To avoid allocating all the memory in advance, the rdma implementation dynamically allocates memory as more QPs are added, however it does not dynamically free the memory. The memory should have been freed on rmmod qedr, but isn't. This patch adds the memory freeing on rmmod qedr (currently it will be freed with qed is removed). An outcome of this bug, is that if qedr is unloaded and loaded without unloaded qed, there will be no more RoCE traffic. The reason these are related, is that the logic of detecting the first QP ever opened is by asking whether ILT memory for RoCE has been allocated. In addition, this patch modifies freeing of the Task context to always use the PROTOCOLID_ROCE and not the protocol passed, this is because task context for iWARP and ROCE both use the ROCE protocol id, as opposed to the connection context. Fixes: dbb799c39717 ("qed: Initialize hardware for new protocols") Signed-off-by: Michal Kalderon Signed-off-by: Ariel Elior --- Difference from V1: Fixes line was truncated --- drivers/net/ethernet/qlogic/qed/qed_cxt.c | 5 ++++- drivers/net/ethernet/qlogic/qed/qed_rdma.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c index 6f546e8..b6f55bc 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c +++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c @@ -2480,7 +2480,10 @@ int qed_cxt_free_proto_ilt(struct qed_hwfn *p_hwfn, enum protocol_type proto) if (rc) return rc; - /* Free Task CXT */ + /* Free Task CXT ( Intentionally RoCE as task-id is shared between + * RoCE and iWARP + */ + proto = PROTOCOLID_ROCE; rc = qed_cxt_free_ilt_range(p_hwfn, QED_ELEM_TASK, 0, qed_cxt_get_proto_tid_count(p_hwfn, proto)); if (rc) diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c index 5d040b8..f3ee653 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c +++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c @@ -380,6 +380,7 @@ static void qed_rdma_free(struct qed_hwfn *p_hwfn) qed_rdma_free_reserved_lkey(p_hwfn); qed_rdma_resc_free(p_hwfn); + qed_cxt_free_proto_ilt(p_hwfn, p_hwfn->p_rdma_info->proto); } static void qed_rdma_get_guid(struct qed_hwfn *p_hwfn, u8 *guid)