From patchwork Thu Aug 15 07:16:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764449 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2048.outbound.protection.outlook.com [40.107.244.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F269145B2C for ; Thu, 15 Aug 2024 07:18:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.48 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723706309; cv=fail; b=gk8Lr76OyjDArTfKC0K380oWih5kTli5PGtoYrQQAI9gNb/37zFi7bSWYyP+l+pGqFOgRkkqi5rv2nobCV0DwPAHHsJ/sLpbUfgduItkHduW3RIXfU+TxU0dKj7nviPUZX06MYxKKV3slihu3E8Klx65DBbnv0PCkTk1cOAosUo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723706309; c=relaxed/simple; bh=LeW2AziNUsLDi+2ySYaNj5cOWaKaNIFZxDrcxpbopqA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=i0Pqc3zAsCs/F4qGKQRLoWJmdt5yoto8N8uteaIvE11iCS1Hp6Otc8zDOt6IpHfDT9E16S7XeP/XiRNlXO6ZiXRqyLGUEWgagQpWIuyKJB28ne6R8VD9ZcqfnBuDjurRC2zrUbKXsp+2uHxR8xU1aYGjmD84vV1f9RT49LoXMKU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=lq3dDn7C; arc=fail smtp.client-ip=40.107.244.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="lq3dDn7C" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ENI/VQp1fVoodJRwpxbTcyN9eJ5Er6UwimWLCexx2EF7tSF/GKMuLFC+yiyLkFXD6Ip2BVCd2r1CJiBGu9NJDVnnoxQEO0G15wOZ01SHBq7hJvmIVS28TarYORbH0lFvxWTXQZot/mY11cCAjLLyOl49xCyVJSB1iZga6Vm2b3xlmvpxl4KCkktg2f4YbQ9ASqtkhxHcR8rqkCIuUIUoDmk5oHTQ4ciDRvU7dciuXXCPirGcfC/rDREcGiavknlU++ajhBnrjjXL+GGjcuZY1dri1D5opCUsUmFgG9KRE+QNxq9n/B3NnbH/D3syQIVIb6AI7NAiGGPWx3L3s/XC/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wnEHs5qN75UExovDalECOn7B7tzDrriTS5gkR+K+afw=; b=VwN3prfw1dvjdJvj/NXSkU6e5Q1Hfu4477TwUudkAx8ufHDOCve18JzDwlZRxBxz+X6SOmg398YRWnlEQMgJYPxd2WyVY4vRBzPuXyxrzq6512qFA7qZMUhtHtx20Ytvlt2HJ+lMqSkJDAxEfu8jMEbFZs0YXckiWySOufC6mZtE5sdzwfa5uJ5b8v2ND9eSLLUyZBVas9Cp95K09hduzvpcow2eLRXZnLweEp9Q+QZjZaJACAHmNeik6r5AWaJniLldGVSm/PAoG3KKSRbT6l2a7jn6cnd321LYONwEML0i5LMA1wzNMk1sqbIWMUOFpAlfb5BPuFTOAz03mLBsMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wnEHs5qN75UExovDalECOn7B7tzDrriTS5gkR+K+afw=; b=lq3dDn7C4as2TDLVhw3fMYE3EZ13uXREwfXZZZBMnmgqkt+aWAVPq5r4aekPFF1Kl4/mhgMQOLLI37rnpCWfCw7IynArun6CmpBgUfzFywoM80HKunHmItpER4lmY5/lNO9XFC0vYM0vtQb818jKNqbj/bI8pkE+Oxu0WuqECcM7mu4WC+tuOD3LkyDRnsGhcsEsYgZcs0U+M5Xv3f4wkQvqdEnseB7USDoYpgIKbed+/PJKt3090og0r6xeCRQhzamPt1YU9Z4gHL0CXsiOnrW/jKt92HwTqTgzPUIF6haeEseOnrxioe1+nZ6RelezkU5Ck9mFXQRjtDspFi36zQ== Received: from SA1P222CA0125.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:3c5::19) by SJ2PR12MB8011.namprd12.prod.outlook.com (2603:10b6:a03:4c8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.18; Thu, 15 Aug 2024 07:18:22 +0000 Received: from SA2PEPF00003AE6.namprd02.prod.outlook.com (2603:10b6:806:3c5:cafe::34) by SA1P222CA0125.outlook.office365.com (2603:10b6:806:3c5::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.24 via Frontend Transport; Thu, 15 Aug 2024 07:18:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SA2PEPF00003AE6.mail.protection.outlook.com (10.167.248.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 07:18:21 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 15 Aug 2024 00:18:10 -0700 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 15 Aug 2024 00:18:09 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Thu, 15 Aug 2024 00:18:07 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Dragos Tatulea , Tariq Toukan Subject: [PATCH net 1/4] net/mlx5e: SHAMPO, Fix page leak Date: Thu, 15 Aug 2024 10:16:08 +0300 Message-ID: <20240815071611.2211873-2-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815071611.2211873-1-tariqt@nvidia.com> References: <20240815071611.2211873-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003AE6:EE_|SJ2PR12MB8011:EE_ X-MS-Office365-Filtering-Correlation-Id: 2432da12-e688-4ca1-2be0-08dcbcfa7199 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026; X-Microsoft-Antispam-Message-Info: NUaKqURTEFR00YaPJzW1UtgGBdYT0w/xbrNhBsXdb8ldVFV6O7EQLattXxJkC+RvtpMlyDnMvbrSdA4lLMNPoy2mfjZrOsYnlqCZ6SIqfZkLFMUIb014oSOg8gpf+PV61Jkm9Ext3B8Oi4ITLEMYt7Bw1WUkUaEm9KBg4foHt5YEda4Cz3npR0BxI9azSOfgKWTydmQG0d3s7myvNkQeyxAFb6ziw847qkZTBai4yc+VDaaYA/ONDXk44eWWbMO+2Nvv6i1i2O87+om1R0rf6tamR4qOxfsEOjKKvH8YqGXxBC+cLPL+0Fyov51e5IOOs+PVrHf95PLrWQQXOXQhntppnbT90z6ME+H1iUI8uzuPh9jrArstdyYwLDHITgXuEG62heUlvhOJIUaAmFe8IXmezq8ZC3uS9/xXYJ8tQEjAlyzUFP1VEEfU48xBqWOq92omu9/sCSzD/+O6EDN/t8DnP1RyE8WHXeaquElSrAkNbqJDHDJDH5zTxvEbnAqyWrxNEpZi4kXGTyY3blGqL5mtTLhu8yHFJ49uLQYDxysqM7f0jfUzg94k+UK+v0IsBHjZNb67K9Q32SVcJB9jeoN8jx7x7JQ0dKD51zhjTUHZo+Bk9SFSTCZ7sIQw5XDGFAMyBY+oz+ndlhtW5LkTVkerTNt6MtrOEHS2g+Ly+iaNADmMmw0km5J5YhUFmMVf4HpX+iienlR/eFx2eFuHRrpVgg85AW52dOFJpvpmFygQUi85+pPZZEq90hVLQq9ByrDUexq1mUYcRaj6rFU3F9zfDY/wuImPZwi2dT4gxsYUnZ1jHcdPCgKfGQvduQyfkAAnVBdYLTIhHsLN74RfHObWyS6pIDg9wbyRZ6wWz2WlCDtn+5dlzRnsV9m4CEigAJh4wcHjAzCNpf8/GTdSuGpumBOV9+ig9X9PmzEaHiOHu6usm8izuqLfaqQX+DTRnWmn/23VLWUAxRgYhnLY4EjgW1BypVSR4kEcJ7MyP9I7H7P6Jh4bm3bgG/QnTuJ/5aOpjp7+grLzkjDItOqzcR1oAUc0TYkIl35kCmWxw0hSQDKvNC/63COCYzfpo15aF0NPlT3Snu5jHxItKMoJJuGphCnac5oiB4/egMGDCyg9EdKfc93wEJrUVQwzmwDzX9GL+cMzcss49W1yRVCeK/nUN32sXxLFPrLCINoqgWXCFqHGCkGXBPUM12GPbPVSj6SucmbuSbEcIow4aGCjUWMjHw8aHk7bWeemXI8qB02um8Zh9+vPoHylm8wFH585gT4y8PoRZLvd+y0HydkJx6o2gXU3W1Lm1nN5XUYlts+JaDguds/0gEIxQdVvqMINVrnWZKKYJciTP6WTeVCHAuHXhiOBrXV2ddAIVsUyUyi4hbYJtJrKDMhr4a/NecTo1L1JH9fwEWTWcPaqkmbH71Yjh0f+TCRRVPVlxEIXV/XEJ4PDo9/QL58nswTUYdmv X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 07:18:21.2071 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2432da12-e688-4ca1-2be0-08dcbcfa7199 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003AE6.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8011 X-Patchwork-Delegate: kuba@kernel.org From: Dragos Tatulea When SHAMPO is used, a receive queue currently almost always leaks one page on shutdown. A page has MLX5E_SHAMPO_WQ_HEADER_PER_PAGE (8) headers. These headers are tracked in the SHAMPO bitmap. Each page is released when the last header index in the group is processed. During header allocation, there can be leftovers from a page that will be used in a subsequent allocation. This is normally fine, except for the following scenario (simplified a bit): 1) Allocate N new page fragments, showing only the relevant last 4 fragments: 0: new page 1: new page 2: new page 3: new page 4: page from previous allocation 5: page from previous allocation 6: page from previous allocation 7: page from previous allocation 2) NAPI processes header indices 4-7 because they are the oldest allocated. Bit 7 will be set to 0. 3) Receive queue shutdown occurs. All the remaining bits are being iterated on to release the pages. But the page assigned to header indices 0-3 will not be freed due to what happened in step 2. This patch fixes the issue by making sure that on allocation, header fragments are always allocated in groups of MLX5E_SHAMPO_WQ_HEADER_PER_PAGE so that there is never a partial page left over between allocations. A more appropriate fix would be a refactoring of mlx5e_alloc_rx_hd_mpwqe() and mlx5e_build_shampo_hd_umr(). But this refactoring is too big for net. It will be targeted for net-next. Fixes: e839ac9a89cb ("net/mlx5e: SHAMPO, Simplify header page release in teardown") Signed-off-by: Dragos Tatulea Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 225da8d691fc..23aa555ca0ae 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -735,6 +735,7 @@ static int mlx5e_alloc_rx_hd_mpwqe(struct mlx5e_rq *rq) ksm_entries = bitmap_find_window(shampo->bitmap, shampo->hd_per_wqe, shampo->hd_per_wq, shampo->pi); + ksm_entries = ALIGN_DOWN(ksm_entries, MLX5E_SHAMPO_WQ_HEADER_PER_PAGE); if (!ksm_entries) return 0;