From patchwork Thu Aug 15 05:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764425 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2051.outbound.protection.outlook.com [40.107.93.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DC947E583 for ; Thu, 15 Aug 2024 06:24:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703078; cv=fail; b=WzQn+UzmBROZ8VRZmVTKiLXmJ5q1vp5DyRb+qurLgmE3+QqIizfFqPKzfJ/guRcHCy5R9OJCRnyVK/sTLcG6Nv4kVhlkQdnRumW2JFI2n4OEQAP7fB2QMmo6JtZaa5aS2cuCbUDtlxhxI0kzZDz4sNMmeRP5MjkzjS/f5hqlm9Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703078; c=relaxed/simple; bh=sWM+Lk5br32gYWKhDI4vRuDqzfgpId/QEeR3rzyJILg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FMcUQyUwqphd0IGH5Sn9InziqZDS0bdgnyE8/hdgvubeK2kN3kUI0Fmmaade+CYIQ7oS8MiMtyhOPOjB2FGYVa14bejl4SHTiRKMf/M29W8PHp84UOVZ4gYji+9GOeR+SJlPJbtjRVelU7b9pozT60LsoEqWs15Z6CcUCRTLMXc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=aJAYIYpD; arc=fail smtp.client-ip=40.107.93.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="aJAYIYpD" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qfomjaBWRmewvGVMO7wazOZIxLeA4afhdT2fOMlgJPKIPzCAFO6aoS/TCKm9PAPCT5Zn+Qc7BizWZHKvJJaRYRYlLCPGBGC6S5WFmYynW5edQqy/iSUeZtY0cK2xg8vrMHSTK1yNQ47jwPBNLGqgWPmEtRQTBCYPeRbJcGLU8YBn8d+nMT9VsbIVSE/Ejs27OIa4pHiiUeGUsbU4XcdUM6H6teoutAMgGmqIB2Y06iFrRefWEoNGm6iy8uO3p409EzZLEdzeNj0sTgx3T9pY8MNNEvNF9UaH6x4tyK80d08B8DfNJFnIRGQtmaS7K5U+MFLLen4rRKTD8/Dvvp69Qw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PqT7oeZR3cWGRv+y0zpRGACXh++WwM6/EtkgHpHpCcA=; b=fIDMbO9+78D3qEoh6wWGSw0a3Gpa71LBEZcBSyLSrhrRWfOiTT8UavJfgc1zJa+amdc+LiQ6ApdPjYQq4oH8zwKysUFTg7GoLzTEK9+jtrBR+b3j7Rmi/xtINiw+nftqiGrW53aBQfLEVPL+uVyyX5DlyA5fdSEXKIunXmDQBjztjjqka2RA1GpQxzeoEU0irjySnKP/T+gZRl9QFZNVQ2ZE++rP2eCToQOh1EHmveWEMLApV8OO9NATXcveDpfukOIHFejxbUiNQQrDRVDB7I+x7HGRZNrqU0cUzlEZZhNWj0tm2Xof/qiBERw6VTZRk87TIYI6uTyXzpLkilDlvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PqT7oeZR3cWGRv+y0zpRGACXh++WwM6/EtkgHpHpCcA=; b=aJAYIYpDKPtS/ULlCzxTL0Wu2S/Toc0IKYr6hojv2amFSNgncJ3QnzNMnr4NndyO9If5g45PDW+1D1f6iBP2YBlMrpyvqzUFnEKS4J4dua5PhhFp3Z19EwGO4Auk3vnqVeiRMOZmW9vZ/TeY7vXGlcH6uig83cYGxukJBHM4bnwlU8UvvMIohNdtV8elgPtCec/8ImxdtK3X5nohixgRHRmDVZPxPodiaXS5gVRa7N23177UaDJXcqpTPc1ml/U/DlIExGA7HZwCL+Dr1PNSrmPqy3Ts6616/1ugQQ+3kO4qRMkfX75LCdfjF0z4WYBGHIk3vhon+vLjpUPEj6gGZA== Received: from BN9P221CA0029.NAMP221.PROD.OUTLOOK.COM (2603:10b6:408:10a::27) by DS7PR12MB6142.namprd12.prod.outlook.com (2603:10b6:8:9a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23; Thu, 15 Aug 2024 06:24:31 +0000 Received: from BL6PEPF00020E65.namprd04.prod.outlook.com (2603:10b6:408:10a:cafe::d2) by BN9P221CA0029.outlook.office365.com (2603:10b6:408:10a::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23 via Frontend Transport; Thu, 15 Aug 2024 06:24:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL6PEPF00020E65.mail.protection.outlook.com (10.167.249.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:24:30 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:18 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:17 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:14 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 01/10] net/mlx5: hw counters: Make fc_stats & fc_pool private Date: Thu, 15 Aug 2024 08:46:47 +0300 Message-ID: <20240815054656.2210494-2-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF00020E65:EE_|DS7PR12MB6142:EE_ X-MS-Office365-Filtering-Correlation-Id: da9d2cbd-4c50-4c25-698b-08dcbcf2ec0f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|376014; X-Microsoft-Antispam-Message-Info: 52VSgV6hSCpZYQPl/n/URr71ur1ByARJ+Ccc8ZB2PfzzCxWg1NxpA17K6JDvHLwKLz5xWT2jx5HfsC//4NDxcW2rFT6O9KuE2V9LM1y4d4gfTYvLRoIFD9oUYFPH3bN3mQetIhDGx26z32HKKa+WVGK8YWXHeDlMpjB3xnmPCsRiY7XeEy+rDvZtbqaXIMtWT+7lIPl+7ryoA6iqCMyPWX/SnDGt5ePdGUWDX3xLT2f9bYSet61cmzDcxKbpcO4Se/a6nHewaApn971jmH7b5qdtLkO1pPnd8/6Fa+bsBYMhex9o9Q4KZlVg0UPS7cQ6jvDAukJ2ZkejEhTpLqHRj3SxuL0ytSjHrTt8dvFnopUnrcCZMmfpjEFDrFuPqCDPFd1FHjeGy+8cL1TZa37MqaXRg0zgFWQX39AJt2PjgxgIQrH+5+CiGY0x6uukPOnbfhpmhOMlNTRwaWKGFbUEgN7k9pytXTDTzuezFdEZh/1MJ55F0XX0Zjg/DBFgN0Jk3c/qp+uALipDNJp3niM8h+r7GtIQrdz52EnHhCqDU1CQP8A589tZzkufTTa0bnZLe4nmpxR+jC24jz5HfhmBwo6/tZRYLhKx6ht58c78f3igHE56kvDPMZsh4FjTb1nZNVtUUXTM7yIg/gcLKaCDrk+RNsMEbPgYSidI4WsfbwRXL4S9zK1JagcCLI28H+p4irdqzfMhlif3T8rAtFmeGC7JCkveMjL2HBR4t5U6SYh8UaguPkbweic3Xkv1e91TvcP3MKASzYw/2R4b/hQKNdFQLsi8Uzh2Met1as3YTTpiFs1uBlO0YFONyDDS/HS8RldXPL7b+HjvdsXVvkEtLBYb1X+tUqDHPwVhAyMWcHRtCc/hslSHhkpaDFzB2knTdX8n6h5am41v1CWpKKQhjmekDYvt+Ti6uemoEjOOAbFb29Nkx3wYpi4ikevAkY65xYLpyJnunqw/pDgFpBJF4SLAmJ0I7Vrrv6Ow0WLYZaI5dsGxI5E4xvEmZihhxoQeukUnOHNx5CGhGZxRLg6ppTVRQMrdB6FODMhR4NbIYWI0j+0PQlHyrQEjD7PI21Y1z9urZCjIpYJTzRhpM4kkkT/lQ9n7d5RYJQX3OyqLHyi4uLEdGAh1ucVXA2YKM7AJNTsSmNKUyutLjX/IE3YlRqpSMPO7bdiXyxIOa/vS8eIf/N0f+NznYkXf6RSh7P8ClbgbdR/Y9B5yFUle4o0KReadfbNN5SU4YINIRRZfugtMq3Ll9o+hOIXo2uhIIRk4XqQCzkzdV2kqP+5Ige7efrGdLjOAaAUrkiTtJAzs975HaapIs/Y/Yqo90za9hC/03VZfuUJ5zZEgHhU+Ayrg4y7KdnWqHwDlMZUGI87N4ZbS1cRLsOTt9QUoZr//jzMywmt+F3WAFbSv29mOQsPhN6zrOAfsIqw80nwfbAH8kdEYJOltvwUBebBXeX9ZLmII X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:30.6433 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da9d2cbd-4c50-4c25-698b-08dcbcf2ec0f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF00020E65.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6142 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu The mlx5_fc_stats and mlx5_fc_pool structs are only used from fs_counters.c. As such, make them private there. mlx5_fc_pool is not used or referenced at all outside fs_counters. mlx5_fc_stats is referenced from mlx5_core_dev, so instead of having it as a direct member (which requires exporting it from fs_counters), store a pointer to it, allocate it on init and clear it on destroy. One caveat is that a simple container_of to get from a 'work' struct to the outermost mlx5_core_dev struct directly no longer works, so an extra pointer had to be added to mlx5_fc_stats back to the parent dev. Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- .../ethernet/mellanox/mlx5/core/fs_counters.c | 79 ++++++++++++++----- include/linux/mlx5/driver.h | 33 +------- 2 files changed, 60 insertions(+), 52 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index 0c26d707eed2..7d6174d0f260 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -69,6 +69,36 @@ struct mlx5_fc { struct mlx5_fc_cache cache ____cacheline_aligned_in_smp; }; +struct mlx5_fc_pool { + struct mlx5_core_dev *dev; + struct mutex pool_lock; /* protects pool lists */ + struct list_head fully_used; + struct list_head partially_used; + struct list_head unused; + int available_fcs; + int used_fcs; + int threshold; +}; + +struct mlx5_fc_stats { + spinlock_t counters_idr_lock; /* protects counters_idr */ + struct idr counters_idr; + struct list_head counters; + struct llist_head addlist; + struct llist_head dellist; + + struct workqueue_struct *wq; + struct delayed_work work; + unsigned long next_query; + unsigned long sampling_interval; /* jiffies */ + u32 *bulk_query_out; + int bulk_query_len; + size_t num_counters; + bool bulk_query_alloc_failed; + unsigned long next_bulk_query_alloc; + struct mlx5_fc_pool fc_pool; +}; + static void mlx5_fc_pool_init(struct mlx5_fc_pool *fc_pool, struct mlx5_core_dev *dev); static void mlx5_fc_pool_cleanup(struct mlx5_fc_pool *fc_pool); static struct mlx5_fc *mlx5_fc_pool_acquire_counter(struct mlx5_fc_pool *fc_pool); @@ -109,7 +139,7 @@ static void mlx5_fc_pool_release_counter(struct mlx5_fc_pool *fc_pool, struct ml static struct list_head *mlx5_fc_counters_lookup_next(struct mlx5_core_dev *dev, u32 id) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; unsigned long next_id = (unsigned long)id + 1; struct mlx5_fc *counter; unsigned long tmp; @@ -137,7 +167,7 @@ static void mlx5_fc_stats_insert(struct mlx5_core_dev *dev, static void mlx5_fc_stats_remove(struct mlx5_core_dev *dev, struct mlx5_fc *counter) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; list_del(&counter->list); @@ -178,7 +208,7 @@ static void mlx5_fc_stats_query_counter_range(struct mlx5_core_dev *dev, struct mlx5_fc *first, u32 last_id) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; bool query_more_counters = (first->id <= last_id); int cur_bulk_len = fc_stats->bulk_query_len; u32 *data = fc_stats->bulk_query_out; @@ -225,7 +255,7 @@ static void mlx5_fc_free(struct mlx5_core_dev *dev, struct mlx5_fc *counter) static void mlx5_fc_release(struct mlx5_core_dev *dev, struct mlx5_fc *counter) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; if (counter->bulk) mlx5_fc_pool_release_counter(&fc_stats->fc_pool, counter); @@ -235,7 +265,7 @@ static void mlx5_fc_release(struct mlx5_core_dev *dev, struct mlx5_fc *counter) static void mlx5_fc_stats_bulk_query_size_increase(struct mlx5_core_dev *dev) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; int max_bulk_len = get_max_bulk_query_len(dev); unsigned long now = jiffies; u32 *bulk_query_out_tmp; @@ -270,9 +300,9 @@ static void mlx5_fc_stats_bulk_query_size_increase(struct mlx5_core_dev *dev) static void mlx5_fc_stats_work(struct work_struct *work) { - struct mlx5_core_dev *dev = container_of(work, struct mlx5_core_dev, - priv.fc_stats.work.work); - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = container_of(work, struct mlx5_fc_stats, + work.work); + struct mlx5_core_dev *dev = fc_stats->fc_pool.dev; /* Take dellist first to ensure that counters cannot be deleted before * they are inserted. */ @@ -334,7 +364,7 @@ static struct mlx5_fc *mlx5_fc_single_alloc(struct mlx5_core_dev *dev) static struct mlx5_fc *mlx5_fc_acquire(struct mlx5_core_dev *dev, bool aging) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; struct mlx5_fc *counter; if (aging && MLX5_CAP_GEN(dev, flow_counter_bulk_alloc) != 0) { @@ -349,7 +379,7 @@ static struct mlx5_fc *mlx5_fc_acquire(struct mlx5_core_dev *dev, bool aging) struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) { struct mlx5_fc *counter = mlx5_fc_acquire(dev, aging); - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; int err; if (IS_ERR(counter)) @@ -389,7 +419,7 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) struct mlx5_fc *mlx5_fc_create(struct mlx5_core_dev *dev, bool aging) { struct mlx5_fc *counter = mlx5_fc_create_ex(dev, aging); - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; if (aging) mod_delayed_work(fc_stats->wq, &fc_stats->work, 0); @@ -405,7 +435,7 @@ EXPORT_SYMBOL(mlx5_fc_id); void mlx5_fc_destroy(struct mlx5_core_dev *dev, struct mlx5_fc *counter) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; if (!counter) return; @@ -422,10 +452,14 @@ EXPORT_SYMBOL(mlx5_fc_destroy); int mlx5_init_fc_stats(struct mlx5_core_dev *dev) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats; int init_bulk_len; int init_out_len; + fc_stats = kzalloc(sizeof(*fc_stats), GFP_KERNEL); + if (!fc_stats) + return -ENOMEM; + spin_lock_init(&fc_stats->counters_idr_lock); idr_init(&fc_stats->counters_idr); INIT_LIST_HEAD(&fc_stats->counters); @@ -436,7 +470,7 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) init_out_len = mlx5_cmd_fc_get_bulk_query_out_len(init_bulk_len); fc_stats->bulk_query_out = kzalloc(init_out_len, GFP_KERNEL); if (!fc_stats->bulk_query_out) - return -ENOMEM; + goto err_bulk; fc_stats->bulk_query_len = init_bulk_len; fc_stats->wq = create_singlethread_workqueue("mlx5_fc"); @@ -447,23 +481,27 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) INIT_DELAYED_WORK(&fc_stats->work, mlx5_fc_stats_work); mlx5_fc_pool_init(&fc_stats->fc_pool, dev); + dev->priv.fc_stats = fc_stats; + return 0; err_wq_create: kfree(fc_stats->bulk_query_out); +err_bulk: + kfree(fc_stats); return -ENOMEM; } void mlx5_cleanup_fc_stats(struct mlx5_core_dev *dev) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; struct llist_node *tmplist; struct mlx5_fc *counter; struct mlx5_fc *tmp; - cancel_delayed_work_sync(&dev->priv.fc_stats.work); - destroy_workqueue(dev->priv.fc_stats.wq); - dev->priv.fc_stats.wq = NULL; + cancel_delayed_work_sync(&fc_stats->work); + destroy_workqueue(fc_stats->wq); + fc_stats->wq = NULL; tmplist = llist_del_all(&fc_stats->addlist); llist_for_each_entry_safe(counter, tmp, tmplist, addlist) @@ -475,6 +513,7 @@ void mlx5_cleanup_fc_stats(struct mlx5_core_dev *dev) mlx5_fc_pool_cleanup(&fc_stats->fc_pool); idr_destroy(&fc_stats->counters_idr); kfree(fc_stats->bulk_query_out); + kfree(fc_stats); } int mlx5_fc_query(struct mlx5_core_dev *dev, struct mlx5_fc *counter, @@ -518,7 +557,7 @@ void mlx5_fc_queue_stats_work(struct mlx5_core_dev *dev, struct delayed_work *dwork, unsigned long delay) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; queue_delayed_work(fc_stats->wq, dwork, delay); } @@ -526,7 +565,7 @@ void mlx5_fc_queue_stats_work(struct mlx5_core_dev *dev, void mlx5_fc_update_sampling_interval(struct mlx5_core_dev *dev, unsigned long interval) { - struct mlx5_fc_stats *fc_stats = &dev->priv.fc_stats; + struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; fc_stats->sampling_interval = min_t(unsigned long, interval, fc_stats->sampling_interval); diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 9f42834f57c5..7047df3ad204 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -45,7 +45,6 @@ #include #include #include -#include #include #include #include @@ -474,36 +473,6 @@ struct mlx5_core_sriov { u16 max_ec_vfs; }; -struct mlx5_fc_pool { - struct mlx5_core_dev *dev; - struct mutex pool_lock; /* protects pool lists */ - struct list_head fully_used; - struct list_head partially_used; - struct list_head unused; - int available_fcs; - int used_fcs; - int threshold; -}; - -struct mlx5_fc_stats { - spinlock_t counters_idr_lock; /* protects counters_idr */ - struct idr counters_idr; - struct list_head counters; - struct llist_head addlist; - struct llist_head dellist; - - struct workqueue_struct *wq; - struct delayed_work work; - unsigned long next_query; - unsigned long sampling_interval; /* jiffies */ - u32 *bulk_query_out; - int bulk_query_len; - size_t num_counters; - bool bulk_query_alloc_failed; - unsigned long next_bulk_query_alloc; - struct mlx5_fc_pool fc_pool; -}; - struct mlx5_events; struct mlx5_mpfs; struct mlx5_eswitch; @@ -630,7 +599,7 @@ struct mlx5_priv { struct mlx5_devcom_comp_dev *hca_devcom_comp; struct mlx5_fw_reset *fw_reset; struct mlx5_core_roce roce; - struct mlx5_fc_stats fc_stats; + struct mlx5_fc_stats *fc_stats; struct mlx5_rl_table rl_table; struct mlx5_ft_pool *ft_pool; From patchwork Thu Aug 15 05:46:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764426 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2062.outbound.protection.outlook.com [40.107.237.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E730A176FD3 for ; Thu, 15 Aug 2024 06:24:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703080; cv=fail; b=Xjga2wbAQh8X6WeaDYpye7SwstRbXk627XUEvrIPPw+2+wpuV1WKu7BUYGSZhm1X12TXpyqfsognyACWh0wASXpPb8EZTFgu7o55eHwSpWJKDFSV+U0KeYuRD7B/M+7Rn11TmYFUvmxmGbcnwXLBWZ8AktWAxjcJppUYgj6aG0s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703080; c=relaxed/simple; bh=zNJAjF6vbYmSSRk1USAGGTROR2b9HnW6tIpddi8BBTg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BAbgz5y3qUDWUs+Qe/303md4M/PVMkGWHkCmmtJYHr0QGIIcciEUx9phKuiPbtm62CRluXbBnVbF0YER4+9Oc/YygFMGdmrZpVXmYKIoSGwT+iTehWgspXq0OyA67+MozhfxnkKrEN97reh51UA7soiPvRHs1XfQIiz3e/LOS8Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=cjpzrULs; arc=fail smtp.client-ip=40.107.237.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="cjpzrULs" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sfRH7ZeIdEKxrQO+aWZIPudjn+FLwG95TENoxZURTwDiDt4vMPkwRlDHrIPpAWG8lUWVDaP4lN1iRrT97GFlCQrIsZLK6iA9ZsceVoeaicQReagAMQgKn/JLA/uvw2OIsqBWFFqrW520lOLsWz387rFq9+uG7HfgTu0XX7KLTp0lSItmtHQmUJpb+fZZj+U0SLBZriBI3eSeu9iZOL+uPAWVyxWaVMgaOqKwi68KCpWuLATmNd/8Isy3R6A9zgc8tgtoUZk9hcfkkNjUFzI+4iE7EdMfZWgGQYNIVhPr3flmfA3fEl8RgtNHdVN+XYDiLrUS/1JaZ3atDOJxIWYnXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DJs77ZVRAaH5RwUsIzjYHBc5goBqhixG7mpxSQv771A=; b=X9jzEpfZRsYUaF3R+uVFRvwoKWDhXOCwgpiyCAduUmhUjlQWpncT6DadO1mr7xreNCIabHYVNnIKhDOrD32s/nZ7ZOD3iiPzA55MIY1SWGYzIKh5ufbhH55SXAV9qAObLYZwlFsXBoUK4VrpMUAMb9bXL6lj9GiAnqikzVO9Z6hmN/4bVYcN2XYrvY3IlLfJnonhWAdFZdZg4wTBi3M3vPYk/7zzYx1Q9H93vAq53golyVCrrRunFphVpAY2VCvY22FDbVa8aiXay7L7cLoJ0bXao/p34QDViS73GaHejBZ1Q6q0F+SxKA2t6q1Pq5DnZWJYTQAMHtorNrqlVjtXYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DJs77ZVRAaH5RwUsIzjYHBc5goBqhixG7mpxSQv771A=; b=cjpzrULsVATQ9m1cBWACgMIcOznJ/U6oJCda90q/1SVcfGQSTurtbl8Ym2fGAC4EauLtoaPJv+J4zoDjqXgw1ctDVHzQ42GGJn1PMqeICiMiKddc1YpvreMOArWZGdeik7l7rlez1PUz6M1bPAMC2CA0l7AhU79QZ5m/UMwHWRVCax/6b6kmVaueYxrGuiZXIrs1/aZ8kF4CgThhsimwnT4C0nLWo4GS7aoZ7ROkYrB1kb2WlH+D/sFWJxLYSzumlcSetIf9tjqW350FGXlGD0X8UqRpHYGrfz3Sll8sQsCR+FqTfByQ1q+EMSZ/QYWTfe0VYl+af3c4wEth4+ve3A== Received: from BN9PR03CA0933.namprd03.prod.outlook.com (2603:10b6:408:108::8) by DM6PR12MB4220.namprd12.prod.outlook.com (2603:10b6:5:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.17; Thu, 15 Aug 2024 06:24:35 +0000 Received: from BN2PEPF000044A7.namprd04.prod.outlook.com (2603:10b6:408:108:cafe::e) by BN9PR03CA0933.outlook.office365.com (2603:10b6:408:108::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.33 via Frontend Transport; Thu, 15 Aug 2024 06:24:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044A7.mail.protection.outlook.com (10.167.243.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:24:34 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:21 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:20 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:18 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 02/10] net/mlx5: hw counters: Use kvmalloc for bulk query buffer Date: Thu, 15 Aug 2024 08:46:48 +0300 Message-ID: <20240815054656.2210494-3-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A7:EE_|DM6PR12MB4220:EE_ X-MS-Office365-Filtering-Correlation-Id: 307a2cc6-1ced-4b54-2501-08dcbcf2ee6f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: RbXT667mVLwUnMcN/SsWXsXReS7pBBdcfXH+tQe7T8eybGRwMa4kjwFv+QSoO2cFWcDMY63j78c5IYMDBWH9OT4nuSZX0cilli4Sz4boGtvH9UQQR7IKX7i7gWfphG288f4gs+vCaeATN1JZskSCGkA9wSvzElSd8qIlZFgtZcU6ACaexLNpxPNkipAhNyzfEgSIqxsIYYMi4j+nIbJyV62CZ5jw621isG78EGP0GW5CVmA4ZYVY3GJQ8ldPJQX6+vnrCFI5auf51J03CqpEobGx7xp5L0OfkF9whlVAYPAMgoSkjSRWQIrnn3+yRLhS1itokZVJT1r7VpT65uAieVuGJlRJOrvOfs8fkRnwxAw3gjJm/Q1Miz2RDxikxxFO9i43ZTHm/rXhJ8Sbea/a6w141Kew/NNrIowXDGpbsfHtbdrqziTG0jsDwzAPVsNxOjEi6Ij7v6l6j9SvDc6NJVhF66NDrRNBMPjcDVEiMGYM/JQsC2UX96RPtliSprgbBJ/VxpoUJECRX92Gdx6jqdpu0mNCHI0XcLP7wHsos3GT40K204DLBhPFJG25aXklZhfXoHfToCMQm/o1bZF3zYmjYvHKGzZWtWGLMA55gZ13tmLjHvCrMyKVDzx2N7LgagGpMlf98TcfMS5sgYtoPZvAlpbbQZkq6a4vuncoK95GhigmuFYpP75KZrfaj2U8FJcbRuFsgYLafINkYnZZ/rempkLtGg3ATqAJBNR+osaEMPzw/1MWoZN+mO6KIUoEwS0E62nL4bMh9Ai39/CmOVtTDB/UYLNi2qVgDq8HiFN+uDgyVxH7IEDFUBckLjrISPZSsi74NrxM0FaCR2F64/C4SxVnBQ8ivE9/RAGPKJBDgXfYAXZXJAeUh9q6hf8WcDuVcSZ4B/l9ki/BcXkveYgEfe8XYYVHMF00dDmIRdsTmt/xl82hJZUaRij47vbj36ck7Yb6qcdhVZdd44rYNJj5GdAGZGvdp8UEchkLMfTv1S3QriaEc4sI9vPVi9COEqFwyanb0sL3giqFX0j0uf1gRWzM/JR2IgZFglWwGh9S7cJRzMECqYYXndDS72j+Y99PIWRvf4eS8PNG6MNIVaI6VAaS1Y+e+BPXM5rn+jwsgP7IqxdRkFuWUmEnST0mX5QdZEHG3l7ZEzZTVqCMpYKXAVEIHRxkmmQlZp1SoOfeV9rUCyJL5jjmXalGIaMpP/1xzgobj8wx43x4y4GL6MBgGjJ/zeXUWN2Kk6zSF3ZW01tpI9L9piNc7nLawprzkGLAct7MSOT9qymN81rm1hXi8SnlRkimunHAH2oC5MR+WLvQ2oODjsLlbqnPNNH3Eng5v2siUtFA7aZF7TgZwBNxaOKib7cLVacNYv7HAQs2JthCSAsFfawI1N8ib/hQzoOSEP1Tni3daZDoLgIjpnjc0cp26fPqGb1r0nguJSyDZuARGf6yuVZW1HYTV2aL X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:34.5792 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 307a2cc6-1ced-4b54-2501-08dcbcf2ee6f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A7.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4220 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu The bulk query buffer starts out small (see [1]) and as soon as the number of counters goes past the initial threshold grows to max size (32K entries, 512KB) with a retry scheme. This commit switches to using kvmalloc for the buffer, which has a near zero likelihood of failing, and thus the explicit retry scheme becomes superfluous and is taken out. On the low chance the allocation fails, it will still be retried every sampling_interval, when the wq task runs. [1] commit b247f32aecad ("net/mlx5: Dynamically resize flow counters query buffer") Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- .../ethernet/mellanox/mlx5/core/fs_counters.c | 59 +++++++------------ 1 file changed, 22 insertions(+), 37 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index 7d6174d0f260..9892895da9ee 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -38,7 +38,6 @@ #include "fs_cmd.h" #define MLX5_FC_STATS_PERIOD msecs_to_jiffies(1000) -#define MLX5_FC_BULK_QUERY_ALLOC_PERIOD msecs_to_jiffies(180 * 1000) /* Max number of counters to query in bulk read is 32K */ #define MLX5_SW_MAX_COUNTERS_BULK BIT(15) #define MLX5_INIT_COUNTERS_BULK 8 @@ -263,39 +262,28 @@ static void mlx5_fc_release(struct mlx5_core_dev *dev, struct mlx5_fc *counter) mlx5_fc_free(dev, counter); } -static void mlx5_fc_stats_bulk_query_size_increase(struct mlx5_core_dev *dev) +static void mlx5_fc_stats_bulk_query_buf_realloc(struct mlx5_core_dev *dev, + int bulk_query_len) { struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - int max_bulk_len = get_max_bulk_query_len(dev); - unsigned long now = jiffies; u32 *bulk_query_out_tmp; - int max_out_len; - - if (fc_stats->bulk_query_alloc_failed && - time_before(now, fc_stats->next_bulk_query_alloc)) - return; + int out_len; - max_out_len = mlx5_cmd_fc_get_bulk_query_out_len(max_bulk_len); - bulk_query_out_tmp = kzalloc(max_out_len, GFP_KERNEL); + out_len = mlx5_cmd_fc_get_bulk_query_out_len(bulk_query_len); + bulk_query_out_tmp = kvzalloc(out_len, GFP_KERNEL); if (!bulk_query_out_tmp) { mlx5_core_warn_once(dev, - "Can't increase flow counters bulk query buffer size, insufficient memory, bulk_size(%d)\n", - max_bulk_len); - fc_stats->bulk_query_alloc_failed = true; - fc_stats->next_bulk_query_alloc = - now + MLX5_FC_BULK_QUERY_ALLOC_PERIOD; + "Can't increase flow counters bulk query buffer size, alloc failed, bulk_query_len(%d)\n", + bulk_query_len); return; } - kfree(fc_stats->bulk_query_out); + kvfree(fc_stats->bulk_query_out); fc_stats->bulk_query_out = bulk_query_out_tmp; - fc_stats->bulk_query_len = max_bulk_len; - if (fc_stats->bulk_query_alloc_failed) { - mlx5_core_info(dev, - "Flow counters bulk query buffer size increased, bulk_size(%d)\n", - max_bulk_len); - fc_stats->bulk_query_alloc_failed = false; - } + fc_stats->bulk_query_len = bulk_query_len; + mlx5_core_info(dev, + "Flow counters bulk query buffer size increased, bulk_query_len(%d)\n", + bulk_query_len); } static void mlx5_fc_stats_work(struct work_struct *work) @@ -327,13 +315,14 @@ static void mlx5_fc_stats_work(struct work_struct *work) fc_stats->num_counters--; } - if (fc_stats->bulk_query_len < get_max_bulk_query_len(dev) && - fc_stats->num_counters > get_init_bulk_query_len(dev)) - mlx5_fc_stats_bulk_query_size_increase(dev); - if (time_before(now, fc_stats->next_query) || list_empty(&fc_stats->counters)) return; + + if (fc_stats->bulk_query_len < get_max_bulk_query_len(dev) && + fc_stats->num_counters > get_init_bulk_query_len(dev)) + mlx5_fc_stats_bulk_query_buf_realloc(dev, get_max_bulk_query_len(dev)); + last = list_last_entry(&fc_stats->counters, struct mlx5_fc, list); counter = list_first_entry(&fc_stats->counters, struct mlx5_fc, @@ -453,12 +442,11 @@ EXPORT_SYMBOL(mlx5_fc_destroy); int mlx5_init_fc_stats(struct mlx5_core_dev *dev) { struct mlx5_fc_stats *fc_stats; - int init_bulk_len; - int init_out_len; fc_stats = kzalloc(sizeof(*fc_stats), GFP_KERNEL); if (!fc_stats) return -ENOMEM; + dev->priv.fc_stats = fc_stats; spin_lock_init(&fc_stats->counters_idr_lock); idr_init(&fc_stats->counters_idr); @@ -466,12 +454,10 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) init_llist_head(&fc_stats->addlist); init_llist_head(&fc_stats->dellist); - init_bulk_len = get_init_bulk_query_len(dev); - init_out_len = mlx5_cmd_fc_get_bulk_query_out_len(init_bulk_len); - fc_stats->bulk_query_out = kzalloc(init_out_len, GFP_KERNEL); + /* Allocate initial (small) bulk query buffer. */ + mlx5_fc_stats_bulk_query_buf_realloc(dev, get_init_bulk_query_len(dev)); if (!fc_stats->bulk_query_out) goto err_bulk; - fc_stats->bulk_query_len = init_bulk_len; fc_stats->wq = create_singlethread_workqueue("mlx5_fc"); if (!fc_stats->wq) @@ -481,12 +467,11 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) INIT_DELAYED_WORK(&fc_stats->work, mlx5_fc_stats_work); mlx5_fc_pool_init(&fc_stats->fc_pool, dev); - dev->priv.fc_stats = fc_stats; return 0; err_wq_create: - kfree(fc_stats->bulk_query_out); + kvfree(fc_stats->bulk_query_out); err_bulk: kfree(fc_stats); return -ENOMEM; @@ -512,7 +497,7 @@ void mlx5_cleanup_fc_stats(struct mlx5_core_dev *dev) mlx5_fc_pool_cleanup(&fc_stats->fc_pool); idr_destroy(&fc_stats->counters_idr); - kfree(fc_stats->bulk_query_out); + kvfree(fc_stats->bulk_query_out); kfree(fc_stats); } From patchwork Thu Aug 15 05:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764429 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2065.outbound.protection.outlook.com [40.107.92.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72C9C1448C1 for ; Thu, 15 Aug 2024 06:24:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703089; cv=fail; b=YvkQWBMwZgcoZpcBbikXJxgssdHina71tEJ3OrXvMMyRg9Tg2H7WoH+17wx++cXAtojO7IQfpm5iSqZErl9yY48pDcbay53Cl9ZBiHo2aEaOzf2zzcJgkacim6iUxbP/4fjTQ3na20hMAp7sHXtzTSjokA7+tbELYcZJnqtx5eU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703089; c=relaxed/simple; bh=MdJliMSBFa1EqMvO1S60iXF+vQW0fvBEJ8W6Q2VWv8c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QxnDzYyHHLrLYspJcELAwRx78dG1B6l5EGfQboYkosDjppLvBGKO1IEAnwYoYBo97i2GYiwrTbOnxo3rAzrFEszNXi5W9Lig0w9+NnZiM6xU4KQh58o2NT7dW9yjW/lr0+IPcqTIlmCOfa/KIbku05SnA7NwdACOMh0BLQgnYDM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=EoLZRLRj; arc=fail smtp.client-ip=40.107.92.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="EoLZRLRj" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gsPllM3fSvWUnZeonmLES13LVOsTHYK55nqhUuguqg8ypGVhHBJoLLZPUQ3PCXwRBCH0A86bu0Sjy7TlQozXY5SxTEfDWxZf/CanBazFYQF+ANu9OV37Wms8aJzL448D28GfvLPBSUsPWbLGNowWJBTQHKF/OIU2oyefl1JcUdGZsMpuPIB/YtbnzW5X6PsYaR0IUKEIXiBw5MXx98hrx9hzZBfzsWuR7HklqU1XVg0Mk9Q68ykD6gq5BYsCFUBi3qrcK6z4XCdtrUNQk4HtQahXc/5x0KJBm/ObOS5mnmnCphpyww2tAhEqNI2ckEF+xaorT088L+KPh75sIz5GPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4xd+sSglU+WvtxNUA/9fYIcr4xCYH4R97W7jHquO9ss=; b=JqzVV3dGLAENCDuFOFsRboXPqzvNwTHqE9QjX46w/gKplLzSkDh5M6yt7vOeD4I2Fdq7IdzP4hX6oEXP+d9MDjZli+HeQSqwxDzm9ATRFG6QT0FOVwZIfDtBBR1m5C7YMwMgPddAYg+HBHo9LV+Cl0bRfcsLg3mPAJk6TAhZmlSFX+xG1k/myWOe32aMMTSUvOlYyR6P3sDif8EQEnY3Xu+kI/u2xP8eP1YVlPvxqipYR8jeKRuhsIlyXZ/mcFfYb7ok/6+ZrTzXAYAI0m2uWP/gzV3ywLM1XNtMz68c0CRQETvZRzjWAGU6twHL629oH2LF1T3FAPK6a/YqMG2qIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4xd+sSglU+WvtxNUA/9fYIcr4xCYH4R97W7jHquO9ss=; b=EoLZRLRjNj61sVl5cMlM6V7CXzSlWG4I2vOCmlWC9LHfpYoqW1dCaNxrRjZ8j+CqeRt5DCb+bkjFcIuQBJZHDdLRjtM4sOtGqEvPeFNfZWe8R/2RkLSXC8M47SwEgs5fy71RSPWtOutlViQNb3cxo3BIaeDbe7c5GnQbDJu5fKhm3eMNSr2Tnlo6gUryejH9VJa9EgJQphLynZNgpvgw5+4SAzU/qzdzK4DZGz3Ysnqd7sdsdYlmHvWm0g3KE/sWrY+Z8wVOcfEjEwGTMCfwRQZQDF0L3EKOoqPxp0uV8oIKz8DlOZAXx0WVYtZOa0DbO3XZijG0lCN2huhnYHtd6A== Received: from BN9PR03CA0935.namprd03.prod.outlook.com (2603:10b6:408:108::10) by PH7PR12MB7843.namprd12.prod.outlook.com (2603:10b6:510:27e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23; Thu, 15 Aug 2024 06:24:38 +0000 Received: from BN2PEPF000044A7.namprd04.prod.outlook.com (2603:10b6:408:108:cafe::92) by BN9PR03CA0935.outlook.office365.com (2603:10b6:408:108::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.33 via Frontend Transport; Thu, 15 Aug 2024 06:24:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044A7.mail.protection.outlook.com (10.167.243.101) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:24:38 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:24 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:24 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:21 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 03/10] net/mlx5: hw counters: Replace IDR+lists with xarray Date: Thu, 15 Aug 2024 08:46:49 +0300 Message-ID: <20240815054656.2210494-4-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044A7:EE_|PH7PR12MB7843:EE_ X-MS-Office365-Filtering-Correlation-Id: a53f17bd-bd34-43fd-bdf4-08dcbcf2f08e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: UjyONIrRlBMBtrEmcksQlBiwrS1hM6jQ4V/V9e533DHkYzIU+Pvwr0wyN39Tja3DVRaT+Ldx9/bVLLJ7w+0tIpB0rePg8MgScYdlL3/jw5RBQiRJFd5g4fpJvX46RExIG3poTgWchaLebZ1pAm4Fpdsiv77KAJWCRY6PP2LwubE5WAlOmx8H0AwbIclL3QrbD/cEKTHDx55G45YUdJ48dqyK4YKkPKZdMwotDnkk5QZK2BbS4LfZWtSlcl+E0YCKwGbfEgQtcAbZbYMj4s3mzKdm240kd4pHWTFFGlKVSJfzAF3DinUDoWRAVrdJbzJ7KtREf1VFXPreFqgw3W4W/M/11cCOy69JSRTwTbaQxv59el6J73C7lWAytw0OSxsdiv0WKw6a2ZPW6wiPge5bJJLK8rkTobYJseYPb+F7bWQfHeBVoLejTqiEF+gVvymRiMaQpXzjXOHl4xfso/gEXqw9CVCmCgcArcE+S14Xo6Sl8kCBCfzkgXIsLgeFmEaljc7zRWXXIZaMKJ9uDE1IFv4mC0NPLomrdblp8T+UJJPZkbC7K6WhY7mqpafX/4FLe7hwDmykjsTkduR+gMxzCsAD8W3hPGs0l4FbctfyV+q/oTB33CarhG5BlySe6W/aRsU14RrhM402RrTwwxOHFFsFDzRIEWqNdTaJQId5YtNAtCYgQKjoWiG6O0rTdwvPjH4eDaM2MoChugE9pyfYwKmRk2Dq80Ac1GEU/HBNJ8+Gksr5p+aGUWMoyixn/PNuCfqqzmlovKYUXYr2lu25UuAQQlxZKqKFUlcKByf9t33m/1zPH0gfzCTriLDmfhJgWyEPMQqMjVHOZVcePuMpNykxbIT3wYyOLlELc30LVA0U4KXvg/X6ToFdnemXkux9MSxBQoxkbA1jx1pW7/IPL3TLtcW5C2z8jYofljML6mA7JFOtR+wdENWbDMIsd/UVXNxOvJbRNW87NSAeoFiqYnNYTrneSf414mt3dYSCDrv3AN+Jwrv5MqCSdsH+yjasF5IZMccOvwrBNNTU+BQaVrS9kcrBEIDHjLK6QFfIFmjrkpB0FRW/0fSUF9gkNXf+u4wMXQz05si0F2+uTUk62/zxynqky6wgAjD0PA0Y3ovNBLE6GOuCsDzkEfCYiji8MLYfZ0JZpbQP0w8X04TNKQ4lE43NtPm+Eicmotod6qk+CQGeUyY/ACP51mYr5Ft9HJaD1X6CFznQJLuNz6q6U7yrXV5XII6oBjUI/dOJQ/4h2DVh0GwpeoYdNOcoBYjRUGID3N0vWtEzZ659piKnmqVob3eVq0uWhH9tCGC9RdFMHptnLk9VdJG3jurJ0wmp5iex52EvFGdOVqN4Chp1BzRXJZ7IhdMfrd7iZueNqb9dQTtKb8BX45fe+aj+UQDu4xuzUGXY2LZNG7nC0RiPwDbFvD0g9CmpflEFE08q+V+NcK2n7H0q6+/mnD5yltCS X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:38.1261 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a53f17bd-bd34-43fd-bdf4-08dcbcf2f08e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044A7.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7843 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu Previously, managing counters was a complicated affair involving an IDR, a sorted double linked list, two single linked lists and a complex dance between a non-periodic wq task and users adding/deleting counters. Adding was done by inserting new counters into the IDR and into a single linked list, leaving the wq to process the list and actually add the counters into the double linked list, maintained sorted with the IDR. Deleting involved adding the counter into another single linked list, leaving the wq to actually unlink the counter from the other structures and release it. Dumping the counters is done with the bulk query API, which relies on the counter list being sorted and unmutable during querying to efficiently retrieve cached counter values. Finally, the IDR data struct is deprecated. This commit replaces all of that with an xarray. Adding is now done directly, by using xa_lock. Deleting is also done directly, under the xa_lock. Querying is done from a periodic task running every sampling_interval (default 1s) and uses the bulk query API for efficiency. It works by iterating over the xarray: - when a new bulk needs to be started, the bulk information is computed under the xa_lock. - the xa iteration state is saved and the xa_lock dropped. - the HW is queried for bulk counter values. - the xa_lock is reacquired. - counter caches with ids covered by the bulk response are updated. Querying always requests the max bulk length, for simplicity. Counters could be added/deleted while the HW is queried. This is safe, as the HW API simply returns unknown values for counters not in HW, but those values won't be accessed. Only counters present in xarray before bulk query will actually read queried cache values. This cuts down the size of mlx5_fc by 4 pointers (88->56 bytes), which amounts to ~3MB / 100K counters. But more importantly, this solves the wq spinlock congestion issue seen happening on high-rate counter insertion+deletion. Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- .../ethernet/mellanox/mlx5/core/fs_counters.c | 276 ++++++------------ 1 file changed, 82 insertions(+), 194 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index 9892895da9ee..05d9351ff577 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -32,7 +32,6 @@ #include #include -#include #include "mlx5_core.h" #include "fs_core.h" #include "fs_cmd.h" @@ -51,21 +50,13 @@ struct mlx5_fc_cache { }; struct mlx5_fc { - struct list_head list; - struct llist_node addlist; - struct llist_node dellist; - - /* last{packets,bytes} members are used when calculating the delta since - * last reading - */ - u64 lastpackets; - u64 lastbytes; - - struct mlx5_fc_bulk *bulk; u32 id; bool aging; - + struct mlx5_fc_bulk *bulk; struct mlx5_fc_cache cache ____cacheline_aligned_in_smp; + /* last{packets,bytes} are used for calculating deltas since last reading. */ + u64 lastpackets; + u64 lastbytes; }; struct mlx5_fc_pool { @@ -80,19 +71,14 @@ struct mlx5_fc_pool { }; struct mlx5_fc_stats { - spinlock_t counters_idr_lock; /* protects counters_idr */ - struct idr counters_idr; - struct list_head counters; - struct llist_head addlist; - struct llist_head dellist; + struct xarray counters; struct workqueue_struct *wq; struct delayed_work work; - unsigned long next_query; unsigned long sampling_interval; /* jiffies */ u32 *bulk_query_out; int bulk_query_len; - size_t num_counters; + size_t num_counters; /* Also protected by xarray->xa_lock. */ bool bulk_query_alloc_failed; unsigned long next_bulk_query_alloc; struct mlx5_fc_pool fc_pool; @@ -103,78 +89,6 @@ static void mlx5_fc_pool_cleanup(struct mlx5_fc_pool *fc_pool); static struct mlx5_fc *mlx5_fc_pool_acquire_counter(struct mlx5_fc_pool *fc_pool); static void mlx5_fc_pool_release_counter(struct mlx5_fc_pool *fc_pool, struct mlx5_fc *fc); -/* locking scheme: - * - * It is the responsibility of the user to prevent concurrent calls or bad - * ordering to mlx5_fc_create(), mlx5_fc_destroy() and accessing a reference - * to struct mlx5_fc. - * e.g en_tc.c is protected by RTNL lock of its caller, and will never call a - * dump (access to struct mlx5_fc) after a counter is destroyed. - * - * access to counter list: - * - create (user context) - * - mlx5_fc_create() only adds to an addlist to be used by - * mlx5_fc_stats_work(). addlist is a lockless single linked list - * that doesn't require any additional synchronization when adding single - * node. - * - spawn thread to do the actual destroy - * - * - destroy (user context) - * - add a counter to lockless dellist - * - spawn thread to do the actual del - * - * - dump (user context) - * user should not call dump after destroy - * - * - query (single thread workqueue context) - * destroy/dump - no conflict (see destroy) - * query/dump - packets and bytes might be inconsistent (since update is not - * atomic) - * query/create - no conflict (see create) - * since every create/destroy spawn the work, only after necessary time has - * elapsed, the thread will actually query the hardware. - */ - -static struct list_head *mlx5_fc_counters_lookup_next(struct mlx5_core_dev *dev, - u32 id) -{ - struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - unsigned long next_id = (unsigned long)id + 1; - struct mlx5_fc *counter; - unsigned long tmp; - - rcu_read_lock(); - /* skip counters that are in idr, but not yet in counters list */ - idr_for_each_entry_continue_ul(&fc_stats->counters_idr, - counter, tmp, next_id) { - if (!list_empty(&counter->list)) - break; - } - rcu_read_unlock(); - - return counter ? &counter->list : &fc_stats->counters; -} - -static void mlx5_fc_stats_insert(struct mlx5_core_dev *dev, - struct mlx5_fc *counter) -{ - struct list_head *next = mlx5_fc_counters_lookup_next(dev, counter->id); - - list_add_tail(&counter->list, next); -} - -static void mlx5_fc_stats_remove(struct mlx5_core_dev *dev, - struct mlx5_fc *counter) -{ - struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - - list_del(&counter->list); - - spin_lock(&fc_stats->counters_idr_lock); - WARN_ON(!idr_remove(&fc_stats->counters_idr, counter->id)); - spin_unlock(&fc_stats->counters_idr_lock); -} - static int get_init_bulk_query_len(struct mlx5_core_dev *dev) { return min_t(int, MLX5_INIT_COUNTERS_BULK, @@ -203,47 +117,64 @@ static void update_counter_cache(int index, u32 *bulk_raw_data, cache->lastuse = jiffies; } -static void mlx5_fc_stats_query_counter_range(struct mlx5_core_dev *dev, - struct mlx5_fc *first, - u32 last_id) +/* Synchronization notes + * + * Access to counter array: + * - create - mlx5_fc_create() (user context) + * - inserts the counter into the xarray. + * + * - destroy - mlx5_fc_destroy() (user context) + * - erases the counter from the xarray and releases it. + * + * - query mlx5_fc_query(), mlx5_fc_query_cached{,_raw}() (user context) + * - user should not access a counter after destroy. + * + * - bulk query (single thread workqueue context) + * - create: query relies on 'lastuse' to avoid updating counters added + * around the same time as the current bulk cmd. + * - destroy: destroyed counters will not be accessed, even if they are + * destroyed during a bulk query command. + */ +static void mlx5_fc_stats_query_all_counters(struct mlx5_core_dev *dev) { struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - bool query_more_counters = (first->id <= last_id); - int cur_bulk_len = fc_stats->bulk_query_len; + u32 bulk_len = fc_stats->bulk_query_len; + XA_STATE(xas, &fc_stats->counters, 0); u32 *data = fc_stats->bulk_query_out; - struct mlx5_fc *counter = first; + struct mlx5_fc *counter; + u32 last_bulk_id = 0; + u64 bulk_query_time; u32 bulk_base_id; - int bulk_len; int err; - while (query_more_counters) { - /* first id must be aligned to 4 when using bulk query */ - bulk_base_id = counter->id & ~0x3; - - /* number of counters to query inc. the last counter */ - bulk_len = min_t(int, cur_bulk_len, - ALIGN(last_id - bulk_base_id + 1, 4)); - - err = mlx5_cmd_fc_bulk_query(dev, bulk_base_id, bulk_len, - data); - if (err) { - mlx5_core_err(dev, "Error doing bulk query: %d\n", err); - return; - } - query_more_counters = false; - - list_for_each_entry_from(counter, &fc_stats->counters, list) { - int counter_index = counter->id - bulk_base_id; - struct mlx5_fc_cache *cache = &counter->cache; - - if (counter->id >= bulk_base_id + bulk_len) { - query_more_counters = true; - break; + xas_lock(&xas); + xas_for_each(&xas, counter, U32_MAX) { + if (xas_retry(&xas, counter)) + continue; + if (unlikely(counter->id >= last_bulk_id)) { + /* Start new bulk query. */ + /* First id must be aligned to 4 when using bulk query. */ + bulk_base_id = counter->id & ~0x3; + last_bulk_id = bulk_base_id + bulk_len; + /* The lock is released while querying the hw and reacquired after. */ + xas_unlock(&xas); + /* The same id needs to be processed again in the next loop iteration. */ + xas_reset(&xas); + bulk_query_time = jiffies; + err = mlx5_cmd_fc_bulk_query(dev, bulk_base_id, bulk_len, data); + if (err) { + mlx5_core_err(dev, "Error doing bulk query: %d\n", err); + return; } - - update_counter_cache(counter_index, data, cache); + xas_lock(&xas); + continue; } + /* Do not update counters added after bulk query was started. */ + if (time_after64(bulk_query_time, counter->cache.lastuse)) + update_counter_cache(counter->id - bulk_base_id, data, + &counter->cache); } + xas_unlock(&xas); } static void mlx5_fc_free(struct mlx5_core_dev *dev, struct mlx5_fc *counter) @@ -291,46 +222,19 @@ static void mlx5_fc_stats_work(struct work_struct *work) struct mlx5_fc_stats *fc_stats = container_of(work, struct mlx5_fc_stats, work.work); struct mlx5_core_dev *dev = fc_stats->fc_pool.dev; - /* Take dellist first to ensure that counters cannot be deleted before - * they are inserted. - */ - struct llist_node *dellist = llist_del_all(&fc_stats->dellist); - struct llist_node *addlist = llist_del_all(&fc_stats->addlist); - struct mlx5_fc *counter = NULL, *last = NULL, *tmp; - unsigned long now = jiffies; - - if (addlist || !list_empty(&fc_stats->counters)) - queue_delayed_work(fc_stats->wq, &fc_stats->work, - fc_stats->sampling_interval); - - llist_for_each_entry(counter, addlist, addlist) { - mlx5_fc_stats_insert(dev, counter); - fc_stats->num_counters++; - } + int num_counters; - llist_for_each_entry_safe(counter, tmp, dellist, dellist) { - mlx5_fc_stats_remove(dev, counter); - - mlx5_fc_release(dev, counter); - fc_stats->num_counters--; - } - - if (time_before(now, fc_stats->next_query) || - list_empty(&fc_stats->counters)) - return; + queue_delayed_work(fc_stats->wq, &fc_stats->work, fc_stats->sampling_interval); + /* num_counters is only needed for determining whether to increase the buffer. */ + xa_lock(&fc_stats->counters); + num_counters = fc_stats->num_counters; + xa_unlock(&fc_stats->counters); if (fc_stats->bulk_query_len < get_max_bulk_query_len(dev) && - fc_stats->num_counters > get_init_bulk_query_len(dev)) + num_counters > get_init_bulk_query_len(dev)) mlx5_fc_stats_bulk_query_buf_realloc(dev, get_max_bulk_query_len(dev)); - last = list_last_entry(&fc_stats->counters, struct mlx5_fc, list); - - counter = list_first_entry(&fc_stats->counters, struct mlx5_fc, - list); - if (counter) - mlx5_fc_stats_query_counter_range(dev, counter, last->id); - - fc_stats->next_query = now + fc_stats->sampling_interval; + mlx5_fc_stats_query_all_counters(dev); } static struct mlx5_fc *mlx5_fc_single_alloc(struct mlx5_core_dev *dev) @@ -374,7 +278,6 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) if (IS_ERR(counter)) return counter; - INIT_LIST_HEAD(&counter->list); counter->aging = aging; if (aging) { @@ -384,18 +287,15 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) counter->lastbytes = counter->cache.bytes; counter->lastpackets = counter->cache.packets; - idr_preload(GFP_KERNEL); - spin_lock(&fc_stats->counters_idr_lock); + xa_lock(&fc_stats->counters); - err = idr_alloc_u32(&fc_stats->counters_idr, counter, &id, id, - GFP_NOWAIT); - - spin_unlock(&fc_stats->counters_idr_lock); - idr_preload_end(); - if (err) + err = xa_err(__xa_store(&fc_stats->counters, id, counter, GFP_KERNEL)); + if (err != 0) { + xa_unlock(&fc_stats->counters); goto err_out_alloc; - - llist_add(&counter->addlist, &fc_stats->addlist); + } + fc_stats->num_counters++; + xa_unlock(&fc_stats->counters); } return counter; @@ -407,12 +307,7 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) struct mlx5_fc *mlx5_fc_create(struct mlx5_core_dev *dev, bool aging) { - struct mlx5_fc *counter = mlx5_fc_create_ex(dev, aging); - struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - - if (aging) - mod_delayed_work(fc_stats->wq, &fc_stats->work, 0); - return counter; + return mlx5_fc_create_ex(dev, aging); } EXPORT_SYMBOL(mlx5_fc_create); @@ -430,11 +325,11 @@ void mlx5_fc_destroy(struct mlx5_core_dev *dev, struct mlx5_fc *counter) return; if (counter->aging) { - llist_add(&counter->dellist, &fc_stats->dellist); - mod_delayed_work(fc_stats->wq, &fc_stats->work, 0); - return; + xa_lock(&fc_stats->counters); + fc_stats->num_counters--; + __xa_erase(&fc_stats->counters, counter->id); + xa_unlock(&fc_stats->counters); } - mlx5_fc_release(dev, counter); } EXPORT_SYMBOL(mlx5_fc_destroy); @@ -448,11 +343,7 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) return -ENOMEM; dev->priv.fc_stats = fc_stats; - spin_lock_init(&fc_stats->counters_idr_lock); - idr_init(&fc_stats->counters_idr); - INIT_LIST_HEAD(&fc_stats->counters); - init_llist_head(&fc_stats->addlist); - init_llist_head(&fc_stats->dellist); + xa_init(&fc_stats->counters); /* Allocate initial (small) bulk query buffer. */ mlx5_fc_stats_bulk_query_buf_realloc(dev, get_init_bulk_query_len(dev)); @@ -467,7 +358,7 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) INIT_DELAYED_WORK(&fc_stats->work, mlx5_fc_stats_work); mlx5_fc_pool_init(&fc_stats->fc_pool, dev); - + queue_delayed_work(fc_stats->wq, &fc_stats->work, MLX5_FC_STATS_PERIOD); return 0; err_wq_create: @@ -480,23 +371,20 @@ int mlx5_init_fc_stats(struct mlx5_core_dev *dev) void mlx5_cleanup_fc_stats(struct mlx5_core_dev *dev) { struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; - struct llist_node *tmplist; struct mlx5_fc *counter; - struct mlx5_fc *tmp; + unsigned long id; cancel_delayed_work_sync(&fc_stats->work); destroy_workqueue(fc_stats->wq); fc_stats->wq = NULL; - tmplist = llist_del_all(&fc_stats->addlist); - llist_for_each_entry_safe(counter, tmp, tmplist, addlist) - mlx5_fc_release(dev, counter); - - list_for_each_entry_safe(counter, tmp, &fc_stats->counters, list) + xa_for_each(&fc_stats->counters, id, counter) { + xa_erase(&fc_stats->counters, id); mlx5_fc_release(dev, counter); + } + xa_destroy(&fc_stats->counters); mlx5_fc_pool_cleanup(&fc_stats->fc_pool); - idr_destroy(&fc_stats->counters_idr); kvfree(fc_stats->bulk_query_out); kfree(fc_stats); } From patchwork Thu Aug 15 05:46:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764427 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2045.outbound.protection.outlook.com [40.107.94.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E409C143748 for ; Thu, 15 Aug 2024 06:24:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.45 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703088; cv=fail; b=VzeZRBzPTm8yKzPh83/1OZMPPN+Dd1AEImrEER2J0DCiHKz6m64FfXlo+AISdsQJELeOXIK4NPqAI0peRVya/SitrBBcsRfUhzp6ZQi7BiLuBKoeHjv2pIENQiaPGC2p0YOsj9tgZiTGqSI33XZVW/1Pw/Nyvd2TojsdnD1Etps= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703088; c=relaxed/simple; bh=5n3sZH2bRntkMmZIJLXG0+s33H09Q+wSr8cLPAA1sAQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=a6KbLappMNAMS9mFGabgUoRanN6ZNRcCnNHk6KDlIhTvvpWDvcJEyqq3A/bnoAsRYDkqLjZluVi3tR0g3Lim0Rcu5v5uSRjHO8ebL7QcTkn8ehZs77ufuUmvnMpCx06fXDvhQx+mfAT/pzRcMG+A/Fj0KiN/w/9PZRWtuFdCW2Y= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Rm5tRIDb; arc=fail smtp.client-ip=40.107.94.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Rm5tRIDb" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Q8+Bqop4anVK5JX9RaDLVZ3+ucrVaQ3EKexKuNbMsyLW0iDzx28Fjg6l//kwnJ6zK4gc22pmePdtiLFaLgWuBNxmiyyOSItvnKcfi69Gw7C/AIWVRxbi01vEHXqRa7OTIX8u2uNjeoy60qjktkxEpwD8DFgKG00Psa8ybA6fX3zYrlCjhuXSjM8zIh8cbsnYE+mj4W/rf6Seppz2HocGBxk0MUTK0s2mDIhjWZsagQfDj4sWDvxn8FBka95Iw3UqHSSNmChlSyWBjyL+5yvao5Fcq6Aw3Wpg6SjX5TCDQdR+XKbXaqRvOWf9Y3K5/Iv6hM6f3E+mGmtIBJ6F3hX36Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cWeXu8oB5hWGn88YtOrb9nDr5pwy8kUIM1NvQU0vqcg=; b=Sc7vEFFcAfV6/0wI7QUl1zrWvv+yo4o/7y10MsA1++hKMuxgN8jM7D1dTnGTBclRV1vGZp2+amG3idynJ1Ul0NFakXdtHR3MIV0YlAn0ZY7L3F+cY0Py1bexnHu78rWUPIXEfKEwd012kEK5mP9b61zklDy8PiLYMKdvI4obc7WZI9yRHdsMbaRsy3+siJ9vD8R8YvsjksFkRhGxg36OXJps4osaz82nFvTryx1ddGtlrpf3sPT/GTVBRzPInywxuaWA9QPpqxOL88KL/6LBJwG3o2Kyx5+Z+TisRtJ4LNRPTupmfXmDBew2b3tbs6v6OjOueMZajoGS8WB0De084A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cWeXu8oB5hWGn88YtOrb9nDr5pwy8kUIM1NvQU0vqcg=; b=Rm5tRIDbMgVgCHlFPs6ks7pCiPswGwhXQd9FmpiTkSeeT6vctqqhz3y5mJuO86E8ti3jQCr87F0V5+yaoe8FvrIzjFCgNGBCLWQe3t1TVgyrDvioJGhT57R8T7qerFp2jRFM2S3FuesAXeR8E+rxb+gWxIRj7O7YRuLBDMU4kfK3bg3OHUhnuP8p4qe5tSGSeegjicXE/e9wXXtQ0UDz7vtVYxEKRE3yLYZZAwjqVfSnRuEInFzoafu+3QRbkTRGZhbJZxvJE9WGPtgjy5fgEI8ZT8qEhzlIDMGQP0S97LgmYot+ysZg/+esnIHLeBiQ6r8th/V/gK1ktVoh4+IYtA== Received: from BN8PR03CA0026.namprd03.prod.outlook.com (2603:10b6:408:94::39) by DM6PR12MB4123.namprd12.prod.outlook.com (2603:10b6:5:21f::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.17; Thu, 15 Aug 2024 06:24:41 +0000 Received: from BN2PEPF000055DE.namprd21.prod.outlook.com (2603:10b6:408:94:cafe::58) by BN8PR03CA0026.outlook.office365.com (2603:10b6:408:94::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.33 via Frontend Transport; Thu, 15 Aug 2024 06:24:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000055DE.mail.protection.outlook.com (10.167.245.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.4 via Frontend Transport; Thu, 15 Aug 2024 06:24:41 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:28 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:27 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:25 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 04/10] net/mlx5: hw counters: Drop unneeded cacheline alignment Date: Thu, 15 Aug 2024 08:46:50 +0300 Message-ID: <20240815054656.2210494-5-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000055DE:EE_|DM6PR12MB4123:EE_ X-MS-Office365-Filtering-Correlation-Id: 77fec92e-6081-4a2b-cfbc-08dcbcf2f257 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|376014|1800799024|82310400026; X-Microsoft-Antispam-Message-Info: 0KdI5VVMT24f/Bls83pTkVJGOK82Ysyf7OAOrnBrT9BMFyJrkGMEt76P+nejLFw4x3CpJ4xL5kezvNukQvIG3uUbSA3L4MaTPsF4gmdOOs3PpZvkVKMl2IL2GqNQX8uvzmWvXv2Uxeyief+d9t5FaoM7z1hqM4V8KWDjIRloy/1EQEYAG45aAxjLUvhjdV5z9LYRzOFort6CJkZM2TaiW5JB5jTWwJEoKXZlaB0S7fCcoTWh4ab69kGOYtpBMSOixGKDnkbSLjo0lcGlJTtecJbNnqrdulHtfCkfvEqrO8xL4e0EBodgc15epKIrEp+8cfmdTP5+UbXp2kOz2UEowF3E6U72oBnU1dJVQi5TJPxU7iKtf6ZdSjjIDyr2C5l0OUZPZGHwnGEEOlUY3seM9BD5vHDMuaTnVL3eNwSOZ5Y++UrdCRH0l3U5WxC/rA6Sw64g4CIpguUptYq5yBK1JzHS548uVi0Ssx/Cqm8apSmaSvMk6bNAqcaSblB0InxSBxgjyhtr7bD3Re7JhR5csiyJSygIl4YskP55hVs/oHoXVxOfwQjiVaH5JYtzJcsSSUGKdFbib6yVCzJjy8Y2GeT38hZ3afKPiBkzRXSaK8Tw98oT1Lp+hB6K5mpWA6nGKfoHve+9+8uyJe8OlVVCoWzHwK7eVDwks0b8QxJo0lx6RYfBzJQENsGN3gD+b/dOlvikXm3cqUUNqNHcMcnErkFcok5jW2AqW2QKRLbRMAIt2uexfPCdQLE2zAOsO7Nyv+vU4rO3VqK+wTqiTIpa5GNgDWemq5b7imHcNiIJjPchYWyc6g6H4iUTRnyojR//T1e/ENs2bxdN8pHd7G2K27/GILPPwCk3F47DvqUzF7VXGETczZw6Hb+zMd/XJYaUqmlVVBvmtJL0vEBd+YkGgA9/1xciEVYBtOVqyLaIQyci8o2asU2hucz8V5ZLUY/nzPAjK/b07MWu7sL2/TUK+BxqfA7FvuYLjKBbKZC6gSCasQSWKL1SYFprC7miVcBTKVLg33nyhEWjk2pEpdxYr3pAGwzMPPyVl+Ly+k7dkPPUGApkTA6Lb2mFy18TUPXB4zsZvyvdDYV802tI/Oh66CEJJfwf5c6i81eHqMc09k43J3gAIyC02feBZjIEjqL4/O2wlNTGslUqJLY0Zn/w1a3c8yUQE4oG1PB0l+md6+vUBzu+gF7k0hYkI294rDyvuyvLA55cBvgmr0MBb04c/y71jS4NmQK4b5zEFd3CWVtdnCPU/d6bHYszIZWxDvVdpqOteKpEigF8anAr9d4DhKb9QqKuYNXeWv+y8TcMGvg1qoMp7pdR5qHvN7x+VM92iLhQMiq0wT/eEnrmKL2XG8etBGP1UfRfw/p7KVGk8NmbjRIybkIMM6h+U0VaA/m7yisTPYjt8xJQuEv2bG2YlyQWQIBXDtcSGBcheMgopiR68xWk7cLyLs045ztuZSdM X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(376014)(1800799024)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:41.1372 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 77fec92e-6081-4a2b-cfbc-08dcbcf2f257 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000055DE.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4123 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu The mlx5_fc struct has a cache for values queried from hw, which is cacheline aligned. On x86_64, this results in: struct mlx5_fc { u32 id; /* 0 4 */ bool aging; /* 4 1 */ /* XXX 3 bytes hole, try to pack */ struct mlx5_fc_bulk * bulk; /* 8 8 */ /* XXX 48 bytes hole, try to pack */ /* --- cacheline 1 boundary (64 bytes) --- */ struct mlx5_fc_cache cache __attribute__((__aligned__(64))); /* 64 24 */ u64 lastpackets; /* 88 8 */ u64 lastbytes; /* 96 8 */ /* size: 128, cachelines: 2, members: 6 */ /* sum members: 53, holes: 2, sum holes: 51 */ /* padding: 24 */ /* forced aligns: 1, forced holes: 1, sum forced holes: 48 */ } __attribute__((__aligned__(64))); (output from pahole). ...So a 48+24=72 byte waste. As far as I can determine, this serves no purpose other than maybe making sure that the values in the cache do not span two cachelines in the worst case scenario, but that's not a valid enough reason to waste 72 bytes per counter, especially since this code is not performance-critical. There could potentially be hundreds of thousands of counters (e.g. for connection-tracking), so this quickly adds up to multiple MB wasted. This commit removes the alignment, resulting in: struct mlx5_fc { [...] /* size: 56, cachelines: 1, members: 6 */ /* sum members: 53, holes: 1, sum holes: 3 */ /* last cacheline: 56 bytes */ }; Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index 05d9351ff577..ef13941e55c2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -53,7 +53,7 @@ struct mlx5_fc { u32 id; bool aging; struct mlx5_fc_bulk *bulk; - struct mlx5_fc_cache cache ____cacheline_aligned_in_smp; + struct mlx5_fc_cache cache; /* last{packets,bytes} are used for calculating deltas since last reading. */ u64 lastpackets; u64 lastbytes; From patchwork Thu Aug 15 05:46:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764428 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2047.outbound.protection.outlook.com [40.107.93.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5DD317625E for ; Thu, 15 Aug 2024 06:24:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.47 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703089; cv=fail; b=aXqyXsMVGMvM1nF4Ii60x9C2g4vd6DAt0RZHsauQqTiwBFj9UonOcAZeSiNNzLmzN8m7S6FfUF0VBQLkYLopCJ0kTD3IEzMLYAqzG8omOIKK6vI6TAUFz4+XwW1SRmuvMpBYrxwmIuXhCvcibj9szbRYgnT0L8dDznOFW5aATPM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703089; c=relaxed/simple; bh=2jypo7DGpI8g0wA9U8X8BmO5SBEhWXsYOAXjBc5/h3Q=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j9GB7yGRlgyS9y6vG0dmIRCxtf8FsUUuOaqbUbjwNexR1m8C9pMuHIb575b0SdTwrQYH/fNETRAXN7CohLlFLDMGLWMIx976mbLBL0XGyuEmAwYiTSkZrajGc5IRszHge/9zCxJWg5orEv/0NmKzpnIBCvvStNVsPhGjWLbC484= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=rpb9Umpz; arc=fail smtp.client-ip=40.107.93.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="rpb9Umpz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=I8TB1fHIBRPP+1sQG3sZHATBsN1bKLW6ybtBIa4VaFlyvBqpqoWpaiv0N0XvuCOcdCkP2aqOR8bzFvBPdrlxX4uezjaUuEmNk5rZSj2xcSkudqUm4PwAayYd224fwCv5U5eb/alEFbBPLPVf0bKoTW79sJMMHHTlZJuTIzgF+togeI1k5w580SzyI9GIAsxnRXN1PmZKM/P8iFd4e0U82HB2ozcBTfx86tSoXpz9oCalUsWNcVnHkYfMeF1UeZZToZvT6IRZTcuT7CR+q+2gpH/N21RKmEUwLvX5JsFPqsse50Whh7/TOHZJqlIly3DGIGli0fR1ZgMHyKBK8sXNYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lRX3CrtQll6TLsWQLDqiZ188Kb1dwWlFcTPrC/ac9uI=; b=lSixnLn+St/2VNo5YQ9D96uwAMRhCkkLU2MXl3hZ0VCIPci7ThE2ltTWgdr/R2+/FFVtX9ZO5gwCwiV97viVzGJaOQrR9V56qgJDWLwV0xipLMUULGU3rxRWCr8e+V0IPb/k5qgkvCZhzatxPoZspz5tzHSadTlMyhORIulfL1vSMxycNsO1HKlsskN/uEuw60Oh87CA1RCsNslD6uYs6ii3QsCPce2C0hw66fB7+/UUGVn9zo21QSb1e5x2HcnUxy4UvfreR+ytqZcWp1DWXw5Dg2NPmRVhSzuj8k6yqZqZ+qAbCWRU4QTiqYwqOZdVxMg7kVctB/p7D67G7iqLuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lRX3CrtQll6TLsWQLDqiZ188Kb1dwWlFcTPrC/ac9uI=; b=rpb9UmpzcQeeq5KTT85YkGaxnHlcK0YpUvvWb18yvY+qFhJ+6u6ZMhPwaGAwFbCx3BpqsC5hzCK9nhFo3c4LL7iGHQF3Y5mI3wmtR6CloD0saaPjHGa8TZSW6TyF8B9Zgj4HcntdVquNV54iP5okegbBxzaPijaVX37hQH/BNdFrSOYGJmsd2ZLbn/wn9sbFPE2n3vfHiwgl1yD8eg//MySoob9bWl8MS/LiXrMwl4amKDgQRxf4ThmMiWSGx3SWLmATDMIvn3dX2ayy487+DxGWHUJkAtC9oqm+fN2vFKzQIM6NvLmphiWHLbV6sw0sMrbQa13ArZAh+L1RvTo74g== Received: from BL1P222CA0030.NAMP222.PROD.OUTLOOK.COM (2603:10b6:208:2c7::35) by MW3PR12MB4425.namprd12.prod.outlook.com (2603:10b6:303:5e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.18; Thu, 15 Aug 2024 06:24:43 +0000 Received: from BN2PEPF000044AC.namprd04.prod.outlook.com (2603:10b6:208:2c7:cafe::76) by BL1P222CA0030.outlook.office365.com (2603:10b6:208:2c7::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.24 via Frontend Transport; Thu, 15 Aug 2024 06:24:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044AC.mail.protection.outlook.com (10.167.243.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:24:43 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:31 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:31 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:28 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 05/10] net/mlx5: hw counters: Don't maintain a counter count Date: Thu, 15 Aug 2024 08:46:51 +0300 Message-ID: <20240815054656.2210494-6-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044AC:EE_|MW3PR12MB4425:EE_ X-MS-Office365-Filtering-Correlation-Id: 64b1f6ca-f558-4d49-72fa-08dcbcf2f37a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: ooFrEaW+ibsA0oiDYtFJkLSJauapE0FRPoFBOLvemJKdnpH4r0F54+Vv+JH6FU/MBZgj52n5JDADE+p3oHLy1G2m1qsmxqUJlobLhi0AwY4ozDpNz69mIY4YmcBKou1FP32NTTZl3/4ZvhTkqZoYJ2tApJMM2scG8MaLzCeoFf6deq5Sgll9vwGQ3/LkeLyRIRWNvyC6ofJbGegvcHWgTO/ZX+YP3UR+7W8afNoWYPnUXwNVcOMOq8bZF+ksmpYrSZEXOvnfUbtCZSmZqBAOnrwpxdgfLu/XNB/URo9bsD7yji6LF/aQYM3io0t/VIrDFnBhxZl8Tzi9X4mabIEDOkFltO6/dQUXzFzKhUJXBdoV3su3V2J8EMthpEiRMxwF0qr9yKuoD8lsW3AFcvdamh6hM/ATn5uStcilY2n1g5zCl9VB3wmaxmNauRJ3stzjLCEJpmp87LusMJsKBwvaardErUd2XR3kxymwJE9ej7vWwpQu851224MzyKni4trj/q5Vo8gWBXCv6uodADwU4ide+28wnJrOdC3REPPm7GwlBbiunxN8gnso919fyswbmJY83/5vTWlpg1txNyZ5Ct7dxnhyJTZWGhBDsvvrHLoocIMnhpADrphFaC9Ywvp06HRz74ghrnNIYi845+eON9xtNOM+IXnhFv6CbG9b5aqWdmTluhncmkmLDlnACAT8xJyLiSmWV9OxJCcKHqg923nhCvFEUduUVGA9M2PAwbNO/hx6UFMgIxH+k2cVOkm5W+H9a+C9IGgIroiTRvRDbL+LVjHI3mvCgiUfLnDTa57zjI4vdzgSSDGL6RWuN6gW882TbVFkY/cQvfsc9pp4w3wnmtkV73CBS2iLG3/eAweDcBTV68fWpxOdL+wVNiy0TS5KuuJZrttBwRpfsV58JfpnVRFcZISavd2xp8NxLKy2wRLXV5tM412/fvGO7gnopw3jJ/YAEVM5q8VPIwVMimcZ+4nhg86HjK1GkD2hFprfPe5RWr/KFQ9VyUHjVbXcfNJ39nb6lDwOMa9UmPzassH3nJE2wrP9yu+VKvQtY7OiR51XfO1p2K1q1IsxxVdsdjhzNriLlsUx3X2amWiqpibVDZAew/ItLQCfK3W7N1Sa1AqMmQQvx1MRCCRQzlMK7Z6XSEh3I/MbwzSEhASA6Y1EO7JLAouZg4YtGyvVNeQNOsLZeWJfmZ4mCwQy4U8BdgQaTqv+UPcnFDHexkUjCGv/SN/TardnU40AV23x2fSzDAoGicsLKh0AYwv+uo7n92gJjbvmo3fLnbjy7DBW3/yfuyKJWcDwmYLCjjWHtgdwWzgNEyZHJDnQMMeFUMWxosCy9ZcrPUQg39d2luw/tKBM1dngKjJOKDepeItqcfs6/Y9Hm/fb69ImiUEmM5mGfxH7avq44YL+5+6DIeKjAATRRWKHaFecAE4GlMQ6dJH/pUA9WFXcUdSZgS8UeNnj X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:43.0419 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 64b1f6ca-f558-4d49-72fa-08dcbcf2f37a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044AC.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4425 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu num_counters is only used for deciding whether to grow the bulk query buffer, which is done once more counters than a small initial threshold are present. After that, maintaining num_counters serves no purpose. This commit replaces that with an actual xarray traversal to count the counters. This appears expensive at first sight, but is only done when the number of counters is less than the initial threshold (8) and only once every sampling interval. Once the number of counters goes above the threshold, the bulk query buffer is grown to max size and the xarray traversal is never done again. Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- .../ethernet/mellanox/mlx5/core/fs_counters.c | 40 +++++++++---------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index ef13941e55c2..0b80c33cba5f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -78,7 +78,6 @@ struct mlx5_fc_stats { unsigned long sampling_interval; /* jiffies */ u32 *bulk_query_out; int bulk_query_len; - size_t num_counters; /* Also protected by xarray->xa_lock. */ bool bulk_query_alloc_failed; unsigned long next_bulk_query_alloc; struct mlx5_fc_pool fc_pool; @@ -217,21 +216,28 @@ static void mlx5_fc_stats_bulk_query_buf_realloc(struct mlx5_core_dev *dev, bulk_query_len); } +static int mlx5_fc_num_counters(struct mlx5_fc_stats *fc_stats) +{ + struct mlx5_fc *counter; + int num_counters = 0; + unsigned long id; + + xa_for_each(&fc_stats->counters, id, counter) + num_counters++; + return num_counters; +} + static void mlx5_fc_stats_work(struct work_struct *work) { struct mlx5_fc_stats *fc_stats = container_of(work, struct mlx5_fc_stats, work.work); struct mlx5_core_dev *dev = fc_stats->fc_pool.dev; - int num_counters; queue_delayed_work(fc_stats->wq, &fc_stats->work, fc_stats->sampling_interval); - /* num_counters is only needed for determining whether to increase the buffer. */ - xa_lock(&fc_stats->counters); - num_counters = fc_stats->num_counters; - xa_unlock(&fc_stats->counters); - if (fc_stats->bulk_query_len < get_max_bulk_query_len(dev) && - num_counters > get_init_bulk_query_len(dev)) + /* Grow the bulk query buffer to max if not maxed and enough counters are present. */ + if (unlikely(fc_stats->bulk_query_len < get_max_bulk_query_len(dev) && + mlx5_fc_num_counters(fc_stats) > get_init_bulk_query_len(dev))) mlx5_fc_stats_bulk_query_buf_realloc(dev, get_max_bulk_query_len(dev)); mlx5_fc_stats_query_all_counters(dev); @@ -287,15 +293,9 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) counter->lastbytes = counter->cache.bytes; counter->lastpackets = counter->cache.packets; - xa_lock(&fc_stats->counters); - - err = xa_err(__xa_store(&fc_stats->counters, id, counter, GFP_KERNEL)); - if (err != 0) { - xa_unlock(&fc_stats->counters); + err = xa_err(xa_store(&fc_stats->counters, id, counter, GFP_KERNEL)); + if (err != 0) goto err_out_alloc; - } - fc_stats->num_counters++; - xa_unlock(&fc_stats->counters); } return counter; @@ -324,12 +324,8 @@ void mlx5_fc_destroy(struct mlx5_core_dev *dev, struct mlx5_fc *counter) if (!counter) return; - if (counter->aging) { - xa_lock(&fc_stats->counters); - fc_stats->num_counters--; - __xa_erase(&fc_stats->counters, counter->id); - xa_unlock(&fc_stats->counters); - } + if (counter->aging) + xa_erase(&fc_stats->counters, counter->id); mlx5_fc_release(dev, counter); } EXPORT_SYMBOL(mlx5_fc_destroy); From patchwork Thu Aug 15 05:46:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764430 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53F25177992 for ; Thu, 15 Aug 2024 06:24:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703097; cv=fail; b=TJkGQpufxC2oy0WWUiVD0t5xe53FY24Bbou64nnIR7XpmwVAHvo+SCfC7Ay2+keRZJjj2fH2xIRQQ07NJZUEkNhEpyD/pyKvIUq0hEer9BWEVJH4GXZNNSk9gP9ELQJSB47wStTbKmfaFSpvnD8Zhq9ikIe3MROoFNoHcvRmJHA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703097; c=relaxed/simple; bh=2L2yvoNcrtB2UH/lbpc4I5BOOpAXzT4/sWacMmDUXIQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bbJjUx6lp0wzEc9O1bBZ7hLqyMCmEBM3x/T//rgEuyegkoFNQUspV0mRjqNDORJpt4MUySVvhh3VTaFoshJdIJdlFHYdsOL305Y9z1uTeo8Z+57czx+S8HPa+xMnj/L9KmDnknH8dSLREe01Fxcor35EOsB8RufUN0wWhPk/BvQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=NzwuVNSS; arc=fail smtp.client-ip=40.107.93.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="NzwuVNSS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=AiZMc103Y2NgeK+zlbFnxA3xDfFJglwUeOxvr4JDz8xpypvSYunPKyT7699rndkPOpHx0NFloC5+27iv9GchqSUWetpGVGjQGHLgv7tOFd+fAz3xTWsQCQbmoZauFr0Wbe3X+lCPRvBlN9c/F9GblpEGhos7tcOov4HWwoJrHy667pr9RtWS36vMMr9W2OF9RCu01/sXyr7haGlLRO4RFJ6fACaCzPD1QYJURKqu49pE3hIShgPuxneDlu9MoG9M8AqhztFUccXHB+pTy3Smeq/E+uCGnlmQiBlXC/JM5REgTktk9KhDKyoOZtya51GdnA+G2Ig61BnsIv7x7s3wPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6DvvZJ+8cNipjUCHumR2S3iCbwH6JDI9zKHQf3fzn1w=; b=TooYBCStonnxlxN5A0jnKZ4NnTxbhfK51/7XJg/+VXZu9v4WXHOSYauwh2DNZpv1jFRwpqspGIMb9CtE2U6qWFS4JJHj5r3O2hyfTXgApleH3J5vB2gRKCgirYvuaxV9qGdnWGPcCuhrrvNYV1YmSmi30nTlaMbHa/F99o4Sv5fibAMffrQGUh6JNLR9poq7E5T0jn9d+A6teIt5WHeWK8YsmSIeCUn7jCjyNSUtswfPaZDmiz4SGdzy8JfrB4bcV6zZCCdsYy7WMxVxkqcy0pRmSSV/kEn8CJF2jzQlYIdRkmpO0qDTiFNkSNhbtt9RGESBSLDyv5Wtuhaoe/l8Sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6DvvZJ+8cNipjUCHumR2S3iCbwH6JDI9zKHQf3fzn1w=; b=NzwuVNSS2ez+aoKNB8fVuXIML70CFCcj+6gRIXmAUZTLeD0W1eisuSCWbeVqQXBH/LGUl+QORXmK3NUbY1EA4nAapy5+Erw4DS4d80tLbn7omeYS6mW6GJ3rUy9BCGXcoE31PLTKaAiGSMjTjk8vLzWRcME/JDrS9yaLmJBD2ZTcKJLZmDh4lm633vIrpgWm2MnKXlnP08reJegdPaV/y5dnd8v0F+P8Q8vAre04LkY3vrLmGP8P1fO85KXiCAUhzoOQ0TGmGm2aAVNYOacRuXKm33jC2lHAxWJRZTsubzczQrZuYqGI+crEe1WhCrfXStLeXzNiTH0MvsgSIwYVPg== Received: from BN1PR14CA0019.namprd14.prod.outlook.com (2603:10b6:408:e3::24) by LV8PR12MB9269.namprd12.prod.outlook.com (2603:10b6:408:1fe::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.18; Thu, 15 Aug 2024 06:24:52 +0000 Received: from BN2PEPF000044AA.namprd04.prod.outlook.com (2603:10b6:408:e3:cafe::a2) by BN1PR14CA0019.outlook.office365.com (2603:10b6:408:e3::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.20 via Frontend Transport; Thu, 15 Aug 2024 06:24:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044AA.mail.protection.outlook.com (10.167.243.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:24:51 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:34 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:34 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:31 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Tariq Toukan Subject: [PATCH net-next 06/10] net/mlx5: hw counters: Remove mlx5_fc_create_ex Date: Thu, 15 Aug 2024 08:46:52 +0300 Message-ID: <20240815054656.2210494-7-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044AA:EE_|LV8PR12MB9269:EE_ X-MS-Office365-Filtering-Correlation-Id: 2c244203-0b5f-4b33-477b-08dcbcf2f8ba X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|376014|82310400026; X-Microsoft-Antispam-Message-Info: Q/pD5+SkW3nrAdmI0NZj7UuhCoi26o4e+nEHf70rqYRdH1L52W8y9jw6+fsd7t3hnfOVtQtPuGZQ/V4hiI1mqf1DrsjdUTEkolBADJYpLnhLxxCKaaiV+t4j31digbtzUfLL+EGaR31UIJv+guLlwMWjluf+9jjeJzsO2b6DswKiVuUjZ7s1zcqMiLoeOThL9siiXNaxG1DoIwaPuPo7569qfvDp72KgdtukAfSFnDVbIciegI9xv7bGeGFHG9HKZ+ddtcQnzOCVgRKk0EfV7TpWr0k2YVOh3tvvS7oeZYMlIS4/iwMMYgsbz4yARVEddAkjRckJKJ4RH5GhCp5JSH1QHxD6LXHNbShe6PVwb2SRbfEwEcNMZSiZZv0jtYBGJnDHsaZ2olXp3lnKzvcq3KqvokVGwD+vZ7npoZ9uhacXsxokT3wbeYo4x1btFl8AFgihsuvzP3i1MdDJMDOy62SQvlCQ1XMWvUfNLxfnHAI6lhHtKRgpFff/cGoGeq506uJ2YSH/rZDXt/yyaTO0BfP1mNBJnA3qA3wl02BttVFuX5n9TxtsXLM42DFNrTnKW+1pg1eNlS0fp7OIPoef0tqkqWSX/OKjOR2PmIzMuJUby10zZajAOCJXOiyMdRyGdyiHWUUnYIWHWJ50vXbP3Zm6IMcBSZe+F/KeGGvLfhM63VmZPoE10jnbFc/sNuNgtgIqKwBUYAbtIpVKeOc4WiAxvYmaFyzCthJU06aYOE7BiEpP07GkOqPAsYD2hk6ErtlfuqWJ9blV9jXnMHw/zYCsIo0/VkeZBoVqEI/J272ibizzRjikWzyV8epdT/1vTQo8vdVjsaZxauhmYKNJtZEFUdgp/GDbTrK5EHOMbxip+cw8awh+DckW3eRZHIX7hQ1rCMI93YSn4NFcbQi7yllDbcCJHky4tm6FJMejFfQQHemovkaTWCqlf72h+6MEvMF2T9e2q3ynvkIIwb1ieq2mco9KpXj3beQhTrRWGu94eBXH9lLeaaGlP88RWaZ97JEZgA4J7C0uNfJIbbJtqMAQz8uZKfeCPOh2Bvhz2ECmakOhjZNxvV/O9pITRbGsAXud2dga5CiNiDopTbnKnIIX3g3Cqetk8xhmbem8xwklgj0YZq3o2K6tsRb11TU9uL1D3fAhnvO9+ynr8ut78Kv79RNRzEvW/dSvPd87xkxDontcyemLszu4GGssHCWO7MQHMsvTUr1oqk3AVs76G0M+SoFrupvuEyXMyMyYYtw0krdvqLjnId9rb9SthT+mQkMAx+jeMLexkrEdDoVEyJg51/o1c9r7bwpbDyZr6ywaRBAa4iCnZntmW4oauCoIDvc3PNlMdZrsufIiszuLXtUDpSDsphH2PbU9RC8n4XR97wLirM6ucv9rf+9BkYl/bnWwQIa4blzQrs6142n6BkUP8+Fn/EKo5FiqWNZlwloAOi90TY9CcPN7bTuMH5pq X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(376014)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:51.8489 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2c244203-0b5f-4b33-477b-08dcbcf2f8ba X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044AA.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9269 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu It no longer serves any purpose and is identical to mlx5_fc_create upon which it was originally based of. Signed-off-by: Cosmin Ratiu Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c | 2 +- drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c | 7 +------ include/linux/mlx5/fs.h | 3 --- 3 files changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c index dcfccaaa8d91..4877a9d86807 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c @@ -1026,7 +1026,7 @@ mlx5_tc_ct_counter_create(struct mlx5_tc_ct_priv *ct_priv) return ERR_PTR(-ENOMEM); counter->is_shared = false; - counter->counter = mlx5_fc_create_ex(ct_priv->dev, true); + counter->counter = mlx5_fc_create(ct_priv->dev, true); if (IS_ERR(counter->counter)) { ct_dbg("Failed to create counter for ct entry"); ret = PTR_ERR(counter->counter); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c index 0b80c33cba5f..62d0c689796b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c @@ -275,7 +275,7 @@ static struct mlx5_fc *mlx5_fc_acquire(struct mlx5_core_dev *dev, bool aging) return mlx5_fc_single_alloc(dev); } -struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) +struct mlx5_fc *mlx5_fc_create(struct mlx5_core_dev *dev, bool aging) { struct mlx5_fc *counter = mlx5_fc_acquire(dev, aging); struct mlx5_fc_stats *fc_stats = dev->priv.fc_stats; @@ -304,11 +304,6 @@ struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging) mlx5_fc_release(dev, counter); return ERR_PTR(err); } - -struct mlx5_fc *mlx5_fc_create(struct mlx5_core_dev *dev, bool aging) -{ - return mlx5_fc_create_ex(dev, aging); -} EXPORT_SYMBOL(mlx5_fc_create); u32 mlx5_fc_id(struct mlx5_fc *counter) diff --git a/include/linux/mlx5/fs.h b/include/linux/mlx5/fs.h index 3fb428ce7d1c..e73a852f51f4 100644 --- a/include/linux/mlx5/fs.h +++ b/include/linux/mlx5/fs.h @@ -298,9 +298,6 @@ int mlx5_modify_rule_destination(struct mlx5_flow_handle *handler, struct mlx5_fc *mlx5_fc_create(struct mlx5_core_dev *dev, bool aging); -/* As mlx5_fc_create() but doesn't queue stats refresh thread. */ -struct mlx5_fc *mlx5_fc_create_ex(struct mlx5_core_dev *dev, bool aging); - void mlx5_fc_destroy(struct mlx5_core_dev *dev, struct mlx5_fc *counter); u64 mlx5_fc_query_lastuse(struct mlx5_fc *counter); void mlx5_fc_query_cached(struct mlx5_fc *counter, From patchwork Thu Aug 15 05:46:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764431 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2071.outbound.protection.outlook.com [40.107.92.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ED9D53365 for ; Thu, 15 Aug 2024 06:24:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.71 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703101; cv=fail; b=h58RtbTUH6NcFMMIZTwRkjdm1czroGvOPY9/X1r6H4WUjRmcyb8sYgrYs95SZElWSXtteOM5yLaDSrePNHO35Vwfo3tJnXwq3UUu27m8xzgtaEUoRuBW1gGVikl0RKpEeGb6hinWQLsuo17Ctd/TGknTD/Ew0zYIfcr4UeEMGoY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703101; c=relaxed/simple; bh=ScPS69T5mE3qA7ccoxxSRdBEYrALUEz6pFCzcPcIw5Q=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=d7xM18DVOUPDdcQk8U7c3Enm5A9UlDtwpFU17bh8YJFa6kl9NkNAtPpZIxqgrT7y2F9E7wBFxg7LQBJBAW9nyL8LLBuZJitstwrNrCyZXV53Zp6fJRrGSYIktjphi2Vgjr8w4oaVN/++AIR6miG7Gx1EXAnN0sDUZJXIiNxb8wM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=tyH2/gK2; arc=fail smtp.client-ip=40.107.92.71 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="tyH2/gK2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hkbRABk7/MDNauiNf8DpA7Xy6cjZhQBqQVR7giJFabtFhJ9xo8OmiZyhylFwUTwSyIpXmQciEf80AM/V+18+WTZmNEvqPZmDbkok3UFOZkBWJKeIdlp1q6pgVAFR+JmPCy1RkPxtohe4FFDGKoBiOsNcbz3Vd/SSQIC89ZMXQICnGx3kX/dCYZGvo8bxbWicYAlumI+RlkugeXsXpzi+8ba0NMvy14GGfp7308ZCHjH4DAgOy0hCqMx9enroByF18lwHY/UGshkkDOSDHLx2Pn70Ng3ep5xcza+j3mTNKxJt0ps8s505pWccT6R5P5gQ7w3gT83FzujN3TIbVvtlKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EOeBQDgwPBaVGM19+Amf34frifWgoE6ExOx5JP36VIs=; b=KMopEqpmakewJR4trwqeM6ERYnfPdK4kaJzRSQoYDhhSWJOX5+qcOrFNsI375sXsqS2Hxjaca2Sg3GhpOYoIlicr6z7oUjpqfKJezPoxqNbbWaXGlKp8tcmJ8G3hDOqxexaHtwwzFAIOHssTdlfFECzpt2ubI9/7FPVWBU3rsQOS8sMD6suz+WDrHgzbq45w5a+B/Kjcp8niHqU1Sv50pOoDUkXFIWsSIhe6BbCL9pHv7C3BJZ2rMfQylUyFF1izS5/dDgw1FG8QCZqPLgGMpHy/wanZHAN6YyES0qW5dTvpE6ga/fOBYKamklrkaWH3BLYB3uc13ha8NvpRA1EYWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EOeBQDgwPBaVGM19+Amf34frifWgoE6ExOx5JP36VIs=; b=tyH2/gK2+xljs5fCCwYR3tdDQDhGfsRdnaIZVvM9JHnNYyyBGyfTx3yeZ4rCnIHSBwO9F/cWOSI5Y7GCb99mim0igGkyHV2moF0Y0YGSbmmsB7bKSTAMW7Gwd7Y1RIgZW0qv31vxbrwYMX5JB9Z3zFLEALfbhs6zMzyQPqXxGqMb7rfAcOXXlQkx8JLIkGadU3ZAKlwaUncgC2YCJABuOCJH/cq9OiwmMvtcocmZnFzuUl98IvdZdEw3zQmaEN9J8G6b4BOOOHthr0bNe6b33/aSrXvvdpHtVY9vq9JA5pDpgxt4BuVN0QJ+7G5Y/gLmOFogWoYS3F8wZ5Z4mwRR6w== Received: from BL0PR05CA0021.namprd05.prod.outlook.com (2603:10b6:208:91::31) by DM4PR12MB6040.namprd12.prod.outlook.com (2603:10b6:8:af::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.19; Thu, 15 Aug 2024 06:24:53 +0000 Received: from BN2PEPF000055DF.namprd21.prod.outlook.com (2603:10b6:208:91:cafe::ca) by BL0PR05CA0021.outlook.office365.com (2603:10b6:208:91::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.15 via Frontend Transport; Thu, 15 Aug 2024 06:24:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000055DF.mail.protection.outlook.com (10.167.245.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.4 via Frontend Transport; Thu, 15 Aug 2024 06:24:52 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:38 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:37 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:34 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Shay Drory , Tariq Toukan Subject: [PATCH net-next 07/10] net/mlx5: Allow users to configure affinity for SFs Date: Thu, 15 Aug 2024 08:46:53 +0300 Message-ID: <20240815054656.2210494-8-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000055DF:EE_|DM4PR12MB6040:EE_ X-MS-Office365-Filtering-Correlation-Id: d71500a2-813b-47b3-b672-08dcbcf2f90e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: hUBNvyh3WVsUwAmVhEYn9KhJy2OlgtRbZq/gfUqZoV9DCSC55fot+Q6VoFlbv2dETWU2omvXaksdwR0ux8mcjw8e3w/DrijIDen9JUrm7dleJgX3uuzhHStf4IYDxbwR+bJag0bbpF4kQwfpN6YzvBwPU31L5QvfG9jb+JORyBDJ1lTlyKRZbGBxaJFVGlMnQLgVLeRagaACMWPSNgn+7fY5F4ZyhfnHhI9byobVn45T9nnQgMasb+QdlrQC3945Uda09VDCSnp1LMCbbDXYFoCcHnJEd7Ow+j8l7gGdWHhJy83F+UdA2DqFa+E3kpJHvQk+oEsVSc6sSrELIoMVbcvGxp3SqVFxwNsNDuHVKDT/sj2m16JZkfgOojHHEvruQzPvQawMw5D6/rkFrJ5PSPplER2/lkM/rIVxiojSduhpbFEH3GQ0nFZeV8sJ4TYSHlvob1eB2inekEv/02FvRKYArbQ1VetzH9hDN009k3NxwXTd8Ob86bFgqwyveIR9pUVhatu2Ec4mN6H1PGGfKEU/WYQTi2WRnXBIUBfmQeZACf2MOraHqpVR7ng6R0EUCHbG3TYv2MQhF3wMVhBAH7UsLDV4NtEjpvIVhJ/Jd1ydVUkhRAwZJoguoXeqOsN8Qscp8vts7T5/dy1f3E+/SefBjHjMTx4P8O6umidlsVyYR9fyOdoWzJRyT2tRGr+zLbZp+mbh1ZdLEqKHMVHyuxHF4tiS+PoY1qG2f17ZhcRi6VQVvmJ5Jcj0rERvyOkQhrhXi2iTs9/fHQm0Hlk5tpo38T2A5ZnOcDcS1tBvMVPIHuL6QvhSCtALc8hdQV7PUSV6BMMh/2nwoA2AR/n4PdYgcT16bp/p+xdKNqK05qZF/XuZSs2EMaiXE/oi8L7/eaGe4U3nKuopXeVn2dp75PejR4nFIwPKIEE5TkJfpfLMyezfiIvcfaw6CnA290OESwGPLQO6WBGREJ1wDeyVpjWChrNRAWcDgglqcP/HhZ1qDSSzQkEg5bhGZU7Y5oGxiFxfa2BuPY7/td+d15OdoMO/1fC+s3jLQKzF+mthc0Sua0sOkeIed/23eixwyXCEJPA2BhkNqay2L5LlyYfNbgmTyDFvw0aosdbdD00jO9SJz74k1imj91nCDeLObHSBy9qXCBmt7SP6eyX5L6g8DWpmshX4gIsqvO20bI0FbGi68JqIXAFLSLKLsH3pzkCyTPbM+GJXWST8LkV4yalBMAJo4Nq4/RWNLA5I5rdZZhJpNB/licGfo0uIZAG4OTyzK0sJXJjjmDDTuvD1387bIPVqO7K4IiQ/W8zHBoFCeS+WBCPCnBaIG66E9+tcB6orpU0c7aV4+q74O2iErdiEn+ufbtJTaXPachlwB/adx/wEyzI4dru8g6GL7TDC3c1be/rRd8R9GG/RlQnwPIFwz4m4OkT4Z86d/ajSEmtY3T5mKCYoIK0BGt/7ifYeYpfZ X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(376014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:24:52.2928 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d71500a2-813b-47b3-b672-08dcbcf2f90e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000055DF.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6040 X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory SFs didn't allow to configure IRQ affinity for its vectors. Allow users to configure the affinity of the SFs irqs. Signed-off-by: Shay Drory Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/eq.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eq.c b/drivers/net/ethernet/mellanox/mlx5/core/eq.c index cb7e7e4104af..66bc5a32ea3d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eq.c @@ -915,7 +915,7 @@ static int comp_irq_request_sf(struct mlx5_core_dev *dev, u16 vecidx) if (!mlx5_irq_pool_is_sf_pool(pool)) return comp_irq_request_pci(dev, vecidx); - af_desc.is_managed = 1; + af_desc.is_managed = false; cpumask_copy(&af_desc.mask, cpu_online_mask); cpumask_andnot(&af_desc.mask, &af_desc.mask, &table->used_cpus); irq = mlx5_irq_affinity_request(dev, pool, &af_desc); From patchwork Thu Aug 15 05:46:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764432 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2069.outbound.protection.outlook.com [40.107.237.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A50D853365 for ; Thu, 15 Aug 2024 06:25:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703106; cv=fail; b=K1KvN5b5dLnjsSzW5I3TNtqiYLYP8dbhIICm5ojb42d7AqczDAUgKDfL4MKE4Rgnz6X/76iLGIsUvdF8j3wWXkCKmRxZ4/erIpsmrc/JgzOmCJfFFvezNw1CGWhTH3c3zSKUCRd11FOp6sP3XUjrihGT6reWt9ahnHhNVUS0Py8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703106; c=relaxed/simple; bh=3qLR1HOzA7KXGPzPHs9IllVyCNzs+3ovsPUATPBYLJg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fjK/2ggGiH3NnpAnnTOlRSKLFmT5PBgMteiJINvzdJYjDBEKUnyvquRFVIno+m/MDNnHXnSYfGilFT4sf8ymeqZkL0mpGukOmFvqpNwWWfkqlHu+ZXSyrc+15SlKMd9WwjQCwD7x5nHD1Y4b5e/y4Un7bm6BtlC/kyAlhQcGhrE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=pJK9uc/Q; arc=fail smtp.client-ip=40.107.237.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="pJK9uc/Q" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dHf2TCvkU8Wy0XfgfnCjE8tZyAjNPG2dZq6uJ5DwVh6wetXBE5CVgdKiMEDY9hUe6HcPDuPnc/mbsAeoVx8CKMk7v2+VvFFx9W2jCVdRl3sBcFI2ejb271cyljHATpdQ81v10/v63/7/yhz8TOYrS8546mGym3YEgO7g8WC63BX9u591vcUI/C/J4SIh9jc5j6lJuE3GUCD5U3T7lGuLfw1qA7bRKa/R1PZlWHahsDSNtU54SRKK43rs6ld8z+gPIGzR2gupKcsQytLrUqMrKCtcDnjT7WEWE0cbEiIwtUHJ9WauII1IE1fFEQMe/buZ2ck+k5e0/PHhbAWb8jwEoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LqdNv4xduBCzwabNZTwSsZsQNC6fZXLLBnCrxvKFGwQ=; b=oeS2yyVf8DB+qmLPd/WZ4HjW/eF9Xjo0yJWhmKPHSZWv67gaUVDsxO3xMmaTdw67aeckotdy0JDprprOBuuII/Qdkc3S1HSMlC3nKhD63duEsSmBYKITFlwyHeOJ3wnm9Qh2k94nQoQ5B9AFffp457mfS1okCl5uB2UKk09uA9BOjcutl9iIVh0CWbTnJvaCUoJvTyAL2PToKfHNeySOYOgJQnVt36eKY1efK8RskuV7nR6WIoN8zSdTwD5gYelTtoMYebKyWgf0400S6wwf5ZGGr/7AQu+gV2rmaJY8ft3zyDQvpbBjTFKe1Y3L2KGUyd2/L4uUZHn2qCCw6S5U8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LqdNv4xduBCzwabNZTwSsZsQNC6fZXLLBnCrxvKFGwQ=; b=pJK9uc/QZJs2fZJPTbC6jk4fCZMDKR0hyT0PVSC3oO8skjVErihKaVUvPUa6eE+rTgrMsFRRe6pKrqus8FS9GcMUJOP8NnpmWrge8DALx4OIN6/NyKvQPImeZ0MpL9uiu4yEyrGMzeQxKtEWoRdASv1VdFRs02EuQA+kf7AqgaZB74msTUYmJQJ8slZc8n12aF1os88xihNG2HEtyzdxJuDDQ1R6YGWr5NGvcUXccXMbIXn0BpQD2PcVbdNn3msMkg3Td7kVSZREhYXPGdWFlmG4+M2sZUMckwansaXpz1YdNvJ9gymGgGUXzR+DKmPDs8CQi1RwvD0XeK7jK4mUNQ== Received: from BN9PR03CA0774.namprd03.prod.outlook.com (2603:10b6:408:13a::29) by DM6PR12MB4281.namprd12.prod.outlook.com (2603:10b6:5:21e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.18; Thu, 15 Aug 2024 06:25:01 +0000 Received: from BN2PEPF000044AB.namprd04.prod.outlook.com (2603:10b6:408:13a:cafe::40) by BN9PR03CA0774.outlook.office365.com (2603:10b6:408:13a::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.33 via Frontend Transport; Thu, 15 Aug 2024 06:25:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044AB.mail.protection.outlook.com (10.167.243.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:25:00 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:41 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:40 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:38 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Shay Drory , Tariq Toukan Subject: [PATCH net-next 08/10] net/mlx5: Add NOT_READY command return status Date: Thu, 15 Aug 2024 08:46:54 +0300 Message-ID: <20240815054656.2210494-9-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044AB:EE_|DM6PR12MB4281:EE_ X-MS-Office365-Filtering-Correlation-Id: c345f131-d35d-46ca-5fc6-08dcbcf2fdf8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: 7JHMAg9jrtjmHIyLwF2xtj2JbVMv6b0kHVaA6xlt/5jIff9yYlmsY67E9k/7bNSHGVhF2s5OIxiXvz31FMGVea8kU0vwtda9XSbHyD79NjV+Yg9AIvB881QTAjDji/ks2jGZmUveWqTEij5rcsHSJn6rLsaR9cXGkMy5BgNVHg4J+3ZbdMUM+leHTi5cIGJQGtgD5uhWUhKGNo+DS3mcWI2OKbzZPSxYufVI/6d75mlybbQO64nVrp5+x1XPLQC9SLgshwnX6/YN6o+ZlegJsTicpI4cp+RQ+h0jNYbP4Z8p+cOoqZsUPB0POep1cClPGxK/WA+D43qyZiHSLmYIvJmTy0jDfdd4vBzRbabH5fdKY9RlRlsnceP1MmADdGyPjOYhKkg/onmnxoWZSLHTEMacVtMsXGg7XX35xx+KrQNYoV63RNmlTUVmB6IdLIQlP4Qn6LhZeBtSoNA/f/TGYRowNFJZY7i1sUHQvOmwEdsJJ8c1On+vJXsZ8o7wwZRsOc7skdCAsyvf5AeWFloRuS6s+7c9mmSueifdE72fsu0AnxAc43KgQxdD+MEPvT3ZFVoAqHBPpXFRBoanPFKG+ul+76vnEi3YUuxszRxbkGeMdEfUoxaKKx64jDI1St6Lh09XgX2OBiUnqQDH6SwyTOphysuSqc7FvfuLMcs5t4TaHsfGwXhblelM+tKgfMd1enZqBFGnHu+XMPptXuuD4p1pvOJvQPlbLJIRBfdqCederco0bK7IrAuvvigIhT5MLTqAYaFhu5Cb4snZ3mh9ja1DWKomSxmBjKi1feGcCmEnlkir898bJmQy0FzlQnjo4l0IBFk8xAi0YmgUrso1bHzJ8/LtJKO4FWW8GEMo1aEwS2ZBRx2Gv8nCfuB+aJ2UoaOavvb/u0td44QXff/QAOTJrlHqmpjbq3obqBFCLSiskZ8YN6tIUu9ehbCboatXBqJxoocSfaXomzhvWT6Vt9eMjTHuHlypW8Asm3w2gf0TvlBOxTfqJwQFQvizxkJHBt1/3WKz9gcsJF6oqE67krJ47ZM1UkCLfeH9dWea0xAB3YBXoj/ema1WARoS0nkRRMhsk35oxzoCq+2lrZ2kfyY0m9jEHBXQ0wkJGXe/ZT04D+JIoHPrYJ7Fhpn7FLrBVKcf4X3DwM8TNCdCJldc2WuEGU24ZGN7FuD0B+okCmWDkdsLcgolbWjbYxd0sajcadHxXNshJ1UzJSVVnGxoCC97dxRjT12FxP+eGJf3izGC42Gb6xuCbZKuswnZ8mtnFxMKo3wTdDdOTlDwXfmhPqGVhp6vqcMZYjLEMea211P8hRR092iGBzNG/HYvrK1Hpnb4GO1GoCWpJBS5l3UJheXzcQdGKvC394ifRkOeHgCMvYlRQxgFrH1XntC9449mBVJ3pHNJGjKr5UBopft5sfgYiCewnj2f1FbBscw/PFbRGWWCmMFFe7vL+PoD9Dwz X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:25:00.6444 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c345f131-d35d-46ca-5fc6-08dcbcf2fdf8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044AB.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4281 X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory Add a new command status MLX5_CMD_STAT_NOT_READY to handle cases where the firmware is not ready. Signed-off-by: Shay Drory Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/cmd.c | 7 ++++++- include/linux/mlx5/device.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c index 20768ef2e9d2..9af8ddb4a78f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c @@ -754,6 +754,8 @@ static const char *cmd_status_str(u8 status) return "bad resource"; case MLX5_CMD_STAT_RES_BUSY: return "resource busy"; + case MLX5_CMD_STAT_NOT_READY: + return "FW not ready"; case MLX5_CMD_STAT_LIM_ERR: return "limits exceeded"; case MLX5_CMD_STAT_BAD_RES_STATE_ERR: @@ -787,6 +789,7 @@ static int cmd_status_to_err(u8 status) case MLX5_CMD_STAT_BAD_SYS_STATE_ERR: return -EIO; case MLX5_CMD_STAT_BAD_RES_ERR: return -EINVAL; case MLX5_CMD_STAT_RES_BUSY: return -EBUSY; + case MLX5_CMD_STAT_NOT_READY: return -EAGAIN; case MLX5_CMD_STAT_LIM_ERR: return -ENOMEM; case MLX5_CMD_STAT_BAD_RES_STATE_ERR: return -EINVAL; case MLX5_CMD_STAT_IX_ERR: return -EINVAL; @@ -815,14 +818,16 @@ EXPORT_SYMBOL(mlx5_cmd_out_err); static void cmd_status_print(struct mlx5_core_dev *dev, void *in, void *out) { u16 opcode, op_mod; + u8 status; u16 uid; opcode = in_to_opcode(in); op_mod = MLX5_GET(mbox_in, in, op_mod); uid = MLX5_GET(mbox_in, in, uid); + status = MLX5_GET(mbox_out, out, status); if (!uid && opcode != MLX5_CMD_OP_DESTROY_MKEY && - opcode != MLX5_CMD_OP_CREATE_UCTX) + opcode != MLX5_CMD_OP_CREATE_UCTX && status != MLX5_CMD_STAT_NOT_READY) mlx5_cmd_out_err(dev, opcode, op_mod, out); } diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index a94bc9e3af96..d0f7d1f36c5e 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -1449,6 +1449,7 @@ enum { MLX5_CMD_STAT_BAD_SYS_STATE_ERR = 0x4, MLX5_CMD_STAT_BAD_RES_ERR = 0x5, MLX5_CMD_STAT_RES_BUSY = 0x6, + MLX5_CMD_STAT_NOT_READY = 0x7, MLX5_CMD_STAT_LIM_ERR = 0x8, MLX5_CMD_STAT_BAD_RES_STATE_ERR = 0x9, MLX5_CMD_STAT_IX_ERR = 0xa, From patchwork Thu Aug 15 05:46:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764433 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2053.outbound.protection.outlook.com [40.107.236.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E05F176FA2 for ; Thu, 15 Aug 2024 06:25:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703112; cv=fail; b=qRgYZjyeYgkVFV+L+sIRx2m6xv3fywFo0VJKqnppObDb8ZIk2OxCojdvvU91p2ujiKx7vETrrH4hMEuuKXe6e0LI4ynnhyHwazsbX+XX7zLF5JpsD4eIjpObj4YXs7QRqZv+DqjqluezRv0o2YOLQo42KRrdvMoV0GbwYYUO0Ho= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703112; c=relaxed/simple; bh=uskiWukM+M3TVU+rZ6kxlMLLssI+Wfm1Q5GeRyFkpp0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=O6ZeFR4R3EuTKySQ/EY9jHv8ZeJlD325ilfkNUxGZ+LRshpkDNLradfJDes1DVKg9Gg9T1dc0LKJCmeucaGln0NEY2euu3cieKnPM+fzkDH6xnuQa9mBnQ4PFgWPvyLUnipBAzUEUqrf1wVFAxseC5ClmERraApUXAGxHtNfMn4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ryQFGrMM; arc=fail smtp.client-ip=40.107.236.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ryQFGrMM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qI5WVRH3c0oJIzvhwpcif7pwhP+VLhsR+uLgSrZ4a7h7dgp5b6Ft3OLrJV+2uGMuhFXZaabTMWsbaqwXiAjzhTBW1gUksboxpN2fbpfBMHVt60aqUAb2n9UraOR5Jy6BTqfK/dRkX9vTX5vgI1rnujzRXz1ux+rgVr7auhO7goV1+zLLv8IXRfP7vdiZnjuU05L5yglah1TAmmAkqPabG49GzNyAJxPDrPLmw7EZnO/ADbHj7+jlK1JGBArxkQm/r3KrGmu1ubr3mX2PWEkZq1x4RWLCLEFm0vyYpXje98rg8AnMVDBWTmSdX3dWDUUfpToWsggbiYQQW3vpN9bo3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4yRNU2yc1MeixEpUrqMoY0hLv9tT1Q0wGITbp62jifs=; b=IH1qt20bdiLzBikx4KeKFAD3b6qi1ScqQcEVVZq5VmPJNy3x2jUnKcS1N1M8GF2xMP19j+Vp4jCHa0ZlYJjsceHiMj8RNQQT4Iml5CZhUI30+Bfz2LsS+Ty0yXwiElM7NnZrTtr6LUpISbbnHFy+QLC90aAJq9WCMLK4lgjFRxUhILoAHoM0Q6F3riySJ2Sd54vSA35JI5UBfbptiGuJOzPC0oiViCfUv1qwpdjWafatUJBtqiKx1oC4aifPJHFwv45QDsJ09nNpXB8+8zb3SwNBwq0UqafZF6IFw0OTmchGMdXSWG2p70/oP0cTBptyJPGqMU3O/QqO4RYj+/A9gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4yRNU2yc1MeixEpUrqMoY0hLv9tT1Q0wGITbp62jifs=; b=ryQFGrMMjkuviJxG99f07S7zVqwJX2s8SgJkdlsdQ+wx47SakibdOu9jgDBFXqmjujRySyNDbT2IGM7AApxisUwQSpFptemJbplJF/tUv7a/teWrKH/mLYpsS/1vOAXlrXtH7XFYn0ykZttX+ICGxtWKuMgMMkQ51GLHpoYRVxf+HhTCmPq9g34pY63sRSjzgpe33FOFhlyR0N2bhlpx18JXi3OnLz3z0Ar/Cxv2W1lxEJ1KuOHSloM1pdq5Ag1W8Gp7EzTH3XNzVuao96QPN1sAGt2BrByFXBFqJBz+E+QuuhG2vtpDZ5GMyWCFwDfqz18srUgNlB9uy8vXxdJTCA== Received: from BN0PR02CA0057.namprd02.prod.outlook.com (2603:10b6:408:e5::32) by DM4PR12MB6469.namprd12.prod.outlook.com (2603:10b6:8:b6::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23; Thu, 15 Aug 2024 06:25:03 +0000 Received: from BN2PEPF000055E0.namprd21.prod.outlook.com (2603:10b6:408:e5:cafe::b1) by BN0PR02CA0057.outlook.office365.com (2603:10b6:408:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23 via Frontend Transport; Thu, 15 Aug 2024 06:25:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN2PEPF000055E0.mail.protection.outlook.com (10.167.245.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.4 via Frontend Transport; Thu, 15 Aug 2024 06:25:02 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:44 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:44 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:41 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Dragos Tatulea , Tariq Toukan Subject: [PATCH net-next 09/10] net/mlx5e: SHAMPO, Add no-split ethtool counters for header/data split Date: Thu, 15 Aug 2024 08:46:55 +0300 Message-ID: <20240815054656.2210494-10-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000055E0:EE_|DM4PR12MB6469:EE_ X-MS-Office365-Filtering-Correlation-Id: 4435c1bd-0ca2-47d4-fd4e-08dcbcf2fed0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|376014|1800799024; X-Microsoft-Antispam-Message-Info: pTiidSJiHj5KTM1D1aravYpxXmSt8a2/t1RdDRoXplNc46JRyP4HZVTSi9r6Od/iaJwhgKsnsEGvf2TfwwekCM+UmB0DQA0bMeKzB/rZTCYztjGvOUDy/n1paHlAQ2zIehL89DNs1koeoTz5SLh7mI0vAgVF2jy6DTrBzM3IciPTY30fDNgAb6d2/zUNZQUabD/IkIaMFKJN5/4aizD4IhJvshm1XfWqWtWJG4MZGdapJkxG4u4uchqyCc3kVO9beeJcX0v5iflV1/A/A7fdlTArVj0bNzH+Iw8Suenypnz4SRvHjV04+GQUrgx5zPaTC+I5YcUV4YRfNPzh+XZD5xWUxhbM8Uh4VL4OMOB2kS+fdwvCwK9dVaRHi/2ajUOugKV+i3SDF9T2I0Op8rGwkW/+CQR/0sxHkdr2sOwsyGdRXmruNYDXig1oKAIoi8DN17b67usdx9r1VLmNvJ56GMHtW+XgcOs4MZf85iPAUPNY/lMoOcjHlmZQOm6oO6cOQIgnI/F4FUGFyB+zKLamDE7ksRfG3por5OtpweUwUUnDHGLXYFeSvPMP2bdgoERVLOypDvqdRDQBC+nfsu9GjU5gCUAO+XjPwRYFNxTJY7jW8tOOqKJPbmzmJCuf59+G6m4rBKkroO456vmn2IG8sPFJ4NsKvj8yqNkmbDjkVwkHTwLiHBfehCmn1HEnRx+QXENCz3RuRoMeUM9/5Ee+Lgb/g9dqEyLFRvvW0AB9ADqH3hp8Q50tLoR3soFE4yyEA644EgM6tiS64GHwI+PCTn/vPxGne8P1vv8fA33pwD9Fwh3qrnSClg6XEPoCZxgTvVSaxCVHPiqO/MIds+x+T08wm3Bhm5aHkSrVktc+QGdWA+feD3OoWXb+lM2swf9/nm885lu2LXJLtwBXTgMVVcqN7AWmMzIk6cQ2UQ8XsHEuNZPs0tG/HfJH+TjjEZars86nukmPl3jgSuiOt/GkWLG3SJvDYy/uIe8Us6k5cyDWSqxukC6ZivB6duoGurNQ3SLiiIvilR+VYvbiaHLKq3uxdzJIOXHbc12eM6uSlTHUZpJa8YVYFvgaU9Msv7qYqsfB+2V8yGHyvgrReCBTzg/1kmOzx0ODirsrCPTPsu5nFtoezdpR/7MrR9U2PB81TjkDdaD4jAf+D3/NmoUQUTzM/lTQl8Zqjay4Q/HId/pi8Rm36TpK8xzIuG81BhivXqYbxRsgS/0TYg4LERTWtrAwdkkRhrquh52R7H65CmK2p8V+rGJ9REbeseH22kztkjH+Cnbz3zSjkPHYbfmSG88nEzt5/XYI6i/RJDpNLeqNTz0HniMBj9tkYfD7QJktRetmJLLc+FPl+1Rs7vTMsFGIKgLxSQF4VbmybRO8nCYrR1ct+DzFG8KSQxT4v6bDD+/C7P/cetha//m1yW3knocvbgkjb4evQA4f/OIXQNbABnXRt9D3MTXEg78uHwDW X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:25:02.0640 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4435c1bd-0ca2-47d4-fd4e-08dcbcf2fed0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000055E0.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6469 X-Patchwork-Delegate: kuba@kernel.org From: Dragos Tatulea When SHAMPO can't identify the protocol/header of a packet, it will yield a packet that is not split - all the packet is in the data part. Count this value in packets and bytes. Signed-off-by: Dragos Tatulea Signed-off-by: Tariq Toukan --- .../ethernet/mellanox/mlx5/counters.rst | 16 ++++++++++++++++ drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 3 +++ .../net/ethernet/mellanox/mlx5/core/en_stats.c | 6 ++++++ .../net/ethernet/mellanox/mlx5/core/en_stats.h | 4 ++++ 4 files changed, 29 insertions(+) diff --git a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst index 3bd72577af9a..99d95be4d159 100644 --- a/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst +++ b/Documentation/networking/device_drivers/ethernet/mellanox/mlx5/counters.rst @@ -218,6 +218,22 @@ the software port. [#accel]_. - Informative + * - `rx[i]_hds_nosplit_packets` + - Number of packets that were not split in header/data split mode. A + packet will not get split when the hardware does not support its + protocol splitting. An example such a protocol is ICMPv4/v6. Currently + TCP and UDP with IPv4/IPv6 are supported for header/data split + [#accel]_. + - Informative + + * - `rx[i]_hds_nosplit_bytes` + - Number of bytes for packets that were not split in header/data split + mode. A packet will not get split when the hardware does not support its + protocol splitting. An example such a protocol is ICMPv4/v6. Currently + TCP and UDP with IPv4/IPv6 are supported for header/data split + [#accel]_. + - Informative + * - `rx[i]_lro_packets` - The number of LRO packets received on ring i [#accel]_. - Acceleration diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 225da8d691fc..1db26a2f237b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -2340,6 +2340,9 @@ static void mlx5e_handle_rx_cqe_mpwrq_shampo(struct mlx5e_rq *rq, struct mlx5_cq stats->hds_nodata_packets++; stats->hds_nodata_bytes += head_size; } + } else { + stats->hds_nosplit_packets++; + stats->hds_nosplit_bytes += data_bcnt; } mlx5e_shampo_complete_rx_cqe(rq, cqe, cqe_bcnt, *skb); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c index e7a3290a708a..611ec4b6f370 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.c @@ -144,6 +144,8 @@ static const struct counter_desc sw_stats_desc[] = { { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_gro_large_hds) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_hds_nodata_packets) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_hds_nodata_bytes) }, + { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_hds_nosplit_packets) }, + { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_hds_nosplit_bytes) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_ecn_mark) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_removed_vlan_packets) }, { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_csum_unnecessary) }, @@ -347,6 +349,8 @@ static void mlx5e_stats_grp_sw_update_stats_rq_stats(struct mlx5e_sw_stats *s, s->rx_gro_large_hds += rq_stats->gro_large_hds; s->rx_hds_nodata_packets += rq_stats->hds_nodata_packets; s->rx_hds_nodata_bytes += rq_stats->hds_nodata_bytes; + s->rx_hds_nosplit_packets += rq_stats->hds_nosplit_packets; + s->rx_hds_nosplit_bytes += rq_stats->hds_nosplit_bytes; s->rx_ecn_mark += rq_stats->ecn_mark; s->rx_removed_vlan_packets += rq_stats->removed_vlan_packets; s->rx_csum_none += rq_stats->csum_none; @@ -2062,6 +2066,8 @@ static const struct counter_desc rq_stats_desc[] = { { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, gro_large_hds) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nodata_packets) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nodata_bytes) }, + { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nosplit_packets) }, + { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, hds_nosplit_bytes) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, ecn_mark) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, removed_vlan_packets) }, { MLX5E_DECLARE_RX_STAT(struct mlx5e_rq_stats, wqe_err) }, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h index 4c5858c1dd82..5961c569cfe0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h @@ -156,6 +156,8 @@ struct mlx5e_sw_stats { u64 rx_gro_large_hds; u64 rx_hds_nodata_packets; u64 rx_hds_nodata_bytes; + u64 rx_hds_nosplit_packets; + u64 rx_hds_nosplit_bytes; u64 rx_mcast_packets; u64 rx_ecn_mark; u64 rx_removed_vlan_packets; @@ -356,6 +358,8 @@ struct mlx5e_rq_stats { u64 gro_large_hds; u64 hds_nodata_packets; u64 hds_nodata_bytes; + u64 hds_nosplit_packets; + u64 hds_nosplit_bytes; u64 mcast_packets; u64 ecn_mark; u64 removed_vlan_packets; From patchwork Thu Aug 15 05:46:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tariq Toukan X-Patchwork-Id: 13764434 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2069.outbound.protection.outlook.com [40.107.96.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0432017839E for ; Thu, 15 Aug 2024 06:25:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703115; cv=fail; b=lcTmC9IKJlJeJMCNI4lI8h08n0wFXwYwMa/pPBfNPrXsAX8XknKPS+FuPvbr0qoEWVrTHrLDXyzG/ZYQP4Bx4HMDWeEBVD4nCWsYbYlDMg0IAp6T90s++SvYhgeaOxGq4V0ZprivxkGytAm8xqzZsdHTbxPzeU4S4Y7haO1hz/4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723703115; c=relaxed/simple; bh=lDSA8zofTDjkLDuvb4aaybMz3vkifkcOjUdKsX7dg2c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VA3IMmdKV/6e41GcjgwedcYYIZXkTOWZhcRWEpAY0XRE5r4zGw6OmspchZ0MJFdEJ5+e8YKueTukyLz5pbuuN71a2XqkCau3U8gj4iwfwvWo0PXNM0i/2osOQFz2tHRhFQQ5F08whC04R+MWd9WqCT1TGPyIfFRfbOMQ/4EVqsA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=JOrO7esS; arc=fail smtp.client-ip=40.107.96.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="JOrO7esS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hwV/FVUdbvHdJCRzSMyvixLqvFILIqANeCso9pCQDZfw/G6eLJ0D3igII+doy5rMmIamF070b/ZgpVBcFrBbNXbs8iXL1TYtIxM8BR7XjNhZv4Pw23FInGPi7SsCxc5GQ4bwxSWYksqRnPmHEKk+HtKZaMkfVpg6QyLUqDVm6q39RAABtlAxrKgusRsv1VQsdJIq2aywh3X7VuQ+yXGITqo4ZvyYKBluXuAHpVxN/U9DxiN2Mwe9AjMHBS5JMySyTKeyk2oW4cXImivZdYWQ2dE5YT3pVNNGEel7MZY6iwCBnlgNGaU5Ho0wYGTXQmizzBM9n0AUMRkrJjKRZIISog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NwIfJ8S7YR3M0nLfI0R4TNtWeYB2JD7dzT9mfDP1m+g=; b=paRk0hhPipKxfpuA7+2JJc++CbmrawOUcmxMGyVHQCfJ83MUJ/3fBEXJLotpEObPgAbdXIclwRqHTQq0ohLu09G/R7fHJs8CAFVbGdISY8nxzFmwL3vupCT9HzRaIC2MuiL3ZnnNJEJQz/64dN7/WNtY74z3O9MjNdUktVjCiGXmnLa7B7x1AD84FE2LeCMi+GAHEUUzl+J3MC1P+UGfYaTo9OoN/I6qTRpTJPAOVM4BTQXoOE3kqlH4F5Doh8x0zBZVSBl25+oyd1BqpXml92yOWIoThWyub6u/AgTSxb/K0VGFVILk+1yjRaMBjB4vem424T5XNbKMndIw7IBJXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=davemloft.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NwIfJ8S7YR3M0nLfI0R4TNtWeYB2JD7dzT9mfDP1m+g=; b=JOrO7esSI1WwARihfmfyrhFDlujGf9udxgNGnVekOu4n3ZJgBhfhdvLOgqasQYdxy8Sin7DVrgJJEYntqLiPu2iEWHvXrtMDvTGWveG8+a/Wk0y2U+dvqSGvXUt02VuMLLKzg0n6fdDcJkychzcHg9bPpqa50MiMbJryHt2pRgPRvvhe+nXrUFlJEbjm6sMJrolOUmQunAGOoFQKtYCBdFKViJ95W4Va42TaHKXDZHr4xBCOa2aZXY2WuWxIRwriA4iAZxmTEefmxBnm7/p9+oJgldwcwmjTsn6p1emAid9Hx5VZIWfC30DY0HisajE3li08DD4/uhhENfsFLVSgkw== Received: from BN9PR03CA0761.namprd03.prod.outlook.com (2603:10b6:408:13a::16) by DM6PR12MB4417.namprd12.prod.outlook.com (2603:10b6:5:2a4::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.18; Thu, 15 Aug 2024 06:25:08 +0000 Received: from BN2PEPF000044AB.namprd04.prod.outlook.com (2603:10b6:408:13a:cafe::a3) by BN9PR03CA0761.outlook.office365.com (2603:10b6:408:13a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.33 via Frontend Transport; Thu, 15 Aug 2024 06:25:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN2PEPF000044AB.mail.protection.outlook.com (10.167.243.106) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 06:25:07 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:48 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 23:24:47 -0700 Received: from vdi.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 23:24:44 -0700 From: Tariq Toukan To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet CC: , Saeed Mahameed , Gal Pressman , Leon Romanovsky , Rahul Rameshbabu , Tariq Toukan Subject: [PATCH net-next 10/10] net/mlx5e: Match cleanup order in mlx5e_free_rq in reverse of mlx5e_alloc_rq Date: Thu, 15 Aug 2024 08:46:56 +0300 Message-ID: <20240815054656.2210494-11-tariqt@nvidia.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240815054656.2210494-1-tariqt@nvidia.com> References: <20240815054656.2210494-1-tariqt@nvidia.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN2PEPF000044AB:EE_|DM6PR12MB4417:EE_ X-MS-Office365-Filtering-Correlation-Id: 03302f0c-593b-42e4-b535-08dcbcf3023e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: ST9UaXYeZsK/F32EvLaRTICPAvDkk1yZwwm+M++kzCHwsEhbZczg6bULIvqUM8EuJYR9UzJuznpUA4RW0NDiYD9QyTt6xotYT2NBzwkZnHjwvYK8QK6ntZfgv7E1tBH7lHsMZNXAaGcMARUt2BYSiQyX+T4uCBnemUDNzR1eFKlG8gRCCOxCwvEo2zIBfRVLAgn2qU52u07jBWjTYziWlGH2RJ6TkDZQ+FR/WWPtbUt/9UPcr6P9gG91zKrR717D3rofYkYtfI4DLGfKPZRyGSoz113TkkR7oMJntu19vgyenH9lTe/TUFWrsnR74fS3ok6mQdUg9hbgVU+7pDQ8dHGE9cnmGM/TYd3BBTrp+48mLU84wmqZbiNo+NgYRQaICnIuDr8V+ltD1ywv536LswM7a4xpaLLKDWsXq+EuM7iYk3wHIHX/lz3hhvSc/93zfVSi5g4z0YAPDCOVavIqM+ehlmLeY5Bfit477wMgrHBvE/W/ro88HalnaTvSjJXBf2sXu4ctaNASmy6pMs9SimkrDpLUNbGRsYTq5FJktdTqwcz0Lm1+95PR3v1mNArdROccIcV5+iupW8DfraJtl7FJhqLoZZVYrXrz9NNAGFZv1POupIF4pYj2VwuMnv/Dkrjv8zIRIiUqWNLf3RMMAwCxNcCuYy+CkIgj8l+EdC4ax8Z5Qpl7XEqyOapHcqxouTzetJJdPw214WDooBeQQpw4UyPJ8e0qJsdGwAfbyMYRaefAYdPqggG0b9RXuTD4iJp1eZTawyj3KVEdKXy/+qQ0HknYjbJ6LIdf0NMaZ0dCwASJ3Ej+YHeBvWlhrF70snYAL+7gd9+1U6rQA3kGFByW+nKM1SGdl773fBzCOlHJ1Jb5DuHdoY797b8HKyvUF1l4iC6kHasujAcrNIejNX5z756UGC/et3GyQyT1l32giaG/6gXynqZod43wUM3PbHWIcFCqvovUsdfr7XObUehmeWI3oujkPrrDe2nZtht6pFXFc62hDJO5KXFCLSeuVfIWGE3qQMsZmMq1kvcjADk602KhQdOSPApb8l+2vrcFBH7o5I4egfBlkGtzVkwDPuAFRaHcYzmnt8CmEXZplOvznvOPJboKRFtNTQfPUPutAhsNuOvfewZ4J+HyLa+xIdEiP5jNj/z3AlxKfAsKtvF6snUlZpLpGQIoWXCSqtYBj2Jyuy40R0L23dDBKfCSBwrCVgc7SSAVFy7EOuTjkuwvuQdBbrIYT63RFylO70PmnYZ7y2H8lbFHp7ENfb2a6AqaCtl/410YdDQPTJNs26onBz2Bi/UjYWFfMnjh+uJun15VQRlYKnspS5WJSd5XC5X3vlzXQk2F5BaGvqZmWkbWDr3DbNhEPcvMeWCl3qOSW6M6Dzrpd52E6zNibyQS86CBiglmJC548l/lGKVAuWju8FrD2q0QpYiyGeIJC72dkvhmIHU89Q/VAcoVvQLa X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 06:25:07.7694 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 03302f0c-593b-42e4-b535-08dcbcf3023e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN2PEPF000044AB.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4417 X-Patchwork-Delegate: kuba@kernel.org From: Rahul Rameshbabu mlx5e_free_rq previously cleaned resources in an order that was not the reverse of the resource allocation order in mlx5e_alloc_rq. Signed-off-by: Rahul Rameshbabu Signed-off-by: Tariq Toukan --- .../net/ethernet/mellanox/mlx5/core/en_main.c | 25 ++++++++++--------- 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 6f686fabed44..621c7451d029 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1016,30 +1016,31 @@ static int mlx5e_alloc_rq(struct mlx5e_params *params, static void mlx5e_free_rq(struct mlx5e_rq *rq) { - struct bpf_prog *old_prog; - - if (xdp_rxq_info_is_reg(&rq->xdp_rxq)) { - old_prog = rcu_dereference_protected(rq->xdp_prog, - lockdep_is_held(&rq->priv->state_lock)); - if (old_prog) - bpf_prog_put(old_prog); - } + kvfree(rq->dim); + page_pool_destroy(rq->page_pool); switch (rq->wq_type) { case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ: + mlx5e_rq_free_shampo(rq); kvfree(rq->mpwqe.info); mlx5_core_destroy_mkey(rq->mdev, be32_to_cpu(rq->mpwqe.umr_mkey_be)); mlx5e_free_mpwqe_rq_drop_page(rq); - mlx5e_rq_free_shampo(rq); break; default: /* MLX5_WQ_TYPE_CYCLIC */ mlx5e_free_wqe_alloc_info(rq); } - kvfree(rq->dim); - xdp_rxq_info_unreg(&rq->xdp_rxq); - page_pool_destroy(rq->page_pool); mlx5_wq_destroy(&rq->wq_ctrl); + + if (xdp_rxq_info_is_reg(&rq->xdp_rxq)) { + struct bpf_prog *old_prog; + + old_prog = rcu_dereference_protected(rq->xdp_prog, + lockdep_is_held(&rq->priv->state_lock)); + if (old_prog) + bpf_prog_put(old_prog); + } + xdp_rxq_info_unreg(&rq->xdp_rxq); } int mlx5e_create_rq(struct mlx5e_rq *rq, struct mlx5e_rq_param *param, u16 q_counter)