From patchwork Fri Oct 8 12:24:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545193 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF727C433EF for ; Fri, 8 Oct 2021 12:26:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CA7B260F48 for ; Fri, 8 Oct 2021 12:26:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241456AbhJHM1z (ORCPT ); Fri, 8 Oct 2021 08:27:55 -0400 Received: from mail-bn8nam11on2042.outbound.protection.outlook.com ([40.107.236.42]:13408 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241478AbhJHM1x (ORCPT ); Fri, 8 Oct 2021 08:27:53 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QQ8pDa6I1pDu7cCybpE6M8Ry7h+oYq7jQ8SWglshacgwFpDcUhRtRckREWZNAiewX1Wv5nbMl70YDjDkEwJ0N+X0+zKmXDsKuWewvLRwBUPmFp+eJUAbtMLOR24EN1CTurATWyu50WoGzmn12I2eqDixh4qCRMKLS6MccK1Jb0eaWvcqt75NWabyiNnx6MmDuGvyrZJizRoDH3gz2o6g2foacWxHFKcQQvRdF3j0bqP9cEhtVgAaYqV/t9OvtbIfhWIoEyE2TJs1/9MTPma91n3RBQZgTiMYanHtwkvBTeVu69U0DO2XLWmqBiBKR34QSC4NI1fYuEptdw5H4SLIew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ff6U3ShTiGEQ6LvjofAbm13twHjNdcdk7hFjjPKWnDw=; b=EpzO8YLgIKaJszKxvL4dj1C5PU8q7v1PpRcm/0xKto66i5DXJ2f4uDNiKsDCOU99xWfI65KpuxYMGhtgRmeaNm20saEk8ERZ8B5hAhUtXN7jBzTNd2ZvXnCUAstFMYHE7BRrWvvSAanMvUq1H1sCYYpB+2RZdgFIjbSn6r7ppQ1jiEnIh6iu+jADldJRtze2hIO6SgSXiOM9IOIzIUMhkXvoN+1mcox5nA/eW0riLOkJQ2rV5iiFUvgp8vJBR2AYQAii5WFb3sxvvRKM5XUKcNi9YTPtdacOp8WetLv58lduOz9eGwjiwmZa7JqO8nCYQdKBxa7GqGog3SutclaCrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ff6U3ShTiGEQ6LvjofAbm13twHjNdcdk7hFjjPKWnDw=; b=OYoqu15TTLT+cK7JPgRobVbpHulZ16YwKd3ZPDXGC1zAy3UfFlHGvjkHRHz9EiI9T2FV8ZFkHdzrdNm0HJQu4wdVYYEQ7s1TwcNthE6U263xVYxzd4BwZoTs6sI5mAX3zbVYYuLiPnA9YL2EJPNy5ApN3Pq8V07R8MWrP3QujgRA6gcwJauRME16l04Uq0ucKnvxzgIs7QcknkaVV4w8ZY4XC/k9a+lmfmc5SD9XP9DcqB3WaIPFN/9A9ConL7C5TfdbDiz2KaqEiosf9uPTX/vXryWl/q3V04+5hWfx4wTlbdDwU46+kNy2Z8TC9lVJiw9Mkq/QZz1LCXjLAX7vjg== Received: from MW4PR04CA0190.namprd04.prod.outlook.com (2603:10b6:303:86::15) by MN2PR12MB4605.namprd12.prod.outlook.com (2603:10b6:208:3d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.20; Fri, 8 Oct 2021 12:25:56 +0000 Received: from CO1NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:303:86:cafe::1b) by MW4PR04CA0190.outlook.office365.com (2603:10b6:303:86::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:25:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by CO1NAM11FT064.mail.protection.outlook.com (10.13.175.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:25:55 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 05:25:54 -0700 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:25:50 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 01/13] net/mlx5: Add ifc bits to support optional counters Date: Fri, 8 Oct 2021 15:24:27 +0300 Message-ID: <20211008122439.166063-2-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a48433c6-b8f3-4ace-3d5e-08d98a56c6f8 X-MS-TrafficTypeDiagnostic: MN2PR12MB4605: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wWclcCpprx1x7jk6SsIrBFB2xwEvs4PGRVdsnL1PFJ+A6HDUQp9LnhY5D6crrG3oKUjbCrOCFkGzRt6/JxIY3LrYGioYpiWTI7obpkhMC99SC9iQcVp0keJm/T3qlxRVgMbBvPYfDOrbRAqONFWJJu1yFcCiuP1Jy9ktTDRO2dZ0QywvS2nvfc92r4WE8gOIkw0d8l7xJeJYTPU8cPHLyYP3gD3fITRytlhwIaq96BMXHfopIY247806eOVPc356Wos60aKw+ioL4bwtPN1lmobFvs8meGx+JoUKJOCPgB1mNRktPj2V1GmQsPerJ/W2gfbhVvW7YNrmMYN9nVrxlEBGOehC8FDY0LVbmUb96qNw4QxIft5WgY3lfseAc1hPZhw1BuxBYjdR/1YGmffzdfNaSACmpna/c+5QXM0JcC+p+Cj2DQKKYcny1mGbxhyl4LO8otaOB4lRAVn2GUZCOPFiE7ifqj8lc+mIQPxKjNLbdC5Seni5LdUNnu8hfFio1BpqpYHRZXQFIWpgxLlgAMm4b0ZnvlkW97HsFc5Q/ibkgx399KBS+2E2JVYuW8JnDtBsJ7Civm+Z0GFsY78nyLfy2XfFUUyEupuZ56goigWY5r6Yogq/mXAYq1nkroISzZVsrOYtdMK/nzVk+LAGszc3xwXyqqAC073Gvg+QseqiCppd6DK5WQmnIqsxCjMesg6eIuVECrmu4AyZYbvwPQ== X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(7696005)(54906003)(8936002)(70586007)(4326008)(47076005)(107886003)(8676002)(6666004)(110136005)(36756003)(356005)(2616005)(426003)(336012)(316002)(1076003)(508600001)(5660300002)(186003)(36860700001)(7416002)(2906002)(7636003)(83380400001)(86362001)(82310400003)(70206006)(6636002)(26005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:25:55.8044 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a48433c6-b8f3-4ace-3d5e-08d98a56c6f8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4605 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Adding bth_opcode field and the relevant bits. This field will be used to capture and count congestion notification packets (CNP). Adding source_vhca_port support bit. This field will be used to check the capability to use the source_vhca_port as a match criteria in cases of dual port. Signed-off-by: Aharon Landau Reviewed-by: Maor Gottlieb Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- include/linux/mlx5/mlx5_ifc.h | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 96f5fb2af811..34254dbe7117 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -342,7 +342,7 @@ struct mlx5_ifc_flow_table_fields_supported_bits { u8 outer_geneve_oam[0x1]; u8 outer_geneve_protocol_type[0x1]; u8 outer_geneve_opt_len[0x1]; - u8 reserved_at_1e[0x1]; + u8 source_vhca_port[0x1]; u8 source_eswitch_port[0x1]; u8 inner_dmac[0x1]; @@ -393,6 +393,14 @@ struct mlx5_ifc_flow_table_fields_supported_bits { u8 metadata_reg_c_0[0x1]; }; +struct mlx5_ifc_flow_table_fields_supported_2_bits { + u8 reserved_at_0[0xe]; + u8 bth_opcode[0x1]; + u8 reserved_at_f[0x11]; + + u8 reserved_at_20[0x60]; +}; + struct mlx5_ifc_flow_table_prop_layout_bits { u8 ft_support[0x1]; u8 reserved_at_1[0x1]; @@ -539,7 +547,7 @@ struct mlx5_ifc_fte_match_set_misc_bits { union mlx5_ifc_gre_key_bits gre_key; u8 vxlan_vni[0x18]; - u8 reserved_at_b8[0x8]; + u8 bth_opcode[0x8]; u8 geneve_vni[0x18]; u8 reserved_at_d8[0x7]; @@ -756,7 +764,15 @@ struct mlx5_ifc_flow_table_nic_cap_bits { struct mlx5_ifc_flow_table_prop_layout_bits flow_table_properties_nic_transmit_sniffer; - u8 reserved_at_e00[0x1200]; + u8 reserved_at_e00[0x700]; + + struct mlx5_ifc_flow_table_fields_supported_2_bits ft_field_support_2_nic_receive_rdma; + + u8 reserved_at_1580[0x280]; + + struct mlx5_ifc_flow_table_fields_supported_2_bits ft_field_support_2_nic_transmit_rdma; + + u8 reserved_at_1880[0x780]; u8 sw_steering_nic_rx_action_drop_icm_address[0x40]; From patchwork Fri Oct 8 12:24:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545195 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAAB1C433EF for ; Fri, 8 Oct 2021 12:26:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D2C1A60FD8 for ; Fri, 8 Oct 2021 12:26:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241485AbhJHM16 (ORCPT ); Fri, 8 Oct 2021 08:27:58 -0400 Received: from mail-sn1anam02on2059.outbound.protection.outlook.com ([40.107.96.59]:7621 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241478AbhJHM16 (ORCPT ); Fri, 8 Oct 2021 08:27:58 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KKC6hhIo9Y1fw0FzIbd+x5Gda8sZ0eJQGjAs3FojRbKhyIQAv2GEM4UclzAAtmlND+AJ4FKEt+6Lg3vKW8OF7+vbDV0+qxt+uRdl5o0+dxi11WSSiEv/1Xpo79G5XERICIbpCUadh9BV248oMdBXOHkjzDNb5kX1U3zEu4LjqP1oYyyMhW4n3ypDjKKSUWw1rh9atatCRjl3gKSkCoG24VAlkWOGz6tjxOVTGHCOPy74jOnDtXSfTp1wW6JW/svuxXNdq2Q8qlETEaAGJWJ87ipxUiQUjszZVWHN5ReP1ac08x3M4jyqDZY0G1LfOmo3Y3DNN1wIR9vTrL/7frxWSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CJeF8rsZbSjZJ8vBXHRnEokE9p6ICrdoro9XuaDbr4c=; b=W9bdgrk2QWNF903YaTHqQdb8SeBTt8cjkZY4CQ7lqhYdmEuFFmZCaiN8sYlqH4ryzMf9cWUnaN9KTXPhXZcfwZBy0xPfVTPs2qlU+G9mAkLeW+uSK1iOctWQUOjxwRG0qbW72bDGQgPMutr/32edghNmo0OkyOuyKiSzxYd5EGmSlI6M7q6LjvvA4vz/Mkq0BmwCeBy/+sN87Q/2id9JvWGWS9vabgR4FZZmGXhH+EEDnvmRzJYJ6H+QJwvg+iqEZEiXHnJgZyj8PyEALXW1Gtqfe5C+vdoWIuLaducOU4QAJgzU29nc7+wppFl9MjkitkdduTcokzxq+C0YJa/Peg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CJeF8rsZbSjZJ8vBXHRnEokE9p6ICrdoro9XuaDbr4c=; b=PwndSWj/5iCD3l0Zy4nF0+Zy0XEPfGzrj7OWdhn57gBeeHC4X9MmNV6x/gKD8whDgR/Uprfyoz0ZSKgP90QgFewh8SHPJ5wHJr595JwLPbWRUx65TPwpAmbbUCf2GIMqK8PNpqaNvjN5BEqlZW4DribfzeJEME0KozVIbMsw8CpuQEgK8Dmx6xSd+48/CrmRyok7GC9Mfqsgh6PR9cMyz6nd9WvdV1JVuuuWwH2zzsHwc+f5EiddOuMZc8V6+N8uyEQqil1/N4iaw+yWT5V08VTGjy6ex4dJ7edfMqKy4GVTkxpTWulsM39WkVo1OW3uovpfi4XxgbKrwNWNSCjziQ== Received: from MWHPR12CA0072.namprd12.prod.outlook.com (2603:10b6:300:103::34) by MN2PR12MB4335.namprd12.prod.outlook.com (2603:10b6:208:1d4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.22; Fri, 8 Oct 2021 12:26:01 +0000 Received: from CO1NAM11FT068.eop-nam11.prod.protection.outlook.com (2603:10b6:300:103:cafe::9c) by MWHPR12CA0072.outlook.office365.com (2603:10b6:300:103::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT068.mail.protection.outlook.com (10.13.175.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:00 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:25:59 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:25:55 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 02/13] net/mlx5: Add priorities for counters in RDMA namespaces Date: Fri, 8 Oct 2021 15:24:28 +0300 Message-ID: <20211008122439.166063-3-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 517d0802-c923-49e7-2fbc-08d98a56c9bf X-MS-TrafficTypeDiagnostic: MN2PR12MB4335: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ujsesgAaucipN3pD8JvqzSDYIJ6l4jxRYgkwrSOqwpsuR1VaGknHIsftgefDDaiCItHIU6TgifmQ28tX4xnjC/XX1VWxIreEUP4ur7QeFMGHxcgCEVTPwsxMRzDUkgNl4/FodfKnh67EnDVZp+WHg7cViqvDZaI6eA5hPV8yR5z419kvm54J8GZl9GLkeHo/RylpgFxJBzSeOcNDxtyv8ephN19XwPI1XLBm/TeiZstXQOpHkDWbH8fbKTIyX2RkNsEfA3z0AJeC136l59i+ZMFl9O5WMHpuSPwIdGzI9/XzBUIRQm+bSFb4eTyWD/cNCsLWXe1B3g4xnDkNDtQB954zHTFzei+u2JlBBu8n1S+IPQoes4iM+FKu8lEeHNQI1/nYCwFCWajY1CNIImMBfvp49Rqd1BR9D3GCFrOymHDb3FKXID6Sjav0o4BoseGhR0ArHZVg9JWoOAUDLyjYssOSovurYwiNu4yBo2aB4rijRlXVn2R5w9oLaza3eh3tin6SwoTCYMg6wS71zQqLfKu4nZTuBM0rfCZqRQQ06KEC6e+VkDiPx/PolNMZnHV3urNEsQb5x+oMcmXXzx+tNcJFXCFa2bgMcPR5AqYOa1qBcnoOapeolV2ZfqTXPfvnaIdsh0mRiif98jdKuIUexaBMXJzz7/ZS2CQRrAs3afpy/Vt3zJFFK41nIzzGo2U7nIyXMdT94y8R4Q7B3a8yAg== X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(8936002)(2906002)(82310400003)(36860700001)(186003)(1076003)(26005)(508600001)(4326008)(86362001)(36756003)(8676002)(6666004)(107886003)(7696005)(336012)(54906003)(47076005)(2616005)(70586007)(316002)(83380400001)(426003)(70206006)(6636002)(7416002)(7636003)(5660300002)(356005)(110136005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:00.5047 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 517d0802-c923-49e7-2fbc-08d98a56c9bf X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT068.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4335 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Add additional flow steering priorities in the RDMA namespace. This allows adding flow counters to count filtered RDMA traffic and then continue processing in the regular RDMA steering flow. Signed-off-by: Aharon Landau Reviewed-by: Maor Gottlieb Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- .../net/ethernet/mellanox/mlx5/core/fs_core.c | 54 ++++++++++++++++--- include/linux/mlx5/device.h | 2 + include/linux/mlx5/fs.h | 2 + 3 files changed, 50 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index fe501ba88bea..71a08f84d49d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -99,6 +99,9 @@ #define LEFTOVERS_NUM_LEVELS 1 #define LEFTOVERS_NUM_PRIOS 1 +#define RDMA_RX_COUNTERS_PRIO_NUM_LEVELS 1 +#define RDMA_TX_COUNTERS_PRIO_NUM_LEVELS 1 + #define BY_PASS_PRIO_NUM_LEVELS 1 #define BY_PASS_MIN_LEVEL (ETHTOOL_MIN_LEVEL + MLX5_BY_PASS_NUM_PRIOS +\ LEFTOVERS_NUM_PRIOS) @@ -206,34 +209,63 @@ static struct init_tree_node egress_root_fs = { } }; -#define RDMA_RX_BYPASS_PRIO 0 -#define RDMA_RX_KERNEL_PRIO 1 +enum { + RDMA_RX_COUNTERS_PRIO, + RDMA_RX_BYPASS_PRIO, + RDMA_RX_KERNEL_PRIO, +}; + +#define RDMA_RX_BYPASS_MIN_LEVEL MLX5_BY_PASS_NUM_REGULAR_PRIOS +#define RDMA_RX_KERNEL_MIN_LEVEL (RDMA_RX_BYPASS_MIN_LEVEL + 1) +#define RDMA_RX_COUNTERS_MIN_LEVEL (RDMA_RX_KERNEL_MIN_LEVEL + 2) + static struct init_tree_node rdma_rx_root_fs = { .type = FS_TYPE_NAMESPACE, - .ar_size = 2, + .ar_size = 3, .children = (struct init_tree_node[]) { + [RDMA_RX_COUNTERS_PRIO] = + ADD_PRIO(0, RDMA_RX_COUNTERS_MIN_LEVEL, 0, + FS_CHAINING_CAPS, + ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, + ADD_MULTIPLE_PRIO(MLX5_RDMA_RX_NUM_COUNTERS_PRIOS, + RDMA_RX_COUNTERS_PRIO_NUM_LEVELS))), [RDMA_RX_BYPASS_PRIO] = - ADD_PRIO(0, MLX5_BY_PASS_NUM_REGULAR_PRIOS, 0, + ADD_PRIO(0, RDMA_RX_BYPASS_MIN_LEVEL, 0, FS_CHAINING_CAPS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_REGULAR_PRIOS, BY_PASS_PRIO_NUM_LEVELS))), [RDMA_RX_KERNEL_PRIO] = - ADD_PRIO(0, MLX5_BY_PASS_NUM_REGULAR_PRIOS + 1, 0, + ADD_PRIO(0, RDMA_RX_KERNEL_MIN_LEVEL, 0, FS_CHAINING_CAPS, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_SWITCH_DOMAIN, ADD_MULTIPLE_PRIO(1, 1))), } }; +enum { + RDMA_TX_COUNTERS_PRIO, + RDMA_TX_BYPASS_PRIO, +}; + +#define RDMA_TX_BYPASS_MIN_LEVEL MLX5_BY_PASS_NUM_PRIOS +#define RDMA_TX_COUNTERS_MIN_LEVEL (RDMA_TX_BYPASS_MIN_LEVEL + 1) + static struct init_tree_node rdma_tx_root_fs = { .type = FS_TYPE_NAMESPACE, - .ar_size = 1, + .ar_size = 2, .children = (struct init_tree_node[]) { - ADD_PRIO(0, MLX5_BY_PASS_NUM_PRIOS, 0, + [RDMA_TX_COUNTERS_PRIO] = + ADD_PRIO(0, RDMA_TX_COUNTERS_MIN_LEVEL, 0, + FS_CHAINING_CAPS, + ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, + ADD_MULTIPLE_PRIO(MLX5_RDMA_TX_NUM_COUNTERS_PRIOS, + RDMA_TX_COUNTERS_PRIO_NUM_LEVELS))), + [RDMA_TX_BYPASS_PRIO] = + ADD_PRIO(0, RDMA_TX_BYPASS_MIN_LEVEL, 0, FS_CHAINING_CAPS_RDMA_TX, ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, - ADD_MULTIPLE_PRIO(MLX5_BY_PASS_NUM_PRIOS, + ADD_MULTIPLE_PRIO(RDMA_TX_BYPASS_MIN_LEVEL, BY_PASS_PRIO_NUM_LEVELS))), } }; @@ -2215,6 +2247,12 @@ struct mlx5_flow_namespace *mlx5_get_flow_namespace(struct mlx5_core_dev *dev, prio = RDMA_RX_KERNEL_PRIO; } else if (type == MLX5_FLOW_NAMESPACE_RDMA_TX) { root_ns = steering->rdma_tx_root_ns; + } else if (type == MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS) { + root_ns = steering->rdma_rx_root_ns; + prio = RDMA_RX_COUNTERS_PRIO; + } else if (type == MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS) { + root_ns = steering->rdma_tx_root_ns; + prio = RDMA_TX_COUNTERS_PRIO; } else { /* Must be NIC RX */ root_ns = steering->root_ns; prio = type; diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index 66eaf0aa7f69..ed0230ff9422 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -1456,6 +1456,8 @@ static inline u16 mlx5_to_sw_pkey_sz(int pkey_sz) return MLX5_MIN_PKEY_TABLE_SIZE << pkey_sz; } +#define MLX5_RDMA_RX_NUM_COUNTERS_PRIOS 2 +#define MLX5_RDMA_TX_NUM_COUNTERS_PRIOS 1 #define MLX5_BY_PASS_NUM_REGULAR_PRIOS 16 #define MLX5_BY_PASS_NUM_DONT_TRAP_PRIOS 16 #define MLX5_BY_PASS_NUM_MULTICAST_PRIOS 1 diff --git a/include/linux/mlx5/fs.h b/include/linux/mlx5/fs.h index 0106c67e8ccb..f2c3da2006d9 100644 --- a/include/linux/mlx5/fs.h +++ b/include/linux/mlx5/fs.h @@ -83,6 +83,8 @@ enum mlx5_flow_namespace_type { MLX5_FLOW_NAMESPACE_RDMA_RX, MLX5_FLOW_NAMESPACE_RDMA_RX_KERNEL, MLX5_FLOW_NAMESPACE_RDMA_TX, + MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS, + MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS, }; enum { From patchwork Fri Oct 8 12:24:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545199 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC4DEC433F5 for ; Fri, 8 Oct 2021 12:26:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A070C60F44 for ; Fri, 8 Oct 2021 12:26:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241603AbhJHM2R (ORCPT ); Fri, 8 Oct 2021 08:28:17 -0400 Received: from mail-bn8nam11on2063.outbound.protection.outlook.com ([40.107.236.63]:21736 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241521AbhJHM2F (ORCPT ); Fri, 8 Oct 2021 08:28:05 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k8gImP5OmyK1/nKBgXda/UPRDdpjFuVbQG0+an1nSDw9nlD3KWKQBVBWy6j5r0YPeDQm1ONREvkzqPD6wt8IyqfLL3fle+CB5RyN6n3ov49octPuYIIzSkahGB5VN+C+k0ecp8aAyaubCZZ/o28HZy/kv8UhFK5+nNTro9gpkc3UhlBCK6GoNYiAVtJ1ADB7spZoDhBn5ON2miQ84m3g1Uw4bhnKStgj+pJ09QYBKTzXLA9NzPn/2NpLFOROGxPe+BzDQfLXXY+NHPPbMaG9XH4bN6dxA5fA1DprxZv3RFTCBOeR3mxf6yFi3DsCNsJ/rrxXBtvOeO3Jy7ivmotBtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=r5bCLEkTrNeZznxloQOcWO7MuBTNU5DUhudCS1K2pIU=; b=R0XK3avzjh+gt1uqktnHPY4usG2n0Wav7omVRHU666b8GiTARiBzKjaqE5JxhR+TpRgvF9giYhXi50ZSB8jY10KECdrM8ICyOu8tHUbH76AcW+5m/8+8cnbPrtTB7ogo2oY3FCCylVoJ+qQgfkiI/hG7BULltUr/AhU7Y2JEynjP3YJGt9GHB0m4ac3rfr0tAzlcTVzX68gJanRYYpeEoWuG7evoLqiEoyS06shAJKhxJo4U4ctex2biOSlgBX6RBZLKCdR4cDWesElIQooYNlNRQfx1pKcG1JmWiVVjNAu+i11Q8S2WKZBkumKHTrfwq0pWAWvUs0OXgL6TxROlBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=r5bCLEkTrNeZznxloQOcWO7MuBTNU5DUhudCS1K2pIU=; b=T0/QCZGuU4WKxTKYpl/2r7imxgMQP0e3vqAzfw2bp9q2lY0R7a3XCBnrXFvjxsoupf032i6iG0Nf5IZAxWGZo/dnuowMmNBrCVqAVzoOtoCJ3QU+E050RnpkkyNxFuPxEbPIoNUoUPSfk9/BZ6dVEJ3OE+7vMrV/RvxYxhXRsuD6iK7+HUZnQKSeMrBszIGI/Rtc6frBUgLtYUENyHZNXUNDUaAnO++Xx/q6kWj5uLhseffobCw9AY60ZGpexH4fHzneh+GgMjM24CW55lKCAnpTGcaQK0kzKgcufsUHhqo+LWOIX/3pmpWRiISDwDqPmCblCFBLHIWKGEAYe6Ucnw== Received: from MW4PR04CA0282.namprd04.prod.outlook.com (2603:10b6:303:89::17) by DM5PR12MB2488.namprd12.prod.outlook.com (2603:10b6:4:b5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.20; Fri, 8 Oct 2021 12:26:06 +0000 Received: from CO1NAM11FT035.eop-nam11.prod.protection.outlook.com (2603:10b6:303:89:cafe::11) by MW4PR04CA0282.outlook.office365.com (2603:10b6:303:89::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT035.mail.protection.outlook.com (10.13.175.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:05 +0000 Received: from HQMAIL105.nvidia.com (172.20.187.12) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:04 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:04 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:25:59 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 03/13] RDMA/counter: Add a descriptor in struct rdma_hw_stats Date: Fri, 8 Oct 2021 15:24:29 +0300 Message-ID: <20211008122439.166063-4-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bdd0ccd6-6e6a-456a-66ee-08d98a56ccbc X-MS-TrafficTypeDiagnostic: DM5PR12MB2488: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:67; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6z7krKb6cvmX4mJFcsQGnccru9BUaHDZ2wLJ29Jg7i3WI88buozHhr047rLsFX3StfJlOkE1GQo8gGwzUdiGS5druWJ84x/4V0+grNm6GLc5DKzByRIE2svVyyhQc4HXhMr+ZIguZdGXTYiDt0AveRkQxj9CXLYKFmhTH5EfL0y29VF2XcJFgQJGSseQEUJ+wCqcwx8MsOl2riq1vNOzGbKi0MCIW4XeW0A1nSjymutR/YZsZXwlq3JBgWLui6TcF91ijrbnYrjWvkdyMy4L2iP1tfpVNbnn+yIDH9REGF5zli2rm3WQP+E1tH0RCrLjlGLs4OShGLMsFvEDbgF1/F8pUvJYfkl5wuS/srSS5+pmrEPjXI49tqIOV/S3qA1vblyhyhHUGFNSU0ServZb0AIlKfHUI67QlpZvSrA8elSpVZvH6D1Ch99Z53P80rrJppp/qYhsTAsZ3LlRa5T02C64P9Wdend22IHYWxHbaSTwleTnoayMK31QreQRLImm8FRIxrwhqtzFvS0mwE6XzRLpvQL+RKKKxPU8BvyaDa8ubB4WyLwhOkeBeEkr9v7V6Gr43o3nth7vveap5cc8dCcuhYj3v3Vsc1hPFZReTdPGBvumFPlqDGvINfcbIyu9DYoGqFduYigm+u/2Y7TFSImjrUtTlRUq2r06IWHzkFKfh+DUnV7vd3UUoX5x/0rQmm62XwnuEXEBnWllJ1BVXQ== X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(70586007)(4326008)(426003)(6666004)(54906003)(26005)(107886003)(6636002)(2616005)(82310400003)(2906002)(8676002)(36860700001)(7696005)(36756003)(86362001)(336012)(8936002)(7416002)(5660300002)(30864003)(186003)(47076005)(316002)(356005)(110136005)(83380400001)(1076003)(7636003)(508600001)(579004);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:05.4174 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bdd0ccd6-6e6a-456a-66ee-08d98a56ccbc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT035.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2488 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Add a counter statistic descriptor structure in rdma_hw_stats. In addition to the counter name, more meta-information will be added. This code extension is needed for optional-counter support in the following patches. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/nldev.c | 6 +- drivers/infiniband/core/sysfs.c | 8 +- drivers/infiniband/hw/bnxt_re/hw_counters.c | 137 ++++++++++---------- drivers/infiniband/hw/cxgb4/provider.c | 22 ++-- drivers/infiniband/hw/efa/efa_verbs.c | 19 +-- drivers/infiniband/hw/hfi1/verbs.c | 53 ++++---- drivers/infiniband/hw/irdma/verbs.c | 100 +++++++------- drivers/infiniband/hw/mlx4/main.c | 44 +++---- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx5/counters.c | 41 +++--- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- drivers/infiniband/sw/rxe/rxe_hw_counters.c | 42 +++--- include/rdma/ib_verbs.h | 21 ++- 13 files changed, 252 insertions(+), 245 deletions(-) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index e9b4b2cccaa0..3f6b98a87566 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -969,7 +969,8 @@ static int fill_stat_counter_hwcounters(struct sk_buff *msg, return -EMSGSIZE; for (i = 0; i < st->num_counters; i++) - if (rdma_nl_stat_hwcounter_entry(msg, st->names[i], st->value[i])) + if (rdma_nl_stat_hwcounter_entry(msg, st->descs[i].name, + st->value[i])) goto err; nla_nest_end(msg, table_attr); @@ -2105,7 +2106,8 @@ static int stat_get_doit_default_counter(struct sk_buff *skb, for (i = 0; i < num_cnts; i++) { v = stats->value[i] + rdma_counter_get_hwstat_value(device, port, i); - if (rdma_nl_stat_hwcounter_entry(msg, stats->names[i], v)) { + if (rdma_nl_stat_hwcounter_entry(msg, + stats->descs[i].name, v)) { ret = -EMSGSIZE; goto err_table; } diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c index 6146c3c1cbe5..c3663cfdcd52 100644 --- a/drivers/infiniband/core/sysfs.c +++ b/drivers/infiniband/core/sysfs.c @@ -895,7 +895,7 @@ alloc_hw_stats_device(struct ib_device *ibdev) stats = ibdev->ops.alloc_hw_device_stats(ibdev); if (!stats) return ERR_PTR(-ENOMEM); - if (!stats->names || stats->num_counters <= 0) + if (!stats->descs || stats->num_counters <= 0) goto err_free_stats; /* @@ -957,7 +957,7 @@ int ib_setup_device_attrs(struct ib_device *ibdev) for (i = 0; i < data->stats->num_counters; i++) { attr = &data->attrs[i]; sysfs_attr_init(&attr->attr.attr); - attr->attr.attr.name = data->stats->names[i]; + attr->attr.attr.name = data->stats->descs[i].name; attr->attr.attr.mode = 0444; attr->attr.show = hw_stat_device_show; attr->show = show_hw_stats; @@ -994,7 +994,7 @@ alloc_hw_stats_port(struct ib_port *port, struct attribute_group *group) stats = ibdev->ops.alloc_hw_port_stats(port->ibdev, port->port_num); if (!stats) return ERR_PTR(-ENOMEM); - if (!stats->names || stats->num_counters <= 0) + if (!stats->descs || stats->num_counters <= 0) goto err_free_stats; /* @@ -1047,7 +1047,7 @@ static int setup_hw_port_stats(struct ib_port *port, for (i = 0; i < data->stats->num_counters; i++) { attr = &data->attrs[i]; sysfs_attr_init(&attr->attr.attr); - attr->attr.attr.name = data->stats->names[i]; + attr->attr.attr.name = data->stats->descs[i].name; attr->attr.attr.mode = 0444; attr->attr.show = hw_stat_port_show; attr->show = show_hw_stats; diff --git a/drivers/infiniband/hw/bnxt_re/hw_counters.c b/drivers/infiniband/hw/bnxt_re/hw_counters.c index 1c06c9cf234b..78ca6dfd182b 100644 --- a/drivers/infiniband/hw/bnxt_re/hw_counters.c +++ b/drivers/infiniband/hw/bnxt_re/hw_counters.c @@ -57,74 +57,72 @@ #include "bnxt_re.h" #include "hw_counters.h" -static const char * const bnxt_re_stat_name[] = { - [BNXT_RE_ACTIVE_PD] = "active_pds", - [BNXT_RE_ACTIVE_AH] = "active_ahs", - [BNXT_RE_ACTIVE_QP] = "active_qps", - [BNXT_RE_ACTIVE_SRQ] = "active_srqs", - [BNXT_RE_ACTIVE_CQ] = "active_cqs", - [BNXT_RE_ACTIVE_MR] = "active_mrs", - [BNXT_RE_ACTIVE_MW] = "active_mws", - [BNXT_RE_RX_PKTS] = "rx_pkts", - [BNXT_RE_RX_BYTES] = "rx_bytes", - [BNXT_RE_TX_PKTS] = "tx_pkts", - [BNXT_RE_TX_BYTES] = "tx_bytes", - [BNXT_RE_RECOVERABLE_ERRORS] = "recoverable_errors", - [BNXT_RE_RX_ERRORS] = "rx_roce_errors", - [BNXT_RE_RX_DISCARDS] = "rx_roce_discards", - [BNXT_RE_TO_RETRANSMITS] = "to_retransmits", - [BNXT_RE_SEQ_ERR_NAKS_RCVD] = "seq_err_naks_rcvd", - [BNXT_RE_MAX_RETRY_EXCEEDED] = "max_retry_exceeded", - [BNXT_RE_RNR_NAKS_RCVD] = "rnr_naks_rcvd", - [BNXT_RE_MISSING_RESP] = "missing_resp", - [BNXT_RE_UNRECOVERABLE_ERR] = "unrecoverable_err", - [BNXT_RE_BAD_RESP_ERR] = "bad_resp_err", - [BNXT_RE_LOCAL_QP_OP_ERR] = "local_qp_op_err", - [BNXT_RE_LOCAL_PROTECTION_ERR] = "local_protection_err", - [BNXT_RE_MEM_MGMT_OP_ERR] = "mem_mgmt_op_err", - [BNXT_RE_REMOTE_INVALID_REQ_ERR] = "remote_invalid_req_err", - [BNXT_RE_REMOTE_ACCESS_ERR] = "remote_access_err", - [BNXT_RE_REMOTE_OP_ERR] = "remote_op_err", - [BNXT_RE_DUP_REQ] = "dup_req", - [BNXT_RE_RES_EXCEED_MAX] = "res_exceed_max", - [BNXT_RE_RES_LENGTH_MISMATCH] = "res_length_mismatch", - [BNXT_RE_RES_EXCEEDS_WQE] = "res_exceeds_wqe", - [BNXT_RE_RES_OPCODE_ERR] = "res_opcode_err", - [BNXT_RE_RES_RX_INVALID_RKEY] = "res_rx_invalid_rkey", - [BNXT_RE_RES_RX_DOMAIN_ERR] = "res_rx_domain_err", - [BNXT_RE_RES_RX_NO_PERM] = "res_rx_no_perm", - [BNXT_RE_RES_RX_RANGE_ERR] = "res_rx_range_err", - [BNXT_RE_RES_TX_INVALID_RKEY] = "res_tx_invalid_rkey", - [BNXT_RE_RES_TX_DOMAIN_ERR] = "res_tx_domain_err", - [BNXT_RE_RES_TX_NO_PERM] = "res_tx_no_perm", - [BNXT_RE_RES_TX_RANGE_ERR] = "res_tx_range_err", - [BNXT_RE_RES_IRRQ_OFLOW] = "res_irrq_oflow", - [BNXT_RE_RES_UNSUP_OPCODE] = "res_unsup_opcode", - [BNXT_RE_RES_UNALIGNED_ATOMIC] = "res_unaligned_atomic", - [BNXT_RE_RES_REM_INV_ERR] = "res_rem_inv_err", - [BNXT_RE_RES_MEM_ERROR] = "res_mem_err", - [BNXT_RE_RES_SRQ_ERR] = "res_srq_err", - [BNXT_RE_RES_CMP_ERR] = "res_cmp_err", - [BNXT_RE_RES_INVALID_DUP_RKEY] = "res_invalid_dup_rkey", - [BNXT_RE_RES_WQE_FORMAT_ERR] = "res_wqe_format_err", - [BNXT_RE_RES_CQ_LOAD_ERR] = "res_cq_load_err", - [BNXT_RE_RES_SRQ_LOAD_ERR] = "res_srq_load_err", - [BNXT_RE_RES_TX_PCI_ERR] = "res_tx_pci_err", - [BNXT_RE_RES_RX_PCI_ERR] = "res_rx_pci_err", - [BNXT_RE_OUT_OF_SEQ_ERR] = "oos_drop_count", - [BNXT_RE_TX_ATOMIC_REQ] = "tx_atomic_req", - [BNXT_RE_TX_READ_REQ] = "tx_read_req", - [BNXT_RE_TX_READ_RES] = "tx_read_resp", - [BNXT_RE_TX_WRITE_REQ] = "tx_write_req", - [BNXT_RE_TX_SEND_REQ] = "tx_send_req", - [BNXT_RE_RX_ATOMIC_REQ] = "rx_atomic_req", - [BNXT_RE_RX_READ_REQ] = "rx_read_req", - [BNXT_RE_RX_READ_RESP] = "rx_read_resp", - [BNXT_RE_RX_WRITE_REQ] = "rx_write_req", - [BNXT_RE_RX_SEND_REQ] = "rx_send_req", - [BNXT_RE_RX_ROCE_GOOD_PKTS] = "rx_roce_good_pkts", - [BNXT_RE_RX_ROCE_GOOD_BYTES] = "rx_roce_good_bytes", - [BNXT_RE_OOB] = "rx_out_of_buffer" +static const struct rdma_stat_desc bnxt_re_stat_descs[] = { + [BNXT_RE_ACTIVE_QP].name = "active_qps", + [BNXT_RE_ACTIVE_SRQ].name = "active_srqs", + [BNXT_RE_ACTIVE_CQ].name = "active_cqs", + [BNXT_RE_ACTIVE_MR].name = "active_mrs", + [BNXT_RE_ACTIVE_MW].name = "active_mws", + [BNXT_RE_RX_PKTS].name = "rx_pkts", + [BNXT_RE_RX_BYTES].name = "rx_bytes", + [BNXT_RE_TX_PKTS].name = "tx_pkts", + [BNXT_RE_TX_BYTES].name = "tx_bytes", + [BNXT_RE_RECOVERABLE_ERRORS].name = "recoverable_errors", + [BNXT_RE_RX_ERRORS].name = "rx_roce_errors", + [BNXT_RE_RX_DISCARDS].name = "rx_roce_discards", + [BNXT_RE_TO_RETRANSMITS].name = "to_retransmits", + [BNXT_RE_SEQ_ERR_NAKS_RCVD].name = "seq_err_naks_rcvd", + [BNXT_RE_MAX_RETRY_EXCEEDED].name = "max_retry_exceeded", + [BNXT_RE_RNR_NAKS_RCVD].name = "rnr_naks_rcvd", + [BNXT_RE_MISSING_RESP].name = "missing_resp", + [BNXT_RE_UNRECOVERABLE_ERR].name = "unrecoverable_err", + [BNXT_RE_BAD_RESP_ERR].name = "bad_resp_err", + [BNXT_RE_LOCAL_QP_OP_ERR].name = "local_qp_op_err", + [BNXT_RE_LOCAL_PROTECTION_ERR].name = "local_protection_err", + [BNXT_RE_MEM_MGMT_OP_ERR].name = "mem_mgmt_op_err", + [BNXT_RE_REMOTE_INVALID_REQ_ERR].name = "remote_invalid_req_err", + [BNXT_RE_REMOTE_ACCESS_ERR].name = "remote_access_err", + [BNXT_RE_REMOTE_OP_ERR].name = "remote_op_err", + [BNXT_RE_DUP_REQ].name = "dup_req", + [BNXT_RE_RES_EXCEED_MAX].name = "res_exceed_max", + [BNXT_RE_RES_LENGTH_MISMATCH].name = "res_length_mismatch", + [BNXT_RE_RES_EXCEEDS_WQE].name = "res_exceeds_wqe", + [BNXT_RE_RES_OPCODE_ERR].name = "res_opcode_err", + [BNXT_RE_RES_RX_INVALID_RKEY].name = "res_rx_invalid_rkey", + [BNXT_RE_RES_RX_DOMAIN_ERR].name = "res_rx_domain_err", + [BNXT_RE_RES_RX_NO_PERM].name = "res_rx_no_perm", + [BNXT_RE_RES_RX_RANGE_ERR].name = "res_rx_range_err", + [BNXT_RE_RES_TX_INVALID_RKEY].name = "res_tx_invalid_rkey", + [BNXT_RE_RES_TX_DOMAIN_ERR].name = "res_tx_domain_err", + [BNXT_RE_RES_TX_NO_PERM].name = "res_tx_no_perm", + [BNXT_RE_RES_TX_RANGE_ERR].name = "res_tx_range_err", + [BNXT_RE_RES_IRRQ_OFLOW].name = "res_irrq_oflow", + [BNXT_RE_RES_UNSUP_OPCODE].name = "res_unsup_opcode", + [BNXT_RE_RES_UNALIGNED_ATOMIC].name = "res_unaligned_atomic", + [BNXT_RE_RES_REM_INV_ERR].name = "res_rem_inv_err", + [BNXT_RE_RES_MEM_ERROR].name = "res_mem_err", + [BNXT_RE_RES_SRQ_ERR].name = "res_srq_err", + [BNXT_RE_RES_CMP_ERR].name = "res_cmp_err", + [BNXT_RE_RES_INVALID_DUP_RKEY].name = "res_invalid_dup_rkey", + [BNXT_RE_RES_WQE_FORMAT_ERR].name = "res_wqe_format_err", + [BNXT_RE_RES_CQ_LOAD_ERR].name = "res_cq_load_err", + [BNXT_RE_RES_SRQ_LOAD_ERR].name = "res_srq_load_err", + [BNXT_RE_RES_TX_PCI_ERR].name = "res_tx_pci_err", + [BNXT_RE_RES_RX_PCI_ERR].name = "res_rx_pci_err", + [BNXT_RE_OUT_OF_SEQ_ERR].name = "oos_drop_count", + [BNXT_RE_TX_ATOMIC_REQ].name = "tx_atomic_req", + [BNXT_RE_TX_READ_REQ].name = "tx_read_req", + [BNXT_RE_TX_READ_RES].name = "tx_read_resp", + [BNXT_RE_TX_WRITE_REQ].name = "tx_write_req", + [BNXT_RE_TX_SEND_REQ].name = "tx_send_req", + [BNXT_RE_RX_ATOMIC_REQ].name = "rx_atomic_req", + [BNXT_RE_RX_READ_REQ].name = "rx_read_req", + [BNXT_RE_RX_READ_RESP].name = "rx_read_resp", + [BNXT_RE_RX_WRITE_REQ].name = "rx_write_req", + [BNXT_RE_RX_SEND_REQ].name = "rx_send_req", + [BNXT_RE_RX_ROCE_GOOD_PKTS].name = "rx_roce_good_pkts", + [BNXT_RE_RX_ROCE_GOOD_BYTES].name = "rx_roce_good_bytes", + [BNXT_RE_OOB].name = "rx_out_of_buffer" }; static void bnxt_re_copy_ext_stats(struct bnxt_re_dev *rdev, @@ -322,7 +320,6 @@ struct rdma_hw_stats *bnxt_re_ib_alloc_hw_port_stats(struct ib_device *ibdev, else num_counters = BNXT_RE_NUM_STD_COUNTERS; - return rdma_alloc_hw_stats_struct(bnxt_re_stat_name, - num_counters, + return rdma_alloc_hw_stats_struct(bnxt_re_stat_descs, num_counters, RDMA_HW_STATS_DEFAULT_LIFESPAN); } diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c index e7337662aff8..0c8fd5a85fcb 100644 --- a/drivers/infiniband/hw/cxgb4/provider.c +++ b/drivers/infiniband/hw/cxgb4/provider.c @@ -366,23 +366,23 @@ enum counters { NR_COUNTERS }; -static const char * const names[] = { - [IP4INSEGS] = "ip4InSegs", - [IP4OUTSEGS] = "ip4OutSegs", - [IP4RETRANSSEGS] = "ip4RetransSegs", - [IP4OUTRSTS] = "ip4OutRsts", - [IP6INSEGS] = "ip6InSegs", - [IP6OUTSEGS] = "ip6OutSegs", - [IP6RETRANSSEGS] = "ip6RetransSegs", - [IP6OUTRSTS] = "ip6OutRsts" +static const struct rdma_stat_desc cxgb4_descs[] = { + [IP4INSEGS].name = "ip4InSegs", + [IP4OUTSEGS].name = "ip4OutSegs", + [IP4RETRANSSEGS].name = "ip4RetransSegs", + [IP4OUTRSTS].name = "ip4OutRsts", + [IP6INSEGS].name = "ip6InSegs", + [IP6OUTSEGS].name = "ip6OutSegs", + [IP6RETRANSSEGS].name = "ip6RetransSegs", + [IP6OUTRSTS].name = "ip6OutRsts" }; static struct rdma_hw_stats *c4iw_alloc_device_stats(struct ib_device *ibdev) { - BUILD_BUG_ON(ARRAY_SIZE(names) != NR_COUNTERS); + BUILD_BUG_ON(ARRAY_SIZE(cxgb4_descs) != NR_COUNTERS); /* FIXME: these look like port stats */ - return rdma_alloc_hw_stats_struct(names, NR_COUNTERS, + return rdma_alloc_hw_stats_struct(cxgb4_descs, NR_COUNTERS, RDMA_HW_STATS_DEFAULT_LIFESPAN); } diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c index e5f9d90aad5e..35d818b38e77 100644 --- a/drivers/infiniband/hw/efa/efa_verbs.c +++ b/drivers/infiniband/hw/efa/efa_verbs.c @@ -60,13 +60,14 @@ struct efa_user_mmap_entry { op(EFA_RDMA_READ_RESP_BYTES, "rdma_read_resp_bytes") \ #define EFA_STATS_ENUM(ename, name) ename, -#define EFA_STATS_STR(ename, name) [ename] = name, +#define EFA_STATS_STR(ename, nam) \ + [ename].name = nam, enum efa_hw_device_stats { EFA_DEFINE_DEVICE_STATS(EFA_STATS_ENUM) }; -static const char *const efa_device_stats_names[] = { +static const struct rdma_stat_desc efa_device_stats_descs[] = { EFA_DEFINE_DEVICE_STATS(EFA_STATS_STR) }; @@ -74,7 +75,7 @@ enum efa_hw_port_stats { EFA_DEFINE_PORT_STATS(EFA_STATS_ENUM) }; -static const char *const efa_port_stats_names[] = { +static const struct rdma_stat_desc efa_port_stats_descs[] = { EFA_DEFINE_PORT_STATS(EFA_STATS_STR) }; @@ -1906,15 +1907,15 @@ int efa_destroy_ah(struct ib_ah *ibah, u32 flags) struct rdma_hw_stats *efa_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) { - return rdma_alloc_hw_stats_struct(efa_port_stats_names, - ARRAY_SIZE(efa_port_stats_names), + return rdma_alloc_hw_stats_struct(efa_port_stats_descs, + ARRAY_SIZE(efa_port_stats_descs), RDMA_HW_STATS_DEFAULT_LIFESPAN); } struct rdma_hw_stats *efa_alloc_hw_device_stats(struct ib_device *ibdev) { - return rdma_alloc_hw_stats_struct(efa_device_stats_names, - ARRAY_SIZE(efa_device_stats_names), + return rdma_alloc_hw_stats_struct(efa_device_stats_descs, + ARRAY_SIZE(efa_device_stats_descs), RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -1939,7 +1940,7 @@ static int efa_fill_device_stats(struct efa_dev *dev, stats->value[EFA_CREATE_AH_ERR] = atomic64_read(&s->create_ah_err); stats->value[EFA_MMAP_ERR] = atomic64_read(&s->mmap_err); - return ARRAY_SIZE(efa_device_stats_names); + return ARRAY_SIZE(efa_device_stats_descs); } static int efa_fill_port_stats(struct efa_dev *dev, struct rdma_hw_stats *stats, @@ -1988,7 +1989,7 @@ static int efa_fill_port_stats(struct efa_dev *dev, struct rdma_hw_stats *stats, stats->value[EFA_RDMA_READ_WR_ERR] = rrs->read_wr_err; stats->value[EFA_RDMA_READ_RESP_BYTES] = rrs->read_resp_bytes; - return ARRAY_SIZE(efa_port_stats_names); + return ARRAY_SIZE(efa_port_stats_descs); } int efa_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index 26bea51869bf..ed9fa0d84e9e 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -1602,8 +1602,8 @@ static const char * const driver_cntr_names[] = { }; static DEFINE_MUTEX(cntr_names_lock); /* protects the *_cntr_names bufers */ -static const char **dev_cntr_names; -static const char **port_cntr_names; +static struct rdma_stat_desc *dev_cntr_descs; +static struct rdma_stat_desc *port_cntr_descs; int num_driver_cntrs = ARRAY_SIZE(driver_cntr_names); static int num_dev_cntrs; static int num_port_cntrs; @@ -1614,13 +1614,12 @@ static int cntr_names_initialized; * strings. Optionally some entries can be reserved in the array to hold extra * external strings. */ -static int init_cntr_names(const char *names_in, - const size_t names_len, - int num_extra_names, - int *num_cntrs, - const char ***cntr_names) +static int init_cntr_names(const char *names_in, const size_t names_len, + int num_extra_names, int *num_cntrs, + struct rdma_stat_desc **cntr_descs) { - char *names_out, *p, **q; + struct rdma_stat_desc *q; + char *names_out, *p; int i, n; n = 0; @@ -1628,26 +1627,28 @@ static int init_cntr_names(const char *names_in, if (names_in[i] == '\n') n++; - names_out = kmalloc((n + num_extra_names) * sizeof(char *) + names_len, - GFP_KERNEL); + names_out = + kmalloc((n + num_extra_names) * sizeof(struct rdma_stat_desc) + + names_len, + GFP_KERNEL); if (!names_out) { *num_cntrs = 0; - *cntr_names = NULL; + *cntr_descs = NULL; return -ENOMEM; } - p = names_out + (n + num_extra_names) * sizeof(char *); + p = names_out + (n + num_extra_names) * sizeof(struct rdma_stat_desc); memcpy(p, names_in, names_len); - q = (char **)names_out; + q = (struct rdma_stat_desc *)names_out; for (i = 0; i < n; i++) { - q[i] = p; + q[i].name = p; p = strchr(p, '\n'); *p++ = '\0'; } *num_cntrs = n; - *cntr_names = (const char **)names_out; + *cntr_descs = (struct rdma_stat_desc *)names_out; return 0; } @@ -1661,18 +1662,18 @@ static int init_counters(struct ib_device *ibdev) goto out_unlock; err = init_cntr_names(dd->cntrnames, dd->cntrnameslen, num_driver_cntrs, - &num_dev_cntrs, &dev_cntr_names); + &num_dev_cntrs, &dev_cntr_descs); if (err) goto out_unlock; for (i = 0; i < num_driver_cntrs; i++) - dev_cntr_names[num_dev_cntrs + i] = driver_cntr_names[i]; + dev_cntr_descs[num_dev_cntrs + i].name = driver_cntr_names[i]; err = init_cntr_names(dd->portcntrnames, dd->portcntrnameslen, 0, - &num_port_cntrs, &port_cntr_names); + &num_port_cntrs, &port_cntr_descs); if (err) { - kfree(dev_cntr_names); - dev_cntr_names = NULL; + kfree(dev_cntr_descs); + dev_cntr_descs = NULL; goto out_unlock; } cntr_names_initialized = 1; @@ -1686,7 +1687,7 @@ static struct rdma_hw_stats *hfi1_alloc_hw_device_stats(struct ib_device *ibdev) { if (init_counters(ibdev)) return NULL; - return rdma_alloc_hw_stats_struct(dev_cntr_names, + return rdma_alloc_hw_stats_struct(dev_cntr_descs, num_dev_cntrs + num_driver_cntrs, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -1696,7 +1697,7 @@ static struct rdma_hw_stats *hfi_alloc_hw_port_stats(struct ib_device *ibdev, { if (init_counters(ibdev)) return NULL; - return rdma_alloc_hw_stats_struct(port_cntr_names, num_port_cntrs, + return rdma_alloc_hw_stats_struct(port_cntr_descs, num_port_cntrs, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -1921,10 +1922,10 @@ void hfi1_unregister_ib_device(struct hfi1_devdata *dd) verbs_txreq_exit(dev); mutex_lock(&cntr_names_lock); - kfree(dev_cntr_names); - kfree(port_cntr_names); - dev_cntr_names = NULL; - port_cntr_names = NULL; + kfree(dev_cntr_descs); + kfree(port_cntr_descs); + dev_cntr_descs = NULL; + port_cntr_descs = NULL; cntr_names_initialized = 0; mutex_unlock(&cntr_names_lock); } diff --git a/drivers/infiniband/hw/irdma/verbs.c b/drivers/infiniband/hw/irdma/verbs.c index 4fc323402073..382f87ea7155 100644 --- a/drivers/infiniband/hw/irdma/verbs.c +++ b/drivers/infiniband/hw/irdma/verbs.c @@ -3644,89 +3644,89 @@ static int irdma_iw_port_immutable(struct ib_device *ibdev, u32 port_num, return 0; } -static const char *const irdma_hw_stat_names[] = { +static const struct rdma_stat_desc irdma_hw_stat_descs[] = { /* 32bit names */ - [IRDMA_HW_STAT_INDEX_RXVLANERR] = "rxVlanErrors", - [IRDMA_HW_STAT_INDEX_IP4RXDISCARD] = "ip4InDiscards", - [IRDMA_HW_STAT_INDEX_IP4RXTRUNC] = "ip4InTruncatedPkts", - [IRDMA_HW_STAT_INDEX_IP4TXNOROUTE] = "ip4OutNoRoutes", - [IRDMA_HW_STAT_INDEX_IP6RXDISCARD] = "ip6InDiscards", - [IRDMA_HW_STAT_INDEX_IP6RXTRUNC] = "ip6InTruncatedPkts", - [IRDMA_HW_STAT_INDEX_IP6TXNOROUTE] = "ip6OutNoRoutes", - [IRDMA_HW_STAT_INDEX_TCPRTXSEG] = "tcpRetransSegs", - [IRDMA_HW_STAT_INDEX_TCPRXOPTERR] = "tcpInOptErrors", - [IRDMA_HW_STAT_INDEX_TCPRXPROTOERR] = "tcpInProtoErrors", - [IRDMA_HW_STAT_INDEX_RXRPCNPHANDLED] = "cnpHandled", - [IRDMA_HW_STAT_INDEX_RXRPCNPIGNORED] = "cnpIgnored", - [IRDMA_HW_STAT_INDEX_TXNPCNPSENT] = "cnpSent", + [IRDMA_HW_STAT_INDEX_RXVLANERR].name = "rxVlanErrors", + [IRDMA_HW_STAT_INDEX_IP4RXDISCARD].name = "ip4InDiscards", + [IRDMA_HW_STAT_INDEX_IP4RXTRUNC].name = "ip4InTruncatedPkts", + [IRDMA_HW_STAT_INDEX_IP4TXNOROUTE].name = "ip4OutNoRoutes", + [IRDMA_HW_STAT_INDEX_IP6RXDISCARD].name = "ip6InDiscards", + [IRDMA_HW_STAT_INDEX_IP6RXTRUNC].name = "ip6InTruncatedPkts", + [IRDMA_HW_STAT_INDEX_IP6TXNOROUTE].name = "ip6OutNoRoutes", + [IRDMA_HW_STAT_INDEX_TCPRTXSEG].name = "tcpRetransSegs", + [IRDMA_HW_STAT_INDEX_TCPRXOPTERR].name = "tcpInOptErrors", + [IRDMA_HW_STAT_INDEX_TCPRXPROTOERR].name = "tcpInProtoErrors", + [IRDMA_HW_STAT_INDEX_RXRPCNPHANDLED].name = "cnpHandled", + [IRDMA_HW_STAT_INDEX_RXRPCNPIGNORED].name = "cnpIgnored", + [IRDMA_HW_STAT_INDEX_TXNPCNPSENT].name = "cnpSent", /* 64bit names */ - [IRDMA_HW_STAT_INDEX_IP4RXOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4RXOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4InOctets", - [IRDMA_HW_STAT_INDEX_IP4RXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4RXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4InPkts", - [IRDMA_HW_STAT_INDEX_IP4RXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4RXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4InReasmRqd", - [IRDMA_HW_STAT_INDEX_IP4RXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4RXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4InMcastOctets", - [IRDMA_HW_STAT_INDEX_IP4RXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4RXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4InMcastPkts", - [IRDMA_HW_STAT_INDEX_IP4TXOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4TXOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4OutOctets", - [IRDMA_HW_STAT_INDEX_IP4TXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4TXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4OutPkts", - [IRDMA_HW_STAT_INDEX_IP4TXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4TXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4OutSegRqd", - [IRDMA_HW_STAT_INDEX_IP4TXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4TXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4OutMcastOctets", - [IRDMA_HW_STAT_INDEX_IP4TXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP4TXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip4OutMcastPkts", - [IRDMA_HW_STAT_INDEX_IP6RXOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6RXOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6InOctets", - [IRDMA_HW_STAT_INDEX_IP6RXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6RXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6InPkts", - [IRDMA_HW_STAT_INDEX_IP6RXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6RXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6InReasmRqd", - [IRDMA_HW_STAT_INDEX_IP6RXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6RXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6InMcastOctets", - [IRDMA_HW_STAT_INDEX_IP6RXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6RXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6InMcastPkts", - [IRDMA_HW_STAT_INDEX_IP6TXOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6TXOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6OutOctets", - [IRDMA_HW_STAT_INDEX_IP6TXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6TXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6OutPkts", - [IRDMA_HW_STAT_INDEX_IP6TXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6TXFRAGS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6OutSegRqd", - [IRDMA_HW_STAT_INDEX_IP6TXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6TXMCOCTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6OutMcastOctets", - [IRDMA_HW_STAT_INDEX_IP6TXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_IP6TXMCPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "ip6OutMcastPkts", - [IRDMA_HW_STAT_INDEX_TCPRXSEGS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_TCPRXSEGS + IRDMA_HW_STAT_INDEX_MAX_32].name = "tcpInSegs", - [IRDMA_HW_STAT_INDEX_TCPTXSEG + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_TCPTXSEG + IRDMA_HW_STAT_INDEX_MAX_32].name = "tcpOutSegs", - [IRDMA_HW_STAT_INDEX_RDMARXRDS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMARXRDS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwInRdmaReads", - [IRDMA_HW_STAT_INDEX_RDMARXSNDS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMARXSNDS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwInRdmaSends", - [IRDMA_HW_STAT_INDEX_RDMARXWRS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMARXWRS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwInRdmaWrites", - [IRDMA_HW_STAT_INDEX_RDMATXRDS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMATXRDS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwOutRdmaReads", - [IRDMA_HW_STAT_INDEX_RDMATXSNDS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMATXSNDS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwOutRdmaSends", - [IRDMA_HW_STAT_INDEX_RDMATXWRS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMATXWRS + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwOutRdmaWrites", - [IRDMA_HW_STAT_INDEX_RDMAVBND + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMAVBND + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwRdmaBnd", - [IRDMA_HW_STAT_INDEX_RDMAVINV + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_RDMAVINV + IRDMA_HW_STAT_INDEX_MAX_32].name = "iwRdmaInv", - [IRDMA_HW_STAT_INDEX_UDPRXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_UDPRXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "RxUDP", - [IRDMA_HW_STAT_INDEX_UDPTXPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = + [IRDMA_HW_STAT_INDEX_UDPTXPKTS + IRDMA_HW_STAT_INDEX_MAX_32].name = "TxUDP", - [IRDMA_HW_STAT_INDEX_RXNPECNMARKEDPKTS + IRDMA_HW_STAT_INDEX_MAX_32] = - "RxECNMrkd", + [IRDMA_HW_STAT_INDEX_RXNPECNMARKEDPKTS + IRDMA_HW_STAT_INDEX_MAX_32] + .name = "RxECNMrkd", }; static void irdma_get_dev_fw_str(struct ib_device *dev, char *str) @@ -3750,10 +3750,10 @@ static struct rdma_hw_stats *irdma_alloc_hw_port_stats(struct ib_device *ibdev, IRDMA_HW_STAT_INDEX_MAX_64; unsigned long lifespan = RDMA_HW_STATS_DEFAULT_LIFESPAN; - BUILD_BUG_ON(ARRAY_SIZE(irdma_hw_stat_names) != + BUILD_BUG_ON(ARRAY_SIZE(irdma_hw_stat_descs) != (IRDMA_HW_STAT_INDEX_MAX_32 + IRDMA_HW_STAT_INDEX_MAX_64)); - return rdma_alloc_hw_stats_struct(irdma_hw_stat_names, num_counters, + return rdma_alloc_hw_stats_struct(irdma_hw_stat_descs, num_counters, lifespan); } diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index f367f4a4abff..38742e233915 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -2105,10 +2105,10 @@ mlx4_ib_alloc_hw_device_stats(struct ib_device *ibdev) struct mlx4_ib_dev *dev = to_mdev(ibdev); struct mlx4_ib_diag_counters *diag = dev->diag_counters; - if (!diag[0].name) + if (!diag[0].descs) return NULL; - return rdma_alloc_hw_stats_struct(diag[0].name, diag[0].num_counters, + return rdma_alloc_hw_stats_struct(diag[0].descs, diag[0].num_counters, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -2118,10 +2118,10 @@ mlx4_ib_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) struct mlx4_ib_dev *dev = to_mdev(ibdev); struct mlx4_ib_diag_counters *diag = dev->diag_counters; - if (!diag[1].name) + if (!diag[1].descs) return NULL; - return rdma_alloc_hw_stats_struct(diag[1].name, diag[1].num_counters, + return rdma_alloc_hw_stats_struct(diag[1].descs, diag[1].num_counters, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -2151,10 +2151,8 @@ static int mlx4_ib_get_hw_stats(struct ib_device *ibdev, } static int __mlx4_ib_alloc_diag_counters(struct mlx4_ib_dev *ibdev, - const char ***name, - u32 **offset, - u32 *num, - bool port) + struct rdma_stat_desc **pdescs, + u32 **offset, u32 *num, bool port) { u32 num_counters; @@ -2166,46 +2164,46 @@ static int __mlx4_ib_alloc_diag_counters(struct mlx4_ib_dev *ibdev, if (!port) num_counters += ARRAY_SIZE(diag_device_only); - *name = kcalloc(num_counters, sizeof(**name), GFP_KERNEL); - if (!*name) + *pdescs = kcalloc(num_counters, sizeof(struct rdma_stat_desc), + GFP_KERNEL); + if (!*pdescs) return -ENOMEM; *offset = kcalloc(num_counters, sizeof(**offset), GFP_KERNEL); if (!*offset) - goto err_name; + goto err; *num = num_counters; return 0; -err_name: - kfree(*name); +err: + kfree(*pdescs); return -ENOMEM; } static void mlx4_ib_fill_diag_counters(struct mlx4_ib_dev *ibdev, - const char **name, - u32 *offset, - bool port) + struct rdma_stat_desc *descs, + u32 *offset, bool port) { int i; int j; for (i = 0, j = 0; i < ARRAY_SIZE(diag_basic); i++, j++) { - name[i] = diag_basic[i].name; + descs[i].name = diag_basic[i].name; offset[i] = diag_basic[i].offset; } if (ibdev->dev->caps.flags2 & MLX4_DEV_CAP_FLAG2_DIAG_PER_PORT) { for (i = 0; i < ARRAY_SIZE(diag_ext); i++, j++) { - name[j] = diag_ext[i].name; + descs[j].name = diag_ext[i].name; offset[j] = diag_ext[i].offset; } } if (!port) { for (i = 0; i < ARRAY_SIZE(diag_device_only); i++, j++) { - name[j] = diag_device_only[i].name; + descs[j].name = diag_device_only[i].name; offset[j] = diag_device_only[i].offset; } } @@ -2233,13 +2231,13 @@ static int mlx4_ib_alloc_diag_counters(struct mlx4_ib_dev *ibdev) if (i && !per_port) continue; - ret = __mlx4_ib_alloc_diag_counters(ibdev, &diag[i].name, + ret = __mlx4_ib_alloc_diag_counters(ibdev, &diag[i].descs, &diag[i].offset, &diag[i].num_counters, i); if (ret) goto err_alloc; - mlx4_ib_fill_diag_counters(ibdev, diag[i].name, + mlx4_ib_fill_diag_counters(ibdev, diag[i].descs, diag[i].offset, i); } @@ -2249,7 +2247,7 @@ static int mlx4_ib_alloc_diag_counters(struct mlx4_ib_dev *ibdev) err_alloc: if (i) { - kfree(diag[i - 1].name); + kfree(diag[i - 1].descs); kfree(diag[i - 1].offset); } @@ -2262,7 +2260,7 @@ static void mlx4_ib_diag_cleanup(struct mlx4_ib_dev *ibdev) for (i = 0; i < MLX4_DIAG_COUNTERS_TYPES; i++) { kfree(ibdev->diag_counters[i].offset); - kfree(ibdev->diag_counters[i].name); + kfree(ibdev->diag_counters[i].descs); } } diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index c60f6e9ac640..d84023b4b1b8 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -601,7 +601,7 @@ struct mlx4_ib_counters { #define MLX4_DIAG_COUNTERS_TYPES 2 struct mlx4_ib_diag_counters { - const char **name; + struct rdma_stat_desc *descs; u32 *offset; u32 num_counters; }; diff --git a/drivers/infiniband/hw/mlx5/counters.c b/drivers/infiniband/hw/mlx5/counters.c index 224ba36f2946..caa35ea14b48 100644 --- a/drivers/infiniband/hw/mlx5/counters.c +++ b/drivers/infiniband/hw/mlx5/counters.c @@ -167,7 +167,7 @@ mlx5_ib_alloc_hw_device_stats(struct ib_device *ibdev) struct mlx5_ib_dev *dev = to_mdev(ibdev); const struct mlx5_ib_counters *cnts = &dev->port[0].cnts; - return rdma_alloc_hw_stats_struct(cnts->names, + return rdma_alloc_hw_stats_struct(cnts->descs, cnts->num_q_counters + cnts->num_cong_counters + cnts->num_ext_ppcnt_counters, @@ -180,7 +180,7 @@ mlx5_ib_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) struct mlx5_ib_dev *dev = to_mdev(ibdev); const struct mlx5_ib_counters *cnts = &dev->port[port_num - 1].cnts; - return rdma_alloc_hw_stats_struct(cnts->names, + return rdma_alloc_hw_stats_struct(cnts->descs, cnts->num_q_counters + cnts->num_cong_counters + cnts->num_ext_ppcnt_counters, @@ -302,7 +302,7 @@ mlx5_ib_counter_alloc_stats(struct rdma_counter *counter) const struct mlx5_ib_counters *cnts = get_counters(dev, counter->port - 1); - return rdma_alloc_hw_stats_struct(cnts->names, + return rdma_alloc_hw_stats_struct(cnts->descs, cnts->num_q_counters + cnts->num_cong_counters + cnts->num_ext_ppcnt_counters, @@ -371,57 +371,55 @@ static int mlx5_ib_counter_unbind_qp(struct ib_qp *qp) return mlx5_ib_qp_set_counter(qp, NULL); } - static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, - const char **names, - size_t *offsets) + struct rdma_stat_desc *descs, size_t *offsets) { int i; int j = 0; for (i = 0; i < ARRAY_SIZE(basic_q_cnts); i++, j++) { - names[j] = basic_q_cnts[i].name; + descs[j].name = basic_q_cnts[i].name; offsets[j] = basic_q_cnts[i].offset; } if (MLX5_CAP_GEN(dev->mdev, out_of_seq_cnt)) { for (i = 0; i < ARRAY_SIZE(out_of_seq_q_cnts); i++, j++) { - names[j] = out_of_seq_q_cnts[i].name; + descs[j].name = out_of_seq_q_cnts[i].name; offsets[j] = out_of_seq_q_cnts[i].offset; } } if (MLX5_CAP_GEN(dev->mdev, retransmission_q_counters)) { for (i = 0; i < ARRAY_SIZE(retrans_q_cnts); i++, j++) { - names[j] = retrans_q_cnts[i].name; + descs[j].name = retrans_q_cnts[i].name; offsets[j] = retrans_q_cnts[i].offset; } } if (MLX5_CAP_GEN(dev->mdev, enhanced_error_q_counters)) { for (i = 0; i < ARRAY_SIZE(extended_err_cnts); i++, j++) { - names[j] = extended_err_cnts[i].name; + descs[j].name = extended_err_cnts[i].name; offsets[j] = extended_err_cnts[i].offset; } } if (MLX5_CAP_GEN(dev->mdev, roce_accl)) { for (i = 0; i < ARRAY_SIZE(roce_accl_cnts); i++, j++) { - names[j] = roce_accl_cnts[i].name; + descs[j].name = roce_accl_cnts[i].name; offsets[j] = roce_accl_cnts[i].offset; } } if (MLX5_CAP_GEN(dev->mdev, cc_query_allowed)) { for (i = 0; i < ARRAY_SIZE(cong_cnts); i++, j++) { - names[j] = cong_cnts[i].name; + descs[j].name = cong_cnts[i].name; offsets[j] = cong_cnts[i].offset; } } if (MLX5_CAP_PCAM_FEATURE(dev->mdev, rx_icrc_encapsulated_counter)) { for (i = 0; i < ARRAY_SIZE(ext_ppcnt_cnts); i++, j++) { - names[j] = ext_ppcnt_cnts[i].name; + descs[j].name = ext_ppcnt_cnts[i].name; offsets[j] = ext_ppcnt_cnts[i].offset; } } @@ -457,20 +455,21 @@ static int __mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev, cnts->num_ext_ppcnt_counters = ARRAY_SIZE(ext_ppcnt_cnts); num_counters += ARRAY_SIZE(ext_ppcnt_cnts); } - cnts->names = kcalloc(num_counters, sizeof(*cnts->names), GFP_KERNEL); - if (!cnts->names) + cnts->descs = kcalloc(num_counters, + sizeof(struct rdma_stat_desc), GFP_KERNEL); + if (!cnts->descs) return -ENOMEM; cnts->offsets = kcalloc(num_counters, sizeof(*cnts->offsets), GFP_KERNEL); if (!cnts->offsets) - goto err_names; + goto err; return 0; -err_names: - kfree(cnts->names); - cnts->names = NULL; +err: + kfree(cnts->descs); + cnts->descs = NULL; return -ENOMEM; } @@ -491,7 +490,7 @@ static void mlx5_ib_dealloc_counters(struct mlx5_ib_dev *dev) dev->port[i].cnts.set_id); mlx5_cmd_exec_in(dev->mdev, dealloc_q_counter, in); } - kfree(dev->port[i].cnts.names); + kfree(dev->port[i].cnts.descs); kfree(dev->port[i].cnts.offsets); } } @@ -514,7 +513,7 @@ static int mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev) if (err) goto err_alloc; - mlx5_ib_fill_counters(dev, dev->port[i].cnts.names, + mlx5_ib_fill_counters(dev, dev->port[i].cnts.descs, dev->port[i].cnts.offsets); MLX5_SET(alloc_q_counter_in, in, uid, diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index bf20a388eabe..6f5451d96dd7 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -798,7 +798,7 @@ struct mlx5_ib_resources { }; struct mlx5_ib_counters { - const char **names; + struct rdma_stat_desc *descs; size_t *offsets; u32 num_q_counters; u32 num_cong_counters; diff --git a/drivers/infiniband/sw/rxe/rxe_hw_counters.c b/drivers/infiniband/sw/rxe/rxe_hw_counters.c index d5ceb706d964..a012522b577a 100644 --- a/drivers/infiniband/sw/rxe/rxe_hw_counters.c +++ b/drivers/infiniband/sw/rxe/rxe_hw_counters.c @@ -6,22 +6,22 @@ #include "rxe.h" #include "rxe_hw_counters.h" -static const char * const rxe_counter_name[] = { - [RXE_CNT_SENT_PKTS] = "sent_pkts", - [RXE_CNT_RCVD_PKTS] = "rcvd_pkts", - [RXE_CNT_DUP_REQ] = "duplicate_request", - [RXE_CNT_OUT_OF_SEQ_REQ] = "out_of_seq_request", - [RXE_CNT_RCV_RNR] = "rcvd_rnr_err", - [RXE_CNT_SND_RNR] = "send_rnr_err", - [RXE_CNT_RCV_SEQ_ERR] = "rcvd_seq_err", - [RXE_CNT_COMPLETER_SCHED] = "ack_deferred", - [RXE_CNT_RETRY_EXCEEDED] = "retry_exceeded_err", - [RXE_CNT_RNR_RETRY_EXCEEDED] = "retry_rnr_exceeded_err", - [RXE_CNT_COMP_RETRY] = "completer_retry_err", - [RXE_CNT_SEND_ERR] = "send_err", - [RXE_CNT_LINK_DOWNED] = "link_downed", - [RXE_CNT_RDMA_SEND] = "rdma_sends", - [RXE_CNT_RDMA_RECV] = "rdma_recvs", +static const struct rdma_stat_desc rxe_counter_descs[] = { + [RXE_CNT_SENT_PKTS].name = "sent_pkts", + [RXE_CNT_RCVD_PKTS].name = "rcvd_pkts", + [RXE_CNT_DUP_REQ].name = "duplicate_request", + [RXE_CNT_OUT_OF_SEQ_REQ].name = "out_of_seq_request", + [RXE_CNT_RCV_RNR].name = "rcvd_rnr_err", + [RXE_CNT_SND_RNR].name = "send_rnr_err", + [RXE_CNT_RCV_SEQ_ERR].name = "rcvd_seq_err", + [RXE_CNT_COMPLETER_SCHED].name = "ack_deferred", + [RXE_CNT_RETRY_EXCEEDED].name = "retry_exceeded_err", + [RXE_CNT_RNR_RETRY_EXCEEDED].name = "retry_rnr_exceeded_err", + [RXE_CNT_COMP_RETRY].name = "completer_retry_err", + [RXE_CNT_SEND_ERR].name = "send_err", + [RXE_CNT_LINK_DOWNED].name = "link_downed", + [RXE_CNT_RDMA_SEND].name = "rdma_sends", + [RXE_CNT_RDMA_RECV].name = "rdma_recvs", }; int rxe_ib_get_hw_stats(struct ib_device *ibdev, @@ -34,18 +34,18 @@ int rxe_ib_get_hw_stats(struct ib_device *ibdev, if (!port || !stats) return -EINVAL; - for (cnt = 0; cnt < ARRAY_SIZE(rxe_counter_name); cnt++) + for (cnt = 0; cnt < ARRAY_SIZE(rxe_counter_descs); cnt++) stats->value[cnt] = atomic64_read(&dev->stats_counters[cnt]); - return ARRAY_SIZE(rxe_counter_name); + return ARRAY_SIZE(rxe_counter_descs); } struct rdma_hw_stats *rxe_ib_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) { - BUILD_BUG_ON(ARRAY_SIZE(rxe_counter_name) != RXE_NUM_OF_COUNTERS); + BUILD_BUG_ON(ARRAY_SIZE(rxe_counter_descs) != RXE_NUM_OF_COUNTERS); - return rdma_alloc_hw_stats_struct(rxe_counter_name, - ARRAY_SIZE(rxe_counter_name), + return rdma_alloc_hw_stats_struct(rxe_counter_descs, + ARRAY_SIZE(rxe_counter_descs), RDMA_HW_STATS_DEFAULT_LIFESPAN); } diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 4b50d9a3018a..aa1e1029b736 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -545,6 +545,14 @@ enum ib_port_speed { IB_SPEED_NDR = 128, }; +/** + * struct rdma_stat_desc + * @name - The name of the counter + */ +struct rdma_stat_desc { + const char *name; +}; + /** * struct rdma_hw_stats * @lock - Mutex to protect parallel write access to lifespan and values @@ -555,8 +563,8 @@ enum ib_port_speed { * should be before being updated again. Stored in jiffies, defaults * to 10 milliseconds, drivers can override the default be specifying * their own value during their allocation routine. - * @name - Array of pointers to static names used for the counters in - * directory. + * @descs - Array of pointers to static descriptors used for the counters + * in directory. * @num_counters - How many hardware counters there are. If name is * shorter than this number, a kernel oops will result. Driver authors * are encouraged to leave BUILD_BUG_ON(ARRAY_SIZE(@name) < num_counters) @@ -568,7 +576,7 @@ struct rdma_hw_stats { struct mutex lock; /* Protect lifespan and values[] */ unsigned long timestamp; unsigned long lifespan; - const char * const *names; + const struct rdma_stat_desc *descs; int num_counters; u64 value[]; }; @@ -577,12 +585,12 @@ struct rdma_hw_stats { /** * rdma_alloc_hw_stats_struct - Helper function to allocate dynamic struct * for drivers. - * @names - Array of static const char * + * @descs - Array of static descriptors * @num_counters - How many elements in array * @lifespan - How many milliseconds between updates */ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct( - const char * const *names, int num_counters, + const struct rdma_stat_desc *descs, int num_counters, unsigned long lifespan) { struct rdma_hw_stats *stats; @@ -591,7 +599,8 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct( GFP_KERNEL); if (!stats) return NULL; - stats->names = names; + + stats->descs = descs; stats->num_counters = num_counters; stats->lifespan = msecs_to_jiffies(lifespan); From patchwork Fri Oct 8 12:24:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545197 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA1CEC433EF for ; Fri, 8 Oct 2021 12:26:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ACAC560F44 for ; Fri, 8 Oct 2021 12:26:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240861AbhJHM2S (ORCPT ); Fri, 8 Oct 2021 08:28:18 -0400 Received: from mail-bn8nam12on2047.outbound.protection.outlook.com ([40.107.237.47]:13537 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241534AbhJHM2I (ORCPT ); Fri, 8 Oct 2021 08:28:08 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K7dpu2iQlsoETyHOm5IvpvEirsGKzsMPDQOyjcozEkL8YvwPMDFsVdugvmGnp5mn834l6M1/IMy2MS/JrAfFLBokeMwxFKOy7lO1yeoTij+BCkmcO4XvTO6Lt/Jp/4/2xS6mHXNCRQ34XOHN8ovao8fCrphkHMFRqyNWhSzN9mq9q0KW+Xeo61gz3gZddqtstPsjeAlAj1K8gJaKSjBr+pEJJlfIiKyVNuWQ6ohUFG1IL+DoNGpfACZEp9F+T412K7wLEeqZePzW63RCtA3nZI/0/NXbZu340I8WxtKdOrQBAThAmrAeLbGw0+bPwDMZw9ybR/L+nYobvBgQ9uvDlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BghE2GqHU7Hb7GZa4ODZP82j76YI+rGy+6vXlPLUJjc=; b=lzzX7+8GbUT9/fpc53NbZArgOzgEFKgG4uEmdqVvCTBB8IcpOCrssvIN+OVIU14OUcJLgGCtrjne61nUBBBs8JZqpwG9G9Zuz6tYekNxIWFOswZrU6NNudUqzuIydpIIILYJZNAiHIPU+OAAQyl8ZM2NlI4mw8RXOAFT7qARnb+uA4fTp9I5S6m4j7iNGjTSFC0zcIZUrgIaDvn9jbZrsz3ogfoF8umRjqg8SKkk6ya9Zfzo2YXN0aUJV1ExYS0F5XN3cAACebzLz78JvNiLwzUPIwPXQhrq90xp8oRIwnGi6KreH3OIbrQ3MVg+PKPoR8Tyrq0HCE4IrA876KgLBw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BghE2GqHU7Hb7GZa4ODZP82j76YI+rGy+6vXlPLUJjc=; b=kRXZBprzHbTrlpaZ2TJjSxQLVexDwDcM2cq8mVIoErQKkXKlAUthWfKSH/NlzRWPFTo6d+KeHOk4Bn0iFvcYff4undUDXxYG12miXzx0tM0WXGx6lMUVgcZRITc3rJ0QwHbU3xgYdbnyQs5xyZa6LbdwEVgfeKqSBsl0rhJ0nRuI/Lutv0VGd29dWrDBdwV/NF7dL07AtKqNo9LDR1xwHw7ebqLHO89W/v9jwlwja9ef9YcaT60AxFGVwtdqPzSbIXOdqKdsOqyBod9Li40HIcruu1nMDHYJmEzRmgiKNX7GSz+7rUfadsMhH73deWBrsdp3DTWNR2Scg+7I1cG11w== Received: from BN9PR03CA0801.namprd03.prod.outlook.com (2603:10b6:408:13f::26) by MN2PR12MB4344.namprd12.prod.outlook.com (2603:10b6:208:26e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18; Fri, 8 Oct 2021 12:26:11 +0000 Received: from BN8NAM11FT049.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13f:cafe::b2) by BN9PR03CA0801.outlook.office365.com (2603:10b6:408:13f::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.22 via Frontend Transport; Fri, 8 Oct 2021 12:26:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT049.mail.protection.outlook.com (10.13.177.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:10 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:09 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:04 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 04/13] RDMA/core: Add a helper API rdma_free_hw_stats_struct Date: Fri, 8 Oct 2021 15:24:30 +0300 Message-ID: <20211008122439.166063-5-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ba26bd6b-01ad-4080-556e-08d98a56cfc6 X-MS-TrafficTypeDiagnostic: MN2PR12MB4344: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dx4rhDu2WzF705NGhP2eis0IDlfiLEAL8EpfWYfCKmVhrtau9ZL5dSh2v9RSvQJdz2lBMO6bpMY7zOZuwK5XhcnQ90vSP/7l3LKzjCFLzSUPur/j2zNIpEJN0ngyl/YMfqrZ4bc9zJbmKSpmQwD0mvTI9mXFyRTzacbfmarD1XcdOqQyH8xus62ZEGPI0/hH9Ap+8DA7baNxdk7fJ+tKQGzBGCwf70ANZ1/mBYJvoT8CUSIGgRUYBX8LMnZfkWWLY019umhxeKoMm6uIjcm0mokYQU4WE88tok+Q9n3vWWMj0AV84LJJzR4pSMoRzETVXtchthkfnDLzGUXC1BfYj+P2z17kdhZVo+aX5JVCZlv7Js7d2A9aWqtg3SnHTuZZ0RJYqGf3MrqssHwVF0wgeuDv9nMe7CZ6G2E1KrXBaiIO2xWDMp890Ve7RsWT9CghQ2/0zqfptBjqeTl8Sl7aq5pIikzNeQ5mHPj9euKpjH4FEQeczLbczHI/LdsB2VrBo+HUbnMcigxCcBTxyTEngJ9h9DSwUowCPwe7HP5UQY193M9P+gWGS6qGuuxS2Z1JP4Y0rBhlr9Hc53DLCP6OjG+r4MVQaRyrWIYnaxAHFZTVrErLl24G/svHQjtoJxO3GHXC/1YRJ1JFX7M5Cbb8mZccjK8C5y/uaAX5p05/gwzuqzK/DGwA3uj56xxnqIIjX9Vij81Pt0z34xyGwZnILw== X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid02.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(107886003)(82310400003)(8936002)(316002)(54906003)(110136005)(36860700001)(70206006)(6666004)(36756003)(26005)(1076003)(8676002)(186003)(5660300002)(2616005)(426003)(336012)(70586007)(47076005)(508600001)(7416002)(356005)(7636003)(7696005)(2906002)(4326008)(6636002)(83380400001)(86362001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:10.5039 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ba26bd6b-01ad-4080-556e-08d98a56cfc6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT049.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4344 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Add a new API rdma_free_hw_stats_struct to pair with rdma_alloc_hw_stats_struct (which is also de-inlined). This will be useful when there are more alloc/free works in following patches. Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/counters.c | 8 +++---- drivers/infiniband/core/sysfs.c | 8 +++---- drivers/infiniband/core/verbs.c | 35 ++++++++++++++++++++++++++++++ include/rdma/ib_verbs.h | 27 ++++------------------- 4 files changed, 47 insertions(+), 31 deletions(-) diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c index df9e6c5e4ddf..331cd29f0d61 100644 --- a/drivers/infiniband/core/counters.c +++ b/drivers/infiniband/core/counters.c @@ -165,7 +165,7 @@ static struct rdma_counter *alloc_and_bind(struct ib_device *dev, u32 port, return counter; err_mode: - kfree(counter->stats); + rdma_free_hw_stats_struct(counter->stats); err_stats: rdma_restrack_put(&counter->res); kfree(counter); @@ -186,7 +186,7 @@ static void rdma_counter_free(struct rdma_counter *counter) mutex_unlock(&port_counter->lock); rdma_restrack_del(&counter->res); - kfree(counter->stats); + rdma_free_hw_stats_struct(counter->stats); kfree(counter); } @@ -618,7 +618,7 @@ void rdma_counter_init(struct ib_device *dev) fail: for (i = port; i >= rdma_start_port(dev); i--) { port_counter = &dev->port_data[port].port_counter; - kfree(port_counter->hstats); + rdma_free_hw_stats_struct(port_counter->hstats); port_counter->hstats = NULL; mutex_destroy(&port_counter->lock); } @@ -631,7 +631,7 @@ void rdma_counter_release(struct ib_device *dev) rdma_for_each_port(dev, port) { port_counter = &dev->port_data[port].port_counter; - kfree(port_counter->hstats); + rdma_free_hw_stats_struct(port_counter->hstats); mutex_destroy(&port_counter->lock); } } diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c index c3663cfdcd52..8d831d4fd2ad 100644 --- a/drivers/infiniband/core/sysfs.c +++ b/drivers/infiniband/core/sysfs.c @@ -755,7 +755,7 @@ static void ib_port_release(struct kobject *kobj) for (i = 0; i != ARRAY_SIZE(port->groups); i++) kfree(port->groups[i].attrs); if (port->hw_stats_data) - kfree(port->hw_stats_data->stats); + rdma_free_hw_stats_struct(port->hw_stats_data->stats); kfree(port->hw_stats_data); kfree(port); } @@ -919,14 +919,14 @@ alloc_hw_stats_device(struct ib_device *ibdev) err_free_data: kfree(data); err_free_stats: - kfree(stats); + rdma_free_hw_stats_struct(stats); return ERR_PTR(-ENOMEM); } void ib_device_release_hw_stats(struct hw_stats_device_data *data) { kfree(data->group.attrs); - kfree(data->stats); + rdma_free_hw_stats_struct(data->stats); kfree(data); } @@ -1018,7 +1018,7 @@ alloc_hw_stats_port(struct ib_port *port, struct attribute_group *group) err_free_data: kfree(data); err_free_stats: - kfree(stats); + rdma_free_hw_stats_struct(stats); return ERR_PTR(-ENOMEM); } diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 89a2b21976d6..8e72290d6b38 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -2976,3 +2976,38 @@ bool __rdma_block_iter_next(struct ib_block_iter *biter) return true; } EXPORT_SYMBOL(__rdma_block_iter_next); + +/** + * rdma_alloc_hw_stats_struct - Helper function to allocate dynamic struct + * for the drivers. + * @descs: array of static descriptors + * @num_counters: number of elements in array + * @lifespan: milliseconds between updates + */ +struct rdma_hw_stats *rdma_alloc_hw_stats_struct( + const struct rdma_stat_desc *descs, int num_counters, + unsigned long lifespan) +{ + struct rdma_hw_stats *stats; + + stats = kzalloc(struct_size(stats, value, num_counters), GFP_KERNEL); + if (!stats) + return NULL; + + stats->descs = descs; + stats->num_counters = num_counters; + stats->lifespan = msecs_to_jiffies(lifespan); + + return stats; +} +EXPORT_SYMBOL(rdma_alloc_hw_stats_struct); + +/** + * rdma_free_hw_stats_struct - Helper function to release rdma_hw_stats + * @stats: statistics to release + */ +void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats) +{ + kfree(stats); +} +EXPORT_SYMBOL(rdma_free_hw_stats_struct); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index aa1e1029b736..938c0c0a1c19 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -582,31 +582,12 @@ struct rdma_hw_stats { }; #define RDMA_HW_STATS_DEFAULT_LIFESPAN 10 -/** - * rdma_alloc_hw_stats_struct - Helper function to allocate dynamic struct - * for drivers. - * @descs - Array of static descriptors - * @num_counters - How many elements in array - * @lifespan - How many milliseconds between updates - */ -static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct( - const struct rdma_stat_desc *descs, int num_counters, - unsigned long lifespan) -{ - struct rdma_hw_stats *stats; - stats = kzalloc(sizeof(*stats) + num_counters * sizeof(u64), - GFP_KERNEL); - if (!stats) - return NULL; - - stats->descs = descs; - stats->num_counters = num_counters; - stats->lifespan = msecs_to_jiffies(lifespan); - - return stats; -} +struct rdma_hw_stats *rdma_alloc_hw_stats_struct( + const struct rdma_stat_desc *descs, int num_counters, + unsigned long lifespan); +void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats); /* Define bits for the various functionality this port needs to be supported by * the core. From patchwork Fri Oct 8 12:24:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545201 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CF98C433FE for ; Fri, 8 Oct 2021 12:26:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 56D7560FDC for ; Fri, 8 Oct 2021 12:26:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241498AbhJHM2U (ORCPT ); Fri, 8 Oct 2021 08:28:20 -0400 Received: from mail-bn7nam10on2081.outbound.protection.outlook.com ([40.107.92.81]:38688 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241582AbhJHM2O (ORCPT ); Fri, 8 Oct 2021 08:28:14 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jPBGrkkOJNvCTRZ/Bq/mSm3QWHHQWmxkRFKr7McY7xALHmVze5tKVh4w7e1gDTXGkfwRH2ItX2zIJTpXm8P1dRiO1A64Nh2+AO+tWjN1oGQxAp42tboeqLxv5GPCLtcl2IjLkYar59GYAMQPw1NmlT+KbtPsI9rzumF63RM0z5fjxpDSI9n33sJFD3bsfuI/v5UeXH2Xb/BPsdEaqNLe1D7MJlnAU9IdFkaaV2WLNqg+cR6v6ep4dSyw2KWJo56I0hJS16ZqLq+V5xTzloNzrmdWSJcSu0/OkwNjlB/Hc9YCRm69JAuxwjz/jYHyciMOL0LLMpm35x8FrgWb/7D4MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4dK3GIoiKN0D5DHbI88X0iH3TPKz45yTldo9SrMkQhs=; b=JTpN7Ybfv2kjhgrrQIjwE9wO7rv8u55NDtzIMUxVw4DC0qpPz4aCP5wWh/2Xg22vhyQpBya0KsYLrVJCgivZlLhPOkeC5HfFB9oM9TwVMSiIhRszS0IOrcb7GyIGgCwKaRk/ysCwKZMeLW+yQp7ivKf+pH/BTWnxuje4sZtTlC1Liu05GUALjrVXQKhitu7YZsAs7gkfLAh2zwOoiiLMkcjGu/rFG4j/f+2F5+BrRPh9NH6t1t5Pkh1aznRES9xY6qLTyxVGzXUluFL+GxIP45khRBASCFd8aVniI1beQXC1Bga5rTEzhyRvhEEsokyOnzXWMQaQ/H7o4bY3EekU3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4dK3GIoiKN0D5DHbI88X0iH3TPKz45yTldo9SrMkQhs=; b=TFvvu9yACXwGbC5QeRy6axjcVhXFsJzeEQ5Fawn09fehky+1+xmVdl6+uB5ykEgd40ILAoqkrfsM3X/66DD96DI19kPoyspn/SbmgQmKacEa6TYPkAIbjqzl7Zd8YGQnuBmTPn9VHi9H75Hjnhb+sza5jSTjX95b9C9OxNb1TZQ7VSlbsVNoqNIwAnJ1E1EECo77xaN8UlcrB3LwsmCu79UVCARnoHB2MiJ0P+10z9E94TvxP4KgfknXukcbb4eQnlOfhz6GusWSh1BiNKgGtunRG81nW8aQAywPanWkTlNRLwE6qvrK6TFbFSek3VHDTXM04XgORayB3AGrdjzjbQ== Received: from BN6PR19CA0073.namprd19.prod.outlook.com (2603:10b6:404:133::11) by MN2PR12MB3006.namprd12.prod.outlook.com (2603:10b6:208:cd::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.19; Fri, 8 Oct 2021 12:26:15 +0000 Received: from BN8NAM11FT052.eop-nam11.prod.protection.outlook.com (2603:10b6:404:133:cafe::b7) by BN6PR19CA0073.outlook.office365.com (2603:10b6:404:133::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.16 via Frontend Transport; Fri, 8 Oct 2021 12:26:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT052.mail.protection.outlook.com (10.13.177.210) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:14 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:13 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:09 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 05/13] RDMA/counter: Add an is_disabled field in struct rdma_hw_stats Date: Fri, 8 Oct 2021 15:24:31 +0300 Message-ID: <20211008122439.166063-6-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1c08853f-1bbc-4c4f-5825-08d98a56d265 X-MS-TrafficTypeDiagnostic: MN2PR12MB3006: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GeAdQKOudAZxVbphak8ZcxZWnDcW5PULcVA0GHg5YJ1WmUCCh1Th4xrpJl9zwIyjwHZry2EVFGnptMoPxF7ip/9r+48F102jjWtT9QK9P7XHVL2SydbJOClsu3WXK6KLo/ua9UzXg+YBRyr5JMDwLtm/VNGP85cAaSTTI6nHtNT8C2YlB7qs+9UKmfsWb1NVsY2NwbBr0Bc50A1Yyvd5z2QBT8Opnm2XKfMeVqCkWd9u09v2+IqmR6JVPBGC1qHq5sFmuvAEdtS/3Bc/8RM32KYQ1MsXKyjGufloli9dwl5VXM5epY5WXS1jxl11qJTLvPvfGW53lT4vhM++ui2sf00Gqj4eoqBfDrSI8BKaIFcUt1dO0BmsIDtakf9GqE3nOuhSG1PocAufJLZ1K/w7bIGlrHJmxJQJe/fm5WfPNljx4faorKc+e7+2wfd1WKUqSSlt0EqAIopbZlBRoFWjmipxffIpxF6auIdY9aXapfTw0xlPZZxyneg9aqygsVbR4upEynwh8j3LYn5QUG86mXeTFIkpylrdAYPahqqarFPI1v7Q6lNqLJY1JE2bu/6OkNZCCXLXrchEBdhWKCAhjs2KPUUV/lIKEOOUgAcbGLCHxHl7zf++Sg6HATtpIILNtiRlJ4zO2A5maLsyi2fzSEbsJcmgl+4krVGVUzDut9O9+zYdK75+UItAEUN8mUJQpLDgEaHQ++kb9Qzm0atqzQ== X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid04.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(8936002)(6636002)(47076005)(8676002)(4326008)(83380400001)(508600001)(36756003)(107886003)(5660300002)(336012)(426003)(86362001)(2616005)(26005)(7636003)(186003)(70206006)(2906002)(7696005)(356005)(36860700001)(110136005)(6666004)(7416002)(70586007)(54906003)(1076003)(316002)(82310400003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:14.9774 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1c08853f-1bbc-4c4f-5825-08d98a56d265 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT052.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3006 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Add a bitmap in rdma_hw_stat structure, with each bit indicates whether the corresponding counter is currently disabled or not. By default hwcounters are enabled. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/nldev.c | 11 ++++++++++- drivers/infiniband/core/verbs.c | 13 +++++++++++++ include/rdma/ib_verbs.h | 3 +++ 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 3f6b98a87566..67519730b1ac 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -968,15 +968,21 @@ static int fill_stat_counter_hwcounters(struct sk_buff *msg, if (!table_attr) return -EMSGSIZE; - for (i = 0; i < st->num_counters; i++) + mutex_lock(&st->lock); + for (i = 0; i < st->num_counters; i++) { + if (test_bit(i, st->is_disabled)) + continue; if (rdma_nl_stat_hwcounter_entry(msg, st->descs[i].name, st->value[i])) goto err; + } + mutex_unlock(&st->lock); nla_nest_end(msg, table_attr); return 0; err: + mutex_unlock(&st->lock); nla_nest_cancel(msg, table_attr); return -EMSGSIZE; } @@ -2104,6 +2110,9 @@ static int stat_get_doit_default_counter(struct sk_buff *skb, goto err_stats; } for (i = 0; i < num_cnts; i++) { + if (test_bit(i, stats->is_disabled)) + continue; + v = stats->value[i] + rdma_counter_get_hwstat_value(device, port, i); if (rdma_nl_stat_hwcounter_entry(msg, diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 8e72290d6b38..47cf273d0678 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -2994,11 +2994,20 @@ struct rdma_hw_stats *rdma_alloc_hw_stats_struct( if (!stats) return NULL; + stats->is_disabled = kcalloc(BITS_TO_LONGS(num_counters), + sizeof(*stats->is_disabled), GFP_KERNEL); + if (!stats->is_disabled) + goto err; + stats->descs = descs; stats->num_counters = num_counters; stats->lifespan = msecs_to_jiffies(lifespan); return stats; + +err: + kfree(stats); + return NULL; } EXPORT_SYMBOL(rdma_alloc_hw_stats_struct); @@ -3008,6 +3017,10 @@ EXPORT_SYMBOL(rdma_alloc_hw_stats_struct); */ void rdma_free_hw_stats_struct(struct rdma_hw_stats *stats) { + if (!stats) + return; + + kfree(stats->is_disabled); kfree(stats); } EXPORT_SYMBOL(rdma_free_hw_stats_struct); diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 938c0c0a1c19..ae467365706b 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -565,6 +565,8 @@ struct rdma_stat_desc { * their own value during their allocation routine. * @descs - Array of pointers to static descriptors used for the counters * in directory. + * @is_disabled - A bitmap to indicate each counter is currently disabled + * or not. * @num_counters - How many hardware counters there are. If name is * shorter than this number, a kernel oops will result. Driver authors * are encouraged to leave BUILD_BUG_ON(ARRAY_SIZE(@name) < num_counters) @@ -577,6 +579,7 @@ struct rdma_hw_stats { unsigned long timestamp; unsigned long lifespan; const struct rdma_stat_desc *descs; + unsigned long *is_disabled; int num_counters; u64 value[]; }; From patchwork Fri Oct 8 12:24:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545203 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 682B8C4332F for ; Fri, 8 Oct 2021 12:26:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C35560F44 for ; Fri, 8 Oct 2021 12:26:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241522AbhJHM2X (ORCPT ); Fri, 8 Oct 2021 08:28:23 -0400 Received: from mail-bn8nam11on2083.outbound.protection.outlook.com ([40.107.236.83]:27872 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241608AbhJHM2R (ORCPT ); Fri, 8 Oct 2021 08:28:17 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bfoSl2rTF6swrgB6YgnfUeuEF/6/1EnW7tB6Slz77vP4jrEoj7zly+njyAJF6aywbIdjIVTTT3jLq3eh3k8sXotyfKPLvN7lZpfwFRYnFQZ/HddwTjgn9HLNoQzVpRF5C2z6wM5lzmELTQit14ose5UbXpAS8bo1gZ8SJt6FZmPdTYMp5xQbpBoAG2dhyn/ZwQPwpzF1z37Qfrbgc41SGQMLDKZZK0dIv62lis07MDtW1B1oF3hz6wjBRcxQTgPldR8d4AU7JH47kkHyQDoX4U99GiGl1FnkNVvaeCMroA9vZ51FeYl19eJLbdpaRfXPRVp3XzUJW6MFJ1WVpl6vHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gmJzmkKWylwu80KmSS+CbRyZubq8LY5B1209kUR/HFo=; b=f7U6/c8vu86mDu/xc5bKZZyn99jVmJNHc21xmgsw8hkE1ca+A4OAZB8bfY2n/cU64cZOXmWN4vU7OWCUAzDDW+21TWhM4kUsq+RGFP53f17saz+N1aqjkAwiYFXguS43u1LzmwXfRpnUSu6OgnXYJlduPuqU+LUt6qrVRUNloMK60uhVw90WAh1Mr5dopJwxGsUD/lWyWSNEMVGwpLZ7s8gpACNpWZIDdb1Vd5BvR9Y1FphzS+LvimYESAQ5axE/LKsdrErDfeog5hTS6PiMeXfVmqWoZHc8t/ReBnZK3MI0S0pxSBhTa15SLY1exkt/va1JdrVL+lB0tfbNPU2N6g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gmJzmkKWylwu80KmSS+CbRyZubq8LY5B1209kUR/HFo=; b=gmrdRi5LMWJ9W6z7wcUggdLq9FN4d90IK53+01TDwy567zez3A8jbv197s+pL06njadJ+sKJLHE5p7ay6hC1rrA/Jq+S1YEFUkTBypZfwPHsBwTYBdy1L2063U/+u4ISoYynkpUrJ9OWW2scCaoxOhDIQM0V4RdyYl0MMsNmnlRpDJ7SopXyMg8A8PVSHmugyyHStRTDT/Q1/U+39C/e9pgdfHQKI0z9xzGVhxKkROR/CBBvUMHm217lm0rCQyyXC90jPK1ZC2Awgoe2tGLb/jBygH1j7VivQlgfG/CzkByhc5UruD5UjolKMJzrFDWYA/iVvhAy6//eEu2nlQF5mg== Received: from MWHPR1701CA0017.namprd17.prod.outlook.com (2603:10b6:301:14::27) by MWHPR1201MB2558.namprd12.prod.outlook.com (2603:10b6:300:e5::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.19; Fri, 8 Oct 2021 12:26:19 +0000 Received: from CO1NAM11FT066.eop-nam11.prod.protection.outlook.com (2603:10b6:301:14:cafe::78) by MWHPR1701CA0017.outlook.office365.com (2603:10b6:301:14::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:19 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by CO1NAM11FT066.mail.protection.outlook.com (10.13.175.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:19 +0000 Received: from HQMAIL105.nvidia.com (172.20.187.12) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:18 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:18 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:14 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 06/13] RDMA/counter: Add optional counter support Date: Fri, 8 Oct 2021 15:24:32 +0300 Message-ID: <20211008122439.166063-7-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a94b8833-bca4-472f-0200-08d98a56d4f4 X-MS-TrafficTypeDiagnostic: MWHPR1201MB2558: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wA9ogn/dfgt2z67fLhsnIE1AHAhNAQ3bHC72OSZl1QC2/UE/vqpqLOhf26lZ+DOHS8Pc5qVIsHnNNCepX8FM3yhy7cvjbLkpbcSTdWyU9YpdbpcNriR+MaS+if2MkVr8akuDUUiFasqulZfL3Ivqpz2qkkPMns7ttWyGMNdO2EZSnlpWKS9Qbivza0Nb44k6URbkH9qAO8+8h0WATJywAeh5GjVdwSTmjOojtKAHfyIujcWEGy3iSMP6U6TgMI2sPyUTTNNyhjTOlBTh+FzmA+elfnoHccUjsQuHFxWM4N683jrXS458pB5dDfZw++OEzjHE4NhKNSF+2DgpkeK09X+rStdkmlY5iWgldYbBE6V1O37uZG/1Os41gq/xPEOtt1fIM4O4/Wvm6miriuhzXQL8JF0GYYNR4RyTNuzs5n5UUmqXQLNHc0Pl/dZcqwknjXWcntAS4uUFd8DdfEjhKV1umrhGl78EUAwa3QIqrsOljHd2YndJ7z3+ClD9fA5b+1K0Pp7EyaKr2onviEytDCcbIcVC1nevCWnoCQSgEV4DV1RRt/byqzCD0Xgy8dsfJZP7IzJ5f3zN4/5ubJiOf3BgOwOc4gGu3XERtJiaIN/lCuOlkdIiko6VW4PYzDa+sI8eW5PaV+xk9oWlFJSbZoqWVlEj8zPKw2crDEHvlXa+7kPyJVKYOKSUFN0xjQuMfDybt0sIwPICgB9Pwk6Qew== X-Forefront-Antispam-Report: CIP:216.228.112.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid05.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(26005)(82310400003)(7696005)(86362001)(5660300002)(36860700001)(70586007)(70206006)(8676002)(4326008)(36906005)(316002)(1076003)(83380400001)(7416002)(7636003)(6636002)(2616005)(110136005)(54906003)(47076005)(36756003)(508600001)(186003)(2906002)(8936002)(356005)(107886003)(426003)(336012)(6666004);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:19.3234 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a94b8833-bca4-472f-0200-08d98a56d4f4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.36];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT066.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB2558 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau An optional counter is a vendor-specific counter that may be dynamically enabled/disabled. This enhancement allows us to expose counters which are for example mutual exclusive and cannot be enabled at the same time, counters that might degrades performance, optional debug counters, etc. Optional counters are marked with IB_STAT_FLAG_OPTIONAL flag. They are not exported in sysfs, and must be at the end of all stats, otherwise the attr->show() in sysfs would get wrong indexes for hwcounters that are behind optional counters. Signed-off-by: Aharon Landau Signed-off-by: Neta Ostrovsky Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/counters.c | 32 ++++++++++++++++++++++++++ drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/sysfs.c | 36 +++++++++++++++++++++--------- include/rdma/ib_verbs.h | 13 +++++++++++ include/rdma/rdma_counter.h | 2 ++ 5 files changed, 74 insertions(+), 10 deletions(-) diff --git a/drivers/infiniband/core/counters.c b/drivers/infiniband/core/counters.c index 331cd29f0d61..af59486fe418 100644 --- a/drivers/infiniband/core/counters.c +++ b/drivers/infiniband/core/counters.c @@ -106,6 +106,38 @@ static int __rdma_counter_bind_qp(struct rdma_counter *counter, return ret; } +int rdma_counter_modify(struct ib_device *dev, u32 port, + unsigned int index, bool enable) +{ + struct rdma_hw_stats *stats; + int ret = 0; + + if (!dev->ops.modify_hw_stat) + return -EOPNOTSUPP; + + stats = ib_get_hw_stats_port(dev, port); + if (!stats || index >= stats->num_counters || + !(stats->descs[index].flags & IB_STAT_FLAG_OPTIONAL)) + return -EINVAL; + + mutex_lock(&stats->lock); + + if (enable != test_bit(index, stats->is_disabled)) + goto out; + + ret = dev->ops.modify_hw_stat(dev, port, index, enable); + if (ret) + goto out; + + if (enable) + clear_bit(index, stats->is_disabled); + else + set_bit(index, stats->is_disabled); +out: + mutex_unlock(&stats->lock); + return ret; +} + static struct rdma_counter *alloc_and_bind(struct ib_device *dev, u32 port, struct ib_qp *qp, enum rdma_nl_counter_mode mode) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index f4814bb7f082..22a4adda7981 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -2676,6 +2676,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) SET_DEVICE_OP(dev_ops, modify_cq); SET_DEVICE_OP(dev_ops, modify_device); SET_DEVICE_OP(dev_ops, modify_flow_action_esp); + SET_DEVICE_OP(dev_ops, modify_hw_stat); SET_DEVICE_OP(dev_ops, modify_port); SET_DEVICE_OP(dev_ops, modify_qp); SET_DEVICE_OP(dev_ops, modify_srq); diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c index 8d831d4fd2ad..1bf3aea4b71e 100644 --- a/drivers/infiniband/core/sysfs.c +++ b/drivers/infiniband/core/sysfs.c @@ -934,7 +934,8 @@ int ib_setup_device_attrs(struct ib_device *ibdev) { struct hw_stats_device_attribute *attr; struct hw_stats_device_data *data; - int i, ret; + bool opstat_skipped = false; + int i, ret, pos = 0; data = alloc_hw_stats_device(ibdev); if (IS_ERR(data)) { @@ -955,16 +956,23 @@ int ib_setup_device_attrs(struct ib_device *ibdev) data->stats->timestamp = jiffies; for (i = 0; i < data->stats->num_counters; i++) { - attr = &data->attrs[i]; + if (data->stats->descs[i].flags & IB_STAT_FLAG_OPTIONAL) { + opstat_skipped = true; + continue; + } + + WARN_ON(opstat_skipped); + attr = &data->attrs[pos]; sysfs_attr_init(&attr->attr.attr); attr->attr.attr.name = data->stats->descs[i].name; attr->attr.attr.mode = 0444; attr->attr.show = hw_stat_device_show; attr->show = show_hw_stats; - data->group.attrs[i] = &attr->attr.attr; + data->group.attrs[pos] = &attr->attr.attr; + pos++; } - attr = &data->attrs[i]; + attr = &data->attrs[pos]; sysfs_attr_init(&attr->attr.attr); attr->attr.attr.name = "lifespan"; attr->attr.attr.mode = 0644; @@ -972,7 +980,7 @@ int ib_setup_device_attrs(struct ib_device *ibdev) attr->show = show_stats_lifespan; attr->attr.store = hw_stat_device_store; attr->store = set_stats_lifespan; - data->group.attrs[i] = &attr->attr.attr; + data->group.attrs[pos] = &attr->attr.attr; for (i = 0; i != ARRAY_SIZE(ibdev->groups); i++) if (!ibdev->groups[i]) { ibdev->groups[i] = &data->group; @@ -1027,7 +1035,8 @@ static int setup_hw_port_stats(struct ib_port *port, { struct hw_stats_port_attribute *attr; struct hw_stats_port_data *data; - int i, ret; + bool opstat_skipped = false; + int i, ret, pos = 0; data = alloc_hw_stats_port(port, group); if (IS_ERR(data)) @@ -1045,16 +1054,23 @@ static int setup_hw_port_stats(struct ib_port *port, data->stats->timestamp = jiffies; for (i = 0; i < data->stats->num_counters; i++) { - attr = &data->attrs[i]; + if (data->stats->descs[i].flags & IB_STAT_FLAG_OPTIONAL) { + opstat_skipped = true; + continue; + } + + WARN_ON(opstat_skipped); + attr = &data->attrs[pos]; sysfs_attr_init(&attr->attr.attr); attr->attr.attr.name = data->stats->descs[i].name; attr->attr.attr.mode = 0444; attr->attr.show = hw_stat_port_show; attr->show = show_hw_stats; - group->attrs[i] = &attr->attr.attr; + group->attrs[pos] = &attr->attr.attr; + pos++; } - attr = &data->attrs[i]; + attr = &data->attrs[pos]; sysfs_attr_init(&attr->attr.attr); attr->attr.attr.name = "lifespan"; attr->attr.attr.mode = 0644; @@ -1062,7 +1078,7 @@ static int setup_hw_port_stats(struct ib_port *port, attr->show = show_stats_lifespan; attr->attr.store = hw_stat_port_store; attr->store = set_stats_lifespan; - group->attrs[i] = &attr->attr.attr; + group->attrs[pos] = &attr->attr.attr; port->hw_stats_data = data; return 0; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index ae467365706b..2207f60b002f 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -545,12 +545,18 @@ enum ib_port_speed { IB_SPEED_NDR = 128, }; +enum ib_stat_flag { + IB_STAT_FLAG_OPTIONAL = 1 << 0, +}; + /** * struct rdma_stat_desc * @name - The name of the counter + * @flags - Flags of the counter; For example, IB_STAT_FLAG_OPTIONAL */ struct rdma_stat_desc { const char *name; + unsigned int flags; }; /** @@ -2562,6 +2568,13 @@ struct ib_device_ops { int (*get_hw_stats)(struct ib_device *device, struct rdma_hw_stats *stats, u32 port, int index); + /** + * modify_hw_stat - Modify the counter configuration + * @enable: true/false when enable/disable a counter + * Return codes - 0 on success or error code otherwise. + */ + int (*modify_hw_stat)(struct ib_device *device, u32 port, + unsigned int counter_index, bool enable); /** * Allows rdma drivers to add their own restrack attributes. */ diff --git a/include/rdma/rdma_counter.h b/include/rdma/rdma_counter.h index 0295b22cd1cd..45d5481a7846 100644 --- a/include/rdma/rdma_counter.h +++ b/include/rdma/rdma_counter.h @@ -63,4 +63,6 @@ int rdma_counter_get_mode(struct ib_device *dev, u32 port, enum rdma_nl_counter_mode *mode, enum rdma_nl_counter_mask *mask); +int rdma_counter_modify(struct ib_device *dev, u32 port, + unsigned int index, bool enable); #endif /* _RDMA_COUNTER_H_ */ From patchwork Fri Oct 8 12:24:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545205 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26C35C433FE for ; Fri, 8 Oct 2021 12:26:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10B83603E7 for ; Fri, 8 Oct 2021 12:26:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241532AbhJHM21 (ORCPT ); Fri, 8 Oct 2021 08:28:27 -0400 Received: from mail-dm6nam11on2062.outbound.protection.outlook.com ([40.107.223.62]:59392 "EHLO NAM11-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241510AbhJHM2V (ORCPT ); Fri, 8 Oct 2021 08:28:21 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UHUnRZOmxAqU/c7CKLCowK1DOjJwekm/AlZ+A4ziHs9cX/qYEfpt2ASqrLknDRKKNqo+O1RXri/RPgCHat8UFUTXbXpNiSwxLzL/vmXhHdT8/z9Ar/e5dFKTToaWWjlE0Bu1jrCplLgkWziQXzkV6NfONMg8zpz1GCq1fJPlaCtD4DFquObz3oPlUD3S8Jw2iryrudVJIQ3k2SxQVIDvfbkeOs123wQnIL5vJR/DQG+dUaqPEoB3lg48LrEzhgCzlZcNld9CpQj6Z7ZWjg3/C2XTg+Eafl4mPce+IYbHzNDnv+xZ7wQQe0uZ++bSFntYb8q/esPcNNKzu0KCByFZ1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZuBrAnYasxb+S6ALBNwEy3sRS3Sh+HWNJkXrPYJrGNU=; b=Zr7wQmuWd4ZoWoGvFz/amd6K9cQKZ4ohvu4EBoIWq8EMGt8U7/KpSME6BlpQD9W5JLoICcpW8mdfcq+0wxqtuHG7AdEeHbnW71GKUTXloxjRBK2D+veRJdzu+M/UI+pmrZyP9E8sF+T+DwDfK/zlX5m3TL7EgLtc/rHA1Pu3d7fs4iO+TQ/+gPmPu7p9UkhFR4KgD+hhWK2LUGrNLs2I0ya7sxlN1xFNIcwgkyFj2PKuPhxxr3qi5dtakY0Zpi6ct6g71c/Xmf7ntJXhl6vRVPY4yg2ftgXlP2k/D8WWqdtm3qktqkljjvOdpS7tAGIAMIrxBjhOF/FizW1Ib4Z68A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZuBrAnYasxb+S6ALBNwEy3sRS3Sh+HWNJkXrPYJrGNU=; b=IyO0g8sxr/dI+de3CZ7DPT+HX4yFWwWRSNVqs5ke7++2+M+4srtKl0vbS6hH+MOO4LpS1EnbdWmozwDx8pZOI9dsiNJLlpcQ+/DI5zTDnnVzwK2rmm9FujNosywh17QAJPJ9rQMIWZjZ2Q1JFI2IvPGwIKvqC7U1Cp1/Gg00wK8VLEjGetEdt72XJmkqIx1AIhir9bXR9wn6VN0HNzi6vlgroS/l/Vot0StHxt8YS6edrweu/Cf9j2DOZLPFpc9Ut1pEyyiy9JlQaDnH70nXNlWktjXVWsPD8sNfzLyA4mK8Oq6jPkq5OspkHKMrtYrmRMww1SS7ZyRPwIeoXpKRMA== Received: from BN9P222CA0014.NAMP222.PROD.OUTLOOK.COM (2603:10b6:408:10c::19) by DM6PR12MB4944.namprd12.prod.outlook.com (2603:10b6:5:1ba::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.19; Fri, 8 Oct 2021 12:26:25 +0000 Received: from BN8NAM11FT033.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10c:cafe::29) by BN9P222CA0014.outlook.office365.com (2603:10b6:408:10c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.22 via Frontend Transport; Fri, 8 Oct 2021 12:26:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT033.mail.protection.outlook.com (10.13.177.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:24 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:23 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:18 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 07/13] RDMA/nldev: Add support to get status of all counters Date: Fri, 8 Oct 2021 15:24:33 +0300 Message-ID: <20211008122439.166063-8-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: eb2bc597-f562-45b8-92ff-08d98a56d803 X-MS-TrafficTypeDiagnostic: DM6PR12MB4944: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2958; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J/D320qVVmpt0IjYyGJXA4U3Kvbizkg/VJmiCjlvy5h8rRxDcnPS8K7Xv1Ym9ai78Xooyp1s9Adp2PXWhUu64V/ejAz9sbuVPp/9UO12krj3pFDHdgvYJQvupd05kxtgDfTOWm+V/U/eAnKFoaGcT7BkcPwY9HeW347JeKABMMb5ErioqEEA98aTQqpxWAc5O3G7x2kxlv9ETEIkC6mWZzpNlCGIbSAmvN3BLmKGkJB1ZNtb9sOWhtWQhResQfiqGJjeKM2RMWnvIULX5XL4eanMJN1MxD//KlIfsRkNMTgi/CDUb3uxZ9WrAPiKKvoz+TYlbDlIHVhxWlbefPEBLCx+R/OpJslC+jA4fihP6laGC6UeO0fxYeEpkAo27W6Cv/yXetelw2FTrZfiU+aPZM6fm9IzTgxrhv+FLWYjDijjI98GogKoM4Qe2ad2On7OhWQ9mjiGF7vcv3GM3uHObmxzz2hGREVVC8ED2QJrWUf7N2TDRqmtXogc2jQyqvLozXWZ3pf3UNjf6XXmS2QE6egBruvCmacHGRbHgzAbelK1DlXIU275Bhd42bZbRKVFB40S0s3YArrYxRbvb9qcGTIPsnKkxZLBIXMFLLCsIC1ldrsdX3s0AR3pOddzl9LxVUt5fdf0pt3jHPoondOpL1BPcd0PLb4teymowOgqGw6E2283ThP+wgxLP1GmAvyNjqes9CKoGou9PbGTPipLzQ== X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid04.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(508600001)(2616005)(70206006)(6666004)(70586007)(54906003)(36756003)(426003)(36860700001)(107886003)(83380400001)(8676002)(4326008)(7696005)(2906002)(8936002)(5660300002)(6636002)(316002)(26005)(1076003)(7416002)(110136005)(356005)(82310400003)(86362001)(47076005)(336012)(7636003)(186003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:24.3860 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eb2bc597-f562-45b8-92ff-08d98a56d803 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT033.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4944 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau This patch adds the ability to get the name, index and status of all counters for each link through RDMA netlink. This can be used for user-space to get the current optional-counter mode. Examples: $ rdma statistic mode link rocep8s0f0/1 optional-counters cc_rx_ce_pkts $ rdma statistic mode supported link rocep8s0f0/1 supported optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts,cc_tx_cnp_pkts link rocep8s0f1/1 supported optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts,cc_tx_cnp_pkts Signed-off-by: Aharon Landau Signed-off-by: Neta Ostrovsky Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/nldev.c | 98 ++++++++++++++++++++++++++++++++ include/uapi/rdma/rdma_netlink.h | 5 ++ 2 files changed, 103 insertions(+) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 67519730b1ac..210057fef7bd 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -154,6 +154,8 @@ static const struct nla_policy nldev_policy[RDMA_NLDEV_ATTR_MAX] = { [RDMA_NLDEV_NET_NS_FD] = { .type = NLA_U32 }, [RDMA_NLDEV_SYS_ATTR_NETNS_MODE] = { .type = NLA_U8 }, [RDMA_NLDEV_SYS_ATTR_COPY_ON_FORK] = { .type = NLA_U8 }, + [RDMA_NLDEV_ATTR_STAT_HWCOUNTER_INDEX] = { .type = NLA_U32 }, + [RDMA_NLDEV_ATTR_STAT_HWCOUNTER_DYNAMIC] = { .type = NLA_U8 }, }; static int put_driver_name_print_type(struct sk_buff *msg, const char *name, @@ -2264,6 +2266,99 @@ static int nldev_stat_get_dumpit(struct sk_buff *skb, return ret; } +static int nldev_stat_get_counter_status_doit(struct sk_buff *skb, + struct nlmsghdr *nlh, + struct netlink_ext_ack *extack) +{ + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX], *table, *entry; + struct rdma_hw_stats *stats; + struct ib_device *device; + struct sk_buff *msg; + u32 devid, port; + int ret, i; + + ret = nlmsg_parse(nlh, 0, tb, RDMA_NLDEV_ATTR_MAX - 1, + nldev_policy, extack); + if (ret || !tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_PORT_INDEX]) + return -EINVAL; + + devid = nla_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); + device = ib_device_get_by_index(sock_net(skb->sk), devid); + if (!device) + return -EINVAL; + + port = nla_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]); + if (!rdma_is_port_valid(device, port)) { + ret = -EINVAL; + goto err; + } + + stats = ib_get_hw_stats_port(device, port); + if (!stats) { + ret = -EINVAL; + goto err; + } + + msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); + if (!msg) { + ret = -ENOMEM; + goto err; + } + + nlh = nlmsg_put( + msg, NETLINK_CB(skb).portid, nlh->nlmsg_seq, + RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_STAT_GET_STATUS), + 0, 0); + + ret = -EMSGSIZE; + if (fill_nldev_handle(msg, device) || + nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port)) + goto err_msg; + + table = nla_nest_start(msg, RDMA_NLDEV_ATTR_STAT_HWCOUNTERS); + if (!table) + goto err_msg; + + mutex_lock(&stats->lock); + for (i = 0; i < stats->num_counters; i++) { + entry = nla_nest_start(msg, + RDMA_NLDEV_ATTR_STAT_HWCOUNTER_ENTRY); + if (!entry) + goto err_msg_table; + + if (nla_put_string(msg, + RDMA_NLDEV_ATTR_STAT_HWCOUNTER_ENTRY_NAME, + stats->descs[i].name) || + nla_put_u32(msg, RDMA_NLDEV_ATTR_STAT_HWCOUNTER_INDEX, i)) + goto err_msg_entry; + + if ((stats->descs[i].flags & IB_STAT_FLAG_OPTIONAL) && + (nla_put_u8(msg, RDMA_NLDEV_ATTR_STAT_HWCOUNTER_DYNAMIC, + !test_bit(i, stats->is_disabled)))) + goto err_msg_entry; + + nla_nest_end(msg, entry); + } + mutex_unlock(&stats->lock); + + nla_nest_end(msg, table); + nlmsg_end(msg, nlh); + ib_device_put(device); + return rdma_nl_unicast(sock_net(skb->sk), msg, NETLINK_CB(skb).portid); + +err_msg_entry: + nla_nest_cancel(msg, entry); +err_msg_table: + mutex_unlock(&stats->lock); + nla_nest_cancel(msg, table); +err_msg: + nlmsg_free(msg); +err: + ib_device_put(device); + return ret; +} + static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { [RDMA_NLDEV_CMD_GET] = { .doit = nldev_get_doit, @@ -2353,6 +2448,9 @@ static const struct rdma_nl_cbs nldev_cb_table[RDMA_NLDEV_NUM_OPS] = { .dump = nldev_res_get_mr_raw_dumpit, .flags = RDMA_NL_ADMIN_PERM, }, + [RDMA_NLDEV_CMD_STAT_GET_STATUS] = { + .doit = nldev_stat_get_counter_status_doit, + }, }; void __init nldev_init(void) diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h index 75a1ae2311d8..e50c357367db 100644 --- a/include/uapi/rdma/rdma_netlink.h +++ b/include/uapi/rdma/rdma_netlink.h @@ -297,6 +297,8 @@ enum rdma_nldev_command { RDMA_NLDEV_CMD_RES_SRQ_GET, /* can dump */ + RDMA_NLDEV_CMD_STAT_GET_STATUS, + RDMA_NLDEV_NUM_OPS }; @@ -549,6 +551,9 @@ enum rdma_nldev_attr { RDMA_NLDEV_SYS_ATTR_COPY_ON_FORK, /* u8 */ + RDMA_NLDEV_ATTR_STAT_HWCOUNTER_INDEX, /* u32 */ + RDMA_NLDEV_ATTR_STAT_HWCOUNTER_DYNAMIC, /* u8 */ + /* * Always the end */ From patchwork Fri Oct 8 12:24:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545207 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06C0EC433EF for ; Fri, 8 Oct 2021 12:26:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E698560F48 for ; Fri, 8 Oct 2021 12:26:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241654AbhJHM2c (ORCPT ); Fri, 8 Oct 2021 08:28:32 -0400 Received: from mail-dm6nam12on2055.outbound.protection.outlook.com ([40.107.243.55]:29094 "EHLO NAM12-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241464AbhJHM20 (ORCPT ); Fri, 8 Oct 2021 08:28:26 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LSVteQYsfBuL1JrVvOaa7VKFgERqeHdtRC82VFfwWfIFiwn/tFwSAps2xXT1AxKUPrI5OePdV9wWTPtKeQ31gc6DmY/XqRctPhx2Ii7vARM/zM61a+tQLPptv/Chp97+9PECD1+vPT/Z7jKRrh4Bs2Rqz111+uD61VatTIdHVUOC/xBFyr/LL0sAid0n8ywY1r0dqXA5k4MpOlTH0RpXmfdKbz572pb3kTjKA95iRLvYQJuN65IjmM+DpTmqiWLDYEvnWtmFXriUVnJW9YowqxVLahjV8gECEY8hgD4p0Xo3XV0W7/7oMDgvj+bgKLODDPW4WTJltR1w2nVIocOaoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=65AI7+lpV1NaLgLYGMwm5DekiaFkNTo8msZrsc0SH98=; b=aY8HgR+chcWW+P5DdjLfcCmr0VyeJ8bgFsQXa0Ya4qOFtpWu/fb3bYtpJgTx29/qw/nW8K5EFhuid3SPjhvUgKdcr9epQrlK2qOQqN49Yp+mfPUP1V9Z9ufh3n+mOkJdTm0M/uVKUi5BjEexmRUuxHa+Q/ItiLu6+zDDTsT7z3A6g3RvVF0hMG+934sKNz+QOrnPTBDX0oB2skMgjkyMkeGY5iGAjdwtkvBGqGBI6UXzrJxdXAgbxYPTonnekSiluTUxkGLThXHmxDBCB7JuxTTGeBe0mkqzUdRFOcLM6W7XySclyclk/2GEIxoT/fPrbw0/7xq3nTdyva/2FQ8T2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=65AI7+lpV1NaLgLYGMwm5DekiaFkNTo8msZrsc0SH98=; b=YOQSjmbn970Ht4HOckdPUdEegPfeddZGAsn9auXjc+DnK+RXUgJtd/2e6VABDm+Cm5nGcVYFPOeaxroziysDNDX+RzJxdzeEMoXQDaGeQi7boPXpFPtR56VX5ryRSGJ7qkgeC5laVAFulZnl5v/r9aW8KKgIKNwerv7WUwQLV6hMgljNS/eN+JEKFy+tzDZHPeLNL0NWFcxAgtkg5hCbzlfwtQsgQxQHAQb2fT0LyDD76DLqc/vyqqHSC5ebbjpVqB/zcsTXcT/vYyFo+ZKF1f2HYIoPAwY0j6tNtTkaNWaUGV6b5xV4pSga9zJ/doQSN5d1A8fQpqoBhMx4yBV6Ww== Received: from BN8PR12CA0011.namprd12.prod.outlook.com (2603:10b6:408:60::24) by DM6PR12MB2923.namprd12.prod.outlook.com (2603:10b6:5:180::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.22; Fri, 8 Oct 2021 12:26:29 +0000 Received: from BN8NAM11FT017.eop-nam11.prod.protection.outlook.com (2603:10b6:408:60:cafe::22) by BN8PR12CA0011.outlook.office365.com (2603:10b6:408:60::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT017.mail.protection.outlook.com (10.13.177.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:29 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:28 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:23 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 08/13] RDMA/nldev: Split nldev_stat_set_mode_doit out of nldev_stat_set_doit Date: Fri, 8 Oct 2021 15:24:34 +0300 Message-ID: <20211008122439.166063-9-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6c77bf34-5861-4bd7-6222-08d98a56dad4 X-MS-TrafficTypeDiagnostic: DM6PR12MB2923: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DzWm4rcGfe7MUxJ6Ulis6IOmwWchgG2JRa2oq59ekDfl904kXlAVfnNM6ntY1uum68odyi6fJfoLAfzAgPGFTpI2wA2Yf+qsQJPnY3XEBgYyJcR+kuD94T4OH6htJ3VjbJAZsXqHq60RtppL950pPZNJwCYxuZHRSSNDf06JzxHVmpw8rgeoEE5BYhuEv7GXqlXnG8HQMmpaJGCYepVHjNzy4zoww9KoCepFwDkvNtlDiI2IVwACuvmaf/IBP8O/Lt5xaDoxSkuXMd9pqK6NWvjlUafa4MVXt6UkvkkMFbQC1Sr9xamsBF2wBODNxpEwVLS15kS7vuPkZLSipR8UiwTyVZ2EfgHTPsPWJgJqYh85JVddXzdCwSd2ScPq4IudefT7fsjNfn/wAGUWcpWADwl3FYdmmUUWVfhjVHQBawgDtEO1FMb5DcACkpwqRXtBj14nXYTH4B81ualpK2GUjx0J58Jh4gxWrPl1JpJevt8nqKKl15fhnp9JaLHcGAUJj9FHI2V0xIk32mFdKTjO2l82QzqCRvzV3eYDNSbh5b/duepSuSRllGgRZQe3hOx57JwvcGTSyj9MqZJ4GvCCORiQVmncX/LX+GtdmMvJalXSFfK65EMJYIoMfbl2H//IiFZG7pDE0hz5vtxo88726QJLZ+I3bS3Fk81seN6mz9cxPEgyHvoxh+qrDMpNwBQW X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid04.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70206006)(82310400003)(7696005)(47076005)(36860700001)(316002)(83380400001)(4326008)(8936002)(426003)(54906003)(2616005)(110136005)(6636002)(70586007)(336012)(8676002)(5660300002)(26005)(86362001)(107886003)(7636003)(1076003)(7416002)(508600001)(356005)(6666004)(2906002)(186003)(36756003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:29.1149 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c77bf34-5861-4bd7-6222-08d98a56dad4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT017.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2923 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau In order to allow expansion of the set command with more set options, take the set mode out of the main set function. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/nldev.c | 116 +++++++++++++++++++------------- 1 file changed, 70 insertions(+), 46 deletions(-) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 210057fef7bd..8361eb08e13b 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1897,24 +1897,67 @@ static int nldev_set_sys_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, return err; } +static int nldev_stat_set_mode_doit(struct sk_buff *msg, + struct netlink_ext_ack *extack, + struct nlattr *tb[], + struct ib_device *device, u32 port) +{ + u32 mode, mask = 0, qpn, cntn = 0; + int ret; + + /* Currently only counter for QP is supported */ + if (nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_RES]) != RDMA_NLDEV_ATTR_RES_QP) + return -EINVAL; + + mode = nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_MODE]); + if (mode == RDMA_COUNTER_MODE_AUTO) { + if (tb[RDMA_NLDEV_ATTR_STAT_AUTO_MODE_MASK]) + mask = nla_get_u32( + tb[RDMA_NLDEV_ATTR_STAT_AUTO_MODE_MASK]); + return rdma_counter_set_auto_mode(device, port, mask, extack); + } + + if (!tb[RDMA_NLDEV_ATTR_RES_LQPN]) + return -EINVAL; + + qpn = nla_get_u32(tb[RDMA_NLDEV_ATTR_RES_LQPN]); + if (tb[RDMA_NLDEV_ATTR_STAT_COUNTER_ID]) { + cntn = nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_COUNTER_ID]); + ret = rdma_counter_bind_qpn(device, port, qpn, cntn); + if (ret) + return ret; + } else { + ret = rdma_counter_bind_qpn_alloc(device, port, qpn, &cntn); + if (ret) + return ret; + } + + if (nla_put_u32(msg, RDMA_NLDEV_ATTR_STAT_COUNTER_ID, cntn) || + nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_LQPN, qpn)) { + ret = -EMSGSIZE; + goto err_fill; + } + + return 0; + +err_fill: + rdma_counter_unbind_qpn(device, port, qpn, cntn); + return ret; +} + static int nldev_stat_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, struct netlink_ext_ack *extack) { - u32 index, port, mode, mask = 0, qpn, cntn = 0; struct nlattr *tb[RDMA_NLDEV_ATTR_MAX]; struct ib_device *device; struct sk_buff *msg; + u32 index, port; int ret; - ret = nlmsg_parse(nlh, 0, tb, RDMA_NLDEV_ATTR_MAX - 1, - nldev_policy, extack); - /* Currently only counter for QP is supported */ - if (ret || !tb[RDMA_NLDEV_ATTR_STAT_RES] || - !tb[RDMA_NLDEV_ATTR_DEV_INDEX] || - !tb[RDMA_NLDEV_ATTR_PORT_INDEX] || !tb[RDMA_NLDEV_ATTR_STAT_MODE]) - return -EINVAL; - - if (nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_RES]) != RDMA_NLDEV_ATTR_RES_QP) + ret = nlmsg_parse(nlh, 0, tb, RDMA_NLDEV_ATTR_MAX - 1, nldev_policy, + extack); + if (ret || !tb[RDMA_NLDEV_ATTR_DEV_INDEX] || + !tb[RDMA_NLDEV_ATTR_PORT_INDEX]) return -EINVAL; index = nla_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]); @@ -1925,59 +1968,40 @@ static int nldev_stat_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, port = nla_get_u32(tb[RDMA_NLDEV_ATTR_PORT_INDEX]); if (!rdma_is_port_valid(device, port)) { ret = -EINVAL; - goto err; + goto err_put_device; + } + + if (!tb[RDMA_NLDEV_ATTR_STAT_MODE]) { + ret = -EINVAL; + goto err_put_device; } msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL); if (!msg) { ret = -ENOMEM; - goto err; + goto err_put_device; } nlh = nlmsg_put(msg, NETLINK_CB(skb).portid, nlh->nlmsg_seq, RDMA_NL_GET_TYPE(RDMA_NL_NLDEV, RDMA_NLDEV_CMD_STAT_SET), 0, 0); - - mode = nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_MODE]); - if (mode == RDMA_COUNTER_MODE_AUTO) { - if (tb[RDMA_NLDEV_ATTR_STAT_AUTO_MODE_MASK]) - mask = nla_get_u32( - tb[RDMA_NLDEV_ATTR_STAT_AUTO_MODE_MASK]); - ret = rdma_counter_set_auto_mode(device, port, mask, extack); - if (ret) - goto err_msg; - } else { - if (!tb[RDMA_NLDEV_ATTR_RES_LQPN]) - goto err_msg; - qpn = nla_get_u32(tb[RDMA_NLDEV_ATTR_RES_LQPN]); - if (tb[RDMA_NLDEV_ATTR_STAT_COUNTER_ID]) { - cntn = nla_get_u32(tb[RDMA_NLDEV_ATTR_STAT_COUNTER_ID]); - ret = rdma_counter_bind_qpn(device, port, qpn, cntn); - } else { - ret = rdma_counter_bind_qpn_alloc(device, port, - qpn, &cntn); - } - if (ret) - goto err_msg; - - if (fill_nldev_handle(msg, device) || - nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port) || - nla_put_u32(msg, RDMA_NLDEV_ATTR_STAT_COUNTER_ID, cntn) || - nla_put_u32(msg, RDMA_NLDEV_ATTR_RES_LQPN, qpn)) { - ret = -EMSGSIZE; - goto err_fill; - } + if (fill_nldev_handle(msg, device) || + nla_put_u32(msg, RDMA_NLDEV_ATTR_PORT_INDEX, port)) { + ret = -EMSGSIZE; + goto err_free_msg; } + ret = nldev_stat_set_mode_doit(msg, extack, tb, device, port); + if (ret) + goto err_free_msg; + nlmsg_end(msg, nlh); ib_device_put(device); return rdma_nl_unicast(sock_net(skb->sk), msg, NETLINK_CB(skb).portid); -err_fill: - rdma_counter_unbind_qpn(device, port, qpn, cntn); -err_msg: +err_free_msg: nlmsg_free(msg); -err: +err_put_device: ib_device_put(device); return ret; } From patchwork Fri Oct 8 12:24:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545209 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4A04C4332F for ; Fri, 8 Oct 2021 12:26:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C100A60F93 for ; Fri, 8 Oct 2021 12:26:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241662AbhJHM2d (ORCPT ); Fri, 8 Oct 2021 08:28:33 -0400 Received: from mail-sn1anam02on2040.outbound.protection.outlook.com ([40.107.96.40]:22147 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241584AbhJHM2b (ORCPT ); Fri, 8 Oct 2021 08:28:31 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yu83WouqecI2pSjgYjzjp/n1NmkICFK04nLi27Fy+NcAtl1vkHdjPY9UOyX1Ry3Uk/yam42Ob3H7NuW00xW73DcxKulOTKppAdMb3+e41GEed+1dyNExK8Vv7Ub+jR3K2M7oER5c1t2fJD1SO6VkTk4gHJyFLBYFNGfd+vSEnDTqcPT/52+ggwqu6LlB5XK3Fye2yumTUS4JshQlhCjUpWGhVo7NNcvyQNFqn6Wk0BSWCNNsJrDwj/CigwiH997MPfVtZ2wHg09VMHDbpxbBH3X9TinBoCQ+iicUoIPPUZ4Q/K3mS+ooNN4k+YwFtN1xesWMfd8VaGfqASHNwmgcHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Sm//V2ka1xPcvvKhc93o14Xo19TIuAIk6TF+GKlItv4=; b=L9Or6DEbqIPq77OrUCZKSQGrbFvn/bwbqh86t8Do00+aRTvpG4lPreJmqipW9tTYMIwcQ+aCGADGtiVH5SMGXRIxVn/2Rng3giDfNg2t3evWf/dib1fvbs+t3NLL/Rr3/q34dDcfQgsVenSMqYri/5yrBdZQyJi79dKILCUuKucThYKeSIR4aVQNSjHMK9pSE+eB2SoDAx9GSD+Bv4+vEgGopXfm/TLAKKReAz7I6/bYJ61QPToC//Wp/ZHJY3rspZivkahAaCUS/Bztb3K8sgN6p5z71rz16ACSM0NjyTbNtHUqedDVpOAWxLwXUKHAHXog7FJ66/mqsNtJmtjc8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sm//V2ka1xPcvvKhc93o14Xo19TIuAIk6TF+GKlItv4=; b=WNnxmUwFgS8OuIIRwmAr5TFSH1geB6sHJzZ9WrkhQ8243IvKZtMV6NxJMqNIYzR2lASsMsM+1kmiORq6boAJocnMRHQfB8d6iASIvDG4hzArcMQBkWFI4FwkYKGcLczHaETXt/nDlZsXMD6rFxkypSLTEZ8UvdzkHmGcLjQ+qjWMPVTeaji6doWvrNfDWDNiB2vtevdZkNVukUu4AahQJWaDjL6WtZpm4ibKiKkXg4RQWDNglwMobpgi7hjEumV12Wy2pEBAgYRlks81uZuv5/ZZUEI8MPQ5zjtfU51TuVkcsAMXIVkTcM96M2eUdy74GkrUkcFXaUJbUxXlOdPLOA== Received: from MWHPR22CA0019.namprd22.prod.outlook.com (2603:10b6:300:ef::29) by DM5PR12MB1706.namprd12.prod.outlook.com (2603:10b6:3:10f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.20; Fri, 8 Oct 2021 12:26:35 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ef:cafe::49) by MWHPR22CA0019.outlook.office365.com (2603:10b6:300:ef::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.19 via Frontend Transport; Fri, 8 Oct 2021 12:26:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:34 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 05:26:32 -0700 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:28 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , Mark Zhang Subject: [PATCH rdma-next v4 09/13] RDMA/nldev: Allow optional-counter status configuration through RDMA netlink Date: Fri, 8 Oct 2021 15:24:35 +0300 Message-ID: <20211008122439.166063-10-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f176ffa-46f3-4edf-297e-08d98a56ddee X-MS-TrafficTypeDiagnostic: DM5PR12MB1706: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: v2P+IuRwzJwXXVVdd3PJYvcRFm8bYGge60NhL2lDO1XfaIOKHRz0Y7syzp1VstkexMG20Z7+C+ZPI1kvXyn2VqBSz9JxMF8LiOshXXRbP+5fzWS6Gk8s6ILhkftelFD2J/WewcpHYjlFmNx79PHEoVnIZYf6Ktrzixbza2kbVAiM4pbIXLIb4nogkXY0ezNKGQmHN0lD6I83Wu7K4ktWkuv2qvmqdxsJ71T1AFt6HSTpFFP9NSU4L6LOlaCtUuBFomvyM76Lwq0Vkb8WI2M+I+M/9GS3m1sQAUE4j9F8ZijQGD0CIPAbQ5lpP0OLSlJYkYvo3mzTQPiFyXbwGDJNX5zIBTSyxKs/huvg2Nr4/KeGMi37jwGuj5HSYqVMbxv61XbnK0zFVN+BqqxpN12zPfzb1oE0O+NcIQQteiwW9CU/FzePdUbghUcSdLGp+iY0GZhibwR8WhunZkorYuQPTmwCJpwmg8PltpPeX8Cc1IXpIAWLrGd3sA6Fi0U0foKeL2ag0V9xJ8Tnnj70aj5H2ZfSIBXEJoRYRLiLEmnM3fwiClNwaU7TbOfZBW/oWVdDAYZJu0yMiVO/atKc5xtwKdzL04eLnm7a4LbNriYX3pUOn6gmbWK4lVeVkZcRN4bsDWzib0ymizCAbXspsJWGGqCsuVIkLzL+Ljezh+Ugvm2UAD6vb4DpdProTLeGjmteys3hv4tJ4UthjHIj+JeqTg== X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(7416002)(336012)(70586007)(5660300002)(70206006)(2906002)(356005)(508600001)(54906003)(1076003)(426003)(107886003)(316002)(47076005)(6666004)(83380400001)(2616005)(7696005)(4326008)(7636003)(86362001)(36860700001)(8676002)(8936002)(82310400003)(186003)(36756003)(26005)(110136005)(6636002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:34.3719 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f176ffa-46f3-4edf-297e-08d98a56ddee X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1706 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Provide an option to allow users to enable/disable optional counters through RDMA netlink. Limiting it to users with ADMIN capability only. Examples: 1. Enable optional counters cc_rx_ce_pkts and cc_rx_cnp_pkts (and disable all others): $ sudo rdma statistic set link rocep8s0f0/1 optional-counters \ cc_rx_ce_pkts,cc_rx_cnp_pkts 2. Remove all optional counters: $ sudo rdma statistic unset link rocep8s0f0/1 optional-counters Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/core/nldev.c | 61 ++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/core/nldev.c b/drivers/infiniband/core/nldev.c index 8361eb08e13b..fedc0fa6ebf9 100644 --- a/drivers/infiniband/core/nldev.c +++ b/drivers/infiniband/core/nldev.c @@ -1945,6 +1945,50 @@ static int nldev_stat_set_mode_doit(struct sk_buff *msg, return ret; } +static int nldev_stat_set_counter_dynamic_doit(struct nlattr *tb[], + struct ib_device *device, + u32 port) +{ + struct rdma_hw_stats *stats; + int rem, i, index, ret = 0; + struct nlattr *entry_attr; + unsigned long *target; + + stats = ib_get_hw_stats_port(device, port); + if (!stats) + return -EINVAL; + + target = kcalloc(BITS_TO_LONGS(stats->num_counters), + sizeof(*stats->is_disabled), GFP_KERNEL); + if (!target) + return -ENOMEM; + + nla_for_each_nested(entry_attr, tb[RDMA_NLDEV_ATTR_STAT_HWCOUNTERS], + rem) { + index = nla_get_u32(entry_attr); + if ((index >= stats->num_counters) || + !(stats->descs[index].flags & IB_STAT_FLAG_OPTIONAL)) { + ret = -EINVAL; + goto out; + } + + set_bit(index, target); + } + + for (i = 0; i < stats->num_counters; i++) { + if (!(stats->descs[i].flags & IB_STAT_FLAG_OPTIONAL)) + continue; + + ret = rdma_counter_modify(device, port, i, test_bit(i, target)); + if (ret) + goto out; + } + +out: + kfree(target); + return ret; +} + static int nldev_stat_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, struct netlink_ext_ack *extack) { @@ -1971,7 +2015,8 @@ static int nldev_stat_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, goto err_put_device; } - if (!tb[RDMA_NLDEV_ATTR_STAT_MODE]) { + if (!tb[RDMA_NLDEV_ATTR_STAT_MODE] && + !tb[RDMA_NLDEV_ATTR_STAT_HWCOUNTERS]) { ret = -EINVAL; goto err_put_device; } @@ -1991,9 +2036,17 @@ static int nldev_stat_set_doit(struct sk_buff *skb, struct nlmsghdr *nlh, goto err_free_msg; } - ret = nldev_stat_set_mode_doit(msg, extack, tb, device, port); - if (ret) - goto err_free_msg; + if (tb[RDMA_NLDEV_ATTR_STAT_MODE]) { + ret = nldev_stat_set_mode_doit(msg, extack, tb, device, port); + if (ret) + goto err_free_msg; + } + + if (tb[RDMA_NLDEV_ATTR_STAT_HWCOUNTERS]) { + ret = nldev_stat_set_counter_dynamic_doit(tb, device, port); + if (ret) + goto err_free_msg; + } nlmsg_end(msg, nlh); ib_device_put(device); From patchwork Fri Oct 8 12:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545211 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB96EC433F5 for ; Fri, 8 Oct 2021 12:26:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B513560F44 for ; Fri, 8 Oct 2021 12:26:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241510AbhJHM2g (ORCPT ); Fri, 8 Oct 2021 08:28:36 -0400 Received: from mail-bn8nam11on2085.outbound.protection.outlook.com ([40.107.236.85]:39486 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241539AbhJHM2f (ORCPT ); Fri, 8 Oct 2021 08:28:35 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K65rhHJRzR30OJ8U6TuhoFknFq/B+Uj3FXBVQMtwISfdD3jk7LvpQmITVdO1X3A7Hnh+ZB0YSZjvQrPMh5i2Sc3K9UK03eG2GuV0M6iARaObEFIht9zDZj+bfBrKCAQCw8U/dZ0rJrMFU1Wf2s9jGzeOSUKqjr4rIS4kBI8S6VKu27Hd1bnamqyVGTXZIaJP/HKKeQub+8DD5nOWPa9+2xAC7/vuB7yFoNbv7/cnMtCpgPKH7dxcX7EcA57vXnkqJjo9uZ53bDIEo7d9GTXNzd90mBEFMq2m9AsTsEyhc57p2TniGXFT/fHnaNN/O/5WaUquPi2H1uVVZQE4Qi/2Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GURo1LgNFr3uEafZnrXT87Qdbqt9coqgY1D9N+uzT7w=; b=k++I03qs7AwfBcd+58IyjRiphEKeZE2lI7AI+nE4wRewHb/SdWffFG84tnvQtYmHsUyFoLqS2SuF5+pUitEybDq0dcbW/EkEQQ1qMbVszpYT9d+gYz1bfPQO8Nj5blIqCuPtLKM0aLVokrR/m/GWM1VZ9mYOtRuYI9o2BidvA0YxgugTUY3Q1tWvKc/QtgZ0mYhtPPxzr4FoAVypAvMYus88jQGeHsEhEi5snLRBf3BgyIn/pVw7atPzFEdkLDMKVsQcr9vftN31clqoSEabvoIllSkYgW4MXFAMY7T+KPpr+rceFjbO0cR4iJ3tpp0DGZPg1y/VjUsqaS9mLxO5lA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GURo1LgNFr3uEafZnrXT87Qdbqt9coqgY1D9N+uzT7w=; b=emIUyo0GfbYN/cbwy7qTzEv1mhgl9Lj7A7LuwGXSdub+gWGjWeGczAUFod5CdCxICcu96b5Xbkjpnuusc7R1iZAa1qbPAqn6Cmn6q1ZYbQsuRoTraWce75hoYwlTE06LjdtvIybmDoAd6g38fXRHqmgng23J0qg3Gri94JAIx+MgvGkF3MphLMnA6xSgmv2rOGZRqTvcQqI4x5AROMQkeC6SZzGQCgqpfzsNCcS3fXbx/O2S/dNGuWqj6gbxAzdoSYgeltEt8RKyonxaZ2ROGghydDcX4KE8zVP3H4Vq+LIcLXqhAu95J6F6ttUGh9EuL5vXTlIQ97q45TvXSQmoGw== Received: from MWHPR22CA0009.namprd22.prod.outlook.com (2603:10b6:300:ef::19) by CH2PR12MB4907.namprd12.prod.outlook.com (2603:10b6:610:68::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.19; Fri, 8 Oct 2021 12:26:38 +0000 Received: from CO1NAM11FT065.eop-nam11.prod.protection.outlook.com (2603:10b6:300:ef:cafe::b7) by MWHPR22CA0009.outlook.office365.com (2603:10b6:300:ef::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by CO1NAM11FT065.mail.protection.outlook.com (10.13.174.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:38 +0000 Received: from HQMAIL105.nvidia.com (172.20.187.12) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 05:26:37 -0700 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL105.nvidia.com (172.20.187.12) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:37 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:33 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , Mark Zhang Subject: [PATCH rdma-next v4 10/13] RDMA/mlx5: Support optional counters in hw_stats initialization Date: Fri, 8 Oct 2021 15:24:36 +0300 Message-ID: <20211008122439.166063-11-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a52add01-eba8-478a-103c-08d98a56e028 X-MS-TrafficTypeDiagnostic: CH2PR12MB4907: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:935; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uGaUFMRBFWCFoZSSIxPm9QzmCsNuNErga569CATLDT6+oB7dbEBTkpzn8EItt7SuW3uIc8MqQrHQXH7S/90e5SNT1C0NHHFcKpx+Q1YcGxlYmTYb6C+WBgZQZVHMAzij2gqAxA4StSoPlzZ9XMMeNcWhWFnj/PeR7uaGq6AGEt8m6WO8pHv90hpWFReJLrUI5TWB1AoyX/qPt996eRrPFFzWuVfhP2+t7L8LMY0M9rLk/DoDepzOLkN6e9QBHSpkrlDyBjL9iG8gI+jcyOhx11Cdh7rJjl4GgUFttpFUOmRhqjHhOYpfD/k1Ql1QQ+pD/qZrO31+sG69ycrejXu0BGTQeWmIbnLSHdBEkHAGt9iKAGPinaUAIpbxlWpMw+HmNTw3s8L7uFWpFjLkU99a38rlXLTrAYg3aY4FoMKDripkEyndom4Dkcpqs3dyyyc/AnvcFkXmJHhF9XEcDVZBBvRAXqmDx/rb+u31mBWLusSsjiJ56mnGCP77Hcg2heW69pps8KQsN6WK9Vh9Q0mZeqyqAOM5NnBEkb/DjTWsqo/vfzQzZ5xDFZR3PIpjf1C8E9lHtJmvAf3H4TWdIwIfVZgTiC43nahhAf/8a34jAFfJpHwZ7lVAGR2LsmEgzN6P4dggXTHTlr/QNXKLnxehvfOoMRfLiMIt1YOatG6Tos7iQQuwPcaKAe4QO+1Jj56Aoca2gXXgQEiubUu+oh9HeQ== X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(8676002)(86362001)(47076005)(1076003)(426003)(82310400003)(36860700001)(54906003)(316002)(508600001)(83380400001)(110136005)(6636002)(186003)(4326008)(356005)(2906002)(2616005)(70206006)(7636003)(7696005)(26005)(5660300002)(336012)(70586007)(7416002)(36756003)(107886003)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:38.1108 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a52add01-eba8-478a-103c-08d98a56e028 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT065.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4907 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Add optional counter support when allocate and initialize hw_stats structure. Optional counters have IB_STAT_FLAG_OPTIONAL flag set and are disabled by default. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/hw/mlx5/counters.c | 90 ++++++++++++++++++++++----- drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + 2 files changed, 75 insertions(+), 16 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/counters.c b/drivers/infiniband/hw/mlx5/counters.c index caa35ea14b48..8fe7900b29f0 100644 --- a/drivers/infiniband/hw/mlx5/counters.c +++ b/drivers/infiniband/hw/mlx5/counters.c @@ -75,6 +75,21 @@ static const struct mlx5_ib_counter ext_ppcnt_cnts[] = { INIT_EXT_PPCNT_COUNTER(rx_icrc_encapsulated), }; +#define INIT_OP_COUNTER(_name) \ + { .name = #_name } + +static const struct mlx5_ib_counter basic_op_cnts[] = { + INIT_OP_COUNTER(cc_rx_ce_pkts), +}; + +static const struct mlx5_ib_counter rdmarx_cnp_op_cnts[] = { + INIT_OP_COUNTER(cc_rx_cnp_pkts), +}; + +static const struct mlx5_ib_counter rdmatx_cnp_op_cnts[] = { + INIT_OP_COUNTER(cc_tx_cnp_pkts), +}; + static int mlx5_ib_read_counters(struct ib_counters *counters, struct ib_counters_read_attr *read_attr, struct uverbs_attr_bundle *attrs) @@ -161,17 +176,34 @@ u16 mlx5_ib_get_counters_id(struct mlx5_ib_dev *dev, u32 port_num) return cnts->set_id; } +static struct rdma_hw_stats *do_alloc_stats(const struct mlx5_ib_counters *cnts) +{ + struct rdma_hw_stats *stats; + u32 num_hw_counters; + int i; + + num_hw_counters = cnts->num_q_counters + cnts->num_cong_counters + + cnts->num_ext_ppcnt_counters; + stats = rdma_alloc_hw_stats_struct(cnts->descs, + num_hw_counters + + cnts->num_op_counters, + RDMA_HW_STATS_DEFAULT_LIFESPAN); + if (!stats) + return NULL; + + for (i = 0; i < cnts->num_op_counters; i++) + set_bit(num_hw_counters + i, stats->is_disabled); + + return stats; +} + static struct rdma_hw_stats * mlx5_ib_alloc_hw_device_stats(struct ib_device *ibdev) { struct mlx5_ib_dev *dev = to_mdev(ibdev); const struct mlx5_ib_counters *cnts = &dev->port[0].cnts; - return rdma_alloc_hw_stats_struct(cnts->descs, - cnts->num_q_counters + - cnts->num_cong_counters + - cnts->num_ext_ppcnt_counters, - RDMA_HW_STATS_DEFAULT_LIFESPAN); + return do_alloc_stats(cnts); } static struct rdma_hw_stats * @@ -180,11 +212,7 @@ mlx5_ib_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) struct mlx5_ib_dev *dev = to_mdev(ibdev); const struct mlx5_ib_counters *cnts = &dev->port[port_num - 1].cnts; - return rdma_alloc_hw_stats_struct(cnts->descs, - cnts->num_q_counters + - cnts->num_cong_counters + - cnts->num_ext_ppcnt_counters, - RDMA_HW_STATS_DEFAULT_LIFESPAN); + return do_alloc_stats(cnts); } static int mlx5_ib_query_q_counters(struct mlx5_core_dev *mdev, @@ -302,11 +330,7 @@ mlx5_ib_counter_alloc_stats(struct rdma_counter *counter) const struct mlx5_ib_counters *cnts = get_counters(dev, counter->port - 1); - return rdma_alloc_hw_stats_struct(cnts->descs, - cnts->num_q_counters + - cnts->num_cong_counters + - cnts->num_ext_ppcnt_counters, - RDMA_HW_STATS_DEFAULT_LIFESPAN); + return do_alloc_stats(cnts); } static int mlx5_ib_counter_update_stats(struct rdma_counter *counter) @@ -423,13 +447,34 @@ static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, offsets[j] = ext_ppcnt_cnts[i].offset; } } + + for (i = 0; i < ARRAY_SIZE(basic_op_cnts); i++, j++) { + descs[j].name = basic_op_cnts[i].name; + descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + } + + if (MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_receive_rdma.bth_opcode)) { + for (i = 0; i < ARRAY_SIZE(rdmarx_cnp_op_cnts); i++, j++) { + descs[j].name = rdmarx_cnp_op_cnts[i].name; + descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + } + } + + if (MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_transmit_rdma.bth_opcode)) { + for (i = 0; i < ARRAY_SIZE(rdmatx_cnp_op_cnts); i++, j++) { + descs[j].name = rdmatx_cnp_op_cnts[i].name; + descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + } + } } static int __mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev, struct mlx5_ib_counters *cnts) { - u32 num_counters; + u32 num_counters, num_op_counters; num_counters = ARRAY_SIZE(basic_q_cnts); @@ -455,6 +500,19 @@ static int __mlx5_ib_alloc_counters(struct mlx5_ib_dev *dev, cnts->num_ext_ppcnt_counters = ARRAY_SIZE(ext_ppcnt_cnts); num_counters += ARRAY_SIZE(ext_ppcnt_cnts); } + + num_op_counters = ARRAY_SIZE(basic_op_cnts); + + if (MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_receive_rdma.bth_opcode)) + num_op_counters += ARRAY_SIZE(rdmarx_cnp_op_cnts); + + if (MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_transmit_rdma.bth_opcode)) + num_op_counters += ARRAY_SIZE(rdmatx_cnp_op_cnts); + + cnts->num_op_counters = num_op_counters; + num_counters += num_op_counters; cnts->descs = kcalloc(num_counters, sizeof(struct rdma_stat_desc), GFP_KERNEL); if (!cnts->descs) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 6f5451d96dd7..8215d7ab579d 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -803,6 +803,7 @@ struct mlx5_ib_counters { u32 num_q_counters; u32 num_cong_counters; u32 num_ext_ppcnt_counters; + u32 num_op_counters; u16 set_id; }; From patchwork Fri Oct 8 12:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545215 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ADAAC433FE for ; Fri, 8 Oct 2021 12:27:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DFCCC60FF2 for ; Fri, 8 Oct 2021 12:26:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241727AbhJHM2w (ORCPT ); Fri, 8 Oct 2021 08:28:52 -0400 Received: from mail-mw2nam10on2075.outbound.protection.outlook.com ([40.107.94.75]:65089 "EHLO NAM10-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241740AbhJHM2l (ORCPT ); Fri, 8 Oct 2021 08:28:41 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dISF4/g556zWMvUNPyP5Rn316/Ih/+Mhv3MqwYyLodPVb9o/sUVtROVLecmIfE7knSsO8HlO5ANMzkUHuonvUVN3ZcJj1Knodb+/wFsFDSUb3MkyhyHmuOTydvZz2WtnzheoLWZ6l92Qfm2rHBeKpKYrhHwRY9EKcylhQ5xnxzETUG8tnmMbg4eMRb3vNI4SyK31hLIdUP/JOnlH1nfHicPLOuEhSLqnHY9SYOOxm1Ms3hL849HITRWqeAH5UXOCRuPu44nJ6AHSer/Cexzrdhh8pUs/+o8l4dN1hyZibAoWZ5qplPkzxuexv2jEsoqaHDE4ZZrXgveEL1ZkukqJxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QZiHEXyFyddFQS76mKyxB+n7w7kei6ba6k/voWayA2g=; b=kbD4/x5g37xRMTIhAE9ZhImoM7jnn4M35gysUndHZjOKkN56LUOjzCKG+g1Ysvhz66q4cpjwrs6+ef/69ViA4V3lHngtHL9lPJoeM1EtmCdyylHd0jgHO4r04KSPqhA5dN8ddUEVQmNY6SnQk0yT+BFAX68LEiQpYJ0sS5TSzc7aeCunjOIaFqWcsDRBMefunu8XJs5aZ2AsZVjJW8h572Bokeq1XWj5J4dc6cWxtYOPMFmqCbv8UDW7HZXUOuJLv2TG6VvUZfOT5XDJVOqCP/VpkK7ucLLnNqCs2yt1Qkv2BHuPDwsLNG/m6VxL66Fb+jkPPMqYAjkRSKBYrHUCLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QZiHEXyFyddFQS76mKyxB+n7w7kei6ba6k/voWayA2g=; b=YkXCNdigF4MWZ9wV6+cx8PF8Y/EHVniKb2Wvf7Jzc13RmDwYZ/39PPNfQpTt82bNryKoT6Pw//0Fvchq5y0B2iqRvwc5r07fd1+ovxqf6psN9qwWqT3qnU5AKl74AvdeuoDrBB/9VPTywQNElQgTHOVkjAponFn8+A8CS8xVXGoOi/znHbBwNzzxioHgw8urKQE7RstqnilSYJxS7i6M0iCLZ+LuVY8OgCT2kFyO0OKM8xPqRma+hi/wyIhW7iDvbHAX119gtf7ktamqOkxM+DycBZJTsL1SiDn6pxBiX7rovwQkqfDYqV90JdZbmIZouEiCWA/jo1wqjZ2haWIB1Q== Received: from MW4PR03CA0156.namprd03.prod.outlook.com (2603:10b6:303:8d::11) by BL1PR12MB5063.namprd12.prod.outlook.com (2603:10b6:208:31a::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18; Fri, 8 Oct 2021 12:26:43 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8d:cafe::db) by MW4PR03CA0156.outlook.office365.com (2603:10b6:303:8d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.19 via Frontend Transport; Fri, 8 Oct 2021 12:26:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:42 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 05:26:42 -0700 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:37 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , Mark Zhang Subject: [PATCH rdma-next v4 11/13] RDMA/mlx5: Add steering support in optional flow counters Date: Fri, 8 Oct 2021 15:24:37 +0300 Message-ID: <20211008122439.166063-12-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6b31f18e-2937-4766-9ff8-08d98a56e2fa X-MS-TrafficTypeDiagnostic: BL1PR12MB5063: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tdNIgIJ0zHI1hGBfcJ1/vBGk38lFVo8Ixq+BoqCn3IaFGuMeXJdIA2omcnSD2r5pMTfySyk/+cICP9156FpKjmlVuBftI+j8jP20JeLAPUjhbnfgna7phixCflW/ApVBWLjsvd7RgZ5IeDUx3m7G1RI8BsXMN3NMRfBKcZRXWI5YVRfwU85R3u0yoxGkmyDcdwgwftUzLWZpi97foxUkNdsqDC3mq17MnI1bDVvA/mgoCdP47cPbA+1eSu9Se+G3S3Q9uGeVHIXi5EiiXcuC0e+T+YDmEPAhUSYBp6ZNteh8FfCcMA/+gQSBkOTWZtB+25UOdoMNT9JjFDSMBT3f6UZqIU++NRo9Cag5MkJNjYlQvAR8o/sRnxyJ3ULosc0po4CUCC/idh+ungHFNhB1SndvmWUrtbC/FHttpblEJSUbADAbEdCiS3M7hM9Xd/AqhzFTZZl2neMk7Wuf6PJIY4VcwVDHXAp/nDW0lQHSYPwg+3DjVsUMG1SaM3kjzOmpdzU7YN8gncYiT+6GGzhme+ApnSy00Ht/4xzWj5TzOvZ0OptgRxJwhIXVFebmZ8b+KcNe+NfhwNp6LdqDiaD+eDhbYA9AGzP2fZ7jyXnUt3+omk02vUEHO3OkDYbF0ePa05KTvpiEUkajUUYg2ydpoCyzRolFmbd7zefoSd1OGHPM9BepQwkt7LwXxzlo68wF02KkbzU/tA6FnIOFTxPUmg== X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(316002)(2616005)(82310400003)(5660300002)(1076003)(8676002)(107886003)(356005)(8936002)(4326008)(7636003)(36756003)(47076005)(186003)(83380400001)(36860700001)(7416002)(86362001)(6636002)(336012)(2906002)(508600001)(7696005)(54906003)(110136005)(426003)(70206006)(70586007)(26005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:42.8379 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6b31f18e-2937-4766-9ff8-08d98a56e2fa X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5063 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Adding steering infrastructure for adding and removing optional counter. This allows to add and remove the counters dynamically in order not to hurt performance. Signed-off-by: Aharon Landau Reviewed-by: Maor Gottlieb Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/hw/mlx5/fs.c | 187 +++++++++++++++++++++++++++ drivers/infiniband/hw/mlx5/mlx5_ib.h | 24 ++++ include/rdma/ib_hdrs.h | 1 + 3 files changed, 212 insertions(+) diff --git a/drivers/infiniband/hw/mlx5/fs.c b/drivers/infiniband/hw/mlx5/fs.c index 5fbc0a8454b9..b780185d9dc6 100644 --- a/drivers/infiniband/hw/mlx5/fs.c +++ b/drivers/infiniband/hw/mlx5/fs.c @@ -10,12 +10,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include "mlx5_ib.h" #include "counters.h" #include "devx.h" @@ -847,6 +849,191 @@ static struct mlx5_ib_flow_prio *get_flow_table(struct mlx5_ib_dev *dev, return prio; } +enum { + RDMA_RX_ECN_OPCOUNTER_PRIO, + RDMA_RX_CNP_OPCOUNTER_PRIO, +}; + +enum { + RDMA_TX_CNP_OPCOUNTER_PRIO, +}; + +static int set_vhca_port_spec(struct mlx5_ib_dev *dev, u32 port_num, + struct mlx5_flow_spec *spec) +{ + if (!MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, + ft_field_support.source_vhca_port) || + !MLX5_CAP_FLOWTABLE_RDMA_TX(dev->mdev, + ft_field_support.source_vhca_port)) + return -EOPNOTSUPP; + + MLX5_SET_TO_ONES(fte_match_param, &spec->match_criteria, + misc_parameters.source_vhca_port); + MLX5_SET(fte_match_param, &spec->match_value, + misc_parameters.source_vhca_port, port_num); + + return 0; +} + +static int set_ecn_ce_spec(struct mlx5_ib_dev *dev, u32 port_num, + struct mlx5_flow_spec *spec, int ipv) +{ + if (!MLX5_CAP_FLOWTABLE_RDMA_RX(dev->mdev, + ft_field_support.outer_ip_version)) + return -EOPNOTSUPP; + + if (mlx5_core_mp_enabled(dev->mdev) && + set_vhca_port_spec(dev, port_num, spec)) + return -EOPNOTSUPP; + + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, + outer_headers.ip_ecn); + MLX5_SET(fte_match_param, spec->match_value, outer_headers.ip_ecn, + INET_ECN_CE); + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, + outer_headers.ip_version); + MLX5_SET(fte_match_param, spec->match_value, outer_headers.ip_version, + ipv); + + spec->match_criteria_enable = + get_match_criteria_enable(spec->match_criteria); + + return 0; +} + +static int set_cnp_spec(struct mlx5_ib_dev *dev, u32 port_num, + struct mlx5_flow_spec *spec) +{ + if (mlx5_core_mp_enabled(dev->mdev) && + set_vhca_port_spec(dev, port_num, spec)) + return -EOPNOTSUPP; + + MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, + misc_parameters.bth_opcode); + MLX5_SET(fte_match_param, spec->match_value, misc_parameters.bth_opcode, + IB_BTH_OPCODE_CNP); + + spec->match_criteria_enable = + get_match_criteria_enable(spec->match_criteria); + + return 0; +} + +int mlx5_ib_fs_add_op_fc(struct mlx5_ib_dev *dev, u32 port_num, + struct mlx5_ib_op_fc *opfc, + enum mlx5_ib_optional_counter_type type) +{ + enum mlx5_flow_namespace_type fn_type; + int priority, i, err, spec_num; + struct mlx5_flow_act flow_act = {}; + struct mlx5_flow_destination dst; + struct mlx5_flow_namespace *ns; + struct mlx5_ib_flow_prio *prio; + struct mlx5_flow_spec *spec; + + spec = kcalloc(MAX_OPFC_RULES, sizeof(*spec), GFP_KERNEL); + if (!spec) + return -ENOMEM; + + switch (type) { + case MLX5_IB_OPCOUNTER_CC_RX_CE_PKTS: + if (set_ecn_ce_spec(dev, port_num, &spec[0], + MLX5_FS_IPV4_VERSION) || + set_ecn_ce_spec(dev, port_num, &spec[1], + MLX5_FS_IPV6_VERSION)) { + err = -EOPNOTSUPP; + goto free; + } + spec_num = 2; + fn_type = MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS; + priority = RDMA_RX_ECN_OPCOUNTER_PRIO; + break; + + case MLX5_IB_OPCOUNTER_CC_RX_CNP_PKTS: + if (!MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_receive_rdma.bth_opcode) || + set_cnp_spec(dev, port_num, &spec[0])) { + err = -EOPNOTSUPP; + goto free; + } + spec_num = 1; + fn_type = MLX5_FLOW_NAMESPACE_RDMA_RX_COUNTERS; + priority = RDMA_RX_CNP_OPCOUNTER_PRIO; + break; + + case MLX5_IB_OPCOUNTER_CC_TX_CNP_PKTS: + if (!MLX5_CAP_FLOWTABLE(dev->mdev, + ft_field_support_2_nic_transmit_rdma.bth_opcode) || + set_cnp_spec(dev, port_num, &spec[0])) { + err = -EOPNOTSUPP; + goto free; + } + spec_num = 1; + fn_type = MLX5_FLOW_NAMESPACE_RDMA_TX_COUNTERS; + priority = RDMA_TX_CNP_OPCOUNTER_PRIO; + break; + + default: + err = -EOPNOTSUPP; + goto free; + } + + ns = mlx5_get_flow_namespace(dev->mdev, fn_type); + if (!ns) { + err = -EOPNOTSUPP; + goto free; + } + + prio = &dev->flow_db->opfcs[type]; + if (!prio->flow_table) { + prio = _get_prio(ns, prio, priority, + dev->num_ports * MAX_OPFC_RULES, 1, 0); + if (IS_ERR(prio)) { + err = PTR_ERR(prio); + goto free; + } + } + + dst.type = MLX5_FLOW_DESTINATION_TYPE_COUNTER; + dst.counter_id = mlx5_fc_id(opfc->fc); + + flow_act.action = + MLX5_FLOW_CONTEXT_ACTION_COUNT | MLX5_FLOW_CONTEXT_ACTION_ALLOW; + + for (i = 0; i < spec_num; i++) { + opfc->rule[i] = mlx5_add_flow_rules(prio->flow_table, &spec[i], + &flow_act, &dst, 1); + if (IS_ERR(opfc->rule[i])) { + err = PTR_ERR(opfc->rule[i]); + goto del_rules; + } + } + prio->refcount += spec_num; + kfree(spec); + + return 0; + +del_rules: + for (i -= 1; i >= 0; i--) + mlx5_del_flow_rules(opfc->rule[i]); + put_flow_table(dev, prio, false); +free: + kfree(spec); + return err; +} + +void mlx5_ib_fs_remove_op_fc(struct mlx5_ib_dev *dev, + struct mlx5_ib_op_fc *opfc, + enum mlx5_ib_optional_counter_type type) +{ + int i; + + for (i = 0; i < MAX_OPFC_RULES && opfc->rule[i]; i++) { + mlx5_del_flow_rules(opfc->rule[i]); + put_flow_table(dev, &dev->flow_db->opfcs[type], true); + } +} + static void set_underlay_qp(struct mlx5_ib_dev *dev, struct mlx5_flow_spec *spec, u32 underlay_qpn) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 8215d7ab579d..d81ff5078e5e 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -263,6 +263,14 @@ struct mlx5_ib_pp { struct mlx5_core_dev *mdev; }; +enum mlx5_ib_optional_counter_type { + MLX5_IB_OPCOUNTER_CC_RX_CE_PKTS, + MLX5_IB_OPCOUNTER_CC_RX_CNP_PKTS, + MLX5_IB_OPCOUNTER_CC_TX_CNP_PKTS, + + MLX5_IB_OPCOUNTER_MAX, +}; + struct mlx5_ib_flow_db { struct mlx5_ib_flow_prio prios[MLX5_IB_NUM_FLOW_FT]; struct mlx5_ib_flow_prio egress_prios[MLX5_IB_NUM_FLOW_FT]; @@ -271,6 +279,7 @@ struct mlx5_ib_flow_db { struct mlx5_ib_flow_prio fdb; struct mlx5_ib_flow_prio rdma_rx[MLX5_IB_NUM_FLOW_FT]; struct mlx5_ib_flow_prio rdma_tx[MLX5_IB_NUM_FLOW_FT]; + struct mlx5_ib_flow_prio opfcs[MLX5_IB_OPCOUNTER_MAX]; struct mlx5_flow_table *lag_demux_ft; /* Protect flow steering bypass flow tables * when add/del flow rules. @@ -797,6 +806,13 @@ struct mlx5_ib_resources { struct mlx5_ib_port_resources ports[2]; }; +#define MAX_OPFC_RULES 2 + +struct mlx5_ib_op_fc { + struct mlx5_fc *fc; + struct mlx5_flow_handle *rule[MAX_OPFC_RULES]; +}; + struct mlx5_ib_counters { struct rdma_stat_desc *descs; size_t *offsets; @@ -807,6 +823,14 @@ struct mlx5_ib_counters { u16 set_id; }; +int mlx5_ib_fs_add_op_fc(struct mlx5_ib_dev *dev, u32 port_num, + struct mlx5_ib_op_fc *opfc, + enum mlx5_ib_optional_counter_type type); + +void mlx5_ib_fs_remove_op_fc(struct mlx5_ib_dev *dev, + struct mlx5_ib_op_fc *opfc, + enum mlx5_ib_optional_counter_type type); + struct mlx5_ib_multiport_info; struct mlx5_ib_multiport { diff --git a/include/rdma/ib_hdrs.h b/include/rdma/ib_hdrs.h index 7e542205861c..8ae07c0ecdf7 100644 --- a/include/rdma/ib_hdrs.h +++ b/include/rdma/ib_hdrs.h @@ -232,6 +232,7 @@ static inline u32 ib_get_sqpn(struct ib_other_headers *ohdr) #define IB_BTH_SE_SHIFT 23 #define IB_BTH_TVER_MASK 0xf #define IB_BTH_TVER_SHIFT 16 +#define IB_BTH_OPCODE_CNP 0x81 static inline u8 ib_bth_get_pad(struct ib_other_headers *ohdr) { From patchwork Fri Oct 8 12:24:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545213 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3B71C433F5 for ; Fri, 8 Oct 2021 12:26:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BB37E60F44 for ; Fri, 8 Oct 2021 12:26:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241531AbhJHM2v (ORCPT ); Fri, 8 Oct 2021 08:28:51 -0400 Received: from mail-sn1anam02on2040.outbound.protection.outlook.com ([40.107.96.40]:27809 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241815AbhJHM2p (ORCPT ); Fri, 8 Oct 2021 08:28:45 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F70O40m2AJ6I6phKwekeRnqpPPrWmrFBPYlWaF06nRJTwT+yXkNRHfrSFM6qh/uupMXwp9NEPHcxfX8SfxMzvdVSOq2bY5XycFsutGVq2qGgEGoh8DshmhUmis4F3aGv/pJUwMVy3tjHYVnV5eiehMo6+tZfKUwn8MYO9AzLB5PkMMRJiEvjXToGBPCDMgxNsXaRRkPc9atAWjR2IcIpYUz71PYB9AYXh/2FisnLNaGT0RZLbNw2ELhXZqh58W6q4worXskexkDLI88rwwsjPlo3RIWQd7sJR5xepbfoz7R/qldLfXRGkyX+t4Je356atVju3GLeEbgACHdZEJGhsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bFvBgx1x2jxFv68ewEQx8uDbhhJyfNF6dGKK6T+fMds=; b=VWQQ5LnXGm3M+HAZ/sYGy91xz+bcXGuEFUeCffzsJGbX/1P6ofI+N8lnFiCS83nDMRqfxsqFbzzjt61SSNSRNpyfc44iWGqgwXi/IL88bQ1Dl9TIkI6HLqEN/ECqoVjq0wG4DRYo8aQzwc3Fs+Hfr67Lf6eiAs9dKe9GN8mERxz5lVqwsEH5dLw89HKtbvvgrsn/TSwA8LqCJwXq3Bw6wdYm0GTyw/70fPsciu+j9ekaoyr+jINvPRaWGtPQBd5YPGz+UFMmFm7jYjWTcohpGGOR5LxDOKooNMv/i9EyO77WT7SlKZ8yshrCAwt5lhO1YPn6dUQ3Cxbw7/NHxjR7wQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bFvBgx1x2jxFv68ewEQx8uDbhhJyfNF6dGKK6T+fMds=; b=c7IM3sIHrGiwoYacvsFEiNbKNcETJvaglGU7gaJObdRs16CR8sGrLSPweTSVFSgsbuijBZgC68HJpzx8L8T7USQ3L7HFOm3/J61CKED8VGCoLl/x55ZlHum/R0hyU7DqvYy9WD0iJYQ4Tt7I6EHA1GVb9rAr4CjW1IlYowZUYTAQ/Jncwsm5Q3kpQBnwLYE+PDINTiD2DcNw1EFp8nWnyPITqnjLcpt9Jd2SV8LOxq3OmIF9v6iTWg/uSGi3MLq81N89AFlYv8IBeueQwosFNIpixH6iGSZtIeq7GecsyNCnac8l7DjAijICrFJgUXlRuKlsCwOGPLOPCp/UGtRJNg== Received: from BN1PR10CA0016.namprd10.prod.outlook.com (2603:10b6:408:e0::21) by DM5PR1201MB0041.namprd12.prod.outlook.com (2603:10b6:4:56::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.19; Fri, 8 Oct 2021 12:26:48 +0000 Received: from BN8NAM11FT062.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e0:cafe::1e) by BN1PR10CA0016.outlook.office365.com (2603:10b6:408:e0::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT062.mail.protection.outlook.com (10.13.177.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:48 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:47 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:42 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 12/13] RDMA/mlx5: Add modify_op_stat() support Date: Fri, 8 Oct 2021 15:24:38 +0300 Message-ID: <20211008122439.166063-13-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92da45b2-3f0d-45af-3a9a-08d98a56e61e X-MS-TrafficTypeDiagnostic: DM5PR1201MB0041: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1148; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6fiNO3F5TRVjKVZCJwb497X83QuHHpSmLl2GeYfbwBi9xnsOAn1oulLJTbWCd2OxJcWwRsBUytCH9xj5FFAYou/bHPFbRHSwV9CSQRyjk9rTMDpju9sap2I0SuFeO5odWwKlOxj77MRPUtRt/xGt9toMP6jwQ1hx2+rlJ1Jg87ehwm5X0A2nszYd2RdmG9BRIaRZm9YQeTjqqc1mrTr2D5cdzb3SwooRla2ymXYSjPzz/FxuMTyC7QgaDF4x11ZtG4NB+Ri58XLb8BcIXKeds9Ls3CGDVrNRWiW237jol/cgR+67m8uGnMAHfGjGCd1yhLLzeemw77WplEq1fL2kNqPg2yjEcj7lEtWPOEtnYZ3pVbNCWbmC4ZIkB2psEV6D+C9rAtu1MVa8LRsVfGD5mvMLw2w63GrvKhvNTMIr8iwCxwXh0XpPRN6LeeLBK6LdKjwY+emf+d5ZEB9SYX4ebqf7URDfZ/FRZTIru88ch1kHw/sJlp07W3DGp3P/9YTcDvRiZm1nCk5C3yatX3NT+KjPN+4R0goCPnBXz2MJfuh8juvht5+UNO67y6r59q7U7Ixp2QkDhV/SUux+XlgidcCk8FHL3LP9Nyf4NBQfoYsvyyvloFcRZtLqG8z8qdKG2h8EqG8jJ2yTc1ZRdSnaTUyvCM7fRtIW+Ib9ownv4LXaPq+X7pupQrgzZ3ZT2+dMBj+fwgf/WopgsgqKj+LqtA== X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid02.nvidia.com;CAT:NONE;SFS:(4636009)(36840700001)(46966006)(70586007)(36860700001)(107886003)(7696005)(70206006)(4326008)(6636002)(316002)(86362001)(36756003)(5660300002)(356005)(426003)(8676002)(83380400001)(2616005)(7636003)(1076003)(6666004)(54906003)(186003)(508600001)(47076005)(8936002)(2906002)(7416002)(336012)(82310400003)(110136005)(26005);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:48.0538 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 92da45b2-3f0d-45af-3a9a-08d98a56e61e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT062.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0041 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau Add support for ib callback modify_op_stat() to add or remove an optional counter. When adding, a steering flow table is created with a rule that catches and counts all the matching packets; When removing, the table and flow counter are destroyed. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/hw/mlx5/counters.c | 79 +++++++++++++++++++++++++-- drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 + include/rdma/ib_verbs.h | 2 + 3 files changed, 76 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/counters.c b/drivers/infiniband/hw/mlx5/counters.c index 8fe7900b29f0..6ee340c63b20 100644 --- a/drivers/infiniband/hw/mlx5/counters.c +++ b/drivers/infiniband/hw/mlx5/counters.c @@ -12,6 +12,7 @@ struct mlx5_ib_counter { const char *name; size_t offset; + u32 type; }; #define INIT_Q_COUNTER(_name) \ @@ -75,19 +76,19 @@ static const struct mlx5_ib_counter ext_ppcnt_cnts[] = { INIT_EXT_PPCNT_COUNTER(rx_icrc_encapsulated), }; -#define INIT_OP_COUNTER(_name) \ - { .name = #_name } +#define INIT_OP_COUNTER(_name, _type) \ + { .name = #_name, .type = MLX5_IB_OPCOUNTER_##_type} static const struct mlx5_ib_counter basic_op_cnts[] = { - INIT_OP_COUNTER(cc_rx_ce_pkts), + INIT_OP_COUNTER(cc_rx_ce_pkts, CC_RX_CE_PKTS), }; static const struct mlx5_ib_counter rdmarx_cnp_op_cnts[] = { - INIT_OP_COUNTER(cc_rx_cnp_pkts), + INIT_OP_COUNTER(cc_rx_cnp_pkts, CC_RX_CNP_PKTS), }; static const struct mlx5_ib_counter rdmatx_cnp_op_cnts[] = { - INIT_OP_COUNTER(cc_tx_cnp_pkts), + INIT_OP_COUNTER(cc_tx_cnp_pkts, CC_TX_CNP_PKTS), }; static int mlx5_ib_read_counters(struct ib_counters *counters, @@ -451,6 +452,7 @@ static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, for (i = 0; i < ARRAY_SIZE(basic_op_cnts); i++, j++) { descs[j].name = basic_op_cnts[i].name; descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + descs[j].priv = &basic_op_cnts[i].type; } if (MLX5_CAP_FLOWTABLE(dev->mdev, @@ -458,6 +460,7 @@ static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, for (i = 0; i < ARRAY_SIZE(rdmarx_cnp_op_cnts); i++, j++) { descs[j].name = rdmarx_cnp_op_cnts[i].name; descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + descs[j].priv = &rdmarx_cnp_op_cnts[i].type; } } @@ -466,6 +469,7 @@ static void mlx5_ib_fill_counters(struct mlx5_ib_dev *dev, for (i = 0; i < ARRAY_SIZE(rdmatx_cnp_op_cnts); i++, j++) { descs[j].name = rdmatx_cnp_op_cnts[i].name; descs[j].flags |= IB_STAT_FLAG_OPTIONAL; + descs[j].priv = &rdmatx_cnp_op_cnts[i].type; } } } @@ -535,7 +539,7 @@ static void mlx5_ib_dealloc_counters(struct mlx5_ib_dev *dev) { u32 in[MLX5_ST_SZ_DW(dealloc_q_counter_in)] = {}; int num_cnt_ports; - int i; + int i, j; num_cnt_ports = is_mdev_switchdev_mode(dev->mdev) ? 1 : dev->num_ports; @@ -550,6 +554,18 @@ static void mlx5_ib_dealloc_counters(struct mlx5_ib_dev *dev) } kfree(dev->port[i].cnts.descs); kfree(dev->port[i].cnts.offsets); + + for (j = 0; j < MLX5_IB_OPCOUNTER_MAX; j++) { + if (!dev->port[i].cnts.opfcs[j].fc) + continue; + + mlx5_ib_fs_remove_op_fc(dev, + &dev->port[i].cnts.opfcs[j], + j); + mlx5_fc_destroy(dev->mdev, + dev->port[i].cnts.opfcs[j].fc); + dev->port[i].cnts.opfcs[j].fc = NULL; + } } } @@ -729,6 +745,56 @@ void mlx5_ib_counters_clear_description(struct ib_counters *counters) mutex_unlock(&mcounters->mcntrs_mutex); } +static int mlx5_ib_modify_stat(struct ib_device *device, u32 port, + unsigned int index, bool enable) +{ + struct mlx5_ib_dev *dev = to_mdev(device); + struct mlx5_ib_counters *cnts; + struct mlx5_ib_op_fc *opfc; + u32 num_hw_counters, type; + int ret; + + cnts = &dev->port[port - 1].cnts; + num_hw_counters = cnts->num_q_counters + cnts->num_cong_counters + + cnts->num_ext_ppcnt_counters; + if (index < num_hw_counters || + index >= (num_hw_counters + cnts->num_op_counters)) + return -EINVAL; + + if (!(cnts->descs[index].flags & IB_STAT_FLAG_OPTIONAL)) + return -EINVAL; + + type = *(u32 *)cnts->descs[index].priv; + if (type >= MLX5_IB_OPCOUNTER_MAX) + return -EINVAL; + + opfc = &cnts->opfcs[type]; + + if (enable) { + if (opfc->fc) + return -EEXIST; + + opfc->fc = mlx5_fc_create(dev->mdev, false); + if (IS_ERR(opfc->fc)) + return PTR_ERR(opfc->fc); + + ret = mlx5_ib_fs_add_op_fc(dev, port, opfc, type); + if (ret) { + mlx5_fc_destroy(dev->mdev, opfc->fc); + opfc->fc = NULL; + } + return ret; + } + + if (!opfc->fc) + return -EINVAL; + + mlx5_ib_fs_remove_op_fc(dev, opfc, type); + mlx5_fc_destroy(dev->mdev, opfc->fc); + opfc->fc = NULL; + return 0; +} + static const struct ib_device_ops hw_stats_ops = { .alloc_hw_port_stats = mlx5_ib_alloc_hw_port_stats, .get_hw_stats = mlx5_ib_get_hw_stats, @@ -737,6 +803,7 @@ static const struct ib_device_ops hw_stats_ops = { .counter_dealloc = mlx5_ib_counter_dealloc, .counter_alloc_stats = mlx5_ib_counter_alloc_stats, .counter_update_stats = mlx5_ib_counter_update_stats, + .modify_hw_stat = mlx5_ib_modify_stat, }; static const struct ib_device_ops hw_switchdev_stats_ops = { diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index d81ff5078e5e..cf8b0653f0ce 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -821,6 +821,7 @@ struct mlx5_ib_counters { u32 num_ext_ppcnt_counters; u32 num_op_counters; u16 set_id; + struct mlx5_ib_op_fc opfcs[MLX5_IB_OPCOUNTER_MAX]; }; int mlx5_ib_fs_add_op_fc(struct mlx5_ib_dev *dev, u32 port_num, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 2207f60b002f..7688720411a3 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -553,10 +553,12 @@ enum ib_stat_flag { * struct rdma_stat_desc * @name - The name of the counter * @flags - Flags of the counter; For example, IB_STAT_FLAG_OPTIONAL + * @priv - Driver private information; Core code should not use */ struct rdma_stat_desc { const char *name; unsigned int flags; + const void *priv; }; /** From patchwork Fri Oct 8 12:24:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Zhang X-Patchwork-Id: 12545217 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C62C6C433EF for ; Fri, 8 Oct 2021 12:27:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD9DF60FC2 for ; Fri, 8 Oct 2021 12:27:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241701AbhJHM3O (ORCPT ); Fri, 8 Oct 2021 08:29:14 -0400 Received: from mail-dm6nam12on2075.outbound.protection.outlook.com ([40.107.243.75]:47488 "EHLO NAM12-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S242005AbhJHM2u (ORCPT ); Fri, 8 Oct 2021 08:28:50 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BxI2JyCeE57ntCTjb9XTGl/rKnLgCpdI2I64j/LbmzqdAmQE+Ntrd+zeNEWUtxu3Ly66dP6ZOo8J1dCShKVdwwFoTbOI9WbOA+Gg26GT8SRFVu3rT3F/t9I6ERTJwKXu3IeP9vMMn+Cl9MZ5MpfRNZLlTJ6CtBth0j5DXi0vBHS+atJ+QoqRwgAEtTbgrTKWETS1Uv2faHstN8y3WqmhlN8pVwPVNgA8UoSKu/4j0/8IyP51nw0oSncqOJc+99f3zl1kb0qRiXasdh+w4t63DwpqHOem8ae+LKFI+W1ZjY5UND1rFayn5JRB25zWrsdX3Sl4HW68FwnjBugR2eulxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W3rPJQHfvyOG47C86ZWTy/4sGzJkUYPvDjb7YjmoTEU=; b=jP90pLz7dmTtkk8YoI4uxE/fzmD1PgXKD5mMwDH6Cf37E7YCdeb35PGQkx14VNQohjOxKGSbaJj1wQofJKpGfoPZMOKWp7ZvbmlYoAeD5UQNdnEAGYNyhub/ZYL8cxpFCkSWMv88d1u84p6aELq7UqP6+qHyq15HHnXsgwS3gizeDbhQPsZ958rLwYKhqwxnwnW+yBX6Dj7GveCeSHpEDvrZCIZX0xQxUXBOsN82Wjgt62UIzWc8NnsnwgZSDKRL8n8rs+/nyd/m8CnwLVNrR0FDToP75LGcF8pFvvYsXcDTW0icTonz1fATqFEriJEkz4nJS8F4RsSTCHzgAjA+tQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=chelsio.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W3rPJQHfvyOG47C86ZWTy/4sGzJkUYPvDjb7YjmoTEU=; b=gX8E2VhQOxkiolUiQdDZntGV+wvFMaqs8v7DLZdTQWHXDAW4cc32nqEgOsJ3HXxYA4E1WCbDzrKqa0Lwj/Tdrpq+j08V3k/jei+LVPG+lijIOyPc18EARvP/fhbJxHSEfjyXLgNn12YAuQ9oxHRi14PNQ4t5M9YW/yuZdbjJfFOVXcwnsNQKBH/Jp+9/lc1ERy2GuqD6hXt9EgKKJc7iRrHHadZsF1fjjBGB8ltScl16+QVjb8Brf8kZLAihuo/szDp7IxgAPO8OcR7OB2b8u967nO53q0bxlOSs8D1WdUQ6UhXmNTJQcIOI+4mWOqRWX1B4EuChtT9TwiosEaFqLg== Received: from BN9PR03CA0179.namprd03.prod.outlook.com (2603:10b6:408:f4::34) by CH2PR12MB3734.namprd12.prod.outlook.com (2603:10b6:610:2a::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.20; Fri, 8 Oct 2021 12:26:53 +0000 Received: from BN8NAM11FT067.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f4:cafe::33) by BN9PR03CA0179.outlook.office365.com (2603:10b6:408:f4::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.22 via Frontend Transport; Fri, 8 Oct 2021 12:26:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; chelsio.com; dkim=none (message not signed) header.d=none;chelsio.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT067.mail.protection.outlook.com (10.13.177.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4587.18 via Frontend Transport; Fri, 8 Oct 2021 12:26:52 +0000 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 8 Oct 2021 12:26:51 +0000 Received: from vdi.nvidia.com (172.20.187.6) by mail.nvidia.com (172.20.187.15) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Fri, 8 Oct 2021 05:26:47 -0700 From: Mark Zhang To: , , CC: , , , , , , , , , , , , , , , , "Mark Zhang" Subject: [PATCH rdma-next v4 13/13] RDMA/mlx5: Add optional counter support in get_hw_stats callback Date: Fri, 8 Oct 2021 15:24:39 +0300 Message-ID: <20211008122439.166063-14-markzhang@nvidia.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20211008122439.166063-1-markzhang@nvidia.com> References: <20211008122439.166063-1-markzhang@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 98b34a54-bb2d-466f-0fb8-08d98a56e8bc X-MS-TrafficTypeDiagnostic: CH2PR12MB3734: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mf4GAo1VyCj4t3AS68jM6ypPLZ6LPwQRZ+RNzwaXWLmoqSY5Y/mcDe8ZldqmL7eDR6+k4am9/PC/ZL3eVZTLuM3IB/KAxrEdbos+eVAT3obTv1FHIR7yfxGp4LyzI2Iwzn0dS4xqw6LsEWUAvO4dqw4zkPyOQ9fNcWlFDFwOKKWf/YkNeWxnxt8O2pDKVcgZwTVf1mzDYBGf1baX0gFLkKIw3plgyFs9n7dpKCeIHGTaGdYnvHWLXBWC8JY7tPoL4gfNllC2I4+tOGAzxgDSn9NjFipZFl8uie+9hhmVadcz5XbGoNqTu9EHD5eJ7QO5YCRdbQkO9cErvLXnJWSWyqqtvE70/SqoafuGwwVcQ37dUCN0mYf5M92hRv668y6gDqNqcY39opGC0bCymlbYZ7f82agzrE482CU+U9cMYccS358AqPUJbhfHi7v84yUXjE0/XyYI7kmJOHit8Ddu7n9wMIoKeU3x1GsN3oNl/DfeXWJr4U9qZZpFrVnG/Njr+Viwqd1Qui0Ck9t2multrWzT5CHBNuvaV8kyDKzlYlVON+P34ZvYt7wtiHg+tUlaaUupsra3J/1vKXVXBszKMgnGE2nLp9khjJops67siM+Rg9loOz4G+xNZqYuZUzhmLXDhs5Dy1T7TO5bpb3VZiE9nfflkmM4iQOJI+UMs6jbnCNEefsK5i91cS03rq0Ruo3pM30/sMH1NKHJur1biOg== X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid02.nvidia.com;CAT:NONE;SFS:(4636009)(46966006)(36840700001)(36860700001)(508600001)(356005)(47076005)(5660300002)(2906002)(1076003)(70206006)(70586007)(7636003)(8936002)(316002)(4326008)(110136005)(7416002)(82310400003)(6636002)(83380400001)(6666004)(336012)(426003)(186003)(54906003)(36756003)(107886003)(8676002)(26005)(2616005)(7696005)(86362001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Oct 2021 12:26:52.4506 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 98b34a54-bb2d-466f-0fb8-08d98a56e8bc X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT067.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3734 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Aharon Landau When get_hw_stats is called, query and return the optional counter statistic as well. Signed-off-by: Aharon Landau Signed-off-by: Leon Romanovsky Signed-off-by: Mark Zhang --- drivers/infiniband/hw/mlx5/counters.c | 88 ++++++++++++++++++++++++++- 1 file changed, 85 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/counters.c b/drivers/infiniband/hw/mlx5/counters.c index 6ee340c63b20..6f1c4b57110e 100644 --- a/drivers/infiniband/hw/mlx5/counters.c +++ b/drivers/infiniband/hw/mlx5/counters.c @@ -270,9 +270,9 @@ static int mlx5_ib_query_ext_ppcnt_counters(struct mlx5_ib_dev *dev, return ret; } -static int mlx5_ib_get_hw_stats(struct ib_device *ibdev, - struct rdma_hw_stats *stats, - u32 port_num, int index) +static int do_get_hw_stats(struct ib_device *ibdev, + struct rdma_hw_stats *stats, + u32 port_num, int index) { struct mlx5_ib_dev *dev = to_mdev(ibdev); const struct mlx5_ib_counters *cnts = get_counters(dev, port_num - 1); @@ -324,6 +324,88 @@ static int mlx5_ib_get_hw_stats(struct ib_device *ibdev, return num_counters; } +static int do_get_op_stat(struct ib_device *ibdev, + struct rdma_hw_stats *stats, + u32 port_num, int index) +{ + struct mlx5_ib_dev *dev = to_mdev(ibdev); + const struct mlx5_ib_counters *cnts; + const struct mlx5_ib_op_fc *opfcs; + u64 packets = 0, bytes; + u32 type; + int ret; + + cnts = get_counters(dev, port_num - 1); + opfcs = cnts->opfcs; + type = *(u32 *)cnts->descs[index].priv; + if (type >= MLX5_IB_OPCOUNTER_MAX) + return -EINVAL; + + if (!opfcs[type].fc) + goto out; + + ret = mlx5_fc_query(dev->mdev, opfcs[type].fc, + &packets, &bytes); + if (ret) + return ret; + +out: + stats->value[index] = packets; + return index; +} + +static int do_get_op_stats(struct ib_device *ibdev, + struct rdma_hw_stats *stats, + u32 port_num) +{ + struct mlx5_ib_dev *dev = to_mdev(ibdev); + const struct mlx5_ib_counters *cnts; + int index, ret, num_hw_counters; + + cnts = get_counters(dev, port_num - 1); + num_hw_counters = cnts->num_q_counters + cnts->num_cong_counters + + cnts->num_ext_ppcnt_counters; + for (index = num_hw_counters; + index < (num_hw_counters + cnts->num_op_counters); index++) { + ret = do_get_op_stat(ibdev, stats, port_num, index); + if (ret != index) + return ret; + } + + return cnts->num_op_counters; +} + +static int mlx5_ib_get_hw_stats(struct ib_device *ibdev, + struct rdma_hw_stats *stats, + u32 port_num, int index) +{ + int num_counters, num_hw_counters, num_op_counters; + struct mlx5_ib_dev *dev = to_mdev(ibdev); + const struct mlx5_ib_counters *cnts; + + cnts = get_counters(dev, port_num - 1); + num_hw_counters = cnts->num_q_counters + cnts->num_cong_counters + + cnts->num_ext_ppcnt_counters; + num_counters = num_hw_counters + cnts->num_op_counters; + + if (index < 0 || index > num_counters) + return -EINVAL; + else if (index > 0 && index < num_hw_counters) + return do_get_hw_stats(ibdev, stats, port_num, index); + else if (index >= num_hw_counters && index < num_counters) + return do_get_op_stat(ibdev, stats, port_num, index); + + num_hw_counters = do_get_hw_stats(ibdev, stats, port_num, index); + if (num_hw_counters < 0) + return num_hw_counters; + + num_op_counters = do_get_op_stats(ibdev, stats, port_num); + if (num_op_counters < 0) + return num_op_counters; + + return num_hw_counters + num_op_counters; +} + static struct rdma_hw_stats * mlx5_ib_counter_alloc_stats(struct rdma_counter *counter) {