From patchwork Tue Jan 30 01:30:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536642 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2074.outbound.protection.outlook.com [40.107.223.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDEDB364A3 for ; Tue, 30 Jan 2024 01:31:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.74 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578267; cv=fail; b=KSfadqKi3VD3yt+muiw+P42fh3pUFZI9BkBX8JXZLDRktyDxreQ5IzrYwwz3dXGOetZmymR/3Vib4DmkqwnFkD7SH3xWaz0bXPpVtxp9C3taGfg+35l3UOhBc4a7j738JqKD5Jw4fecC+VfkBpDFwXAAr0Rf9olns2NxWDQ9YNE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578267; c=relaxed/simple; bh=sNszYVzy97tZvq6Lf4y5DUNzPnBJqXe4AyLe/nUH2S0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NDd11i73NCDN6RUTjs6AId1tffadXy6JU8gLfpUyvlSb1lEWpMfGCzVLaOQUurIGIS+eArGr8RLmKUJEQjlUvep9Zf6F0l4t9AgIrghGw3UH1Lno6hrj22Bcbi2pCWED2KQeYuGXb4joVAVELpJefylGxLUSYk9vbGhyQP0cj8k= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=w0FQt68b; arc=fail smtp.client-ip=40.107.223.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="w0FQt68b" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PYcuXzbgnIPbY1qpfBWDXN5hCsu4P+haAgpu28c7gnx/qMefS6XW8B0kfi9kGQ7MtdCFeKgJ2OLIzuAsVpjzPR80fk81l4MI0iCN7HI1M2m+mIzjhAy+cYNkCCjNT9A1q8Sl5l6AQYtl1AcJI+WSwvpmG3Sbs1aTDHFPoSHb8V7RZXXQVClBXqiCe4phSTsC1NY20xVz/oUGt3TqVTNTIp7/UZzD5q08VJk2yeCHEbx61pu4BbPO+mRyATie4FM0s7wAixFKb6mi6lx0aaKVw1eIqD4UxhvegiZgJN6X2xb+JEKK6Ku5jw4vKNkXnLfjRoDVO1RY9QOuSgo0zP423w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Iuqd0UJRY/8Pef92ox+L6iZYimdz3aw3Vpdb8kHYg9I=; b=Y2MUlnvWVGtpRD6Lwik+anZZdRq0IwpSKnRJPVh7FkgaGMyS3w6sIbojX1jHxY5KO1xcxrN3Eq2ZC+awPkSYyPlIzs5i8VsIkEqXAxNmnGGLtH3dR6yj08JsAu5EJYwMqqPLeT/MyGU5HylprcEiyFlTSADSJGFv+zMCzaLkl7W5/1WvQdQl5uQHUtarDVdr7z6iYXZ0z3sG4/zTFIjVgwx4jQE44Cg5m9kG+3izSJvfrEWowdUItmb5bD5+wjSGFe+S3ABVn9ja+s+bvXAcItOHONTixqJVKZuKzcuUiWjr4RSzRPn4Ag9GZA1IXVTWI3QtxlvmUD9qNukpivydeg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Iuqd0UJRY/8Pef92ox+L6iZYimdz3aw3Vpdb8kHYg9I=; b=w0FQt68bxL3F3MYLJHaxOnRNLMcBM0M13P3c/88/ScYQfq+6+6ynQ87bzLoXQAvI0dAX+SFgkuXsDKfNEe+KZ2CSVBYUyxyDxv26a1kp2N24ygIPwqwCNaH+iFU+yg8FlBk+IhdV09okJyOv64e5LMoZvmqx8vLarMQ2QEIUtxQ= Received: from BL1P223CA0028.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::33) by SN7PR12MB7836.namprd12.prod.outlook.com (2603:10b6:806:34e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 01:31:02 +0000 Received: from MN1PEPF0000F0E4.namprd04.prod.outlook.com (2603:10b6:208:2c4:cafe::9d) by BL1P223CA0028.outlook.office365.com (2603:10b6:208:2c4::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E4.mail.protection.outlook.com (10.167.242.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:02 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:01 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 1/9] ionic: set adminq irq affinity Date: Mon, 29 Jan 2024 17:30:34 -0800 Message-ID: <20240130013042.11586-2-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E4:EE_|SN7PR12MB7836:EE_ X-MS-Office365-Filtering-Correlation-Id: 36f52653-90f4-4e46-76fe-08dc21331f13 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ClGYvpO7GaehkLkVAoU4ik10iA3Fx/ZdZan1EuCfxNFDt1yblV0grHQeM+5yB+dxk/i8qaA8n7XIHWniUGslW2/1qMaqHnrbrRxnWsLrRVN3BBJlVtx8/h8YlXzxzRpGwdSuavrvKFWMFa19pkvlE4tF5QwQYmwv+ovzG9vpL5AmrfRN+k7mt3jDH+vF2/y0weyceevESwdaZAazVUlvC8M9oNnBGFK68PiuSlmKkEA/z0NwNuIYzVrciT1uYzpqoLPDyCDCKaRATQJBbQDqvol/9k2e4ukLfkiVqgyqqxF7wgBnxN1ixml/6VNKbf7dazyemlDTwHOOU/sJEsOZTG6+ZU+Yr1cL1ktH+oiAKVcY9J+wOFWQ19uCMUJTH72ZyVE0EMMyIMCYidOuc6xWgmM+UGBNyuDufUjELkZyXTaaU3iHUsZp89fkRd1qQxkueTAo5m96r9sOfrBh3maR5JU++QfUf4ejji60SII+P2D5f1Zc0uYjUtMlcs5H8sQIbggorahd454PdF4l9soqKhAqPiSTBpOCnuWHre4kGIw67oqEKi2/JQLRcYVIwCUOLMoXh4+nWTEhTp7dj/0VWS4tVNl/6p3MPAymWhpXc6oUhdtXJv/RtlRNSXq40br+MSQKQcl+IvTgU9ysVyW8uuHKNzT/3oIEkRMDqnTzmN8gVJm8199WgDT655VAC4QxuvSZFfh9QiyQjBDsBH8Jft84eFQM3N/+wMIHydTFzKKgAkRzayj8B9nx6+RkHWkvGBV0EmsfYsOdlbQFfafF+w== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(396003)(136003)(346002)(230922051799003)(64100799003)(82310400011)(451199024)(1800799012)(186009)(40470700004)(46966006)(36840700001)(4326008)(336012)(426003)(36860700001)(47076005)(478600001)(41300700001)(36756003)(86362001)(82740400003)(83380400001)(81166007)(356005)(8936002)(2616005)(110136005)(316002)(70206006)(1076003)(44832011)(70586007)(26005)(4744005)(54906003)(2906002)(6666004)(8676002)(16526019)(5660300002)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:02.7717 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 36f52653-90f4-4e46-76fe-08dc21331f13 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7836 X-Patchwork-Delegate: kuba@kernel.org We claim to have the AdminQ on our irq0 and thus cpu id 0, but we need to be sure we set the affinity hint to try to keep it there. Signed-off-by: Shannon Nelson --- drivers/net/ethernet/pensando/ionic/ionic_lif.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index cf2d5ad7b68c..d92f8734d153 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -3391,9 +3391,12 @@ static int ionic_lif_adminq_init(struct ionic_lif *lif) napi_enable(&qcq->napi); - if (qcq->flags & IONIC_QCQ_F_INTR) + if (qcq->flags & IONIC_QCQ_F_INTR) { + irq_set_affinity_hint(qcq->intr.vector, + &qcq->intr.affinity_mask); ionic_intr_mask(idev->intr_ctrl, qcq->intr.index, IONIC_INTR_MASK_CLEAR); + } qcq->flags |= IONIC_QCQ_F_INITED; From patchwork Tue Jan 30 01:30:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536646 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2069.outbound.protection.outlook.com [40.107.95.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57E0D364AB for ; Tue, 30 Jan 2024 01:31:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.95.69 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578271; cv=fail; b=itehjcwYrT7hczaeRMqXC38QXVzd0CHPXauKJS+J9VWK0kMHwEwwO0DQW+0GMoCTIF1PHOz4/OjcVeyGlhIMRjnqo45K2kQWhMlofN1onmNfWDz9LtVF1mANtJNV2iq8BX6+48bML1quV+Od3froLBJUYRUIaBWPg5Me/L+WDqQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578271; c=relaxed/simple; bh=iL5lXyQWAEFKMD1KCjiJfwYmdVvwvQxMEdCZgH/4mMQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jMtOB/cmL4CwizEVjdwSXOUpRMOXn1RNmQHVh+YWi02Ek1okEPfO5bnKqrN2KHy8VtkoZhOmDAxPAhjHP2Ravv2XStAob5y5VvFq26Uo5g5RTyRWdiAxfqcFDNV1CbPu2HwNb0UA5/rwKdqHurZOdY/lXpGP0iSAI40TOuxPawE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=kcdZjzli; arc=fail smtp.client-ip=40.107.95.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="kcdZjzli" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FNw3TVxbvO6Jq4oHVPbFc0kkiG9qqOr3n0FT7u9mctP5NyNk/d/s9buQfB4MHNXrW/s5un6CwwA7zcfoowu4WE1Eq9ljkpXAy1cr3afxiEhHbI4oKhhKSdeVGg5eM9vdwIQnCFA6nzL0psEjQ8tZhezjP98jH8s2+ifXJqdiV3cSZa8aMXJiA5LdmTRLEJpRvwEw7Z2gsFFnz3RSJ4aH1pwuImCFIK0hLdBAiFm8UhFmVzb/9+zWEkKZ/HGcbYEaytf1V7mxvn1cOBm6jTZ9XcnK9DcQqRioC3e3U6e1UmuM9x2c1TUTa/pPZmDR+PctXCWwWvE2tbZ2cIpJD4uBmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=t301f4OhCMUG6dYYrx2rgTmyo0jKwpEP9FVmIjeyQDU=; b=jiXDrgvDmd572WyuzSVIJCEZRDUuZlTJNDw7Kvj5FdmGKOaJAQnLz3numUeyYfa7b4WgXJaMPsz5Qy+u2oxUSZr/fHOz3Vr1YAhaUG+uNEg6bgynO30pgznVc2Jka510YbGpucdBSHYBafYnxZujoke+Qe2Yb2pe9Upq3WfAN3A1upwsSl351UsLmhQCZ34R2SNTRkFtpz9o2iiwgD+eatbFLfIdc9xtcLKvw/Kq81JdU0rhNbri9Fcu74oZ92aeEW8Cu/kk856i6WRkKypjpIM9MZK9W+aoc6G1nsaOQ6C8DdCfx0j8Hm0HDs1ebKykdQJwT398P8yq21HpDbp4sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=t301f4OhCMUG6dYYrx2rgTmyo0jKwpEP9FVmIjeyQDU=; b=kcdZjzlifxK1HgK2AmA92aBk4gvIkPnCg31w1upBnlAAQcricwXlUqlWnj+abP3BXuMYTW5TtyCk3yw4oSx54tpmeVJ+0mffuPGddH4k171JWae/E8cAkwHHFaBtTVExsOQgyd1SMrqOrg3uEHFDld9I7UPAjM9ycvr1Gkuv4pI= Received: from BL1P223CA0021.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::26) by MW4PR12MB6997.namprd12.prod.outlook.com (2603:10b6:303:20a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.26; Tue, 30 Jan 2024 01:31:04 +0000 Received: from MN1PEPF0000F0E4.namprd04.prod.outlook.com (2603:10b6:208:2c4:cafe::b0) by BL1P223CA0021.outlook.office365.com (2603:10b6:208:2c4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E4.mail.protection.outlook.com (10.167.242.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:04 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:02 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 2/9] ionic: add helpers for accessing buffer info Date: Mon, 29 Jan 2024 17:30:35 -0800 Message-ID: <20240130013042.11586-3-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E4:EE_|MW4PR12MB6997:EE_ X-MS-Office365-Filtering-Correlation-Id: 05946122-b80a-48bd-b912-08dc21332038 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: d8OyDoR6DJGsWgrVnYwVB1Xcc0Gf7Nynm4j8nzVuFRE7kvPvi5T8/UYXb6NhYtQwfxbCx43tQ8HPm0wPSbXViVupQ+eFepiJsrOzrV3QCIS/0w+ytjXnv/8AivFq6oWCMxvEjoqQav76WcCME4wjwGGmstDpowbERW9/YeXP4rkC9nEITcf54Z0EZyWV7d1sWcA7HcqdAJJhxTxiq92XUDA7YUa67cgOdXSuC0YPs4nLSoCtcRqV0K5aPa9I3iPNsYJQfXkkhvHCe42njdXIeBQhCb5EnZcHSiDgC71/kAJ834PWBACHS+n4dG8MgA8fl/XUhw84/49s1/+E6+6l6xA2S0D3IruMVAqfwOb0Vrr3RvmjZRoE696IKqwIhF2BLX4nnZiqzijZsD/eLLfd7Qb9DpEjKlaMNpO0Vh2VXg/+YNhevNdm3Em5bvG/AKz8Hx6lKoqBOITy46yiTdR0HdbKLQMQ++2gNBlT77gFrZZkGwDGuib5eXyoxFfCfT+66Aewth8X0Y4ZhIghHzyAKFe1e1EAsHbaXrgj6efXw5UxO7f8jU24dTZtkOUChk+mizjfUaABrkdtjV5VYhh1SYVPaAhUl8ilrG4Q88/rO7kC2JMyWgUhe5s9XV6cw058Gwn8zRZiRQEYxTjOXFW0UQF1t1wq24wG0Qtn02t466puNyeDfOTtaarUZTBD2r70DJYqhrNsyde4K7+RvvMZZgDjrGm+VThLgc9gcER7F5BugqA5HBkSxfQqBVbMggQZAvAyBI4MKLkMddvkyH+BfQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(39860400002)(346002)(136003)(230922051799003)(451199024)(1800799012)(186009)(82310400011)(64100799003)(46966006)(36840700001)(40470700004)(83380400001)(16526019)(426003)(1076003)(6666004)(336012)(478600001)(36860700001)(316002)(47076005)(8936002)(8676002)(81166007)(5660300002)(4326008)(44832011)(41300700001)(2616005)(70586007)(2906002)(54906003)(110136005)(70206006)(82740400003)(26005)(36756003)(86362001)(356005)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:04.6936 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 05946122-b80a-48bd-b912-08dc21332038 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6997 X-Patchwork-Delegate: kuba@kernel.org These helpers clean up some of the code around DMA mapping and other buffer references, and will be used in the next few patches for the XDP support. Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_txrx.c | 37 ++++++++++++------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 54cd96b035d6..19a7a8a8e1b3 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -88,6 +88,21 @@ static inline struct netdev_queue *q_to_ndq(struct ionic_queue *q) return netdev_get_tx_queue(q->lif->netdev, q->index); } +static inline void *ionic_rx_buf_va(struct ionic_buf_info *buf_info) +{ + return page_address(buf_info->page) + buf_info->page_offset; +} + +static inline dma_addr_t ionic_rx_buf_pa(struct ionic_buf_info *buf_info) +{ + return buf_info->dma_addr + buf_info->page_offset; +} + +static inline unsigned int ionic_rx_buf_size(struct ionic_buf_info *buf_info) +{ + return min_t(u32, IONIC_MAX_BUF_LEN, IONIC_PAGE_SIZE - buf_info->page_offset); +} + static int ionic_rx_page_alloc(struct ionic_queue *q, struct ionic_buf_info *buf_info) { @@ -207,12 +222,11 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, return NULL; } - frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - buf_info->page_offset)); + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; dma_sync_single_for_cpu(dev, - buf_info->dma_addr + buf_info->page_offset, + ionic_rx_buf_pa(buf_info), frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, @@ -262,10 +276,10 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return NULL; } - dma_sync_single_for_cpu(dev, buf_info->dma_addr + buf_info->page_offset, + dma_sync_single_for_cpu(dev, ionic_rx_buf_pa(buf_info), len, DMA_FROM_DEVICE); - skb_copy_to_linear_data(skb, page_address(buf_info->page) + buf_info->page_offset, len); - dma_sync_single_for_device(dev, buf_info->dma_addr + buf_info->page_offset, + skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); + dma_sync_single_for_device(dev, ionic_rx_buf_pa(buf_info), len, DMA_FROM_DEVICE); skb_put(skb, len); @@ -452,9 +466,8 @@ void ionic_rx_fill(struct ionic_queue *q) } /* fill main descriptor - buf[0] */ - desc->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - buf_info->page_offset)); + desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); desc->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; @@ -472,10 +485,8 @@ void ionic_rx_fill(struct ionic_queue *q) } } - sg_elem->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, remain_len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - - buf_info->page_offset)); + sg_elem->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); + frag_len = min_t(u16, remain_len, ionic_rx_buf_size(buf_info)); sg_elem->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; From patchwork Tue Jan 30 01:30:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536644 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2058.outbound.protection.outlook.com [40.107.244.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80FAE36132 for ; Tue, 30 Jan 2024 01:31:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578270; cv=fail; b=V9nx/SDZAkZigMEKzt3dEPZlnJX+Tuk5DftwQKtDcGBlpvwh9+Z7c7ubUy5jasYD9p+88S04epf5o1CK1b6hqQJ5h/ikFFU0+tagODYL4jDToQrgyIbXr+/VKQ9mlpE7BCELkkYQzPWP/4NLe9l2Jj5hGi25OcEAQcvOelPd0Hg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578270; c=relaxed/simple; bh=0wIeKF9Dt1bKwFHXNxtRsnn73s3kJFATJ5BrI5qA6D8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Rq/twHb2iFNFt1V9lCVTJsv/CjMJz/ClSduAQDg6AtcpXuCC/nBSP4N7XCmL2hA5Cwj5ZLhJJzvEoXvbojBFKt0Oc/mjNHsTu9soirDHSZWyLNf3D68lfo3M3wEv4mhYd7LTeZTrwuBecIpN6Zhp7FQVyhHYAjKhb/3BTZi4JG0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=jVZ9MhxA; arc=fail smtp.client-ip=40.107.244.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="jVZ9MhxA" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V+x60dxWj4IdlysnFTx8MpNiy0pIilycHHy2lVBKI+ehGhSN6y2MEXBQzmCeSxlJmoXfeONRniWS9RnzFhWSzpufFO2FPxr/LUyZGMFVWZ4A/vIGR1SjyGjzhwWJ5+daUgt0zPyBnlowDvs3AnErK09YFGcCl3mG9h5KKdfOb3FmB927QVXU9czK5vKYolzjd82qVKS6Uu46KbAoAho+88k/po9H1yIwi9cBcoYgt8jqAdfD2drVIxUaX726z+gyvztnO6OQoLbaLy4jQ2dIaSEPhfDfEQJUIzJIApuI76mGOAeaBIbw3cCMmhjui1n6Mv/7DQJ2dW7PFkP0+Raz0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xB6sc+32yCgiNi4g2j9bFkm4E2q3QAPB3kdRX/SP8G0=; b=k0U9DPsg6kKLD+px/1/SP3ijXczCi9X4bprDraCto2KDOYJmza1Zc7UyUaWwqoX0OyF7Nv3FaSIHl5DWfF4UqY0zRNCGkQj1Hr26koexrgu1HzwGpsXFG8pV8LeDklxtgR0l4RUvpJyZB94BjYGct1ntZWa0yDvz6KDLfdvOUvFYUZsKWxwHzS5r2NMPeg9SMBBydVv+BuJSTF4TqpkgcUFGIQ7L+QRM6KAsUT9D+QC5+V6nyBnA5bTjxVm4QRfp/XiSwKPQcuw22UvSAuZOjU/SuHP+UQYGR8/glYPgyIvFYosf5C87/3ScAVZTeQfRC92/k3teffFRtmWuyZZR8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xB6sc+32yCgiNi4g2j9bFkm4E2q3QAPB3kdRX/SP8G0=; b=jVZ9MhxAguBFW4L4MRDfjXDHrPqq652ejglc8ZcnBOQdsNyEsw12hgSFWMZu0pU/wfKe2p5AkMq2Zlw5Xaf5Z+QYb2/Q/xw6IHfETh18aVOuOkKQKpXl00qIielrlcodk2BKhxlggihT9WP937P0zgQZcXBjRRl3MxOCDqkI8Ys= Received: from BL1P223CA0026.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::31) by DS0PR12MB9039.namprd12.prod.outlook.com (2603:10b6:8:de::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 01:31:05 +0000 Received: from MN1PEPF0000F0E4.namprd04.prod.outlook.com (2603:10b6:208:2c4:cafe::ba) by BL1P223CA0026.outlook.office365.com (2603:10b6:208:2c4::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E4.mail.protection.outlook.com (10.167.242.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:05 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:03 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 3/9] ionic: use dma range APIs Date: Mon, 29 Jan 2024 17:30:36 -0800 Message-ID: <20240130013042.11586-4-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E4:EE_|DS0PR12MB9039:EE_ X-MS-Office365-Filtering-Correlation-Id: 03084e8f-b521-4aaa-177b-08dc21332098 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IiU3BobwZwNdUcTMbFIBO9EZpXcjCLe6MYKpaS0NZxkOCIOP9nvqIR9FokwsRw3xj+tHXCBcBYxdxqyYdeKjz5UXHumMkJJin3hYX3oAohgDtFOyTQ0N1JmwOQR9Zx0lkxuvV479RXWso0Qyd0X66lmiuDoY2t++gNHSqlBql9onSHxyk2oA17wztgvkoUWv14dbkJ3212Q8wZKz8Lkp3qaxdVde1NpHcmQqobfuc8hJqD6zCsONtsyeLHQvRuRpRN7SbUnk4K+zTCHnvhMCXsCKBWq67dQCA0BgQMKRv4FyHv/TEqbTu0H2HM8N/UNwhzPuTdyqiaJuWoMWNDgrkF/wPn1/sVouYsLiywN/MuY7PVFeuu3+egW3Z890ppGYvhzZxtwQyYNxKI8sAHx7dWQwOYV0hJh0klv0DPi3akH6GlJpItt46HsVtdxeZnomOB2YXYUC4lvWqpArN3OOMS09JD+C0O5aM/IChhLvwyYSCMB8rFOEtFBLTfqzfLyF/V2sRvGC01Z8F2iCu+tLJanZ5/kb5fZCr6hw2z4jOxJTNZAr24bAIDzjCqzR5NJyZOddyU9I1rU431kC2SBhM58X0MmEyqyXoGOU1S1VcVVHeg56UyY8GyzLSYXUmyY7Cs8L+y6oMVBibOXQes3hs4WnaGY3qOXOlS/Zuh+wgThvRQ+PmwcdcbcGsracJOdfE71mP/9xyZmFFKkA4ub1lb4RXLKOGzNE+vOPD6Qjur/ISpdtMSI2d7BpGP1xwGbJ+SSNoOUAHBnKKLGiHluYrQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(39860400002)(136003)(376002)(230922051799003)(186009)(64100799003)(1800799012)(82310400011)(451199024)(46966006)(36840700001)(40470700004)(41300700001)(6666004)(478600001)(86362001)(54906003)(2616005)(5660300002)(2906002)(44832011)(70206006)(8936002)(4326008)(110136005)(16526019)(8676002)(70586007)(1076003)(316002)(36860700001)(47076005)(83380400001)(26005)(336012)(426003)(356005)(82740400003)(36756003)(40460700003)(40480700001)(81166007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:05.3342 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 03084e8f-b521-4aaa-177b-08dc21332098 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB9039 X-Patchwork-Delegate: kuba@kernel.org Convert Rx datapath handling to use the DMA range APIs in preparation for adding XDP handling. Signed-off-by: Shannon Nelson --- drivers/net/ethernet/pensando/ionic/ionic_txrx.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 19a7a8a8e1b3..aee38979a9d7 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -225,9 +225,8 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; - dma_sync_single_for_cpu(dev, - ionic_rx_buf_pa(buf_info), - frag_len, DMA_FROM_DEVICE); + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, buf_info->page, buf_info->page_offset, frag_len, @@ -276,11 +275,11 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return NULL; } - dma_sync_single_for_cpu(dev, ionic_rx_buf_pa(buf_info), - len, DMA_FROM_DEVICE); + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, len, DMA_FROM_DEVICE); skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); - dma_sync_single_for_device(dev, ionic_rx_buf_pa(buf_info), - len, DMA_FROM_DEVICE); + dma_sync_single_range_for_device(dev, ionic_rx_buf_pa(buf_info), + 0, len, DMA_FROM_DEVICE); skb_put(skb, len); skb->protocol = eth_type_trans(skb, q->lif->netdev); From patchwork Tue Jan 30 01:30:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536645 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B807360AE for ; Tue, 30 Jan 2024 01:31:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.42 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578271; cv=fail; b=m08cwagXk+m+UBSdEjPpwN7hD9oTIKHQevXEM7/RpXOVyKcoUH1XUm/63sZC4NFJh5wujkqqY6KmKkZqxY83Kmuh1xhTL5EuvnHA53U+PLkFOlOwEzxGy21lNhPzz2O1HCoEZ6F+aQtKvL4MhUi7/G1UefGeiDoQKsTH6YKFFyk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578271; c=relaxed/simple; bh=3h7Fzp2CbrW8gV9ndfPj0L1lcuUSGIvTQB0kerbUmLU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Bcb3XYjGRiHLEsTXYB8G+8t5uj8oMROEBo3jEz265Kuz25/qs6qYX4elrcxHOzfREoMAnoXHeIMOTTsnMIboJZr27sMrXHeJ6Y2D9TX9Worjc+NKSquzY5/hBdaeWbZvY9Z+V8mjneLs8ndr4NIbnHaOCYqHEkbbkqHVhCCP418= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=5iiNLDAS; arc=fail smtp.client-ip=40.107.220.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="5iiNLDAS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VrwnLUMsBvjF4JX2OdGkL2FJvOt+E+FoU0k52qh6tOHbc+r0OqsBHL+0F4xBnZXgjJVWeVsRRE3HVi624fHieJ9RgCcGIUetPMgQ4yNbpyHdifb8x4UstROFEL995AguQ8rct/FjhxW1c7XMkdJ9JcVW0RmAnUFf6UUqiW1PO+Dh3h1fcclO1i3QcPKfnD0TANWJyIPL59ZUGB1/GSGE4ng2U6DiXBdtAQl9D3o7P6ub1p3K1BblQEVNvHJ8zGf5UvgFm6mFAXrVzBDZEAhIbaCp1Pm0qInTHXP9JNtT9aug9OyGFPtpaXRfy++nynCnefZGoL+hKgYVPsP/GfSByg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kbLukNUnI9D4esRWJh3zPT6wqq0cnNU3Lb8JTB3HYDc=; b=X255eQhnHzrV7uEFAk+UJeeAojjeiIAZbea41IeJ8QNQLF69E7YjnzTfT798ygnoQuSHZvGRha22MlQ9Z9hazumjgTZwjSAbQhjpnoamPzxEm0z8HhrXCe2/I+Co/tBkS7MKXoBtSnma+Ypffa3FOk57UwXyjyE4VS7ltTxlClQNE8KaJSeaGNGks8Oggfna3m3lLanceMGmqB/INKCU2E8Mo1dwQf214Hc2ypphWymbSiiB5e+gZxe92kGUV9No91uPTpKBJ0wGibGwyN+wYCgSKZ+kjcuccayMk8ky/zI2W5Qb4Dyj9nzxzeciaF8wUuWWm8l6vGsdi2jqiLqN9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kbLukNUnI9D4esRWJh3zPT6wqq0cnNU3Lb8JTB3HYDc=; b=5iiNLDASAUvn+r+Cn1bYXCvRUOj2b2kQuQYKdxvvc7eGJ55cn6pe5bw44KIYC3tQ2hIB+w/blkWgjVd4hKocsKV6WcWbwlpG/iR37vvqRufyZDzO0HWV+GcCtuWZ45KPIgQbXdtYkTbjAf17YNymOt0Hnx/E95GrJ1zc6ulBEW8= Received: from MN2PR14CA0018.namprd14.prod.outlook.com (2603:10b6:208:23e::23) by CH3PR12MB7643.namprd12.prod.outlook.com (2603:10b6:610:152::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Tue, 30 Jan 2024 01:31:06 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:23e:cafe::f6) by MN2PR14CA0018.outlook.office365.com (2603:10b6:208:23e::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:06 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:04 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 4/9] ionic: add initial framework for XDP support Date: Mon, 29 Jan 2024 17:30:37 -0800 Message-ID: <20240130013042.11586-5-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|CH3PR12MB7643:EE_ X-MS-Office365-Filtering-Correlation-Id: f6612d50-f9ca-4bdc-c63c-08dc2133210f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0qy9Iz6QBpVDqQbGDA/eLEk5rWAAfVr1O20/WJjWtxyXGrIXE0oheEs/ai5ixb/MbHJpukV1uHGj0NxtwowLuBboxDGZHRopzB2HdS1HPVvpn7faduL/n0T7itH4QvalM/q8JxZb5L36RQ2CChmJgu5NLSS9/QLBCV1DnUFGaN8U+ZUhrQ1ZvFpONkvrKn3Wk4iLjBvIAzzvggJRja3GUgb1q8iN/b1vUxZyZeR4bo9CoQRtMZhQ8Q4M6VnxRH/zFXtE9afVqOwpiqCmWobzXsebei8byZBsDb/hGAnHZ33RKKAcv5V2kXZimp2faYiViboV4cgjPkyaI03ZqkYV6TdPEGSxhJJ+GTYJ5sBVh0DSxMzBNsNvMCaokaVrup6MA2qz4iSEZX4/+ZgqwxScXvBAmaIZyklNQUjKMA2C+IRNJBDQIBC/jtGiTLs8RmS5giyNznKYYNfIrWLPrNzrITfynU0bD9BeMjJmlE3b9BDHfyPEiAuMHy3AHNQycjSRsizeJmtwGEqJER3a/ICgKlzvQSq8Exr8X1Kj9OBVToTagKfQGrBKPuS2s41jRHKKYw+J8ApVWKoWQJmHD0dlB2bK13wg3EBp/SE9bd8udpJZOQ5Wwe+avkDxMb4ibKMvcqoQeGD2WnuB1k/OaaTZnVGxBiHDLV1SnAl9k6Op0UI96GRHAtUya6xtAs3IkPCwzM8ZaFTfs5I/ADSIhU1T2Fr1x7pYRM6w9ZuWjTq3+Zm4ZM/L/yFad3VIuBfcnR1UPiv0SIQxaq2xUEO26bBkJw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(39860400002)(396003)(376002)(346002)(230922051799003)(82310400011)(451199024)(186009)(1800799012)(64100799003)(46966006)(36840700001)(40470700004)(36860700001)(36756003)(83380400001)(2616005)(47076005)(316002)(70586007)(8936002)(110136005)(70206006)(8676002)(54906003)(86362001)(478600001)(4326008)(26005)(16526019)(2906002)(1076003)(6666004)(30864003)(5660300002)(426003)(44832011)(336012)(41300700001)(356005)(40460700003)(40480700001)(82740400003)(81166007)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:06.1029 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f6612d50-f9ca-4bdc-c63c-08dc2133210f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7643 X-Patchwork-Delegate: kuba@kernel.org Set up the basics for running Rx packets through XDP programs. Add new queue setup and teardown steps for adding/removing an XDP program, and add the call to run the XDP on a packet. The XDP frame size needs to be the MTU plus standard ethernet header, plus head room for XDP scribblings and tail room for a struct skb_shared_info. Also, at this point, we don't support XDP frags, only a single contiguous Rx buffer. This means that our page splitting is not very useful, so when XDP is in use we need to use the full Rx buffer size and not do sharing. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_dev.h | 7 + .../ethernet/pensando/ionic/ionic_ethtool.c | 5 + .../net/ethernet/pensando/ionic/ionic_lif.c | 172 +++++++++++++++++- .../net/ethernet/pensando/ionic/ionic_lif.h | 7 + .../net/ethernet/pensando/ionic/ionic_stats.c | 9 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 61 ++++++- 6 files changed, 255 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 2667e1cde16b..70f5725fdfe8 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -8,6 +8,7 @@ #include #include #include +#include #include "ionic_if.h" #include "ionic_regs.h" @@ -195,6 +196,11 @@ typedef void (*ionic_desc_cb)(struct ionic_queue *q, #define IONIC_PAGE_GFP_MASK (GFP_ATOMIC | __GFP_NOWARN |\ __GFP_COMP | __GFP_MEMALLOC) +#define IONIC_XDP_MAX_LINEAR_MTU (IONIC_PAGE_SIZE - \ + (VLAN_ETH_HLEN + \ + XDP_PACKET_HEADROOM + \ + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))) + struct ionic_buf_info { struct page *page; dma_addr_t dma_addr; @@ -256,6 +262,7 @@ struct ionic_queue { struct ionic_txq_sg_desc *txq_sgl; struct ionic_rxq_sg_desc *rxq_sgl; }; + struct xdp_rxq_info *xdp_rxq_info; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c index cd3c0b01402e..98df2ee11c51 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c @@ -721,6 +721,11 @@ static int ionic_set_channels(struct net_device *netdev, ionic_init_queue_params(lif, &qparam); + if ((ch->rx_count || ch->tx_count) && lif->xdp_prog) { + netdev_info(lif->netdev, "Split Tx/Rx interrupts not available when using XDP\n"); + return -EOPNOTSUPP; + } + if (ch->rx_count != ch->tx_count) { netdev_info(netdev, "The rx and tx count must be equal\n"); return -EINVAL; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d92f8734d153..6e9065dd149e 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -46,6 +46,9 @@ static int ionic_start_queues(struct ionic_lif *lif); static void ionic_stop_queues(struct ionic_lif *lif); static void ionic_lif_queue_identify(struct ionic_lif *lif); +static int ionic_xdp_queues_config(struct ionic_lif *lif); +static void ionic_xdp_unregister_rxq_info(struct ionic_queue *q); + static void ionic_dim_work(struct work_struct *work) { struct dim *dim = container_of(work, struct dim, work); @@ -422,6 +425,7 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) qcq->sg_base_pa = 0; } + ionic_xdp_unregister_rxq_info(&qcq->q); ionic_qcq_intr_free(lif, qcq); vfree(qcq->cq.info); @@ -862,8 +866,7 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) .type = q->type, .ver = lif->qtype_info[q->type].version, .index = cpu_to_le32(q->index), - .flags = cpu_to_le16(IONIC_QINIT_F_IRQ | - IONIC_QINIT_F_SG), + .flags = cpu_to_le16(IONIC_QINIT_F_IRQ), .intr_index = cpu_to_le16(cq->bound_intr->index), .pid = cpu_to_le16(q->pid), .ring_size = ilog2(q->num_descs), @@ -875,6 +878,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) }; int err; + if (!lif->xdp_prog) + ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); + if (qcq->flags & IONIC_QCQ_F_CMB_RINGS) { ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_CMB); ctx.cmd.q_init.ring_base = cpu_to_le64(qcq->cmb_q_base_pa); @@ -1640,6 +1646,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE; + netdev->xdp_features = NETDEV_XDP_ACT_BASIC; + return 0; } @@ -1777,6 +1785,18 @@ static int ionic_start_queues_reconfig(struct ionic_lif *lif) return err; } +static bool ionic_xdp_is_valid_mtu(struct ionic_lif *lif, u32 mtu, + struct bpf_prog *xdp_prog) +{ + if (!xdp_prog) + return true; + + if (mtu <= IONIC_XDP_MAX_LINEAR_MTU) + return true; + + return false; +} + static int ionic_change_mtu(struct net_device *netdev, int new_mtu) { struct ionic_lif *lif = netdev_priv(netdev); @@ -1789,8 +1809,13 @@ static int ionic_change_mtu(struct net_device *netdev, int new_mtu) .mtu = cpu_to_le32(new_mtu), }, }; + struct bpf_prog *xdp_prog; int err; + xdp_prog = READ_ONCE(lif->xdp_prog); + if (!ionic_xdp_is_valid_mtu(lif, new_mtu, xdp_prog)) + return -EINVAL; + err = ionic_adminq_post_wait(lif, &ctx); if (err) return err; @@ -2166,6 +2191,10 @@ static int ionic_txrx_enable(struct ionic_lif *lif) int derr = 0; int i, err; + err = ionic_xdp_queues_config(lif); + if (err) + return err; + for (i = 0; i < lif->nxqs; i++) { if (!(lif->rxqcqs[i] && lif->txqcqs[i])) { dev_err(lif->ionic->dev, "%s: bad qcq %d\n", __func__, i); @@ -2211,6 +2240,8 @@ static int ionic_txrx_enable(struct ionic_lif *lif) derr = ionic_qcq_disable(lif, lif->rxqcqs[i], derr); } + ionic_xdp_queues_config(lif); + return err; } @@ -2668,11 +2699,147 @@ static void ionic_vf_attr_replay(struct ionic_lif *lif) ionic_vf_start(ionic); } +static void ionic_xdp_unregister_rxq_info(struct ionic_queue *q) +{ + struct xdp_rxq_info *xi; + + if (!q->xdp_rxq_info) + return; + + xi = q->xdp_rxq_info; + q->xdp_rxq_info = NULL; + + xdp_rxq_info_unreg(xi); + kfree(xi); +} + +static int ionic_xdp_register_rxq_info(struct ionic_queue *q, unsigned int napi_id) +{ + struct xdp_rxq_info *rxq_info; + int err; + + rxq_info = kzalloc(sizeof(*rxq_info), GFP_KERNEL); + if (!rxq_info) + return -ENOMEM; + + err = xdp_rxq_info_reg(rxq_info, q->lif->netdev, q->index, napi_id); + if (err) { + dev_err(q->dev, "Queue %d xdp_rxq_info_reg failed, err %d\n", + q->index, err); + kfree(rxq_info); + return err; + } + + err = xdp_rxq_info_reg_mem_model(rxq_info, MEM_TYPE_PAGE_ORDER0, NULL); + if (err) { + dev_err(q->dev, "Queue %d xdp_rxq_info_reg_mem_model failed, err %d\n", + q->index, err); + xdp_rxq_info_unreg(rxq_info); + kfree(rxq_info); + return err; + } + + q->xdp_rxq_info = rxq_info; + + return 0; +} + +static int ionic_xdp_queues_config(struct ionic_lif *lif) +{ + unsigned int i; + int err; + + if (!lif->rxqcqs) + return 0; + + /* There's no need to rework memory if not going to/from NULL program. + * If there is no lif->xdp_prog, there should also be no q.xdp_rxq_info + * This way we don't need to keep an *xdp_prog in every queue struct. + */ + if (!lif->xdp_prog == !lif->rxqcqs[0]->q.xdp_rxq_info) + return 0; + + for (i = 0; i < lif->ionic->nrxqs_per_lif && lif->rxqcqs[i]; i++) { + struct ionic_queue *q = &lif->rxqcqs[i]->q; + + if (q->xdp_rxq_info) { + ionic_xdp_unregister_rxq_info(q); + } else { + err = ionic_xdp_register_rxq_info(q, lif->rxqcqs[i]->napi.napi_id); + if (err) { + dev_err(lif->ionic->dev, "failed to register RX queue %d info for XDP, err %d\n", + i, err); + goto err_out; + } + } + } + + return 0; + +err_out: + for (i = 0; i < lif->ionic->nrxqs_per_lif && lif->rxqcqs[i]; i++) + ionic_xdp_unregister_rxq_info(&lif->rxqcqs[i]->q); + + return err; +} + +static int ionic_xdp_config(struct net_device *netdev, struct netdev_bpf *bpf) +{ + struct ionic_lif *lif = netdev_priv(netdev); + struct bpf_prog *old_prog; + u32 maxfs; + + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { +#define XDP_ERR_SPLIT "XDP not available with split Tx/Rx interrupts" + NL_SET_ERR_MSG_MOD(bpf->extack, XDP_ERR_SPLIT); + netdev_info(lif->netdev, XDP_ERR_SPLIT); + return -EOPNOTSUPP; + } + + if (!ionic_xdp_is_valid_mtu(lif, netdev->mtu, bpf->prog)) { +#define XDP_ERR_MTU "MTU is too large for XDP without frags support" + NL_SET_ERR_MSG_MOD(bpf->extack, XDP_ERR_MTU); + netdev_info(lif->netdev, XDP_ERR_MTU); + return -EINVAL; + } + + maxfs = __le32_to_cpu(lif->identity->eth.max_frame_size) - VLAN_ETH_HLEN; + if (bpf->prog) + maxfs = min_t(u32, maxfs, IONIC_XDP_MAX_LINEAR_MTU); + netdev->max_mtu = maxfs; + + if (!netif_running(netdev)) { + old_prog = xchg(&lif->xdp_prog, bpf->prog); + } else { + mutex_lock(&lif->queue_lock); + ionic_stop_queues_reconfig(lif); + old_prog = xchg(&lif->xdp_prog, bpf->prog); + ionic_start_queues_reconfig(lif); + mutex_unlock(&lif->queue_lock); + } + + if (old_prog) + bpf_prog_put(old_prog); + + return 0; +} + +static int ionic_xdp(struct net_device *netdev, struct netdev_bpf *bpf) +{ + switch (bpf->command) { + case XDP_SETUP_PROG: + return ionic_xdp_config(netdev, bpf); + default: + return -EINVAL; + } +} + static const struct net_device_ops ionic_netdev_ops = { .ndo_open = ionic_open, .ndo_stop = ionic_stop, .ndo_eth_ioctl = ionic_eth_ioctl, .ndo_start_xmit = ionic_start_xmit, + .ndo_bpf = ionic_xdp, .ndo_get_stats64 = ionic_get_stats64, .ndo_set_rx_mode = ionic_ndo_set_rx_mode, .ndo_set_features = ionic_set_features, @@ -2755,6 +2922,7 @@ static void ionic_swap_queues(struct ionic_qcq *a, struct ionic_qcq *b) swap(a->q.base, b->q.base); swap(a->q.base_pa, b->q.base_pa); swap(a->q.info, b->q.info); + swap(a->q.xdp_rxq_info, b->q.xdp_rxq_info); swap(a->q_base, b->q_base); swap(a->q_base_pa, b->q_base_pa); swap(a->q_size, b->q_size); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 61548b3eea93..61fa4ea4f04c 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -51,6 +51,9 @@ struct ionic_rx_stats { u64 alloc_err; u64 hwstamp_valid; u64 hwstamp_invalid; + u64 xdp_drop; + u64 xdp_aborted; + u64 xdp_pass; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -135,6 +138,9 @@ struct ionic_lif_sw_stats { u64 hw_rx_over_errors; u64 hw_rx_missed_errors; u64 hw_tx_aborted_errors; + u64 xdp_drop; + u64 xdp_aborted; + u64 xdp_pass; }; enum ionic_lif_state_flags { @@ -230,6 +236,7 @@ struct ionic_lif { struct ionic_phc *phc; struct dentry *dentry; + struct bpf_prog *xdp_prog; }; struct ionic_phc { diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 1f6022fb7679..2fb20173b2c6 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -27,6 +27,9 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(hw_rx_over_errors), IONIC_LIF_STAT_DESC(hw_rx_missed_errors), IONIC_LIF_STAT_DESC(hw_tx_aborted_errors), + IONIC_LIF_STAT_DESC(xdp_drop), + IONIC_LIF_STAT_DESC(xdp_aborted), + IONIC_LIF_STAT_DESC(xdp_pass), }; static const struct ionic_stat_desc ionic_port_stats_desc[] = { @@ -149,6 +152,9 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(hwstamp_invalid), IONIC_RX_STAT_DESC(dropped), IONIC_RX_STAT_DESC(vlan_stripped), + IONIC_RX_STAT_DESC(xdp_drop), + IONIC_RX_STAT_DESC(xdp_aborted), + IONIC_RX_STAT_DESC(xdp_pass), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -185,6 +191,9 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->rx_csum_error += rxstats->csum_error; stats->rx_hwstamp_valid += rxstats->hwstamp_valid; stats->rx_hwstamp_invalid += rxstats->hwstamp_invalid; + stats->xdp_drop += rxstats->xdp_drop; + stats->xdp_aborted += rxstats->xdp_aborted; + stats->xdp_pass += rxstats->xdp_pass; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index aee38979a9d7..07a17be94d4d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -177,7 +177,7 @@ static bool ionic_rx_buf_recycle(struct ionic_queue *q, if (page_to_nid(buf_info->page) != numa_mem_id()) return false; - size = ALIGN(used, IONIC_PAGE_SPLIT_SZ); + size = ALIGN(used, q->xdp_rxq_info ? IONIC_PAGE_SIZE : IONIC_PAGE_SPLIT_SZ); buf_info->page_offset += size; if (buf_info->page_offset >= IONIC_PAGE_SIZE) return false; @@ -287,6 +287,54 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return skb; } +static bool ionic_run_xdp(struct ionic_rx_stats *stats, + struct net_device *netdev, + struct ionic_queue *rxq, + struct ionic_buf_info *buf_info, + int len) +{ + u32 xdp_action = XDP_ABORTED; + struct bpf_prog *xdp_prog; + struct xdp_buff xdp_buf; + + xdp_prog = READ_ONCE(rxq->lif->xdp_prog); + if (!xdp_prog) + return false; + + xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); + xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), + 0, len, false); + + dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), + 0, len, + DMA_FROM_DEVICE); + + prefetchw(&xdp_buf.data_hard_start); + + xdp_action = bpf_prog_run_xdp(xdp_prog, &xdp_buf); + + switch (xdp_action) { + case XDP_PASS: + stats->xdp_pass++; + return false; /* false = we didn't consume the packet */ + + case XDP_DROP: + ionic_rx_page_free(rxq, buf_info); + stats->xdp_drop++; + break; + + case XDP_REDIRECT: + case XDP_TX: + case XDP_ABORTED: + default: + trace_xdp_exception(netdev, xdp_prog, xdp_action); + ionic_rx_page_free(rxq, buf_info); + stats->xdp_aborted++; + } + + return true; +} + static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info, struct ionic_cq_info *cq_info, @@ -297,6 +345,7 @@ static void ionic_rx_clean(struct ionic_queue *q, struct ionic_rx_stats *stats; struct ionic_rxq_comp *comp; struct sk_buff *skb; + u16 len; comp = cq_info->cq_desc + qcq->cq.desc_size - sizeof(*comp); @@ -307,10 +356,14 @@ static void ionic_rx_clean(struct ionic_queue *q, return; } + len = le16_to_cpu(comp->len); stats->pkts++; - stats->bytes += le16_to_cpu(comp->len); + stats->bytes += len; + + if (ionic_run_xdp(stats, netdev, q, desc_info->bufs, len)) + return; - if (le16_to_cpu(comp->len) <= q->lif->rx_copybreak) + if (len <= q->lif->rx_copybreak) skb = ionic_rx_copybreak(q, desc_info, comp); else skb = ionic_rx_frags(q, desc_info, comp); @@ -380,7 +433,7 @@ static void ionic_rx_clean(struct ionic_queue *q, } } - if (le16_to_cpu(comp->len) <= q->lif->rx_copybreak) + if (len <= q->lif->rx_copybreak) napi_gro_receive(&qcq->napi, skb); else napi_gro_frags(&qcq->napi); From patchwork Tue Jan 30 01:30:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536647 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2049.outbound.protection.outlook.com [40.107.223.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D63FB364C8 for ; Tue, 30 Jan 2024 01:31:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.49 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578273; cv=fail; b=Air5miIx0VO36frMsw0F37/Pn51IErdIbZgsKrob+oLjf16xgUE+Jb+zxPs+e3u+O2urvHuG7wkvM3xQGQjXRtGryCEwh6oM7cW/O0/x/VNLqFR4TfCBtj+m5OqvP1NaXOERFCPGsXBtI4K8PJGDru4xSjN9P/mrhRDQMJ8nUro= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578273; c=relaxed/simple; bh=EpdcKkIgYPpdVeehNi4pzLnExscutvW5b09S9vOvlNQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nOWzDZNd+VhQVJszTJdWXZiSjEXEQM7wtMfFZaazdrY9uV4yr7Du89cI6PoUOwqARo3J4VzPIKDtTkY7ap8itlFvxcs7BgDMgzMzLPK0waQU9TFbFO8E1fx4OUkIv2F+m3B+3P9yH4oRRcSVeOq1112gyrihccXbKSBCCE4hR4s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=fEpAdgdg; arc=fail smtp.client-ip=40.107.223.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="fEpAdgdg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YMQiyJGJC573AlSDpl47GMRZBq5vGHoQMxOq2zgdeJ4vKuchjR9dBhl4SHyqhxDxPviQQxTbaAXNRCkVReczLEdTzkVxiH73L06vmtXtQSGy6GzPbBS2RjjMSUKnfD667zWpG/tiJYZ0tokZVA3M0RvAmYlfei7u0tNQH8Ay+luzuTQD4vu5CnHmjNyOjaHCRGRmYSgb4/h1oMRum2P76nctaRWZm0IjSt96MGMBeBYcdj3Nkfn2G6EySajPJAqbSiF+PXQBPJV7enihhh8GqlVRcx8SafmTTyi5ptm0uSQEFVOunw0di7MNV4rTPrXjQDEVK4nL6kESiEGzCPdfdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4ezuKt1xTbJvXbixYp+Qutaxtz5PU0IPIJB3Ny0uipY=; b=N+C6p0pFV5B1woMwGuO9cYfZBdiJnR843SQLQKh6I9BtmP1X4hVmwfMRzDNEPXS24KFxkGz5AGc7yOvR/2vKKEryinhynyOUwDK7L9eNIyLvD+l5DckWYtpwcluVz3XtMzTbjnyhFdhwOlC36QSDjg9hPKm7fmW3Y5mAy8mOblw5BFfHKse0l0o4m7KJZ9eHMAI3jO7FFbIICWqzmSDLFUmz6EqQOXH+9byJ81/Onpsv9SuLRCqNXRxqbzR3j7TRLcKwKaiaPSWAHW2EdQYdfKgwwIGgf02GLn9WavHtdE++Ymc5v1aUA5fMhXK6Qf7VxXWnzq0+gTM3hphdW3iVyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4ezuKt1xTbJvXbixYp+Qutaxtz5PU0IPIJB3Ny0uipY=; b=fEpAdgdgXsUOu76z45lGL7uK2Vxc9Yt79TM2JLrW3skUtAVWtLgC6FWMf5jipDFSr/YAPV6d+xLOehORkzsSqkBFYehdTaZa7E5ZM5i0D03pslTITRFs5KMVFK7PoN2F4Zn1jf+Z88I5pOcxxPepekQiztjATTXmspoKDPYsVKw= Received: from MN2PR14CA0009.namprd14.prod.outlook.com (2603:10b6:208:23e::14) by SA1PR12MB6728.namprd12.prod.outlook.com (2603:10b6:806:257::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 01:31:07 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:23e:cafe::cd) by MN2PR14CA0009.outlook.office365.com (2603:10b6:208:23e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:07 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:05 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 5/9] ionic: Add XDP packet headroom Date: Mon, 29 Jan 2024 17:30:38 -0800 Message-ID: <20240130013042.11586-6-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|SA1PR12MB6728:EE_ X-MS-Office365-Filtering-Correlation-Id: 825e5987-30ee-4990-aa0c-08dc213321d5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NX5f4RXGXfLkrw6HhVSH38PGuYez9Odq4g17mBs1x9bKa/ylL8opiamPDRUWveW/yf043IL+7jsebCau73cWAXACPNwWNHHhrj/5HjDCUopcC0podPIT2ZUMSZjq4B66LKaAVS86ESVPtf4Vx/Vt6WlScDEaZG9vtpEkYrNje/6uakKfiVrekizFQrCCQoQWV1A0GF19El67n5ZdRxePM+IDOCk9Ul5TRCr2lJ9jA4qkgz62PVx5zJUnXqFCgo49xRZBgiL+fIqa1lFbf2seARcuHQk7XcDWrzNBghdfQQw4piOOOr1Lr49WsiGarpn3Of2duhveJDKUBLD05CgvDjgOIQtoNOOaEtJhxBQrkWAdt+DE8dv4CwUsgLM21QFmSn4c7VulCidSqdSgeLiwOUFYQv9Vcvsb/M0+JkZaKcaDgJTCohBmf0bDs0jtoybgdGleIzt0sNxftKzSol9dHAxeGfN/jHhz2Ev/aF7z78byhDe2YDYLbelYbzNU/9Jd2LWrl2fNEsEPR1Lu+hXMoS5ofmoX5ZRJ8HG6gW2zhIigrHY7OUbGaFXtjXbSE+JyiS16vONTPOqMy0EpLVhBw3ges6FZInwQ8OLhs4dpN/2hufGXHbNvQYHN5GxLEpxaBVOfIbWtnco++OAMB+BG49KzI/t6gg2NYpGhD6u0w6Ryqw5LsWtyWMZMnZbqjXYVkctOc5VxccxYByc2ExCbPTxMbgkMCH6hAbqBV97zMRhXzDqNtye5CwBa7YKUG7DFNB32B+9ZUG0qt5WdEzGhxQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(376002)(39860400002)(346002)(136003)(230922051799003)(451199024)(1800799012)(186009)(82310400011)(64100799003)(40470700004)(36840700001)(46966006)(83380400001)(426003)(1076003)(16526019)(6666004)(336012)(26005)(36860700001)(2616005)(47076005)(8936002)(5660300002)(81166007)(4326008)(8676002)(41300700001)(44832011)(2906002)(478600001)(110136005)(54906003)(316002)(70206006)(70586007)(36756003)(86362001)(82740400003)(356005)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:07.4154 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 825e5987-30ee-4990-aa0c-08dc213321d5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6728 X-Patchwork-Delegate: kuba@kernel.org If an xdp program is loaded, add headroom at the beginning of the frame to allow for editing and insertions that an XDP program might need room for, and tailroom used later for XDP frame tracking. These are only needed in the first Rx buffer in a packet, not for any trailing frags. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_txrx.c | 67 ++++++++++++------- 1 file changed, 44 insertions(+), 23 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 07a17be94d4d..072a9e376b39 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -189,7 +189,9 @@ static bool ionic_rx_buf_recycle(struct ionic_queue *q, static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp) + unsigned int headroom, + unsigned int len, + unsigned int num_sg_elems) { struct net_device *netdev = q->lif->netdev; struct ionic_buf_info *buf_info; @@ -198,12 +200,10 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, struct sk_buff *skb; unsigned int i; u16 frag_len; - u16 len; stats = q_to_rx_stats(q); buf_info = &desc_info->bufs[0]; - len = le16_to_cpu(comp->len); prefetchw(buf_info->page); @@ -215,22 +215,25 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, return NULL; } - i = comp->num_sg_elems + 1; + i = num_sg_elems + 1; do { if (unlikely(!buf_info->page)) { dev_kfree_skb(skb); return NULL; } - frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); + if (headroom) + frag_len = min_t(u16, len, IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN); + else + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, frag_len, DMA_FROM_DEVICE); + headroom, frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, - buf_info->page, buf_info->page_offset, frag_len, - IONIC_PAGE_SIZE); + buf_info->page, buf_info->page_offset + headroom, + frag_len, IONIC_PAGE_SIZE); if (!ionic_rx_buf_recycle(q, buf_info, frag_len)) { dma_unmap_page(dev, buf_info->dma_addr, @@ -238,6 +241,10 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, buf_info->page = NULL; } + /* only needed on the first buffer */ + if (headroom) + headroom = 0; + buf_info++; i--; @@ -248,19 +255,18 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp) + unsigned int headroom, + unsigned int len) { struct net_device *netdev = q->lif->netdev; struct ionic_buf_info *buf_info; struct ionic_rx_stats *stats; struct device *dev = q->dev; struct sk_buff *skb; - u16 len; stats = q_to_rx_stats(q); buf_info = &desc_info->bufs[0]; - len = le16_to_cpu(comp->len); skb = napi_alloc_skb(&q_to_qcq(q)->napi, len); if (unlikely(!skb)) { @@ -276,10 +282,10 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, } dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, len, DMA_FROM_DEVICE); - skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); + headroom, len, DMA_FROM_DEVICE); + skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info) + headroom, len); dma_sync_single_range_for_device(dev, ionic_rx_buf_pa(buf_info), - 0, len, DMA_FROM_DEVICE); + headroom, len, DMA_FROM_DEVICE); skb_put(skb, len); skb->protocol = eth_type_trans(skb, q->lif->netdev); @@ -303,10 +309,10 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), - 0, len, false); + XDP_PACKET_HEADROOM, len, false); dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), - 0, len, + XDP_PACKET_HEADROOM, len, DMA_FROM_DEVICE); prefetchw(&xdp_buf.data_hard_start); @@ -344,6 +350,7 @@ static void ionic_rx_clean(struct ionic_queue *q, struct ionic_qcq *qcq = q_to_qcq(q); struct ionic_rx_stats *stats; struct ionic_rxq_comp *comp; + unsigned int headroom; struct sk_buff *skb; u16 len; @@ -363,10 +370,11 @@ static void ionic_rx_clean(struct ionic_queue *q, if (ionic_run_xdp(stats, netdev, q, desc_info->bufs, len)) return; + headroom = q->xdp_rxq_info ? XDP_PACKET_HEADROOM : 0; if (len <= q->lif->rx_copybreak) - skb = ionic_rx_copybreak(q, desc_info, comp); + skb = ionic_rx_copybreak(q, desc_info, headroom, len); else - skb = ionic_rx_frags(q, desc_info, comp); + skb = ionic_rx_frags(q, desc_info, headroom, len, comp->num_sg_elems); if (unlikely(!skb)) { stats->dropped++; @@ -490,8 +498,9 @@ void ionic_rx_fill(struct ionic_queue *q) unsigned int frag_len; unsigned int nfrags; unsigned int n_fill; - unsigned int i, j; unsigned int len; + unsigned int i; + unsigned int j; n_fill = ionic_q_space_avail(q); @@ -500,9 +509,12 @@ void ionic_rx_fill(struct ionic_queue *q) if (n_fill < fill_threshold) return; - len = netdev->mtu + ETH_HLEN + VLAN_HLEN; + len = netdev->mtu + VLAN_ETH_HLEN; for (i = n_fill; i; i--) { + unsigned int headroom; + unsigned int buf_len; + nfrags = 0; remain_len = len; desc_info = &q->info[q->head_idx]; @@ -517,9 +529,18 @@ void ionic_rx_fill(struct ionic_queue *q) } } - /* fill main descriptor - buf[0] */ - desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); - frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); + /* fill main descriptor - buf[0] + * XDP uses space in the first buffer, so account for + * head room, tail room, and ip header in the first frag size. + */ + headroom = q->xdp_rxq_info ? XDP_PACKET_HEADROOM : 0; + if (q->xdp_rxq_info) + buf_len = IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN; + else + buf_len = ionic_rx_buf_size(buf_info); + frag_len = min_t(u16, len, buf_len); + + desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info) + headroom); desc->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; From patchwork Tue Jan 30 01:30:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536650 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71CCD36AE7 for ; Tue, 30 Jan 2024 01:31:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.40 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578275; cv=fail; b=I/6ThCgFt9XndGqvpfrpHKwwwRLW54Ehfj0YehLKRG24TK3VYNxcp6/KwYfcqVRAckzYy3wLW79rJsuEiGqNZfjdWyFNVuilIPdbrBdedYhHMRhpQruseYRWQyLRDieIPdLfvyHf10iO1gHm12FJPCsugpzv+763bSd4pRLt0Ho= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578275; c=relaxed/simple; bh=1Zy0C2/VyFP3uEkle1GlX3c6PwciAE1V9ygrliXEBGg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=q7rT/1gMFt9ov5MZlQsQw9hwhaF2ZuKTSsWu+sZluxKA5cuPwS1dJ6pqpTZeEYQEjg3ug1Kk/E975HPWei57nEZucVAN1RbB/ZoFSTZDYsp+uhkl0ziqUWG2UVvaymvRf2WzHgXpfyctydHntB7PbAlsW8bVHKcpIHjT35iSLrI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=vlzQYg2k; arc=fail smtp.client-ip=40.107.236.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="vlzQYg2k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i9dPpK2SLZoPX7AwwXo15Uu9hckTgjiIu/8XzMUx9H0AuxrKT1hOTo8xSixxPWwLVpw+c+4Tb8lecTkEongC9/qelm4urzmHF6ASx4nESZvhBZcp+0Dz8dMgTN1JVtNhQX/FCI5BB/zEYhWeB3K0YBCn/hAP5us5WmP1JyO/xqylJ19LQ29df402gX5G8o7n3xN5D1JEsU/1cXlSW/WGFqVAedJVPVJCEuPpmdCkanJhjSMCfU6y4QZ0BVjQqDzSWITf70tHZiun3PCZpr2uKjeeDyWM0KvRabiw6o98CIo3m4Srs+JJh5z3ANdQRhOSoX9sOosbQ+ZQASC5+x1guQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N9eZ+cGTwJ/V53kDPaMRiDUP8h6YHqZUU0v1yBkrKfg=; b=VDVXUJ3qM7ZAc+3qHdKKngjxj1jMI/hRWoGqAhmcrRHfhzjFjfqDQYzKqe7X5pf4OevRxs4ZPyOclKWmZpAUJxlSUvUvBphRCAWkkbsANYQ24gl6MPMM2Kul3450pMRBim0SxEdawi//oGTG6Bqy0JK06HuJkAN/huulezNdp1/QD7dzZKwPDQKaN10fz5V9AfVf5He1sVR9GOE4tXnQvBveX8/Wv5oIZq4CvyWbiipZrGEjMzLoVPRHtbwUGqkcoBqLmdLiYYt8O06qQFM7n63KZZ6uZT/JiEzmKZb6IrNtolQ1/HATu68HEOWHDr8pPjWV9FFEEicto9aln8DWOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N9eZ+cGTwJ/V53kDPaMRiDUP8h6YHqZUU0v1yBkrKfg=; b=vlzQYg2kB20bkfZl4KPhIFd8GK2DavEsuHyHFUxl+0vUYlPT7ET70/28h/A9sPnz2EsoooUq/XJ176s6xGNra0dK8ujdM62BDbEpX07P+n/Lb670ldVA0fcyCCugZUsDvCRKZJpIz6xm5KdTaGLnvCINrctgvJR1/kFJdax8n0w= Received: from MN2PR14CA0007.namprd14.prod.outlook.com (2603:10b6:208:23e::12) by BL1PR12MB5192.namprd12.prod.outlook.com (2603:10b6:208:311::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34; Tue, 30 Jan 2024 01:31:08 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:23e:cafe::c6) by MN2PR14CA0007.outlook.office365.com (2603:10b6:208:23e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32 via Frontend Transport; Tue, 30 Jan 2024 01:31:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:08 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:07 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 6/9] ionic: Add XDP_TX support Date: Mon, 29 Jan 2024 17:30:39 -0800 Message-ID: <20240130013042.11586-7-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|BL1PR12MB5192:EE_ X-MS-Office365-Filtering-Correlation-Id: ebcdcf09-d408-4997-b5fa-08dc21332283 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w2xNfEgn6GgRK277KjOpNN2WTNGNcNPL/sf3m1KOgR3qlz480mIIcy1Wh5NRHffVGqmI1uqXqyhFS6/ugw7BQNtH/Wb4zy+jwklGGM1HPxcVyhb1o8HS90GJs//LXe97vh2SDwQWE/BdnewxaZrOs7q3iMrFZWPvEpRYkr7Sxe71PuWRiDB4Ah0kRJEs0ZaQzRwIcNQgRY8zAdY+8E7z+Fvt2nnJXBPPPvn15wC0KKitHea+A6Su47EbvxOnUe7wFPArKcc61R8df94ow5LOWhsMKuRjh3WFFL5CEnm1TQZoR+QX/9Ixvl44PJcfqFOfyWMPMptl6fMOVSZ41kBbaRAeqdkHL1P7Y/DEAtBlq+7BXACFUaFHlwTTsu28ZDhfGV5h039oL+ZLN8oR8Nc1MXZTOmUAIfArlrFACpQqrdcyD7H9hT+z8OpcwaimyobAaH5hsINja4XNtjxc579mhVMSRRyDHpCs1MgmP1E0ocvZn8SDn3CDaFZB0+Hau/bhhMkgsI5yWNKBRrQz3VWHuDwJXYzXrVZDiHFhLEUHopqy3f1K3TDNxJrLYXuGaCeors1qvj0drRW8qzdihtcj9lPDPYxVdN6sq1aewyrrxgGUYZ69Ng2PHaXyC7LPNGfpKQebjXFf2/5e6A1OX+LsPXn1hghmz6NnxQhJhkZh2r2OcnAjqfL8zWRTRfKaq2nH3696RYDGaiX0Y9E6MvGquzwufgqHa1FwYIup1aybJJlL5YjabCOuA5FJ2TCHnORfdzoskzXf17qNnteU2+2zdA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(136003)(376002)(39860400002)(230922051799003)(186009)(64100799003)(82310400011)(451199024)(1800799012)(40470700004)(46966006)(36840700001)(36860700001)(47076005)(86362001)(83380400001)(36756003)(41300700001)(82740400003)(356005)(81166007)(478600001)(8676002)(4326008)(44832011)(70206006)(70586007)(316002)(110136005)(54906003)(1076003)(2616005)(26005)(336012)(426003)(16526019)(2906002)(5660300002)(6666004)(8936002)(40480700001)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:08.5404 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ebcdcf09-d408-4997-b5fa-08dc21332283 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5192 X-Patchwork-Delegate: kuba@kernel.org The XDP_TX packets get fed back into the Rx queue's partnered Tx queue as an xdp_frame. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_dev.h | 3 + .../net/ethernet/pensando/ionic/ionic_lif.c | 4 + .../net/ethernet/pensando/ionic/ionic_lif.h | 4 + .../net/ethernet/pensando/ionic/ionic_stats.c | 6 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 137 +++++++++++++++++- 5 files changed, 151 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 70f5725fdfe8..76425bb546ba 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -228,6 +228,8 @@ struct ionic_desc_info { struct ionic_buf_info bufs[MAX_SKB_FRAGS + 1]; ionic_desc_cb cb; void *cb_arg; + struct xdp_frame *xdpf; + enum xdp_action act; }; #define IONIC_QUEUE_NAME_MAX_SZ 16 @@ -263,6 +265,7 @@ struct ionic_queue { struct ionic_rxq_sg_desc *rxq_sgl; }; struct xdp_rxq_info *xdp_rxq_info; + struct ionic_queue *partner; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 6e9065dd149e..997141321c40 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -878,6 +878,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) }; int err; + q->partner = &lif->txqcqs[q->index]->q; + q->partner->partner = q; + if (!lif->xdp_prog) ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); @@ -2923,6 +2926,7 @@ static void ionic_swap_queues(struct ionic_qcq *a, struct ionic_qcq *b) swap(a->q.base_pa, b->q.base_pa); swap(a->q.info, b->q.info); swap(a->q.xdp_rxq_info, b->q.xdp_rxq_info); + swap(a->q.partner, b->q.partner); swap(a->q_base, b->q_base); swap(a->q_base_pa, b->q_base_pa); swap(a->q_size, b->q_size); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 61fa4ea4f04c..092ff08fc7e0 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -37,6 +37,7 @@ struct ionic_tx_stats { u64 dma_map_err; u64 hwstamp_valid; u64 hwstamp_invalid; + u64 xdp_frames; }; struct ionic_rx_stats { @@ -54,6 +55,7 @@ struct ionic_rx_stats { u64 xdp_drop; u64 xdp_aborted; u64 xdp_pass; + u64 xdp_tx; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -141,6 +143,8 @@ struct ionic_lif_sw_stats { u64 xdp_drop; u64 xdp_aborted; u64 xdp_pass; + u64 xdp_tx; + u64 xdp_frames; }; enum ionic_lif_state_flags { diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 2fb20173b2c6..5d48226e66cd 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -30,6 +30,8 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(xdp_drop), IONIC_LIF_STAT_DESC(xdp_aborted), IONIC_LIF_STAT_DESC(xdp_pass), + IONIC_LIF_STAT_DESC(xdp_tx), + IONIC_LIF_STAT_DESC(xdp_frames), }; static const struct ionic_stat_desc ionic_port_stats_desc[] = { @@ -138,6 +140,7 @@ static const struct ionic_stat_desc ionic_tx_stats_desc[] = { IONIC_TX_STAT_DESC(csum_none), IONIC_TX_STAT_DESC(csum), IONIC_TX_STAT_DESC(vlan_inserted), + IONIC_TX_STAT_DESC(xdp_frames), }; static const struct ionic_stat_desc ionic_rx_stats_desc[] = { @@ -155,6 +158,7 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(xdp_drop), IONIC_RX_STAT_DESC(xdp_aborted), IONIC_RX_STAT_DESC(xdp_pass), + IONIC_RX_STAT_DESC(xdp_tx), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -177,6 +181,7 @@ static void ionic_add_lif_txq_stats(struct ionic_lif *lif, int q_num, stats->tx_csum += txstats->csum; stats->tx_hwstamp_valid += txstats->hwstamp_valid; stats->tx_hwstamp_invalid += txstats->hwstamp_invalid; + stats->xdp_frames += txstats->xdp_frames; } static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, @@ -194,6 +199,7 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->xdp_drop += rxstats->xdp_drop; stats->xdp_aborted += rxstats->xdp_aborted; stats->xdp_pass += rxstats->xdp_pass; + stats->xdp_tx += rxstats->xdp_tx; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 072a9e376b39..a68f659e416d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -10,6 +10,16 @@ #include "ionic_lif.h" #include "ionic_txrx.h" +static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs); + +static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, + void *data, size_t len); + +static void ionic_tx_clean(struct ionic_queue *q, + struct ionic_desc_info *desc_info, + struct ionic_cq_info *cq_info, + void *cb_arg); + static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell, ionic_desc_cb cb_func, void *cb_arg) { @@ -293,6 +303,75 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return skb; } +static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, + struct ionic_desc_info *desc_info) +{ + unsigned int nbufs = desc_info->nbufs; + struct ionic_buf_info *buf_info; + struct device *dev = q->dev; + + if (!nbufs) + return; + + buf_info = desc_info->bufs; + dma_unmap_single(dev, buf_info->dma_addr, + buf_info->len, DMA_TO_DEVICE); + __free_pages(buf_info->page, 0); + buf_info->page = NULL; + + desc_info->nbufs = 0; + desc_info->xdpf = NULL; + desc_info->act = 0; +} + +static int ionic_xdp_post_frame(struct net_device *netdev, + struct ionic_queue *q, struct xdp_frame *frame, + enum xdp_action act, struct page *page, int off, + bool ring_doorbell) +{ + struct ionic_desc_info *desc_info; + struct ionic_buf_info *buf_info; + struct ionic_tx_stats *stats; + struct ionic_txq_desc *desc; + size_t len = frame->len; + dma_addr_t dma_addr; + u64 cmd; + + desc_info = &q->info[q->head_idx]; + desc = desc_info->txq_desc; + buf_info = desc_info->bufs; + stats = q_to_tx_stats(q); + + dma_addr = ionic_tx_map_single(q, frame->data, len); + if (dma_mapping_error(q->dev, dma_addr)) { + stats->dma_map_err++; + return -EIO; + } + buf_info->dma_addr = dma_addr; + buf_info->len = len; + buf_info->page = page; + buf_info->page_offset = off; + + desc_info->nbufs = 1; + desc_info->xdpf = frame; + desc_info->act = act; + + cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE, + 0, 0, buf_info->dma_addr); + desc->cmd = cpu_to_le64(cmd); + desc->len = cpu_to_le16(len); + desc->csum_start = 0; + desc->csum_offset = 0; + + stats->xdp_frames++; + stats->pkts++; + stats->bytes += len; + + ionic_txq_post(q, ring_doorbell, ionic_tx_clean, NULL); + + return 0; +} + static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct net_device *netdev, struct ionic_queue *rxq, @@ -302,6 +381,10 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, u32 xdp_action = XDP_ABORTED; struct bpf_prog *xdp_prog; struct xdp_buff xdp_buf; + struct ionic_queue *txq; + struct netdev_queue *nq; + struct xdp_frame *xdpf; + int err = 0; xdp_prog = READ_ONCE(rxq->lif->xdp_prog); if (!xdp_prog) @@ -330,14 +413,52 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, break; case XDP_REDIRECT: + goto out_xdp_abort; + case XDP_TX: + xdpf = xdp_convert_buff_to_frame(&xdp_buf); + if (!xdpf) + goto out_xdp_abort; + + txq = rxq->partner; + nq = netdev_get_tx_queue(netdev, txq->index); + __netif_tx_lock(nq, smp_processor_id()); + + if (netif_tx_queue_stopped(nq) || + unlikely(ionic_maybe_stop_tx(txq, 1))) { + __netif_tx_unlock(nq); + goto out_xdp_abort; + } + + dma_unmap_page(rxq->dev, buf_info->dma_addr, + IONIC_PAGE_SIZE, DMA_FROM_DEVICE); + + err = ionic_xdp_post_frame(netdev, txq, xdpf, XDP_TX, + buf_info->page, + buf_info->page_offset, + true); + __netif_tx_unlock(nq); + if (err) { + netdev_dbg(netdev, "tx ionic_xdp_post_frame err %d\n", err); + goto out_xdp_abort; + } + stats->xdp_tx++; + + /* the Tx completion will free the buffers */ + break; + case XDP_ABORTED: default: - trace_xdp_exception(netdev, xdp_prog, xdp_action); - ionic_rx_page_free(rxq, buf_info); - stats->xdp_aborted++; + goto out_xdp_abort; } + return true; + +out_xdp_abort: + trace_xdp_exception(netdev, xdp_prog, xdp_action); + ionic_rx_page_free(rxq, buf_info); + stats->xdp_aborted++; + return true; } @@ -880,6 +1001,16 @@ static void ionic_tx_clean(struct ionic_queue *q, struct sk_buff *skb = cb_arg; u16 qi; + if (desc_info->xdpf) { + ionic_xdp_tx_desc_clean(q->partner, desc_info); + stats->clean++; + + if (unlikely(__netif_subqueue_stopped(q->lif->netdev, q->index))) + netif_wake_subqueue(q->lif->netdev, q->index); + + return; + } + ionic_tx_desc_unmap_bufs(q, desc_info); if (!skb) From patchwork Tue Jan 30 01:30:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536648 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2075.outbound.protection.outlook.com [40.107.92.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3091A364D8 for ; Tue, 30 Jan 2024 01:31:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.75 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578273; cv=fail; b=LMUXNRprhs1isQohUcz+K5GBpIUBIGDq97O0zwnbcworAEeXEtQGn+dppNaJDRTLNcMM8M85yeDOekbsn11so3kp+w2q9t4VLABHfI3LqE0MMMLpzrYv09qwEcjKXSYTwZqBlo+1Gd+US7BipqyIXQ9l/njc3GLsUPMIXr5OAio= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578273; c=relaxed/simple; bh=olqJ6fsH4bqchTy4+JNG/9C7Il0YckzG2Kyl53s76yw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=I7MPSIG0gGtJ93dIvCIxVMmrDRTavbxCV5rUiNH3+EuH7kVSb3MNXlQ1oEVLtFADTI7aykB8mlFLHoCS7bCsUro6pl5HkioO4/hWZbUoLCPkSV8pC7F7G/iDWTgIODdFTNr/3P0DbaQRLxuGRf11BIOfg8cOJY/gssDsjoOzY4A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=KYBsOxrz; arc=fail smtp.client-ip=40.107.92.75 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="KYBsOxrz" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K2Lu+biylSTY+c1imKQZYQAICT70UsDe6HkqEwZmOSl7m5HPSSL1tstRZ1FNUReyKgUie1kjHFfdI5Rz0Bkj0XyKAnxvA994jMB4QuYgS6Bl4umjLqCkZYA/0nPJiCE8LwtyA877MhDm5RJt/eX9VWWjxH1wvbcYzRyi4WWdyCv/fN070GBL2qOvZ7ITnZI8eivv7E3LEESVNKd6IY4FyvBtJK4qMakgMCo+N87tFGhX42ZyGmptbMxTSq4ZO1b3neAR+PaRlSoAdnpep3d6qvyMsMV/lOdSI8depZuwI/SOlsdv+6c/eC45zBPnUFWS0cvHd84olDC298VQ+zzkWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XaH3CEUQZTWFLWGXid1hgMgpt+AZ5G6/5wOvz+wTxLs=; b=ofIRAdAYD50qwu779w2Y5DrRd2g/S4nB+1p77krEW5lipgD9uFiG/iXhCtN/e5OSVraUUEfJaxXVE0ardEgLSelH3t0roLf9k7h02mzn7yqlP+0ptdjdpZ9ZIwE86soYWejoWqEmZy7x57lB0AKDETs2AXp8GW7ddJCGCqDqIcS0SiVFCH9hM/XZvvGdFnJ4zClrIqOXkwgLici2f/jWbdw2mpJcjLF7dvK/9XqaUiVqMI8wSg4vr1zuEBslVlhDICaXSTfEbT/sQV1nVyUsi+aonZ400zIoJGk7rQW4Fr/OLkUlDjxmi33pFbNKB8f7Agp4QCjb4rKDFuguk0Hpnw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XaH3CEUQZTWFLWGXid1hgMgpt+AZ5G6/5wOvz+wTxLs=; b=KYBsOxrzBYoBa6CLdrxxRRTiwuedFi9kwhnao6B5Q4l3Fa5hF18ED+vrdNu6Prx2MtIp/9BpHFXdV3HNwGpZoVkhB3B/RNsZpRrr99E0ChWpZfgWQMLILNa6+T+ls5X25JH6znwg/13KNFy1fVMi8VLOesmknQsuP9gziQPz3r4= Received: from BLAP220CA0028.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:32c::33) by DS0PR12MB8477.namprd12.prod.outlook.com (2603:10b6:8:15b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 01:31:09 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:32c:cafe::2f) by BLAP220CA0028.outlook.office365.com (2603:10b6:208:32c::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:09 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:08 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 7/9] ionic: Add XDP_REDIRECT support Date: Mon, 29 Jan 2024 17:30:40 -0800 Message-ID: <20240130013042.11586-8-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|DS0PR12MB8477:EE_ X-MS-Office365-Filtering-Correlation-Id: 10d7d975-5b50-4e8f-d126-08dc2133231e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HI2DfeYqIATiFhsc60PnKX06kdLaTwr9NK04xm1ulXvEtdUcfOjkjIhTTdRv/CO0VqxRfj+m+qXr26Fho1qw0J7G57rAeMbfO9UgJFlU//2vx7bUJAY8nN7BrRC6LsMXYy91PEylpXkJROBtHG0Oz/Hg8QCbU82ouCh7ZkDBz6BktP9sZ1RsuGUnK2gshQ4Okza+mOgg1q785ZtyI6jOI/I8Zj23jpUdCXS0XLNWj/uig6MbHv2+TiiL4Wg+u9SRru5mwuhtpuU+MyAnTNWp9HqQ/F65aQbbwP6kpRph7kaQWm7ksiOkWFS/CYZZxxYZNf3x1CMPP8I/DtWTYHPj3QfyrO/Cmcrgm5Z+HxokvtWbbhtw9wSucStxMlkuv1lUNQSlT5M6DzXpPYuha+zPvyawMXTzbpivBndnBXXX791oAiDAGXox2yjEOuAKO5+Y0FeCaMoWm8l4i1XnLq+6Ql1VkVSQqVvTtpFsngRe1nFudcpoXiHuxiVBgFfNLRgK7LgdWKzNcJj3Lx9XSESiwduDo82u6tKloyWKVFJU1749P07XYJm0A4dwusbqfDc0vmLOT/AYyqbTd8xL1eOWmtsriMImvukQpuNDtETvz9Z6LAF7ZlMciWtAcpagDojxk4SyzgaEZKJ+JwoAEK9AL2TtocYtV2zQawc537DYLlNK96l5Dc6fWVHsCtN6G6pxA0CmZJ3zE5vC79D69DNeWfpzNNc4slBwsR1ODwB5yOVmWGTXvTZnI2narMwajBI148TZZtX0gwsektyO9hqNEg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(39860400002)(376002)(136003)(230922051799003)(64100799003)(186009)(451199024)(82310400011)(1800799012)(36840700001)(46966006)(40470700004)(26005)(40460700003)(40480700001)(6666004)(36860700001)(426003)(2616005)(1076003)(16526019)(336012)(356005)(81166007)(82740400003)(86362001)(36756003)(41300700001)(70586007)(2906002)(4326008)(478600001)(83380400001)(47076005)(5660300002)(44832011)(110136005)(70206006)(54906003)(8676002)(316002)(8936002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:09.5569 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 10d7d975-5b50-4e8f-d126-08dc2133231e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8477 X-Patchwork-Delegate: kuba@kernel.org The XDP_REDIRECT packets are given to the XDP stack and we drop the use of the related page: it will get freed by the driver that ends up doing the Tx. Because we have some hardware configurations with limited queue resources, we use the existing datapath Tx queues rather than creating and managing a separate set of xdp_tx queues. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_dev.h | 1 + .../net/ethernet/pensando/ionic/ionic_lif.c | 3 ++- .../net/ethernet/pensando/ionic/ionic_lif.h | 2 ++ .../net/ethernet/pensando/ionic/ionic_stats.c | 3 +++ .../net/ethernet/pensando/ionic/ionic_txrx.c | 24 ++++++++++++++++++- 5 files changed, 31 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 76425bb546ba..bfcfc2d7bcbd 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -266,6 +266,7 @@ struct ionic_queue { }; struct xdp_rxq_info *xdp_rxq_info; struct ionic_queue *partner; + bool xdp_flush; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 997141321c40..bd65b4b2c7f8 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -1649,7 +1649,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE; - netdev->xdp_features = NETDEV_XDP_ACT_BASIC; + netdev->xdp_features = NETDEV_XDP_ACT_BASIC | + NETDEV_XDP_ACT_REDIRECT; return 0; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 092ff08fc7e0..42006de8069d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -56,6 +56,7 @@ struct ionic_rx_stats { u64 xdp_aborted; u64 xdp_pass; u64 xdp_tx; + u64 xdp_redirect; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -144,6 +145,7 @@ struct ionic_lif_sw_stats { u64 xdp_aborted; u64 xdp_pass; u64 xdp_tx; + u64 xdp_redirect; u64 xdp_frames; }; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 5d48226e66cd..0107599a9dd4 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -31,6 +31,7 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(xdp_aborted), IONIC_LIF_STAT_DESC(xdp_pass), IONIC_LIF_STAT_DESC(xdp_tx), + IONIC_LIF_STAT_DESC(xdp_redirect), IONIC_LIF_STAT_DESC(xdp_frames), }; @@ -159,6 +160,7 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(xdp_aborted), IONIC_RX_STAT_DESC(xdp_pass), IONIC_RX_STAT_DESC(xdp_tx), + IONIC_RX_STAT_DESC(xdp_redirect), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -200,6 +202,7 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->xdp_aborted += rxstats->xdp_aborted; stats->xdp_pass += rxstats->xdp_pass; stats->xdp_tx += rxstats->xdp_tx; + stats->xdp_redirect += rxstats->xdp_redirect; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index a68f659e416d..c2be2215406a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -413,7 +413,19 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, break; case XDP_REDIRECT: - goto out_xdp_abort; + /* unmap the pages before handing them to a different device */ + dma_unmap_page(rxq->dev, buf_info->dma_addr, + IONIC_PAGE_SIZE, DMA_FROM_DEVICE); + + err = xdp_do_redirect(netdev, &xdp_buf, xdp_prog); + if (err) { + netdev_dbg(netdev, "xdp_do_redirect err %d\n", err); + goto out_xdp_abort; + } + buf_info->page = NULL; + rxq->xdp_flush = true; + stats->xdp_redirect++; + break; case XDP_TX: xdpf = xdp_convert_buff_to_frame(&xdp_buf); @@ -803,6 +815,14 @@ int ionic_tx_napi(struct napi_struct *napi, int budget) return work_done; } +static void ionic_xdp_do_flush(struct ionic_cq *cq) +{ + if (cq->bound_q->xdp_flush) { + xdp_do_flush(); + cq->bound_q->xdp_flush = false; + } +} + int ionic_rx_napi(struct napi_struct *napi, int budget) { struct ionic_qcq *qcq = napi_to_qcq(napi); @@ -820,6 +840,7 @@ int ionic_rx_napi(struct napi_struct *napi, int budget) ionic_rx_fill(cq->bound_q); + ionic_xdp_do_flush(cq); if (work_done < budget && napi_complete_done(napi, work_done)) { ionic_dim_update(qcq, IONIC_LIF_F_RX_DIM_INTR); flags |= IONIC_INTR_CRED_UNMASK; @@ -866,6 +887,7 @@ int ionic_txrx_napi(struct napi_struct *napi, int budget) ionic_rx_fill(rxcq->bound_q); + ionic_xdp_do_flush(rxcq); if (rx_work_done < budget && napi_complete_done(napi, rx_work_done)) { ionic_dim_update(rxqcq, 0); flags |= IONIC_INTR_CRED_UNMASK; From patchwork Tue Jan 30 01:30:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536649 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2070.outbound.protection.outlook.com [40.107.92.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 479E9364AB for ; Tue, 30 Jan 2024 01:31:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578275; cv=fail; b=Kq1B5w6Tf2wc3MCf71COlC+VXYaDqfF2lFrdYu+6osEwI5pApphsPk3A8PKbzpievugxzQnsKPfI7Sm+nXYAaGzs3E5AxqgESb3khJppKOVZcHGJxEHlV4QksCrVoOWN2Jv4BVjPQvuePsJ/zsUD9rXuDUbr0mR0m25BqlP/QDY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578275; c=relaxed/simple; bh=2g8GMJdRZHv9xdYAgNM/7eeUlVcCVapJh7bNUFKmyDU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OCSS5d1FiKSWF9fg037uXQwKP6iiznCFc1Yny5FIwONAPEoB/QFYS0kE+84FwW2Nmxn/oASaR5zPGCDk7iL/MWqxHnIkRAISDCPpcRlPWBCIiYgDzEbnD0nls5MUJGDTf4Coek8gkBlns7jWbd3YUs9iPM2gylim0cybvxI1Wbg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=NtFsXGE0; arc=fail smtp.client-ip=40.107.92.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="NtFsXGE0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m6ZuZNaUE19XoXQgtdUuM5lLW8+chcwuKqBxYHQ51nGIVBpNb0aa5NRvvhTUgGJvPhsHMnjtRgR7SbNSTda6l4biGfUOjJ+9hZOYVFs8/N7HRjOIJRLc59GKj5DTtgq3CvziTXTQNS/EzELgIipUyz+WuyeGG6NlLXknZnJ3yOs7e5uQc5Y8lKtj4z7jcpMEeH84tcM5/z9gx627JqS/BmlmZSgw+iBuYlwsB1cMsYrGbQf1+NYGvVyPyXFA7NJT71GKGmizt37ZJStu6f/AQq9jYodPdYdBB9A5/0/84RNUq0g0kwP9v5enE4tmIKI7WXN1cbwT5iwWCcZvma3W7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5RVok7rC3gV/gSSSVgsw8fEa9vrFF1HHJgTk+BLvU24=; b=RUMo1mSC3mcowl9W1Cx2rT3XG/9TtzgqbmSNAj0pN2Tpk9WEC9qZtL2IDfsLS/VTKoi7iaQ7bv/NVT6PHK0OB9K6mgOQ2B8/UmQiICcyUZw3cy8KZh6NBtiHjT7GHc8+DZwv1Ji7Dh4YYffWk+i9W5OE5eEOsj1ZGP9I4Sb6z27b0s1LMoXMNBp/vy+9THFD9pbWza/1Ss19p4gur5Iz8SAnA8kmCOK3XJ2uPw431lEK29pz5iLKHe697bT1Kv8Cq7JivQkQjjHCpZAPScVsVDOBe8CINUep9nawf3o7sNTZ8Vqd07BErvHUwwFFbY5BTkkOKIUv2m0WZk9F7UItWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5RVok7rC3gV/gSSSVgsw8fEa9vrFF1HHJgTk+BLvU24=; b=NtFsXGE0H0X1vT3LPMxGqlwuwRElQCn3oyOf0wK92ciMQE1mR/AUgmgj0R0oUsGnzR3V1TJMqEJXjWWVBYT8tUvRqRw/Gqzm94s/WnZhrI7r+7ztUHRGtr2iUxrY6qlvF2ZZw+B4pxlkeQ8as4V8yetWjhUskC025VutsZvu934= Received: from BLAP220CA0015.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:32c::20) by SN7PR12MB7420.namprd12.prod.outlook.com (2603:10b6:806:2a7::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.33; Tue, 30 Jan 2024 01:31:10 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:32c:cafe::15) by BLAP220CA0015.outlook.office365.com (2603:10b6:208:32c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:10 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:09 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 8/9] ionic: add ndo_xdp_xmit Date: Mon, 29 Jan 2024 17:30:41 -0800 Message-ID: <20240130013042.11586-9-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|SN7PR12MB7420:EE_ X-MS-Office365-Filtering-Correlation-Id: 44c7e1ea-e2d1-4c2e-f189-08dc213323c0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: g76MlNKc1RdX/Ghsnc3SSxLnuqRPM4g+X8ksdGhW2UOkPW3X/cKchER+PLP+o0O42evSuvgQuDEmJbh8sC4f6KkNy3+UOPMa2kWbeKvpNxFtqKzblg3TA78X3GlnTY285oTCY6M8P686canWDzvIwCVrmf9p6wbLBIitt6YNEucZhuSqUi7o3oNIdJHX3XtyVDtJxcL9X0CZB7ewMoqeH5EaKbMvaL93WCc8FpBRV1oORbgatiImGfNemY1Hzp02CN89zlTTVq68w6MV1T4KmScfV//x4/ieNvgQOo+1W5wqIxVXA7hoJD+B3RMQP2zyhfwVSygjAna3FfbLOxtfloLkVnIQa6+V63kl6+CGkd3ieMx/f/+BwkpOEIhMcr5mgg+IsABBD+0iGKJW0IeXFaGaSMBSI93q6VB+SfQe/x4ijBazm5ij/VqNYBqhl3MBt/J3I4lD/Axt7qLQ0WqSIu+0JsCwOSxeMCICo6KaTTCLXPzs+5YpvURcXllQVLoLfNRX2NjTBbOVVLLcKaejWPUJ8D9DHIgPlKIwByVqCoEKMReAOHXc2knRC64dQ6WQ9SCikjIqL3hI+zIBHHary9hu63YlIGRvbyGFoTvS7s9jNoOqTPbvbzZ7rPt0XrHf4SeGECO5Tu5EUXCAqM+i6XXjG5T++3g26GRb5v6F0HABnwfaY20B88NqhRUX5k0C+Qg01bLuJzTAtGV/V9sDlfnVpM6lsyyVEQhYbOThlGSFGq/VH0NBFi4d7DMXXRpaxTEX3ZuJ3YwdIabOASsgGQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(136003)(396003)(346002)(376002)(230922051799003)(1800799012)(186009)(451199024)(82310400011)(64100799003)(46966006)(36840700001)(40470700004)(47076005)(316002)(6666004)(54906003)(336012)(426003)(86362001)(70586007)(110136005)(70206006)(82740400003)(44832011)(4326008)(1076003)(81166007)(36860700001)(26005)(478600001)(40480700001)(40460700003)(2616005)(16526019)(8676002)(356005)(8936002)(83380400001)(5660300002)(2906002)(41300700001)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:10.6195 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 44c7e1ea-e2d1-4c2e-f189-08dc213323c0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7420 X-Patchwork-Delegate: kuba@kernel.org When our ndo_xdp_xmit is called we mark the buffer with XDP_REDIRECT so we know to return it to the XDP stack for cleaning. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_lif.c | 4 +- .../net/ethernet/pensando/ionic/ionic_txrx.c | 62 ++++++++++++++++++- .../net/ethernet/pensando/ionic/ionic_txrx.h | 1 + 3 files changed, 65 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index bd65b4b2c7f8..ed5d792c4780 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -1650,7 +1650,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) IFF_LIVE_ADDR_CHANGE; netdev->xdp_features = NETDEV_XDP_ACT_BASIC | - NETDEV_XDP_ACT_REDIRECT; + NETDEV_XDP_ACT_REDIRECT | + NETDEV_XDP_ACT_NDO_XMIT; return 0; } @@ -2844,6 +2845,7 @@ static const struct net_device_ops ionic_netdev_ops = { .ndo_eth_ioctl = ionic_eth_ioctl, .ndo_start_xmit = ionic_start_xmit, .ndo_bpf = ionic_xdp, + .ndo_xdp_xmit = ionic_xdp_xmit, .ndo_get_stats64 = ionic_get_stats64, .ndo_set_rx_mode = ionic_ndo_set_rx_mode, .ndo_set_features = ionic_set_features, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index c2be2215406a..618d15fb8d95 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -316,9 +316,13 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, buf_info = desc_info->bufs; dma_unmap_single(dev, buf_info->dma_addr, buf_info->len, DMA_TO_DEVICE); - __free_pages(buf_info->page, 0); + if (desc_info->act == XDP_TX) + __free_pages(buf_info->page, 0); buf_info->page = NULL; + if (desc_info->act == XDP_REDIRECT) + xdp_return_frame(desc_info->xdpf); + desc_info->nbufs = 0; desc_info->xdpf = NULL; desc_info->act = 0; @@ -372,6 +376,62 @@ static int ionic_xdp_post_frame(struct net_device *netdev, return 0; } +int ionic_xdp_xmit(struct net_device *netdev, int n, + struct xdp_frame **xdp_frames, u32 flags) +{ + struct ionic_lif *lif = netdev_priv(netdev); + struct ionic_queue *txq; + struct netdev_queue *nq; + int nxmit; + int space; + int cpu; + int qi; + + if (unlikely(!test_bit(IONIC_LIF_F_UP, lif->state))) + return -ENETDOWN; + + if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) + return -EINVAL; + + /* AdminQ is assumed on cpu 0, while we attempt to affinitize the + * TxRx queue pairs 0..n-1 on cpus 1..n. We try to keep with that + * affinitization here, but of course irqbalance and friends might + * have juggled things anyway, so we have to check for the 0 case. + */ + cpu = smp_processor_id(); + qi = cpu ? (cpu - 1) % lif->nxqs : cpu; + + txq = &lif->txqcqs[qi]->q; + nq = netdev_get_tx_queue(netdev, txq->index); + __netif_tx_lock(nq, cpu); + + if (netif_tx_queue_stopped(nq) || + unlikely(ionic_maybe_stop_tx(txq, 1))) { + __netif_tx_unlock(nq); + return -EIO; + } + + space = min_t(int, n, ionic_q_space_avail(txq)); + for (nxmit = 0; nxmit < space ; nxmit++) { + if (ionic_xdp_post_frame(netdev, txq, xdp_frames[nxmit], + XDP_REDIRECT, + virt_to_page(xdp_frames[nxmit]->data), + 0, false)) { + nxmit--; + break; + } + } + + if (flags & XDP_XMIT_FLUSH) + ionic_dbell_ring(lif->kern_dbpage, txq->hw_type, + txq->dbval | txq->head_idx); + + ionic_maybe_stop_tx(txq, 4); + __netif_tx_unlock(nq); + + return nxmit; +} + static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct net_device *netdev, struct ionic_queue *rxq, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h index d7cbaad8a6fb..82fc38e0f573 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h @@ -17,4 +17,5 @@ netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev); bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); +int ionic_xdp_xmit(struct net_device *netdev, int n, struct xdp_frame **xdp, u32 flags); #endif /* _IONIC_TXRX_H_ */ From patchwork Tue Jan 30 01:30:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13536651 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2084.outbound.protection.outlook.com [40.107.237.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9251936AF9 for ; Tue, 30 Jan 2024 01:31:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.84 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578276; cv=fail; b=deb1X8RqWjs/ZTLMd56L4Xfbb2mwv4w3fYK24fKbke9nE39N1Ko+EG8JRLwVTImcYs2NjurtkLUkvLx7cncI3DX8bcNu2hbzrjtpmBR8cxqtGXb0+GPT23t59MZINpDhDxz3KZZlhUPMuNwWh3ujfhTrnxDHm6a9mWUOlobIQWI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578276; c=relaxed/simple; bh=rTrVQVuSC1trimGoRGMFEaBdqZfASNVDVv4GokqH5lQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=vDiOujfUjzqOU8FmZ10injp4RvUP5UJjVo8/5OCx7Z6Wzt/ISAQk3XAJWAXQ3x1DiUZI736nkyyidw//56Lg9989gFPnIssjiB0ASvWngebKcwfZPenve3YfmkxF9jFdg93YvZxy475H6Y+sL3YqTvARgDxHkgUXoAAe5Ua6eRk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=W8T3lkXg; arc=fail smtp.client-ip=40.107.237.84 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="W8T3lkXg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xpl2QKfKrx4PLFBa6y8I7SBuNJJ5iujlwlJpO3H8dGlzQ8dZdcDPKgA4OD9tYWy6h+brGTp2LRAtePRXG7NjCHvRh+bQPMU2muLcZbOH6znFpNxUe+uB+AhzieuPZU+JNKs6h0gFTQlQWc54Vsx4nbTeSf7cq/2XZ6P0VQJhdIVtcTiIsbcWFsNxNB/+wXHIrtTQHsvVowGfzqwH2pm0DvmPpEAwI45Y40Fr0UYeRnQFVr8hZj369hGCgDM37O42GUdOeGurzAcRUKepotZ4lBWq+F/9r1whE6sA/ibvLja9UaTuRG23mBasNo20nbAY6XAT9qRbwtmdcM8KY8zW2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vB7CWYYDAYBhZgUXqHPwf+6AaIYuCIi8OUqGMl1C40U=; b=RxxPE5GxezzNlzChHDaGr6sXfk9cBtkGqgj3bT2gQMqG3k6O9CPfcIgSXm4B3or9BNy7Sn/w240mWjzT1E3h+114HRnilCQwFYUgQNNvDaS/KDrCb/f/dF/F5fe5omilff9ucFeKhunP2IZD5Ab+/N3zZCt5Llsur+YPdY2dt7urqwyAq2jlc+G7qMI18wiUNkiX7sEVY/iJroubmv+c5lvbyp6oudZtA4nLZ2ggGuJzZvysM4wwfmQ7wQ1GirP4K7Khd7ia7cf/z4HVN3aF4seybNjkg0yNkReJmkHrWabX2l2+Q61V9rre3qgAevu9whMMaZI+0Ic9aBHb4t3iMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vB7CWYYDAYBhZgUXqHPwf+6AaIYuCIi8OUqGMl1C40U=; b=W8T3lkXgV3zx8iOfHryDoqIiHS4g7taO2WAyZuIg+E3aXZO9vU3d1ULe3H2558tM4HrSCkS9B8MrDAvgLa0ryTHQxiKWxD4WvgEHDhy5sjk6LdACuChq2LvYojR2oDx3LN3Y6Nkq3EyeqeyI52af6aisakD0yapJEkW64K+Ibcg= Received: from BL1P223CA0029.NAMP223.PROD.OUTLOOK.COM (2603:10b6:208:2c4::34) by MW5PR12MB5624.namprd12.prod.outlook.com (2603:10b6:303:19d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.32; Tue, 30 Jan 2024 01:31:12 +0000 Received: from MN1PEPF0000F0E4.namprd04.prod.outlook.com (2603:10b6:208:2c4:cafe::9a) by BL1P223CA0029.outlook.office365.com (2603:10b6:208:2c4::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7228.34 via Frontend Transport; Tue, 30 Jan 2024 01:31:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E4.mail.protection.outlook.com (10.167.242.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7249.19 via Frontend Transport; Tue, 30 Jan 2024 01:31:11 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 29 Jan 2024 19:31:10 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH net-next 9/9] ionic: implement xdp frags support Date: Mon, 29 Jan 2024 17:30:42 -0800 Message-ID: <20240130013042.11586-10-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240130013042.11586-1-shannon.nelson@amd.com> References: <20240130013042.11586-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E4:EE_|MW5PR12MB5624:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b8a82a7-2bc7-476f-5192-08dc2133243c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0qOFf6v+o9TQ/V3IWDBfygI4aztmbIpeUu1zpJZ+qANsbO6fhUrQiQ9IorVKnIhub/QuJq3eIDnaAuE2vCT9roDzcm1rc7qO0jQkEk3Q64lKxbwB4fPk2sDD8NdcnHOAlI6AayOq987SBink78qEJIKag1U7DBkUf/nBWdBsUKBMj8v2gfj0s/VHjc2Vv4jaHsyuylwsKstK+Dxk1oYcAuNMKDkdBA2mbq4Yl6s1PO7ePPqh0WvkW3ti4+fCFf/21aKaZxwqD6kBfMOR351sk0ZpssBFIkZYwJE1Wav4F0d9y0r6RtaTw+RhwfuAZ6Rweq8t22BuI79A6rAQthdAx/i8cdpjK7KCDTxG6xWY3kvFkebTkls2ZLgQpMvvLMQf7R3L16FFzcPmFF0zl3e8ATzBlCFS9BZcALsPGzb0+xyw7ll22qOs85P71v/1yA3iHnT4ivSupSPP05nW5ORNNFdsRx533iG8BjjkGYd+dIM7vPA4Q4nX5pbyX+/QiGxfaVIYMcpnKjCANzqaeUDgDqJnWphKzjR8Eqk+fKGKHzAt43ljfp3pRctUHKCE5S9rmbNirqtpABZEFrYVGLZcSs26Vja9nIDNIVgtApBYWB2KXED9rModhlXG2e35hYPuaiPmnjdZtP5mc8zQ1xtTafAeoQ7odwfwOXvoRcqVmZvrZuC/OhvUR6XiPa1cqNtthBq7ZEGhhb+UYeOUng6hZeTXNDrI2rj9z/QTfp7pxyJWZNKCidfHBcJ97VF95GAZBj14TI2OPbCdN6LyqAvvbw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(39860400002)(376002)(396003)(346002)(230922051799003)(1800799012)(82310400011)(186009)(64100799003)(451199024)(36840700001)(40470700004)(46966006)(316002)(54906003)(110136005)(47076005)(83380400001)(70586007)(86362001)(70206006)(478600001)(40460700003)(6666004)(40480700001)(36860700001)(82740400003)(81166007)(1076003)(426003)(336012)(16526019)(2616005)(26005)(8676002)(8936002)(4326008)(356005)(44832011)(5660300002)(2906002)(41300700001)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2024 01:31:11.4280 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4b8a82a7-2bc7-476f-5192-08dc2133243c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E4.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR12MB5624 X-Patchwork-Delegate: kuba@kernel.org Add support for using scatter-gather / frags in XDP in both Rx and Tx paths. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_lif.c | 12 ++- .../net/ethernet/pensando/ionic/ionic_txrx.c | 91 ++++++++++++++++++- 2 files changed, 98 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index ed5d792c4780..11adf3f55fd0 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -881,7 +881,8 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) q->partner = &lif->txqcqs[q->index]->q; q->partner->partner = q; - if (!lif->xdp_prog) + if (!lif->xdp_prog || + (lif->xdp_prog->aux && lif->xdp_prog->aux->xdp_has_frags)) ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); if (qcq->flags & IONIC_QCQ_F_CMB_RINGS) { @@ -1651,7 +1652,9 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | - NETDEV_XDP_ACT_NDO_XMIT; + NETDEV_XDP_ACT_RX_SG | + NETDEV_XDP_ACT_NDO_XMIT | + NETDEV_XDP_ACT_NDO_XMIT_SG; return 0; } @@ -1799,6 +1802,9 @@ static bool ionic_xdp_is_valid_mtu(struct ionic_lif *lif, u32 mtu, if (mtu <= IONIC_XDP_MAX_LINEAR_MTU) return true; + if (xdp_prog->aux && xdp_prog->aux->xdp_has_frags) + return true; + return false; } @@ -2809,7 +2815,7 @@ static int ionic_xdp_config(struct net_device *netdev, struct netdev_bpf *bpf) } maxfs = __le32_to_cpu(lif->identity->eth.max_frame_size) - VLAN_ETH_HLEN; - if (bpf->prog) + if (bpf->prog && !(bpf->prog->aux && bpf->prog->aux->xdp_has_frags)) maxfs = min_t(u32, maxfs, IONIC_XDP_MAX_LINEAR_MTU); netdev->max_mtu = maxfs; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 618d15fb8d95..f80ee7a870fb 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -15,6 +15,13 @@ static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs); static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, void *data, size_t len); +static dma_addr_t ionic_tx_map_frag(struct ionic_queue *q, + const skb_frag_t *frag, + size_t offset, size_t len); + +static void ionic_tx_desc_unmap_bufs(struct ionic_queue *q, + struct ionic_desc_info *desc_info); + static void ionic_tx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info, struct ionic_cq_info *cq_info, @@ -309,6 +316,7 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, unsigned int nbufs = desc_info->nbufs; struct ionic_buf_info *buf_info; struct device *dev = q->dev; + int i; if (!nbufs) return; @@ -320,6 +328,15 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, __free_pages(buf_info->page, 0); buf_info->page = NULL; + buf_info++; + for (i = 1; i < nbufs + 1 && buf_info->page; i++, buf_info++) { + dma_unmap_page(dev, buf_info->dma_addr, + buf_info->len, DMA_TO_DEVICE); + if (desc_info->act == XDP_TX) + __free_pages(buf_info->page, 0); + buf_info->page = NULL; + } + if (desc_info->act == XDP_REDIRECT) xdp_return_frame(desc_info->xdpf); @@ -360,8 +377,38 @@ static int ionic_xdp_post_frame(struct net_device *netdev, desc_info->xdpf = frame; desc_info->act = act; + if (xdp_frame_has_frags(frame)) { + struct ionic_txq_sg_elem *elem; + struct skb_shared_info *sinfo; + struct ionic_buf_info *bi; + skb_frag_t *frag; + int i; + + bi = &buf_info[1]; + sinfo = xdp_get_shared_info_from_frame(frame); + frag = sinfo->frags; + elem = desc_info->txq_sg_desc->elems; + for (i = 0; i < sinfo->nr_frags; i++, frag++, bi++) { + dma_addr = ionic_tx_map_frag(q, frag, 0, skb_frag_size(frag)); + if (dma_mapping_error(q->dev, dma_addr)) { + stats->dma_map_err++; + ionic_tx_desc_unmap_bufs(q, desc_info); + return -EIO; + } + bi->dma_addr = dma_addr; + bi->len = skb_frag_size(frag); + bi->page = skb_frag_page(frag); + + elem->addr = cpu_to_le64(bi->dma_addr); + elem->len = cpu_to_le16(bi->len); + elem++; + + desc_info->nbufs++; + } + } + cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE, - 0, 0, buf_info->dma_addr); + 0, (desc_info->nbufs - 1), buf_info->dma_addr); desc->cmd = cpu_to_le64(cmd); desc->len = cpu_to_le16(len); desc->csum_start = 0; @@ -444,6 +491,8 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct ionic_queue *txq; struct netdev_queue *nq; struct xdp_frame *xdpf; + int remain_len; + int frag_len; int err = 0; xdp_prog = READ_ONCE(rxq->lif->xdp_prog); @@ -451,8 +500,9 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, return false; xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); + frag_len = min_t(u16, len, IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN); xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), - XDP_PACKET_HEADROOM, len, false); + XDP_PACKET_HEADROOM, frag_len, false); dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), XDP_PACKET_HEADROOM, len, @@ -460,6 +510,43 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, prefetchw(&xdp_buf.data_hard_start); + /* We limit MTU size to one buffer if !xdp_has_frags, so + * if the recv len is bigger than one buffer + * then we know we have frag info to gather + */ + remain_len = len - frag_len; + if (remain_len) { + struct skb_shared_info *sinfo; + struct ionic_buf_info *bi; + skb_frag_t *frag; + + bi = buf_info; + sinfo = xdp_get_shared_info_from_buff(&xdp_buf); + sinfo->nr_frags = 0; + sinfo->xdp_frags_size = 0; + xdp_buff_set_frags_flag(&xdp_buf); + + do { + if (unlikely(sinfo->nr_frags >= MAX_SKB_FRAGS)) { + err = -ENOSPC; + goto out_xdp_abort; + } + + frag = &sinfo->frags[sinfo->nr_frags]; + sinfo->nr_frags++; + bi++; + frag_len = min_t(u16, remain_len, ionic_rx_buf_size(bi)); + dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(bi), + 0, frag_len, DMA_FROM_DEVICE); + skb_frag_fill_page_desc(frag, bi->page, 0, frag_len); + sinfo->xdp_frags_size += frag_len; + remain_len -= frag_len; + + if (page_is_pfmemalloc(bi->page)) + xdp_buff_set_frag_pfmemalloc(&xdp_buf); + } while (remain_len > 0); + } + xdp_action = bpf_prog_run_xdp(xdp_prog, &xdp_buf); switch (xdp_action) {