From patchwork Wed Feb 14 17:59:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556880 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2072.outbound.protection.outlook.com [40.107.237.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85CAE128804 for ; Wed, 14 Feb 2024 17:59:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.72 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933572; cv=fail; b=hNTitU+qNgMNS0NThsQG9sw12CrVUxcQZgj0u/wXTFQHs/fZWlHBQrT+GmzfUnQjSqu3DuxFjuHw5hWDQ+jFA8QGeLG+vhR5CGuhr1Y7bdzkwePbxulrjjDi9DsNh8r9iQAa5l48q7ApqDRgthrA+yL0sYOn0ZH5xPUXCValf+g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933572; c=relaxed/simple; bh=r9mZdjKxF+ESrjLt7Fo4bL3aM9vW6towQe6iO2M9DIw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BoxwJaXGqoZSpeAIjU7C9UfGSCS8THHIzefgXPFJIWsOX5iKppKxxm6kRYPHZJv4VZ4UtjzGU3Kqq0ugylXAM1pJi+XVqqQDUqgZiJbk+rtMdi+w99xVPmffs69TuROBr1qJTecNvaMHS4VysKqMThfnTpfPkjK6O1tcYbyV5fE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=SxTpp8gv; arc=fail smtp.client-ip=40.107.237.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="SxTpp8gv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Av5p0VLZaCtlCDQOL/hdSrV07b1lXEbEttqQQMmDmFHsaXNo4c0JYNDMbQiElFIqtCvea6oRjMSPoZ0t9hG9JFx6dU7MYlebJFcXwd8WaTcHP29cay9X9o1HhMuRcYBdQjaVxHUpuZmHnwn3VzC3EUeK+l2ChZmV+glD/7Qx+CfN3qFL+ySD2/3Jip+REIM7MR4RH6WIm6/pFHDt2NBMqnyNZKgyqr8IqiFKP1OdCjl9esE1nFJM/qgFxvPmAgnN+HHTi5EPDYOLvdIEDGHO0TJhX1PUfJfeKOgkvCM4Qmw4xXpgx7l3GePv97ZZuhC1/2kBMfP8eIUFA6AixP6ADQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IlaePoqD63MkIfC/8xk9mww37KcdFE0/JQKsrH9YqVk=; b=fSAR9Cfcg7kAm1pYiHSZcy2DjbuyuQ7P/4SPDvlVmCdvlqLAgezkkgd3ki3e0FJfTni/JKuvVB8ROhBxYZGyZQwzvCqoAdNFrJmv486CMyViIIkP1/k0B9Wx5tJHXux5I6jX7PI3+DIY+AV+vX3xGTK4uos23hymoE927H5ie/qA5+Tpxx5ljhgGGyWylkPter2keH2+Y5xDfXP08XZJ8O99i8Izc3QvtL0FPQd70uR1/jS2bkegYDvvzgGxE+yl6pJUDnRl1K5wWNgKgoFpdayxCUha3vhnoIvnxRJ1tHbjGWiQ46ay/mlvl5qEAY49bztspF/64VdMYOrA2PX/1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IlaePoqD63MkIfC/8xk9mww37KcdFE0/JQKsrH9YqVk=; b=SxTpp8gveShid35TOOGRwLnb5z6UIQOL7zxtRyrcghjvLCfD5pHQBZqvLXnc8eG0cvJILrIpnbzy5igYKRr72NgdUZlctchInV1V/KEu7a1H1sGKbTgKcgdHZE9Fyxn9ZuH0jaXgYckaBPU4Gtwe+S6SqVFBtApiPspEtT0xfus= Received: from BL0PR0102CA0036.prod.exchangelabs.com (2603:10b6:207:18::49) by IA0PR12MB8304.namprd12.prod.outlook.com (2603:10b6:208:3dc::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb 2024 17:59:27 +0000 Received: from BL6PEPF0001AB76.namprd02.prod.outlook.com (2603:10b6:207:18:cafe::7) by BL0PR0102CA0036.outlook.office365.com (2603:10b6:207:18::49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39 via Frontend Transport; Wed, 14 Feb 2024 17:59:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB76.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:27 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:26 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 1/9] ionic: set adminq irq affinity Date: Wed, 14 Feb 2024 09:59:01 -0800 Message-ID: <20240214175909.68802-2-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB76:EE_|IA0PR12MB8304:EE_ X-MS-Office365-Filtering-Correlation-Id: caa9ed3c-44d1-437d-3aca-08dc2d86afc7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Rnu8GQfCLMLzGhuAv7iR3CL4FE7n0s9SaF0oXaWG5jWHdvNYD4TY//Bjp7i+J9nVXP9QGHIRwDQjgDHDM5FkwaD6SKBmBw++1ggayAwBTFPvnb5gSSbse4e5JPJZp+mtwUa3f8hS8fzBz5ATVCSJ0d7+r+0/Ab3Fp8O/SUTeYyMuPTwcFia0oUr8wd80U1MC76XyarPecomqmC1P7sf9dHZz6JFVhLUjL+Brk8kCtQPZInyh70HKJ5MuexsSs2c53OPg4P2p5SGhhvNRfbtGoIqY+0ZpW1QMxYVQ/7uhovyidzbNnuoKKwM/VVZlmRXwtmuDE3Rk/Qz+2caQTzlDcg8PgDhrKoYEmFe9P0Mh7aQtyCRg/UQ8HMSd1xrXakvuyyTBXr4M64s6pOR/JU9GJZMzTNQ2B56cB0TlSv9PQNTDdZGQFObAimF9sRwR0g7Pr/SQpqd+t9CG6+w16zCyv4C8FoKeJwGs4HcfvGYEHsgcXAqUpWTLCpldzsHKUN1e+ybQlaWxtteCKUK4MGmbopiDP8kNs6Ma0mLm+LgBzaWQLMl1lZ2vkR1fZ0OBTERQKRfUHk8VuM/lNAagSY7wdZ3gAAp9jFRDg41Fwb0kBSw= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(396003)(136003)(376002)(39860400002)(230922051799003)(82310400011)(186009)(451199024)(64100799003)(1800799012)(46966006)(36840700001)(40470700004)(8936002)(83380400001)(426003)(41300700001)(2616005)(336012)(26005)(70586007)(16526019)(1076003)(70206006)(4326008)(8676002)(316002)(54906003)(6666004)(36756003)(110136005)(478600001)(356005)(86362001)(82740400003)(81166007)(4744005)(2906002)(44832011)(5660300002);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:27.7392 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: caa9ed3c-44d1-437d-3aca-08dc2d86afc7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB76.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8304 X-Patchwork-Delegate: kuba@kernel.org We claim to have the AdminQ on our irq0 and thus cpu id 0, but we need to be sure we set the affinity hint to try to keep it there. Signed-off-by: Shannon Nelson Reviewed-by: Brett Creeley Reviewed-by: Jacob Keller --- drivers/net/ethernet/pensando/ionic/ionic_lif.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index cf2d5ad7b68c..d92f8734d153 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -3391,9 +3391,12 @@ static int ionic_lif_adminq_init(struct ionic_lif *lif) napi_enable(&qcq->napi); - if (qcq->flags & IONIC_QCQ_F_INTR) + if (qcq->flags & IONIC_QCQ_F_INTR) { + irq_set_affinity_hint(qcq->intr.vector, + &qcq->intr.affinity_mask); ionic_intr_mask(idev->intr_ctrl, qcq->intr.index, IONIC_INTR_MASK_CLEAR); + } qcq->flags |= IONIC_QCQ_F_INITED; From patchwork Wed Feb 14 17:59:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556881 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2056.outbound.protection.outlook.com [40.107.237.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77B69129A9F for ; Wed, 14 Feb 2024 17:59:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933573; cv=fail; b=Mzg7ClubVJcxDNdYRhrT536sVYQPYhiILCuvgfMThoH/3KXzupkYs2Dj54oBtMD0zj7/DHYV8MpySRBSPiBSDrTKNYjXvJ11f4fUqjr8mbWNabR20PZBYo9yqswqjdsUUgQZ9L/El2RmQVCtf2l5aydCMgZapUsvxnJeyzb2R54= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933573; c=relaxed/simple; bh=AxI13vZKuvW8V/NnJc+Sk/ktITj+sfxd8mO2KLf8zDE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QIgaYvYyAK528imdhfrgvUkjOHJNvC2jgKfKJloSJI3siXBdD6z7I4y9CI7a4wSbgqwmyRrNaxVjV9k/hy5Rk1+bUXmpjcPMOb3kykl3Gqci14Rdcua9PhORwLTb+v5okTlo3/5TC5TBIp4bLKCkAp3jHktnHUP6i0hEIisIjDg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=lytCS975; arc=fail smtp.client-ip=40.107.237.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="lytCS975" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ABM8VmLM12Rt5IwtHL+Zn7WONkBQJ5ZVXc0kIeukQZlsr39w1UuA8hhApcFC4ZbiQJL0BbzkYtM1gzUhtE2MAwfmQ5kOK5lKt74qw9nFxJVQMu5B24G1/yynjtFaojaGSBsGEZksZBO/JnaRqORBQCipkX5Nh0LIIYExbPOua2rcqMkg0mVsvlXxxyBbw6iNWf91fB7UcphKNaf9wmuCpJrmdjaBjmwqW5Pw3z1jXp9m+SHjwJ2f1nAKzLqKzldhPhN+GBfsK/KKgEtL5YGcew2sWVC1zK0yfaL5AMDSLRisMCFVQ6pFxKiXR24fOPLIYxOcrlLM0zB3j+xwIadKdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vsZbhbZMnU1dPDHAuSYwocUtSoHA3j02ciZhjiW5ZCg=; b=QDU4k/76QbDpa+PyVE9vRDhcC1iUwmKaCDDeEjbqeBRHYoIeA50fCLyHPRjQqzm7pYVlSLCfWSnWIFc+hgQ6lREJZvGgSYn2K9DB0nYHy2QsdY8lwhZhz5WHjFqZgD61AVqk+tOlh/ryu70UljjfrNEelvun6LBM+8aTYoZejw+YIIYGdmK3rzFEgeLTPNJmS9YHv0IiHF9yZpsG1271hm4askxpPhSAefmbsV5cR8oMCeBVkaxISJGQCVu24XotuxAovGtZWnxBZQDzmR1DK64WA7YhUNAEzaq2NeAo1O+dUjuJMa+H27Crh0wAQ5/+Ny7mpasTVRC3fM0stoxxPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vsZbhbZMnU1dPDHAuSYwocUtSoHA3j02ciZhjiW5ZCg=; b=lytCS975qq02mSpoEeUdX5oq/Yals4k266yKhvmmBjJE4EVprij37uW1epI3w09kfaHINjSrXI++jmXINWqoZur/hU/okznEctfKybTrTQSkr2cMNZgw093VwZl8YfETln4kL6fy5LG9dSFfc3WTTmNbPVls0BSWR0r+oBqL8SM= Received: from BL0PR0102CA0014.prod.exchangelabs.com (2603:10b6:207:18::27) by DM4PR12MB8499.namprd12.prod.outlook.com (2603:10b6:8:181::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26; Wed, 14 Feb 2024 17:59:28 +0000 Received: from BL6PEPF0001AB76.namprd02.prod.outlook.com (2603:10b6:207:18:cafe::ed) by BL0PR0102CA0014.outlook.office365.com (2603:10b6:207:18::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39 via Frontend Transport; Wed, 14 Feb 2024 17:59:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB76.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:28 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:26 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 2/9] ionic: add helpers for accessing buffer info Date: Wed, 14 Feb 2024 09:59:02 -0800 Message-ID: <20240214175909.68802-3-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB76:EE_|DM4PR12MB8499:EE_ X-MS-Office365-Filtering-Correlation-Id: 7a6d1e9a-886c-414f-d3e6-08dc2d86b021 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6ui10VA5vJM7BSQVDVsXg/DlyKpfugnbs9dZFbWrwqwo2uBxkyIoNnckKa4DvMPgRIfUBKj7LO8EgeViAPzz4DspkDQo2jAnO4B210v2cUUOJ835BFrwYFwY2Lk8PdTm10fWi3MwBu6w8ZkasagG88GKZGbNofUhzylFRycnJ2J44rBXh1Qw+lTpNmVg2xaogtCgFusCvcJxbYt+EDy/WNWN7BlzyIiLb/fKJRcn6fP7bjkwJk9aA5cAiWYGwlRJkzeP4l66vxRpzbJ/I0qjgF3L9wR3p/S8p1yZnYXAkYNqz+z0eDDf8uTovvRqrtE3gzQ2m77dWrtccLn4quvkArl/w4XgftblJG30N7i1zbOKTW4TvJLYHDviLcL0EZe6rQzmQ7iiznkgrp4Y2A6a0M+j8F23iR93nmDUuvEPRsfkJtQbG+mkbhk3O18zJRAtyprIvQynyEqun/gOWoZE4O1wOUf4bb5EyEhbfoq3YsALSjnjsyY/HBX182JAhkH7kDMgopI9cKzX4SlCmKeguyRpUAE0TuNvvUBkbtYs1xejNaeV+Eln3hVGupFrmuA0Xt9l3bhkhBwajQ8Qn7GNt1gh6NxmRjGJpmQ5M/ME3HQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(396003)(39860400002)(346002)(230922051799003)(1800799012)(451199024)(186009)(82310400011)(64100799003)(36840700001)(40470700004)(46966006)(83380400001)(86362001)(356005)(82740400003)(81166007)(70586007)(70206006)(478600001)(110136005)(2616005)(1076003)(54906003)(6666004)(336012)(16526019)(316002)(426003)(26005)(36756003)(2906002)(4326008)(5660300002)(8936002)(8676002)(44832011)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:28.3173 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7a6d1e9a-886c-414f-d3e6-08dc2d86b021 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB76.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8499 X-Patchwork-Delegate: kuba@kernel.org These helpers clean up some of the code around DMA mapping and other buffer references, and will be used in the next few patches for the XDP support. Signed-off-by: Shannon Nelson Reviewed-by: Brett Creeley Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_txrx.c | 37 ++++++++++++------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 54cd96b035d6..00d703c027fb 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -88,6 +88,21 @@ static inline struct netdev_queue *q_to_ndq(struct ionic_queue *q) return netdev_get_tx_queue(q->lif->netdev, q->index); } +static void *ionic_rx_buf_va(struct ionic_buf_info *buf_info) +{ + return page_address(buf_info->page) + buf_info->page_offset; +} + +static dma_addr_t ionic_rx_buf_pa(struct ionic_buf_info *buf_info) +{ + return buf_info->dma_addr + buf_info->page_offset; +} + +static unsigned int ionic_rx_buf_size(struct ionic_buf_info *buf_info) +{ + return min_t(u32, IONIC_MAX_BUF_LEN, IONIC_PAGE_SIZE - buf_info->page_offset); +} + static int ionic_rx_page_alloc(struct ionic_queue *q, struct ionic_buf_info *buf_info) { @@ -207,12 +222,11 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, return NULL; } - frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - buf_info->page_offset)); + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; dma_sync_single_for_cpu(dev, - buf_info->dma_addr + buf_info->page_offset, + ionic_rx_buf_pa(buf_info), frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, @@ -262,10 +276,10 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return NULL; } - dma_sync_single_for_cpu(dev, buf_info->dma_addr + buf_info->page_offset, + dma_sync_single_for_cpu(dev, ionic_rx_buf_pa(buf_info), len, DMA_FROM_DEVICE); - skb_copy_to_linear_data(skb, page_address(buf_info->page) + buf_info->page_offset, len); - dma_sync_single_for_device(dev, buf_info->dma_addr + buf_info->page_offset, + skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); + dma_sync_single_for_device(dev, ionic_rx_buf_pa(buf_info), len, DMA_FROM_DEVICE); skb_put(skb, len); @@ -452,9 +466,8 @@ void ionic_rx_fill(struct ionic_queue *q) } /* fill main descriptor - buf[0] */ - desc->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - buf_info->page_offset)); + desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); desc->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; @@ -472,10 +485,8 @@ void ionic_rx_fill(struct ionic_queue *q) } } - sg_elem->addr = cpu_to_le64(buf_info->dma_addr + buf_info->page_offset); - frag_len = min_t(u16, remain_len, min_t(u32, IONIC_MAX_BUF_LEN, - IONIC_PAGE_SIZE - - buf_info->page_offset)); + sg_elem->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); + frag_len = min_t(u16, remain_len, ionic_rx_buf_size(buf_info)); sg_elem->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; From patchwork Wed Feb 14 17:59:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556882 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF99A12A15C for ; Wed, 14 Feb 2024 17:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933575; cv=fail; b=hAQL6SxDWJCP4OImeFq+1ZSIqCd8cXcE7ldu4Z0+Nhc88IiyjOiro9pWoScrBCxPUBNXqb70YQ901UAIt51s+HlgVAUvTVRikhX613iKQKLS9zsE/RE9SZHizOzeTDDuFEnisO8Ve+sPhFRHFwdocxkb0qytJClGFElRcHcrza0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933575; c=relaxed/simple; bh=Pmn9ltZ6zuZaK+B4AaF8IQ8Lqw2t9Bz8w23qquDmaYo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=keVkNnwXAWLNSkDwZ1RVHIzlf792/tjA0V3y3WbBO6pYOfVl8niRJPVpv3m7/VR44dUpkWoFr6I48gNuY7GLhvgmlKDrNCDU6Z3ohJhy/7KikcNF9g7rNSHbutvwySc0nZXY6+WryrC85YFP1pr9D5Qg1IMBnyKRVzbzjnPkHOo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Q5QRyhSy; arc=fail smtp.client-ip=40.107.236.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Q5QRyhSy" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eSwbTp2wLbQzi1hxFKfzVRnAL5fDGA/5aKTvuotMEY3jz4ZGO1Vth00Nk2MW4Y/Kn68wxmA5L93sXst4wQXjz+YqKIy9RMEGXRnoT0VCmPF7Yl/JSNvcJ03HIsyprAqkLCkas6l0RkxruEXDp5IDmIZd5GEfaGUFxVr45LqpfSpLAysSUU28ggavTG/ZtVbDqrFQM5xSpAs4kc1a5HeDRqTgQoGK6G+SkScTxSY/8miYqH8mo/hfE8/dAJBlA8S3lRnoDHl/LXbD9yk5cRJKBJ0qkdPwv3wyZTPYMJbUH/ej74lFOEjW8zuesOpKJp96AXsl15X7bEc8fC7srl1nzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GYpJNQneOdCzvSiZZ7gIS2G6zShEAlX/Pxs+YmCNKvs=; b=VoeLH7Xbv15BIxY+0Qu1YBTzdUqWeNxMunRASh5pXBzzbqmg3rW3brPB8bjxyJYrIsj9wMwuYTW+SrMG4UOjlTCQ7qX96Fuk1uPY/QIOEBHaD0ToFB3YGQUMrqD/9rkBRk3oZYZJ/WNoTB2VEaieYQfb0UQcaCKIsm0rsRwN0Y2OvM1AVR0kl0ECYdbqJORYfvni57U59r8ONLUgfDDaVx0b0Zwtam6EaPEm5qY4IXYRzNGOfwuKAyXgPKK0LuqVA9Nr6pQLBG39dyKPCpA5hsct8Znvd6bhQWu7O6CgPd/H1hqOJmEGzKtbNKB/7s7Y+ZQBr/7zdq+lr6wlU/HZxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GYpJNQneOdCzvSiZZ7gIS2G6zShEAlX/Pxs+YmCNKvs=; b=Q5QRyhSyMdG06G1qzwT5bCjWeZUgHBgc5P9YnfuwWZc3QStzP0kUKSo2s8hFXZW3z4hjfMqbHVWjoHyG8TGPct9n+5/OX8iAZK7Dgev0F+SnAG7cq9JB2TKxKuvkwIQuFO0hjh1+0tNDqLkqWvM/NRvkiUXExQxMNQ+5hy1o+LY= Received: from BL0PR0102CA0010.prod.exchangelabs.com (2603:10b6:207:18::23) by IA1PR12MB6385.namprd12.prod.outlook.com (2603:10b6:208:38b::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb 2024 17:59:29 +0000 Received: from BL6PEPF0001AB76.namprd02.prod.outlook.com (2603:10b6:207:18:cafe::5d) by BL0PR0102CA0010.outlook.office365.com (2603:10b6:207:18::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.41 via Frontend Transport; Wed, 14 Feb 2024 17:59:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB76.mail.protection.outlook.com (10.167.242.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:29 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:27 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 3/9] ionic: use dma range APIs Date: Wed, 14 Feb 2024 09:59:03 -0800 Message-ID: <20240214175909.68802-4-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB76:EE_|IA1PR12MB6385:EE_ X-MS-Office365-Filtering-Correlation-Id: acb5f531-5315-4366-7f91-08dc2d86b08f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2AAQTVPVfTC1RPEcm8lFEVZE2sn6U9UcH0v93REI21XILfBGYqLJ4YwUeVmZaSq4Btt+k5EsLwCyzBl0B8x8sWM3BdUmm7/xjCzInVOW1MR8UWpFN/2+wcLvClA0hzY4mARpLrObDREyFT5avNMUkb7mul8bOnOgghrfBYPVjA1QjrMxPRut2oIn3XXJxGZYr5RE0OsIMZtUoELCODNYKFUv1/Xm3kIIg3CzoNKYby8ba10qvpBCKVVSP3oob0J5QQQZtQaIFGIJGJs48o0ZT/vUeygzPL/qNAH0Ec9M34WzNTwNCnyUA0Q2CqiQD6hn7NJh6TzoZGRMl35Zhj0ZUHL9tOIv/SL3ZVZY6VOavcHVvEjxW1x8QaLxdUvMNi/O5F2nIVUrUQWyPcGCuorQU12SZguMhfA7UQ5H31kyDha7xVptwHcqAvjDXEU/9PkGr/ra2ApKG7GO4z98UxdXLSthgsmGPvYjZNwdylg/jlqkTnye5ObTBOWVDb/CBuyLmSLYl69jFBboWprrEIs+MaOFhsSwIHZm2n31fiLV188sUIuvznANKUiWPMDjS7DyU0utso5fmqcGX0s+oC240E9Q81awx3b0CMMNxDahXx0= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(39860400002)(396003)(346002)(230922051799003)(1800799012)(186009)(82310400011)(64100799003)(451199024)(36840700001)(40470700004)(46966006)(8936002)(2906002)(8676002)(44832011)(5660300002)(4326008)(426003)(336012)(83380400001)(2616005)(86362001)(26005)(16526019)(1076003)(82740400003)(36756003)(356005)(81166007)(70206006)(110136005)(70586007)(316002)(6666004)(54906003)(478600001)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:29.0360 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acb5f531-5315-4366-7f91-08dc2d86b08f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB76.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6385 X-Patchwork-Delegate: kuba@kernel.org Convert Rx datapath handling to use the DMA range APIs in preparation for adding XDP handling. Signed-off-by: Shannon Nelson Reviewed-by: Brett Creeley Reviewed-by: Jacob Keller --- drivers/net/ethernet/pensando/ionic/ionic_txrx.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 00d703c027fb..7f095717595d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -225,9 +225,8 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; - dma_sync_single_for_cpu(dev, - ionic_rx_buf_pa(buf_info), - frag_len, DMA_FROM_DEVICE); + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, buf_info->page, buf_info->page_offset, frag_len, @@ -276,11 +275,11 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return NULL; } - dma_sync_single_for_cpu(dev, ionic_rx_buf_pa(buf_info), - len, DMA_FROM_DEVICE); + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, len, DMA_FROM_DEVICE); skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); - dma_sync_single_for_device(dev, ionic_rx_buf_pa(buf_info), - len, DMA_FROM_DEVICE); + dma_sync_single_range_for_device(dev, ionic_rx_buf_pa(buf_info), + 0, len, DMA_FROM_DEVICE); skb_put(skb, len); skb->protocol = eth_type_trans(skb, q->lif->netdev); From patchwork Wed Feb 14 17:59:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556883 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2066.outbound.protection.outlook.com [40.107.93.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFF3312A15F for ; Wed, 14 Feb 2024 17:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.66 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933576; cv=fail; b=SFO+GPv49sphXpTpbiv+JE02Fy4LplcXQzpRq/e24YXsRS0IFI5ZtDaKgamNEN7lhHL1Rc5/B8gk2SaI/7gho+gTXaSzAaVqAImpaVcYm4P1ctUn3BmPe5Xukajc4lYORiRrlQ2xsmTNiyw+Wg1c4+clzsWyElb6Oa4Ohpf5WP4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933576; c=relaxed/simple; bh=6aUrpcF0DgS277oLOYZHpBK2NWy8VkDCkK1RsglakiI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FHFP76euztx1rOXTy4fo4A1ZVyKsXVEGXcOqn2HyKiDzq7fG6N5yohYz37oIMC97d3InDY1mfEd4Gw+e4OO3i4mgalODr8QHO4VHY0puxsmvpFV33/eSH7XJF3YW2sttV7JP+Zif7XYwcE8Gn0+eFVd21qy42zK0P9z1ivqrCwg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=xrB/kPsM; arc=fail smtp.client-ip=40.107.93.66 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="xrB/kPsM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nRKtNZpup128rnO8aFwsDKHJtU4T0cwJRwcprtJJboK4TwpjwmB7eJX+ppnIyJvPw59fZXViq32EWCtqip/RiYpBkUPwmUlS+Q4lfbZED4iw5ShNKe8DD3b4TvG4dX9paLcia1SY82sQab5tqTi9JpM63nbxNRKlObuG7iJ9M+OfHcg8l4lePE95fwjY3QRTvjlfpFDZ6eNSj5e+eqHyFK9oDOpWP4BaFIo7ctQ9tzn7mUWBaKIZDydt70Gz9X/Ir/RP8WWHob1LyaqaI+vkrGpxGLLJvTZ2Rb85C8GR519Yg+Jy3nzbqp+4gOZ03wZkZDpk971d40pE/n3+1M6EDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QPMT1qlP06nqubL9ZmjbZjwbBxp5J+HBVxtCSLRLm9o=; b=ixnK8ANGL4CP1+lZpF5FikSmOxP/r/3jhuICg8m3Vim4UE3bqfnTlsg/0ZbyJtXPdpQopEMDehfcIitWYuBKadd7QG9OJ+1R4jMYYtzIXribtcH6mBwP9HZ0DMuRHRg1QTSws2yRtXBjLvVEp9+RGzpBEJsOu95n2fW0RfS4vQeGyGmRYP45qJ1yE07Yw8+TztPZEfXa4q9hvhmPLVnoqFGPZKvuzkf8kxJteSXNeRpUQBdTkopJkAzMxzf+CWya8Sh2wMQ+4JeqXiW2xkIvEeQ2doBIAVKyMdQ5M+UQTGZqXzjp80IzLwdtiw9e/U5LGvuu244jx0kx1qugxXbXkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QPMT1qlP06nqubL9ZmjbZjwbBxp5J+HBVxtCSLRLm9o=; b=xrB/kPsMX/DJ7Ao8UyBknh+r/VebvfNGPPmh4KBPjhzkT50n8jh2Y9sLqokqSHfB1s2WKxcFHjG/dvvBgzTi3ikV4jOYgh2XiCgF7iIpHAEL02cPlrh2jmWV5D8tn+BQXR/l7qVDXDA5Z3hUEYG55LduHB8zutOXl7SL8LGgcDU= Received: from BL0PR01CA0035.prod.exchangelabs.com (2603:10b6:208:71::48) by PH0PR12MB5630.namprd12.prod.outlook.com (2603:10b6:510:146::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb 2024 17:59:31 +0000 Received: from BL6PEPF0001AB75.namprd02.prod.outlook.com (2603:10b6:208:71:cafe::28) by BL0PR01CA0035.outlook.office365.com (2603:10b6:208:71::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.40 via Frontend Transport; Wed, 14 Feb 2024 17:59:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB75.mail.protection.outlook.com (10.167.242.168) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:31 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:28 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 4/9] ionic: add initial framework for XDP support Date: Wed, 14 Feb 2024 09:59:04 -0800 Message-ID: <20240214175909.68802-5-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB75:EE_|PH0PR12MB5630:EE_ X-MS-Office365-Filtering-Correlation-Id: 160e7b10-04d1-4a7b-e37c-08dc2d86b1c9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BaeJZbOgWr1Zn1fEumlJDRK4P6e57YExZOuZf9Hrx0agxR/yOHaUhqqfo2MtOZjy8a566txE/h/PwgHnIB3itWbUBR3F2Vs8M3BpoEQPX4SBl5GIPjGEEuliDjvowRg1ggvGRlGoIkoPeBUhd2xC1914ZLBQvlhgoHmsJ/vWJSR+sEKgKZn8N73a7KRN2mCVUhde1DzT+KjccNM08LKdlCjKMbl148w2ojkT8HBHd2kG2TuEyeIjKhYqa/iuXJ/PQRPDJsB4m817TwHCu7WEk1n/+CBLxmgk8TTiSrrElMmHoboJpIgIrldbVgUfhwqY30LO7S/dTiO6mYbNnq2TiRJTR9qUDoCs82OqtIcwHkwJ1Cxn7Br9tpUp59sa3RvI1/zi5LDS33tUGSQpx1acTPsCgghEqUe2KuqVMSaP8wjh9KFJr57p26rwlplSFe4ze5srDYabjXbwqm/jP907bZ3xpqAf8SjdUq0Q4mTxNUDpS4I2JspBVTJlPRvWRsNiptO/d16WZkGDZ6no7OtovRCDt5t+WmOVH233BCsRMUYo7fSfzWaFEe57HX9rnsFKbDV9OQcGiLUP1F5dnd6WpHghwk4Uj8KhXNvT2JaOzgk= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(346002)(136003)(396003)(230922051799003)(82310400011)(451199024)(64100799003)(186009)(1800799012)(46966006)(36840700001)(40470700004)(41300700001)(83380400001)(16526019)(86362001)(316002)(6666004)(1076003)(26005)(110136005)(2616005)(5660300002)(478600001)(2906002)(4326008)(30864003)(36756003)(54906003)(70586007)(336012)(426003)(8676002)(8936002)(44832011)(70206006)(82740400003)(81166007)(356005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:31.1121 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 160e7b10-04d1-4a7b-e37c-08dc2d86b1c9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB75.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB5630 X-Patchwork-Delegate: kuba@kernel.org Set up the basics for running Rx packets through XDP programs. Add new queue setup and teardown steps for adding/removing an XDP program, and add the call to run the XDP on a packet. The XDP frame size needs to be the MTU plus standard ethernet header, plus head room for XDP scribblings and tail room for a struct skb_shared_info. Also, at this point, we don't support XDP frags, only a single contiguous Rx buffer. This means that our page splitting is not very useful, so when XDP is in use we need to use the full Rx buffer size and not do sharing. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_dev.h | 7 + .../ethernet/pensando/ionic/ionic_ethtool.c | 5 + .../net/ethernet/pensando/ionic/ionic_lif.c | 175 +++++++++++++++++- .../net/ethernet/pensando/ionic/ionic_lif.h | 7 + .../net/ethernet/pensando/ionic/ionic_stats.c | 9 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 80 ++++++-- 6 files changed, 269 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 2667e1cde16b..70f5725fdfe8 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -8,6 +8,7 @@ #include #include #include +#include #include "ionic_if.h" #include "ionic_regs.h" @@ -195,6 +196,11 @@ typedef void (*ionic_desc_cb)(struct ionic_queue *q, #define IONIC_PAGE_GFP_MASK (GFP_ATOMIC | __GFP_NOWARN |\ __GFP_COMP | __GFP_MEMALLOC) +#define IONIC_XDP_MAX_LINEAR_MTU (IONIC_PAGE_SIZE - \ + (VLAN_ETH_HLEN + \ + XDP_PACKET_HEADROOM + \ + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))) + struct ionic_buf_info { struct page *page; dma_addr_t dma_addr; @@ -256,6 +262,7 @@ struct ionic_queue { struct ionic_txq_sg_desc *txq_sgl; struct ionic_rxq_sg_desc *rxq_sgl; }; + struct xdp_rxq_info *xdp_rxq_info; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c index cd3c0b01402e..98df2ee11c51 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c @@ -721,6 +721,11 @@ static int ionic_set_channels(struct net_device *netdev, ionic_init_queue_params(lif, &qparam); + if ((ch->rx_count || ch->tx_count) && lif->xdp_prog) { + netdev_info(lif->netdev, "Split Tx/Rx interrupts not available when using XDP\n"); + return -EOPNOTSUPP; + } + if (ch->rx_count != ch->tx_count) { netdev_info(netdev, "The rx and tx count must be equal\n"); return -EINVAL; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d92f8734d153..d8c83ff47b66 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -46,6 +46,9 @@ static int ionic_start_queues(struct ionic_lif *lif); static void ionic_stop_queues(struct ionic_lif *lif); static void ionic_lif_queue_identify(struct ionic_lif *lif); +static int ionic_xdp_queues_config(struct ionic_lif *lif); +static void ionic_xdp_unregister_rxq_info(struct ionic_queue *q); + static void ionic_dim_work(struct work_struct *work) { struct dim *dim = container_of(work, struct dim, work); @@ -422,6 +425,7 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) qcq->sg_base_pa = 0; } + ionic_xdp_unregister_rxq_info(&qcq->q); ionic_qcq_intr_free(lif, qcq); vfree(qcq->cq.info); @@ -862,8 +866,7 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) .type = q->type, .ver = lif->qtype_info[q->type].version, .index = cpu_to_le32(q->index), - .flags = cpu_to_le16(IONIC_QINIT_F_IRQ | - IONIC_QINIT_F_SG), + .flags = cpu_to_le16(IONIC_QINIT_F_IRQ), .intr_index = cpu_to_le16(cq->bound_intr->index), .pid = cpu_to_le16(q->pid), .ring_size = ilog2(q->num_descs), @@ -875,6 +878,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) }; int err; + if (!lif->xdp_prog) + ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); + if (qcq->flags & IONIC_QCQ_F_CMB_RINGS) { ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_CMB); ctx.cmd.q_init.ring_base = cpu_to_le64(qcq->cmb_q_base_pa); @@ -1640,6 +1646,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE; + netdev->xdp_features = NETDEV_XDP_ACT_BASIC; + return 0; } @@ -1777,6 +1785,18 @@ static int ionic_start_queues_reconfig(struct ionic_lif *lif) return err; } +static bool ionic_xdp_is_valid_mtu(struct ionic_lif *lif, u32 mtu, + struct bpf_prog *xdp_prog) +{ + if (!xdp_prog) + return true; + + if (mtu <= IONIC_XDP_MAX_LINEAR_MTU) + return true; + + return false; +} + static int ionic_change_mtu(struct net_device *netdev, int new_mtu) { struct ionic_lif *lif = netdev_priv(netdev); @@ -1789,8 +1809,13 @@ static int ionic_change_mtu(struct net_device *netdev, int new_mtu) .mtu = cpu_to_le32(new_mtu), }, }; + struct bpf_prog *xdp_prog; int err; + xdp_prog = READ_ONCE(lif->xdp_prog); + if (!ionic_xdp_is_valid_mtu(lif, new_mtu, xdp_prog)) + return -EINVAL; + err = ionic_adminq_post_wait(lif, &ctx); if (err) return err; @@ -2166,6 +2191,10 @@ static int ionic_txrx_enable(struct ionic_lif *lif) int derr = 0; int i, err; + err = ionic_xdp_queues_config(lif); + if (err) + return err; + for (i = 0; i < lif->nxqs; i++) { if (!(lif->rxqcqs[i] && lif->txqcqs[i])) { dev_err(lif->ionic->dev, "%s: bad qcq %d\n", __func__, i); @@ -2211,6 +2240,8 @@ static int ionic_txrx_enable(struct ionic_lif *lif) derr = ionic_qcq_disable(lif, lif->rxqcqs[i], derr); } + ionic_xdp_queues_config(lif); + return err; } @@ -2668,11 +2699,150 @@ static void ionic_vf_attr_replay(struct ionic_lif *lif) ionic_vf_start(ionic); } +static void ionic_xdp_unregister_rxq_info(struct ionic_queue *q) +{ + struct xdp_rxq_info *xi; + + if (!q->xdp_rxq_info) + return; + + xi = q->xdp_rxq_info; + q->xdp_rxq_info = NULL; + + xdp_rxq_info_unreg(xi); + kfree(xi); +} + +static int ionic_xdp_register_rxq_info(struct ionic_queue *q, unsigned int napi_id) +{ + struct xdp_rxq_info *rxq_info; + int err; + + rxq_info = kzalloc(sizeof(*rxq_info), GFP_KERNEL); + if (!rxq_info) + return -ENOMEM; + + err = xdp_rxq_info_reg(rxq_info, q->lif->netdev, q->index, napi_id); + if (err) { + dev_err(q->dev, "Queue %d xdp_rxq_info_reg failed, err %d\n", + q->index, err); + goto err_out; + } + + err = xdp_rxq_info_reg_mem_model(rxq_info, MEM_TYPE_PAGE_ORDER0, NULL); + if (err) { + dev_err(q->dev, "Queue %d xdp_rxq_info_reg_mem_model failed, err %d\n", + q->index, err); + xdp_rxq_info_unreg(rxq_info); + goto err_out; + } + + q->xdp_rxq_info = rxq_info; + + return 0; + +err_out: + kfree(rxq_info); + return err; +} + +static int ionic_xdp_queues_config(struct ionic_lif *lif) +{ + unsigned int i; + int err; + + if (!lif->rxqcqs) + return 0; + + /* There's no need to rework memory if not going to/from NULL program. + * If there is no lif->xdp_prog, there should also be no q.xdp_rxq_info + * This way we don't need to keep an *xdp_prog in every queue struct. + */ + if (!lif->xdp_prog == !lif->rxqcqs[0]->q.xdp_rxq_info) + return 0; + + for (i = 0; i < lif->ionic->nrxqs_per_lif && lif->rxqcqs[i]; i++) { + struct ionic_queue *q = &lif->rxqcqs[i]->q; + + if (q->xdp_rxq_info) { + ionic_xdp_unregister_rxq_info(q); + continue; + } + + err = ionic_xdp_register_rxq_info(q, lif->rxqcqs[i]->napi.napi_id); + if (err) { + dev_err(lif->ionic->dev, "failed to register RX queue %d info for XDP, err %d\n", + i, err); + goto err_out; + } + } + + return 0; + +err_out: + for (i = 0; i < lif->ionic->nrxqs_per_lif && lif->rxqcqs[i]; i++) + ionic_xdp_unregister_rxq_info(&lif->rxqcqs[i]->q); + + return err; +} + +static int ionic_xdp_config(struct net_device *netdev, struct netdev_bpf *bpf) +{ + struct ionic_lif *lif = netdev_priv(netdev); + struct bpf_prog *old_prog; + u32 maxfs; + + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { +#define XDP_ERR_SPLIT "XDP not available with split Tx/Rx interrupts" + NL_SET_ERR_MSG_MOD(bpf->extack, XDP_ERR_SPLIT); + netdev_info(lif->netdev, XDP_ERR_SPLIT); + return -EOPNOTSUPP; + } + + if (!ionic_xdp_is_valid_mtu(lif, netdev->mtu, bpf->prog)) { +#define XDP_ERR_MTU "MTU is too large for XDP without frags support" + NL_SET_ERR_MSG_MOD(bpf->extack, XDP_ERR_MTU); + netdev_info(lif->netdev, XDP_ERR_MTU); + return -EINVAL; + } + + maxfs = __le32_to_cpu(lif->identity->eth.max_frame_size) - VLAN_ETH_HLEN; + if (bpf->prog) + maxfs = min_t(u32, maxfs, IONIC_XDP_MAX_LINEAR_MTU); + netdev->max_mtu = maxfs; + + if (!netif_running(netdev)) { + old_prog = xchg(&lif->xdp_prog, bpf->prog); + } else { + mutex_lock(&lif->queue_lock); + ionic_stop_queues_reconfig(lif); + old_prog = xchg(&lif->xdp_prog, bpf->prog); + ionic_start_queues_reconfig(lif); + mutex_unlock(&lif->queue_lock); + } + + if (old_prog) + bpf_prog_put(old_prog); + + return 0; +} + +static int ionic_xdp(struct net_device *netdev, struct netdev_bpf *bpf) +{ + switch (bpf->command) { + case XDP_SETUP_PROG: + return ionic_xdp_config(netdev, bpf); + default: + return -EINVAL; + } +} + static const struct net_device_ops ionic_netdev_ops = { .ndo_open = ionic_open, .ndo_stop = ionic_stop, .ndo_eth_ioctl = ionic_eth_ioctl, .ndo_start_xmit = ionic_start_xmit, + .ndo_bpf = ionic_xdp, .ndo_get_stats64 = ionic_get_stats64, .ndo_set_rx_mode = ionic_ndo_set_rx_mode, .ndo_set_features = ionic_set_features, @@ -2755,6 +2925,7 @@ static void ionic_swap_queues(struct ionic_qcq *a, struct ionic_qcq *b) swap(a->q.base, b->q.base); swap(a->q.base_pa, b->q.base_pa); swap(a->q.info, b->q.info); + swap(a->q.xdp_rxq_info, b->q.xdp_rxq_info); swap(a->q_base, b->q_base); swap(a->q_base_pa, b->q_base_pa); swap(a->q_size, b->q_size); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 61548b3eea93..61fa4ea4f04c 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -51,6 +51,9 @@ struct ionic_rx_stats { u64 alloc_err; u64 hwstamp_valid; u64 hwstamp_invalid; + u64 xdp_drop; + u64 xdp_aborted; + u64 xdp_pass; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -135,6 +138,9 @@ struct ionic_lif_sw_stats { u64 hw_rx_over_errors; u64 hw_rx_missed_errors; u64 hw_tx_aborted_errors; + u64 xdp_drop; + u64 xdp_aborted; + u64 xdp_pass; }; enum ionic_lif_state_flags { @@ -230,6 +236,7 @@ struct ionic_lif { struct ionic_phc *phc; struct dentry *dentry; + struct bpf_prog *xdp_prog; }; struct ionic_phc { diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 1f6022fb7679..2fb20173b2c6 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -27,6 +27,9 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(hw_rx_over_errors), IONIC_LIF_STAT_DESC(hw_rx_missed_errors), IONIC_LIF_STAT_DESC(hw_tx_aborted_errors), + IONIC_LIF_STAT_DESC(xdp_drop), + IONIC_LIF_STAT_DESC(xdp_aborted), + IONIC_LIF_STAT_DESC(xdp_pass), }; static const struct ionic_stat_desc ionic_port_stats_desc[] = { @@ -149,6 +152,9 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(hwstamp_invalid), IONIC_RX_STAT_DESC(dropped), IONIC_RX_STAT_DESC(vlan_stripped), + IONIC_RX_STAT_DESC(xdp_drop), + IONIC_RX_STAT_DESC(xdp_aborted), + IONIC_RX_STAT_DESC(xdp_pass), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -185,6 +191,9 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->rx_csum_error += rxstats->csum_error; stats->rx_hwstamp_valid += rxstats->hwstamp_valid; stats->rx_hwstamp_invalid += rxstats->hwstamp_invalid; + stats->xdp_drop += rxstats->xdp_drop; + stats->xdp_aborted += rxstats->xdp_aborted; + stats->xdp_pass += rxstats->xdp_pass; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 7f095717595d..8a85f846107d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -177,7 +177,7 @@ static bool ionic_rx_buf_recycle(struct ionic_queue *q, if (page_to_nid(buf_info->page) != numa_mem_id()) return false; - size = ALIGN(used, IONIC_PAGE_SPLIT_SZ); + size = ALIGN(used, q->xdp_rxq_info ? IONIC_PAGE_SIZE : IONIC_PAGE_SPLIT_SZ); buf_info->page_offset += size; if (buf_info->page_offset >= IONIC_PAGE_SIZE) return false; @@ -189,7 +189,8 @@ static bool ionic_rx_buf_recycle(struct ionic_queue *q, static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp) + struct ionic_rxq_comp *comp, + bool synced) { struct net_device *netdev = q->lif->netdev; struct ionic_buf_info *buf_info; @@ -225,8 +226,9 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; - dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, frag_len, DMA_FROM_DEVICE); + if (!synced) + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, buf_info->page, buf_info->page_offset, frag_len, @@ -248,7 +250,8 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp) + struct ionic_rxq_comp *comp, + bool synced) { struct net_device *netdev = q->lif->netdev; struct ionic_buf_info *buf_info; @@ -275,8 +278,9 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return NULL; } - dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, len, DMA_FROM_DEVICE); + if (!synced) + dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), + 0, len, DMA_FROM_DEVICE); skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); dma_sync_single_range_for_device(dev, ionic_rx_buf_pa(buf_info), 0, len, DMA_FROM_DEVICE); @@ -287,6 +291,50 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return skb; } +static bool ionic_run_xdp(struct ionic_rx_stats *stats, + struct net_device *netdev, + struct bpf_prog *xdp_prog, + struct ionic_queue *rxq, + struct ionic_buf_info *buf_info, + int len) +{ + u32 xdp_action = XDP_ABORTED; + struct xdp_buff xdp_buf; + + xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); + xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), + 0, len, false); + + dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), + 0, len, + DMA_FROM_DEVICE); + + prefetchw(&xdp_buf.data_hard_start); + + xdp_action = bpf_prog_run_xdp(xdp_prog, &xdp_buf); + + switch (xdp_action) { + case XDP_PASS: + stats->xdp_pass++; + return false; /* false = we didn't consume the packet */ + + case XDP_DROP: + ionic_rx_page_free(rxq, buf_info); + stats->xdp_drop++; + break; + + case XDP_TX: + case XDP_REDIRECT: + case XDP_ABORTED: + default: + trace_xdp_exception(netdev, xdp_prog, xdp_action); + ionic_rx_page_free(rxq, buf_info); + stats->xdp_aborted++; + } + + return true; +} + static void ionic_rx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info, struct ionic_cq_info *cq_info, @@ -296,7 +344,9 @@ static void ionic_rx_clean(struct ionic_queue *q, struct ionic_qcq *qcq = q_to_qcq(q); struct ionic_rx_stats *stats; struct ionic_rxq_comp *comp; + struct bpf_prog *xdp_prog; struct sk_buff *skb; + u16 len; comp = cq_info->cq_desc + qcq->cq.desc_size - sizeof(*comp); @@ -307,13 +357,19 @@ static void ionic_rx_clean(struct ionic_queue *q, return; } + len = le16_to_cpu(comp->len); stats->pkts++; - stats->bytes += le16_to_cpu(comp->len); + stats->bytes += len; + + xdp_prog = READ_ONCE(q->lif->xdp_prog); + if (xdp_prog && + ionic_run_xdp(stats, netdev, xdp_prog, q, desc_info->bufs, len)) + return; - if (le16_to_cpu(comp->len) <= q->lif->rx_copybreak) - skb = ionic_rx_copybreak(q, desc_info, comp); + if (len <= q->lif->rx_copybreak) + skb = ionic_rx_copybreak(q, desc_info, comp, !!xdp_prog); else - skb = ionic_rx_frags(q, desc_info, comp); + skb = ionic_rx_frags(q, desc_info, comp, !!xdp_prog); if (unlikely(!skb)) { stats->dropped++; @@ -380,7 +436,7 @@ static void ionic_rx_clean(struct ionic_queue *q, } } - if (le16_to_cpu(comp->len) <= q->lif->rx_copybreak) + if (len <= q->lif->rx_copybreak) napi_gro_receive(&qcq->napi, skb); else napi_gro_frags(&qcq->napi); From patchwork Wed Feb 14 17:59:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556884 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBD46129A9F for ; Wed, 14 Feb 2024 17:59:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933576; cv=fail; b=o93GInE2U+Mm0ngE8kwjRPwkE3+1DCwSL9z5oVN7mMuF7do76MfqKTT8ayPvwwz56c9P1iutA8q6GRgraJ3MUYPgTylfq98f1Kwlta6TRCTtMCUeeV29zx+0sJNamiWc5TEHrU1j+J8/vSNclcQK8gfsRYMr40XGbNKcWzfM2og= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933576; c=relaxed/simple; bh=WZ8WewXVvK6s+LDiNYUv50k4FkyqyltBwsV6TCVUeVQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=P3SqdRTA7pyJBQPbLszsbccj3niRT9WgQ5lNzbAQd06LjZFHH4bB4stbCKm6Vs+Buhsdeosr1qkVvhI5WtKYju926nTBZfuNCeuTTWdeaCzg1VvbLUdGZvZ0Db0vencjQEneD1oe49qWKaHl7/uJ5MqBR/3w0KqbyLuAPGw6lQs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=fD24xL1x; arc=fail smtp.client-ip=40.107.92.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="fD24xL1x" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z5YDhwyTSlmWzHjyW9cA8s7ZYO6pA1E+tQk6Y66S2bXfnDycnpsfYDYBEqE90N0HnqzDw0cLmOW+Imz6tIThpiH518aebg+Gg21pLkHI7D8V3zyKeI6WjTXtrwu0oMycKVSZRh6pDW/H0IHmGGCn0vidN1uaNI1LWbOpKqQUuL5nEtI5+zw4RiPjNwX7GgnrIwOfO+s3FAiOEhaTXvH/MfTtUAcrGKlf73eq7svaYBH46wm6U5zSeAfBQVpZCaTsPuUPZc13VCrd74z1vLs7b/xm8bJd9oZrPr+ldkbwenNlkVYPJh066QthUWtzE19ngs5GasEqyKVDGid9IB+2nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GqL7s+fJxchqk4+g2GyyNcCaOThEMhgA3cJmbs6LGds=; b=bBvDg2N6GGbQfuuqG4nbAfX4M7CzCxTDA6PE2oLWU0Kf2TQELX4EYjyZ/sSAKCB38u19y8XtMyL+CYOwLOCt6nPcgOh98ZVbAdpw6132/nnYj6IlAQ8xMuuyLfskEq7nWSBDbttb8654tnzyaYELIOWce7oPuFC/Je/2524//QrwLxM0YfHiZbOZmZJiLA/zhqHTwhDFqa/13aKUizZE1WIBgU4335vSshYgo9JJiKKsvCk4dcTul24oq6Nd09vBYC9gM14ddEp31FEq5ZYR/gKvAPH9TqXVpub0OKxZOLNGFTuBw5wdyWWbgLXmqZCHRUbhJwruNaBzxk9iNLnVjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GqL7s+fJxchqk4+g2GyyNcCaOThEMhgA3cJmbs6LGds=; b=fD24xL1xjojtksDvqVb+uiruzHFBX5/kG4zyaeC7zD4AMtGWi78JMhU0CB9s/LOoQAbCOxdiPITdSLASTNyebAi64Llok1CwxeZYnFCl97W60jVfjvkK3Eu7CyDfQj2KsDsh/aM4/bSEogUUWf/Ozo0IlDZXX+yh0AIv7zdyAF8= Received: from BL0PR01CA0008.prod.exchangelabs.com (2603:10b6:208:71::21) by IA0PR12MB7652.namprd12.prod.outlook.com (2603:10b6:208:434::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.28; Wed, 14 Feb 2024 17:59:31 +0000 Received: from BL6PEPF0001AB75.namprd02.prod.outlook.com (2603:10b6:208:71:cafe::d7) by BL0PR01CA0008.outlook.office365.com (2603:10b6:208:71::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.40 via Frontend Transport; Wed, 14 Feb 2024 17:59:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB75.mail.protection.outlook.com (10.167.242.168) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:31 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:30 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 5/9] ionic: Add XDP packet headroom Date: Wed, 14 Feb 2024 09:59:05 -0800 Message-ID: <20240214175909.68802-6-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB75:EE_|IA0PR12MB7652:EE_ X-MS-Office365-Filtering-Correlation-Id: 357332af-999a-445a-b7e7-08dc2d86b21f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AY4HPab5He2CSpF6KSmxM8tWnGWGd9u8VXOeBE/YhLH5AD/j9vE5NvnlOAcNNbnZXhwTzY4cSsmi8A8l2jAjf2i4kez2GOEGiPd5Q9RynMqrjyzCRJfGSo3cVqzBHqYBOV1dpqXt3mixSqSn8dpY4BlZiIj7Bc0XRIDI4+xIHlxuwrWyUiqZS0imnWZBjcsKZHwtHTjTyhVlOu0OIWaimmGhELljc9z3PTlVrsMMKhD2QH6VPYWoBQQukj+JQzfR9yp5Ssx6UKPdY9f4l4ZGMa8FaGLq/P1RP9qJb/XEmVTbrh/M7+H4jy+eU/SZDHieV6UTZmmGxS3FYiaKgfx9rr45nt40VySOt0IsjR85SqgJkeKP7zB0X5k9AjLpoLR2y5/kwpHCsh5rKlk3WucJvbkl8bXPqbzHSfpRv6Zy28K+0r3g6ileItqKKHuzjqV1g/VQoo8a/c9GEY8SNGuY+IsV8a7iI82C2WyTyfRWFjoyjxfoqFUEo43/R45dDUCq8ULutf1nbErkXYk7zViKoztN5jbfeghtfDYPthRcoV0cMrcA53BTgw9Oyo/kKOV0iSZCUum99t9Ti2mvcb0TAFG9I0SUp3ci1AwjLeHU4fE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(396003)(346002)(376002)(136003)(39860400002)(230922051799003)(64100799003)(186009)(1800799012)(82310400011)(451199024)(46966006)(40470700004)(36840700001)(478600001)(41300700001)(4326008)(44832011)(2906002)(5660300002)(8676002)(8936002)(70206006)(110136005)(6666004)(54906003)(316002)(83380400001)(2616005)(70586007)(82740400003)(336012)(426003)(356005)(86362001)(81166007)(26005)(16526019)(1076003)(36756003);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:31.6746 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 357332af-999a-445a-b7e7-08dc2d86b21f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB75.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7652 X-Patchwork-Delegate: kuba@kernel.org If an xdp program is loaded, add headroom at the beginning of the frame to allow for editing and insertions that an XDP program might need room for, and tailroom used later for XDP frame tracking. These are only needed in the first Rx buffer in a packet, not for any trailing frags. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_txrx.c | 68 ++++++++++++------- 1 file changed, 45 insertions(+), 23 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 8a85f846107d..88d4306f013b 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -189,7 +189,9 @@ static bool ionic_rx_buf_recycle(struct ionic_queue *q, static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp, + unsigned int headroom, + unsigned int len, + unsigned int num_sg_elems, bool synced) { struct net_device *netdev = q->lif->netdev; @@ -199,12 +201,10 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, struct sk_buff *skb; unsigned int i; u16 frag_len; - u16 len; stats = q_to_rx_stats(q); buf_info = &desc_info->bufs[0]; - len = le16_to_cpu(comp->len); prefetchw(buf_info->page); @@ -216,23 +216,26 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, return NULL; } - i = comp->num_sg_elems + 1; + i = num_sg_elems + 1; do { if (unlikely(!buf_info->page)) { dev_kfree_skb(skb); return NULL; } - frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); + if (headroom) + frag_len = min_t(u16, len, IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN); + else + frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); len -= frag_len; if (!synced) dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, frag_len, DMA_FROM_DEVICE); + headroom, frag_len, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, - buf_info->page, buf_info->page_offset, frag_len, - IONIC_PAGE_SIZE); + buf_info->page, buf_info->page_offset + headroom, + frag_len, IONIC_PAGE_SIZE); if (!ionic_rx_buf_recycle(q, buf_info, frag_len)) { dma_unmap_page(dev, buf_info->dma_addr, @@ -240,6 +243,10 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, buf_info->page = NULL; } + /* only needed on the first buffer */ + if (headroom) + headroom = 0; + buf_info++; i--; @@ -250,7 +257,8 @@ static struct sk_buff *ionic_rx_frags(struct ionic_queue *q, static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, struct ionic_desc_info *desc_info, - struct ionic_rxq_comp *comp, + unsigned int headroom, + unsigned int len, bool synced) { struct net_device *netdev = q->lif->netdev; @@ -258,12 +266,10 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, struct ionic_rx_stats *stats; struct device *dev = q->dev; struct sk_buff *skb; - u16 len; stats = q_to_rx_stats(q); buf_info = &desc_info->bufs[0]; - len = le16_to_cpu(comp->len); skb = napi_alloc_skb(&q_to_qcq(q)->napi, len); if (unlikely(!skb)) { @@ -280,10 +286,10 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, if (!synced) dma_sync_single_range_for_cpu(dev, ionic_rx_buf_pa(buf_info), - 0, len, DMA_FROM_DEVICE); - skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info), len); + headroom, len, DMA_FROM_DEVICE); + skb_copy_to_linear_data(skb, ionic_rx_buf_va(buf_info) + headroom, len); dma_sync_single_range_for_device(dev, ionic_rx_buf_pa(buf_info), - 0, len, DMA_FROM_DEVICE); + headroom, len, DMA_FROM_DEVICE); skb_put(skb, len); skb->protocol = eth_type_trans(skb, q->lif->netdev); @@ -303,10 +309,10 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), - 0, len, false); + XDP_PACKET_HEADROOM, len, false); dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), - 0, len, + XDP_PACKET_HEADROOM, len, DMA_FROM_DEVICE); prefetchw(&xdp_buf.data_hard_start); @@ -345,6 +351,7 @@ static void ionic_rx_clean(struct ionic_queue *q, struct ionic_rx_stats *stats; struct ionic_rxq_comp *comp; struct bpf_prog *xdp_prog; + unsigned int headroom; struct sk_buff *skb; u16 len; @@ -366,10 +373,12 @@ static void ionic_rx_clean(struct ionic_queue *q, ionic_run_xdp(stats, netdev, xdp_prog, q, desc_info->bufs, len)) return; + headroom = q->xdp_rxq_info ? XDP_PACKET_HEADROOM : 0; if (len <= q->lif->rx_copybreak) - skb = ionic_rx_copybreak(q, desc_info, comp, !!xdp_prog); + skb = ionic_rx_copybreak(q, desc_info, headroom, len, !!xdp_prog); else - skb = ionic_rx_frags(q, desc_info, comp, !!xdp_prog); + skb = ionic_rx_frags(q, desc_info, headroom, len, + comp->num_sg_elems, !!xdp_prog); if (unlikely(!skb)) { stats->dropped++; @@ -493,8 +502,9 @@ void ionic_rx_fill(struct ionic_queue *q) unsigned int frag_len; unsigned int nfrags; unsigned int n_fill; - unsigned int i, j; unsigned int len; + unsigned int i; + unsigned int j; n_fill = ionic_q_space_avail(q); @@ -503,9 +513,12 @@ void ionic_rx_fill(struct ionic_queue *q) if (n_fill < fill_threshold) return; - len = netdev->mtu + ETH_HLEN + VLAN_HLEN; + len = netdev->mtu + VLAN_ETH_HLEN; for (i = n_fill; i; i--) { + unsigned int headroom; + unsigned int buf_len; + nfrags = 0; remain_len = len; desc_info = &q->info[q->head_idx]; @@ -520,9 +533,18 @@ void ionic_rx_fill(struct ionic_queue *q) } } - /* fill main descriptor - buf[0] */ - desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info)); - frag_len = min_t(u16, len, ionic_rx_buf_size(buf_info)); + /* fill main descriptor - buf[0] + * XDP uses space in the first buffer, so account for + * head room, tail room, and ip header in the first frag size. + */ + headroom = q->xdp_rxq_info ? XDP_PACKET_HEADROOM : 0; + if (q->xdp_rxq_info) + buf_len = IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN; + else + buf_len = ionic_rx_buf_size(buf_info); + frag_len = min_t(u16, len, buf_len); + + desc->addr = cpu_to_le64(ionic_rx_buf_pa(buf_info) + headroom); desc->len = cpu_to_le16(frag_len); remain_len -= frag_len; buf_info++; From patchwork Wed Feb 14 17:59:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556885 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2088.outbound.protection.outlook.com [40.107.223.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AA3312A17A for ; Wed, 14 Feb 2024 17:59:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.88 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933578; cv=fail; b=cxOol3yDPDxz8tJZpNmSyKlfTTYzObVWKFbrcrc6Aj6qjZoJnHn1GKvwwMPIMtDrYbURsoYoHyv6haKZmVJpmyt3LFAQF8RjOSKQNLsjUxlli+CngwL+uPAkLxcbi79DrQutitc75HezUaxYqT7namK0rieaN3xmby5QS+SOGZ0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933578; c=relaxed/simple; bh=fPBbHEswNKPax68zeryP4NKy3kXuiB8HMWtvU791v0U=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S1IEQvCDeHKp3zgnMV9Peh1B7U9meGMV9bKgGB2wQKyIt+Z6KpX57cVfvPwM+l/AR5bOLAkQSfybUuDidiMwS9IAEQ2XfkAy70f5kY+UkWo+ySA4AIM8cHPmqzuIAUoHC+s3CJqytU26ph3fsNQxyMWf9wvcchzTLv5tw1Xm8to= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=S58rcKrN; arc=fail smtp.client-ip=40.107.223.88 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="S58rcKrN" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fhaIlnUJrTVI/2FC2+35xywGP3I060txHFG6sE4SA4nWt2A6+cVOm89tp7YYpOOuvR9SiD0UcggZSwkmbvD72jkPf/+uBBBGW1W+VoSiP4iPJwYFMlf6riBKfmnlkZIb+ah9WExxTtA1c9xIM9aG+6CDffV4e1wK39F8bryWYZNpHX8p8rFASypXCJbhK1K2CYN/7H67XxJVXX6uuYcnklTwapJiXGAh9S08FidDbJA1xlcg9QRC5idfqvzF4CPFPPHmUEggot77MCycO8tUocgsx83/5o4E8l5upi4saNUBXil9WU3SygEfVkxdFVAOUWJgMkJT+vSfPBVs+5tTjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JaRMtRguAKNPvZ4lKokSgzS6teu9EB+srTy3cexDOUY=; b=j0+L/f1/03pXa91IwmCoOhhFm+R87Qoe3spbSVWU0eXRyiPaOf5UKvrtahF42lmVc1SBdNf4QQfxlV40WVMQiEpAzPyd+GU489RgQ/qQtZ48Hex2YQ+xL9JTK93kVdFBX2EGbGs7wcnQGlztfmvyqaijn5JKCcg4VbiURFPcy2M+wiuHxmZWZSyAnNt/cfUEblahdQhFPufmLIc0JTGpNvKprgj0/uoGSuehm59ADOJc5i87GiDAUxra/hM9UKbMJJgoqutL5kArgBFv0wOKqREycdeZ04soLT7GQPsawSnSyGLVUUyskQ9oEd40BrsIas6Mkq4Hlz5Bh5uA8z2MAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JaRMtRguAKNPvZ4lKokSgzS6teu9EB+srTy3cexDOUY=; b=S58rcKrNMxavIhn/kSZrMNs0MW+Er0IWYfyIrYwXfFhOqACj7ZVz9aujjie0PVfq25Ozs04h4R/6SocrQyMWMZ/NEFhxf9zv/y1Lh9OGOnpN7tZf7JRYBf/Wdm8wsIJoDwu+EHTXsSKvF3Zfw4fPzeXbbtG/aLG5+j4Mgvp3ucA= Received: from BL1P221CA0017.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::35) by CYYPR12MB8963.namprd12.prod.outlook.com (2603:10b6:930:c3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb 2024 17:59:33 +0000 Received: from BL6PEPF0001AB71.namprd02.prod.outlook.com (2603:10b6:208:2c5:cafe::b1) by BL1P221CA0017.outlook.office365.com (2603:10b6:208:2c5::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend Transport; Wed, 14 Feb 2024 17:59:33 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB71.mail.protection.outlook.com (10.167.242.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:33 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:31 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 6/9] ionic: Add XDP_TX support Date: Wed, 14 Feb 2024 09:59:06 -0800 Message-ID: <20240214175909.68802-7-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB71:EE_|CYYPR12MB8963:EE_ X-MS-Office365-Filtering-Correlation-Id: dfc45a53-2335-4f9c-0767-08dc2d86b2ff X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7ztyOWk/FcC/ui3FL0QUEyk8rzGCvofCBrzih/2hwX5WXGgEOn69KA3gMDqPEM+2TarlfY3we5xBXdcpSAgZSfKVyu/g8jqhVnh9B62BakfXjJGLi9DBe4mAk3qkqYWS0Td1aLGDxI1SKRI2FYb9tpI7XSCo3vd5mUmGb4RclSXSzbBvXk4//2Hi/o5Vl3W1ODy2ZkgAE37mHbDHfhlS0D5Rq624Lurtx286oBf3EqgLqeKmNClLF6bMo2CsO1T9ZpryAcltRbUkSdIjxVmJiThTLaRPNMLmxW0jTVJzykHuvxLsHHUtWKm3U9nnpu63MZuOuH/OjNNny2mYy1rzS/08hX3Uykosv5eZgq2YFSIcirU+MLYYN7kihqVcbGwUEQ7dW/PVQ5rssqYNDVXSTUNpXN4JtR1DYBLCBCAcZXSrME2VB2Mbf/+95q5K+kfrpfw2FQjhpXX5UMdrVuuYoyONsf+JVsG72wtBYoFJbr6qXZNvpfHwoAS/9vrfvcaNldf/3kT3gI0xX362+oBbsNxRq8PqfkTcTw06498H+FbfdRm7XnkAwupie8JOtLMz4/ksJ4gglHclZu1JwhBcN+k0TZcvQ/hxSWQdJIzXOpc= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(376002)(346002)(396003)(136003)(230922051799003)(1800799012)(186009)(82310400011)(451199024)(64100799003)(46966006)(40470700004)(36840700001)(8936002)(4326008)(8676002)(5660300002)(2906002)(44832011)(426003)(336012)(26005)(36756003)(81166007)(2616005)(356005)(86362001)(82740400003)(16526019)(1076003)(70206006)(70586007)(110136005)(54906003)(316002)(83380400001)(6666004)(478600001)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:33.1224 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dfc45a53-2335-4f9c-0767-08dc2d86b2ff X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB71.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8963 X-Patchwork-Delegate: kuba@kernel.org The XDP_TX packets get fed back into the Rx queue's partnered Tx queue as an xdp_frame. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_dev.h | 3 + .../net/ethernet/pensando/ionic/ionic_lif.c | 4 + .../net/ethernet/pensando/ionic/ionic_lif.h | 4 + .../net/ethernet/pensando/ionic/ionic_stats.c | 6 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 136 +++++++++++++++++- 5 files changed, 150 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 70f5725fdfe8..76425bb546ba 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -228,6 +228,8 @@ struct ionic_desc_info { struct ionic_buf_info bufs[MAX_SKB_FRAGS + 1]; ionic_desc_cb cb; void *cb_arg; + struct xdp_frame *xdpf; + enum xdp_action act; }; #define IONIC_QUEUE_NAME_MAX_SZ 16 @@ -263,6 +265,7 @@ struct ionic_queue { struct ionic_rxq_sg_desc *rxq_sgl; }; struct xdp_rxq_info *xdp_rxq_info; + struct ionic_queue *partner; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d8c83ff47b66..72bbab1576ab 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -878,6 +878,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) }; int err; + q->partner = &lif->txqcqs[q->index]->q; + q->partner->partner = q; + if (!lif->xdp_prog) ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); @@ -2926,6 +2929,7 @@ static void ionic_swap_queues(struct ionic_qcq *a, struct ionic_qcq *b) swap(a->q.base_pa, b->q.base_pa); swap(a->q.info, b->q.info); swap(a->q.xdp_rxq_info, b->q.xdp_rxq_info); + swap(a->q.partner, b->q.partner); swap(a->q_base, b->q_base); swap(a->q_base_pa, b->q_base_pa); swap(a->q_size, b->q_size); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 61fa4ea4f04c..092ff08fc7e0 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -37,6 +37,7 @@ struct ionic_tx_stats { u64 dma_map_err; u64 hwstamp_valid; u64 hwstamp_invalid; + u64 xdp_frames; }; struct ionic_rx_stats { @@ -54,6 +55,7 @@ struct ionic_rx_stats { u64 xdp_drop; u64 xdp_aborted; u64 xdp_pass; + u64 xdp_tx; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -141,6 +143,8 @@ struct ionic_lif_sw_stats { u64 xdp_drop; u64 xdp_aborted; u64 xdp_pass; + u64 xdp_tx; + u64 xdp_frames; }; enum ionic_lif_state_flags { diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 2fb20173b2c6..5d48226e66cd 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -30,6 +30,8 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(xdp_drop), IONIC_LIF_STAT_DESC(xdp_aborted), IONIC_LIF_STAT_DESC(xdp_pass), + IONIC_LIF_STAT_DESC(xdp_tx), + IONIC_LIF_STAT_DESC(xdp_frames), }; static const struct ionic_stat_desc ionic_port_stats_desc[] = { @@ -138,6 +140,7 @@ static const struct ionic_stat_desc ionic_tx_stats_desc[] = { IONIC_TX_STAT_DESC(csum_none), IONIC_TX_STAT_DESC(csum), IONIC_TX_STAT_DESC(vlan_inserted), + IONIC_TX_STAT_DESC(xdp_frames), }; static const struct ionic_stat_desc ionic_rx_stats_desc[] = { @@ -155,6 +158,7 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(xdp_drop), IONIC_RX_STAT_DESC(xdp_aborted), IONIC_RX_STAT_DESC(xdp_pass), + IONIC_RX_STAT_DESC(xdp_tx), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -177,6 +181,7 @@ static void ionic_add_lif_txq_stats(struct ionic_lif *lif, int q_num, stats->tx_csum += txstats->csum; stats->tx_hwstamp_valid += txstats->hwstamp_valid; stats->tx_hwstamp_invalid += txstats->hwstamp_invalid; + stats->xdp_frames += txstats->xdp_frames; } static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, @@ -194,6 +199,7 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->xdp_drop += rxstats->xdp_drop; stats->xdp_aborted += rxstats->xdp_aborted; stats->xdp_pass += rxstats->xdp_pass; + stats->xdp_tx += rxstats->xdp_tx; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 88d4306f013b..3f1dee47bfa7 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -10,6 +10,16 @@ #include "ionic_lif.h" #include "ionic_txrx.h" +static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs); + +static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, + void *data, size_t len); + +static void ionic_tx_clean(struct ionic_queue *q, + struct ionic_desc_info *desc_info, + struct ionic_cq_info *cq_info, + void *cb_arg); + static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell, ionic_desc_cb cb_func, void *cb_arg) { @@ -297,6 +307,75 @@ static struct sk_buff *ionic_rx_copybreak(struct ionic_queue *q, return skb; } +static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, + struct ionic_desc_info *desc_info) +{ + unsigned int nbufs = desc_info->nbufs; + struct ionic_buf_info *buf_info; + struct device *dev = q->dev; + + if (!nbufs) + return; + + buf_info = desc_info->bufs; + dma_unmap_single(dev, buf_info->dma_addr, + buf_info->len, DMA_TO_DEVICE); + __free_pages(buf_info->page, 0); + buf_info->page = NULL; + + desc_info->nbufs = 0; + desc_info->xdpf = NULL; + desc_info->act = 0; +} + +static int ionic_xdp_post_frame(struct net_device *netdev, + struct ionic_queue *q, struct xdp_frame *frame, + enum xdp_action act, struct page *page, int off, + bool ring_doorbell) +{ + struct ionic_desc_info *desc_info; + struct ionic_buf_info *buf_info; + struct ionic_tx_stats *stats; + struct ionic_txq_desc *desc; + size_t len = frame->len; + dma_addr_t dma_addr; + u64 cmd; + + desc_info = &q->info[q->head_idx]; + desc = desc_info->txq_desc; + buf_info = desc_info->bufs; + stats = q_to_tx_stats(q); + + dma_addr = ionic_tx_map_single(q, frame->data, len); + if (dma_mapping_error(q->dev, dma_addr)) { + stats->dma_map_err++; + return -EIO; + } + buf_info->dma_addr = dma_addr; + buf_info->len = len; + buf_info->page = page; + buf_info->page_offset = off; + + desc_info->nbufs = 1; + desc_info->xdpf = frame; + desc_info->act = act; + + cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE, + 0, 0, buf_info->dma_addr); + desc->cmd = cpu_to_le64(cmd); + desc->len = cpu_to_le16(len); + desc->csum_start = 0; + desc->csum_offset = 0; + + stats->xdp_frames++; + stats->pkts++; + stats->bytes += len; + + ionic_txq_post(q, ring_doorbell, ionic_tx_clean, NULL); + + return 0; +} + static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct net_device *netdev, struct bpf_prog *xdp_prog, @@ -306,6 +385,10 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, { u32 xdp_action = XDP_ABORTED; struct xdp_buff xdp_buf; + struct ionic_queue *txq; + struct netdev_queue *nq; + struct xdp_frame *xdpf; + int err = 0; xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), @@ -330,14 +413,51 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, break; case XDP_TX: + xdpf = xdp_convert_buff_to_frame(&xdp_buf); + if (!xdpf) + goto out_xdp_abort; + + txq = rxq->partner; + nq = netdev_get_tx_queue(netdev, txq->index); + __netif_tx_lock(nq, smp_processor_id()); + txq_trans_cond_update(nq); + + if (netif_tx_queue_stopped(nq) || + unlikely(ionic_maybe_stop_tx(txq, 1))) { + __netif_tx_unlock(nq); + goto out_xdp_abort; + } + + dma_unmap_page(rxq->dev, buf_info->dma_addr, + IONIC_PAGE_SIZE, DMA_FROM_DEVICE); + + err = ionic_xdp_post_frame(netdev, txq, xdpf, XDP_TX, + buf_info->page, + buf_info->page_offset, + true); + __netif_tx_unlock(nq); + if (err) { + netdev_dbg(netdev, "tx ionic_xdp_post_frame err %d\n", err); + goto out_xdp_abort; + } + stats->xdp_tx++; + + /* the Tx completion will free the buffers */ + break; + case XDP_REDIRECT: case XDP_ABORTED: default: - trace_xdp_exception(netdev, xdp_prog, xdp_action); - ionic_rx_page_free(rxq, buf_info); - stats->xdp_aborted++; + goto out_xdp_abort; } + return true; + +out_xdp_abort: + trace_xdp_exception(netdev, xdp_prog, xdp_action); + ionic_rx_page_free(rxq, buf_info); + stats->xdp_aborted++; + return true; } @@ -884,6 +1004,16 @@ static void ionic_tx_clean(struct ionic_queue *q, struct sk_buff *skb = cb_arg; u16 qi; + if (desc_info->xdpf) { + ionic_xdp_tx_desc_clean(q->partner, desc_info); + stats->clean++; + + if (unlikely(__netif_subqueue_stopped(q->lif->netdev, q->index))) + netif_wake_subqueue(q->lif->netdev, q->index); + + return; + } + ionic_tx_desc_unmap_bufs(q, desc_info); if (!skb) From patchwork Wed Feb 14 17:59:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556887 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2080.outbound.protection.outlook.com [40.107.92.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B723A12AADE for ; Wed, 14 Feb 2024 17:59:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933579; cv=fail; b=G3usB+PEW0FepsUmQuUOPbPWQWLtpjPd62VdgrfoRbtFXGFUI1jmdYvCP9/Zz3I4tRpAYmn0DjpUu5dvA/kfrupWeD893cOCT7z7lIldoauCHgLDz7whcofH5eRsYHfi1myAqQwQT8vyvp5RJYQlK8jP3LNExR//+7yUf3PhHWQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933579; c=relaxed/simple; bh=m2/3V8BKHP1OYrVrB5Wh1JKIuo8pcPuDb6hXF0XLumA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j71xAxWIk10Cq/ua5KN+DJ8IItKn2TiPbtV849DZhkN08THb6+JG+i+4Ghro/N5ocxY1FD1zkI0dwpsp8P/6Q55bTySc5IrQB0iBeOgQNUwI+PH+p1nIRo950UY3O8V0DBP8H00xH7ho/S1WFM20hsXCj5nyhmHAiKgAVx98xBo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=lm31vKT5; arc=fail smtp.client-ip=40.107.92.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="lm31vKT5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hia1CeO0RhyyHH7yM4ekWtflSMt8p7vfjy1FzAo2fkC4XBnKpSq3eAb1cMOQRissWDtjsW32dq1TO5iOYPT0MCpcfNxP653b4qo1MCSZG1ig3V6feodaPA68r4a+IH4q8bvCHqxfAYM2sQCoYmkHYctPaGArom8AfsKulH+Ol45rr2Lb4me0dYr4uMKRdqOmY8zv0wUYzSboejzk3haa45Qpo2I/8vXegxCX73k/cgbadegSc8rryCaODS6vnLH6lKtKwZWoLwTwZ2aSHmuhDMs+RT3xfhpi5I8pjvGFTWWTR1a1j7xbSAWYsCqLf4TQ24dYo3+tpjK3ihttnlMbJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VlEbmXaKfk3epSdpopQ6vA/Ij+bAECDUp++3pYKTuLo=; b=SKsJQrBN2BqswZtobbeyXVnLpe5Z/4MMAqvEg0Dhb2bitYQWC35Vh/6p3slhPd5yWU6/j4nIKhPrV1gxKMnGFhLT2vngce8MtcGh8iXrjwaMbP7Bgx74QTheT0CWd4PstuY0wzycicX8AWoVsaFtRRS8eJhS6M/62PMZA0URk5zWnG5bBqyiesLgXe1f//49xZsXyV/V6jT6ndxEpG84DLH4uKV3j+CRhZeCUjtKJfbQ3G/jFKOgTtwKU+2NREmVtB9+ytmFDwnnZjoTzMRGxEeYZ3eDPxXVfQifvOALS27kwEGoNHi+HbX2SoYoNCL4cdicns0KmKBwbmZhkzuAEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VlEbmXaKfk3epSdpopQ6vA/Ij+bAECDUp++3pYKTuLo=; b=lm31vKT5Os1pyy5eQFcSxcS1WPKdIJNKQ5YSHK9yr22jJla/g1xovlO+GnEqfLc+g0TPPlgkQ9q7sAqDFF4Z4TFeMGkZvFxYR+kMLHQTkRtAJxjJhU0uk0SHB2Ob6unj6G2zAIOzLXEHecZ5ppCChVppXtoD/uryfYSr35azuug= Received: from BL1P221CA0012.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::6) by LV3PR12MB9437.namprd12.prod.outlook.com (2603:10b6:408:21d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26; Wed, 14 Feb 2024 17:59:34 +0000 Received: from BL6PEPF0001AB71.namprd02.prod.outlook.com (2603:10b6:208:2c5:cafe::8a) by BL1P221CA0012.outlook.office365.com (2603:10b6:208:2c5::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.41 via Frontend Transport; Wed, 14 Feb 2024 17:59:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB71.mail.protection.outlook.com (10.167.242.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:34 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:32 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 7/9] ionic: Add XDP_REDIRECT support Date: Wed, 14 Feb 2024 09:59:07 -0800 Message-ID: <20240214175909.68802-8-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB71:EE_|LV3PR12MB9437:EE_ X-MS-Office365-Filtering-Correlation-Id: da89f067-6a59-4f1e-fc10-08dc2d86b3f6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fPLwrhutEmNWgdR7MXza40QfZ0UJu98ial4FgDvVo9HSrWRWglA4YHanaomn/5zC/A2iZ6qYd2htCkKz7Ibu3ma0Gco9Ry/TrYvPEASm5EAApZo9f5PXuXXd+Wj717YvngqFDSYVJ64v+72x/b+eNdYVtyV14i81GEmAmrHUEXWsTM43cckvP939pffI/8bHozZsjppQt7NhksQBzz288tRIqVItYpD4NJmOp9DpINlVboiAiwr3cIi/Cd1xFe/IPsYcGsnGE8mlUI52xrM36G5nZZwHzgFYKew+mFTAZpjvRe4fItc3N2Rh94uoaZwopEkPaI0xNb0BvlfpyLp9AeUXc5UCR1hnLXfTjvrdywu3bbg9VOH99/sB2j7WnYh2SdgzhVFif5PUpCLJj3R89O0ifair/Yw9k7KrqFRH6ofJcpqzqYkzbOfkU86PvaU7TQ/r4wObgbVPkaHBf1tibqSztDZ5Okx8R6FDbgR0Y0cbfryUMGqsaa1j9706S6+o+2mYLpOs/S87loZSBtqkjiZ+FjmgBV8C3+5mvXFS5mgvLKKdzdtPnubOHjL4Ur4pk6+SWvWOanC3RLyeDh911eQci/ujwz4tvIpYpnYi9mw= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(376002)(39860400002)(396003)(136003)(346002)(230922051799003)(186009)(64100799003)(1800799012)(451199024)(82310400011)(36840700001)(40470700004)(46966006)(2906002)(8676002)(8936002)(44832011)(4326008)(5660300002)(336012)(426003)(2616005)(83380400001)(86362001)(16526019)(1076003)(26005)(82740400003)(36756003)(356005)(81166007)(70586007)(70206006)(316002)(54906003)(110136005)(478600001)(6666004)(41300700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:34.7318 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: da89f067-6a59-4f1e-fc10-08dc2d86b3f6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB71.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9437 X-Patchwork-Delegate: kuba@kernel.org The XDP_REDIRECT packets are given to the XDP stack and we drop the use of the related page: it will get freed by the driver that ends up doing the Tx. Because we have some hardware configurations with limited queue resources, we use the existing datapath Tx queues rather than creating and managing a separate set of xdp_tx queues. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_dev.h | 1 + .../net/ethernet/pensando/ionic/ionic_lif.c | 3 ++- .../net/ethernet/pensando/ionic/ionic_lif.h | 2 ++ .../net/ethernet/pensando/ionic/ionic_stats.c | 3 +++ .../net/ethernet/pensando/ionic/ionic_txrx.c | 24 +++++++++++++++++++ 5 files changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index 76425bb546ba..bfcfc2d7bcbd 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -266,6 +266,7 @@ struct ionic_queue { }; struct xdp_rxq_info *xdp_rxq_info; struct ionic_queue *partner; + bool xdp_flush; dma_addr_t base_pa; dma_addr_t cmb_base_pa; dma_addr_t sg_base_pa; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 72bbab1576ab..79bb07083f35 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -1649,7 +1649,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE; - netdev->xdp_features = NETDEV_XDP_ACT_BASIC; + netdev->xdp_features = NETDEV_XDP_ACT_BASIC | + NETDEV_XDP_ACT_REDIRECT; return 0; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 092ff08fc7e0..42006de8069d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -56,6 +56,7 @@ struct ionic_rx_stats { u64 xdp_aborted; u64 xdp_pass; u64 xdp_tx; + u64 xdp_redirect; }; #define IONIC_QCQ_F_INITED BIT(0) @@ -144,6 +145,7 @@ struct ionic_lif_sw_stats { u64 xdp_aborted; u64 xdp_pass; u64 xdp_tx; + u64 xdp_redirect; u64 xdp_frames; }; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_stats.c b/drivers/net/ethernet/pensando/ionic/ionic_stats.c index 5d48226e66cd..0107599a9dd4 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_stats.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_stats.c @@ -31,6 +31,7 @@ static const struct ionic_stat_desc ionic_lif_stats_desc[] = { IONIC_LIF_STAT_DESC(xdp_aborted), IONIC_LIF_STAT_DESC(xdp_pass), IONIC_LIF_STAT_DESC(xdp_tx), + IONIC_LIF_STAT_DESC(xdp_redirect), IONIC_LIF_STAT_DESC(xdp_frames), }; @@ -159,6 +160,7 @@ static const struct ionic_stat_desc ionic_rx_stats_desc[] = { IONIC_RX_STAT_DESC(xdp_aborted), IONIC_RX_STAT_DESC(xdp_pass), IONIC_RX_STAT_DESC(xdp_tx), + IONIC_RX_STAT_DESC(xdp_redirect), }; #define IONIC_NUM_LIF_STATS ARRAY_SIZE(ionic_lif_stats_desc) @@ -200,6 +202,7 @@ static void ionic_add_lif_rxq_stats(struct ionic_lif *lif, int q_num, stats->xdp_aborted += rxstats->xdp_aborted; stats->xdp_pass += rxstats->xdp_pass; stats->xdp_tx += rxstats->xdp_tx; + stats->xdp_redirect += rxstats->xdp_redirect; } static void ionic_get_lif_stats(struct ionic_lif *lif, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 3f1dee47bfa7..5471c182bacc 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -446,6 +446,20 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, break; case XDP_REDIRECT: + /* unmap the pages before handing them to a different device */ + dma_unmap_page(rxq->dev, buf_info->dma_addr, + IONIC_PAGE_SIZE, DMA_FROM_DEVICE); + + err = xdp_do_redirect(netdev, &xdp_buf, xdp_prog); + if (err) { + netdev_dbg(netdev, "xdp_do_redirect err %d\n", err); + goto out_xdp_abort; + } + buf_info->page = NULL; + rxq->xdp_flush = true; + stats->xdp_redirect++; + break; + case XDP_ABORTED: default: goto out_xdp_abort; @@ -806,6 +820,14 @@ int ionic_tx_napi(struct napi_struct *napi, int budget) return work_done; } +static void ionic_xdp_do_flush(struct ionic_cq *cq) +{ + if (cq->bound_q->xdp_flush) { + xdp_do_flush(); + cq->bound_q->xdp_flush = false; + } +} + int ionic_rx_napi(struct napi_struct *napi, int budget) { struct ionic_qcq *qcq = napi_to_qcq(napi); @@ -823,6 +845,7 @@ int ionic_rx_napi(struct napi_struct *napi, int budget) ionic_rx_fill(cq->bound_q); + ionic_xdp_do_flush(cq); if (work_done < budget && napi_complete_done(napi, work_done)) { ionic_dim_update(qcq, IONIC_LIF_F_RX_DIM_INTR); flags |= IONIC_INTR_CRED_UNMASK; @@ -869,6 +892,7 @@ int ionic_txrx_napi(struct napi_struct *napi, int budget) ionic_rx_fill(rxcq->bound_q); + ionic_xdp_do_flush(rxcq); if (rx_work_done < budget && napi_complete_done(napi, rx_work_done)) { ionic_dim_update(rxqcq, 0); flags |= IONIC_INTR_CRED_UNMASK; From patchwork Wed Feb 14 17:59:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556886 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2075.outbound.protection.outlook.com [40.107.244.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7617F132473 for ; Wed, 14 Feb 2024 17:59:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.75 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933579; cv=fail; b=LyDX1O4GtByOwwJWbG35lrzeXAgk6cuZxVdsLIViT2OcP/9fu/Fk7m4+t7G6U7riiXlP8RX2+iSNabx30uPbNxVpdPFCkGolvQwmP/lrcAly09lD0wK3nr/4FJ7fqUWzZ8f9vkdOaDOAEs/dk2fpJCicynCWAfvzsUK20Pb39Yc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933579; c=relaxed/simple; bh=jk4HHPTiEX0yzpe06XcyFcJ1klS9UED153A+03RbjmU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EsHQpIi1eR+DN5C/FdXpWqnX6d4ZHC6y6xFKAF8ho4P5CgSML3f7pPMM5Vd09jc9JcNBVyvMM0rDHs/MnJiFH/th03oM9nA5HdfdoZlF0H3Jb22OFwzy1dWsyO7F6cIJrk2z2r4zs9UbtLxUf30fQx9MRgBkd6Kas3tdt5P2++I= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=kmGLqC6i; arc=fail smtp.client-ip=40.107.244.75 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="kmGLqC6i" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oA3AZNdhyvDaw2y7d1L2jHYVb5euON/04BVvvgU/kbI93OilR1BAU7rEa1Bh/sut+QgcCpbvhtqDPv0Ew4vg9CABH8nlXG9lKGhykLCntHSrQjbkGi+rkTEjks9D2Ssa9vhq38VpmCsGDLfwxHyHd9axgqbNxq85qDAuobpsbJbxf3ESORgvuF2S+yP538OPEFbxtQETtna9oN9UaybFtjLLqxMO7SGOTsuHdtqLe44R5IhveTN3Kcah3nyUu6l9QxieBhFPVT5X7fAn/sddeQu9g1rGHFXxAk3osISXLONYVixWsnIdFkQ7fIuSVwawftkZYTu2kN1YcuDl2p+ZDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=T3ZB8iDOah7e6WRPK3imLQgeUSeNHy7HZSoez22PQUs=; b=f+z72jlm2l6Ds25VOLN6D7SgDEP6Ak/e5zngQyyk+bWIhl5UAmsPaHPW4rKe/yty5RZPwp6Z5qz2QrSP158dnQWQ/IJ8/atclGbTuuYYSBOOKktc0isPV0h2Z9JJHe8+84I6sfViOIbeHna4TIhk1B41z37XNC8Hr+AI5M0xN1FZkLPGM++Uakbm2GPc8QHDyuiILfXYBQaV03rpv28g+SoIkani6FnS8PjPUmla8Pu3dLdlvDRO+6tRPWgxn5gLXXm1bMrkUbA7wwfAcunqompePNGkyiYNi0iXCtunf2plWC9aoGdINnlNmgShnBBeFus8nxmOljAboa7ZXXHR1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T3ZB8iDOah7e6WRPK3imLQgeUSeNHy7HZSoez22PQUs=; b=kmGLqC6inmgwo+C9D0XyuquX9NkrZPj48c0ax7a4x5zpYJJppjl6LEJF+TLGwkz7u+cZhsfc0k7yomVos/o4JLJkzYjMI8xPYdN7UjcnOWLNYlm9ggBuQ4FX/wyUaAMHxiz+zGjVWD4txiJP2NVoA815QXBbx8wnTUrD82L94j4= Received: from BL1P221CA0030.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::21) by DM6PR12MB5007.namprd12.prod.outlook.com (2603:10b6:5:20d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25; Wed, 14 Feb 2024 17:59:35 +0000 Received: from BL6PEPF0001AB71.namprd02.prod.outlook.com (2603:10b6:208:2c5:cafe::36) by BL1P221CA0030.outlook.office365.com (2603:10b6:208:2c5::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.26 via Frontend Transport; Wed, 14 Feb 2024 17:59:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB71.mail.protection.outlook.com (10.167.242.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:35 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:33 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 8/9] ionic: add ndo_xdp_xmit Date: Wed, 14 Feb 2024 09:59:08 -0800 Message-ID: <20240214175909.68802-9-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB71:EE_|DM6PR12MB5007:EE_ X-MS-Office365-Filtering-Correlation-Id: 1748a137-97c5-4f88-6fe9-08dc2d86b42b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ryl3wgq3dMUoVXEK3wutgwdzmAenJaWwbKCbX+I0m2DpTPdPVBYSdyke15fG5yMB57JRhaY7Cv9Ur5+s8/1Flp6YGWmlamkhntNbIdsDMJ9Og5zxx0BFYX2WVt9CujyQzVwtsPwQdfxiGUn8iZLaHC7mvQXVXhqMydLd+/vKDJ9iSPY5N/N/m5QR3nWef690V7qpEvx3J/b//EVGc0111kQlzKUSw8fg9Ob3k8vUa8AvrtGxaEihXWP6KqrMWjJ7B2ucOCHRnP4PnrzDoHbQjFgT9IhS6/eA4Pl1axH4WOz/0V9PQYjZqk5pqcQvyCa3S39laCDGeWOcGavYY5xJ8pWZv0DqhSVnaPXEzjbjah1+EocpjpQYyoQ2r5CtSP0EEuoX/cYYZwFrrh3XlSGtoHs0IAMaJGSbl7bn/D5a5XMzMWLY/3SVAo7snKZxd/EegV23j6NFn2uMAA/YYk0z5FbR0Rkd90qhknSv9z6dMBXBKNcq4q1V5puJmAsBBoQ5zh/sWMLV+6EXZarFOCWg+Bl6guMND3wBmdQB7+jTpZLGcihuyUWkfPl0Y/55PMBvQ6LDl8KK/n39tyURGOTd6cB69tCXepeXj560mfivYQ= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(39860400002)(376002)(396003)(230922051799003)(82310400011)(186009)(451199024)(1800799012)(64100799003)(46966006)(36840700001)(40470700004)(316002)(110136005)(54906003)(6666004)(5660300002)(2906002)(44832011)(70206006)(86362001)(70586007)(4326008)(8676002)(8936002)(336012)(26005)(478600001)(41300700001)(1076003)(36756003)(2616005)(356005)(81166007)(16526019)(426003)(82740400003)(83380400001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:35.1068 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1748a137-97c5-4f88-6fe9-08dc2d86b42b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB71.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB5007 X-Patchwork-Delegate: kuba@kernel.org When our ndo_xdp_xmit is called we mark the buffer with XDP_REDIRECT so we know to return it to the XDP stack for cleaning. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_lif.c | 4 +- .../net/ethernet/pensando/ionic/ionic_txrx.c | 63 ++++++++++++++++++- .../net/ethernet/pensando/ionic/ionic_txrx.h | 1 + 3 files changed, 66 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 79bb07083f35..d26ea697804d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -1650,7 +1650,8 @@ static int ionic_init_nic_features(struct ionic_lif *lif) IFF_LIVE_ADDR_CHANGE; netdev->xdp_features = NETDEV_XDP_ACT_BASIC | - NETDEV_XDP_ACT_REDIRECT; + NETDEV_XDP_ACT_REDIRECT | + NETDEV_XDP_ACT_NDO_XMIT; return 0; } @@ -2847,6 +2848,7 @@ static const struct net_device_ops ionic_netdev_ops = { .ndo_eth_ioctl = ionic_eth_ioctl, .ndo_start_xmit = ionic_start_xmit, .ndo_bpf = ionic_xdp, + .ndo_xdp_xmit = ionic_xdp_xmit, .ndo_get_stats64 = ionic_get_stats64, .ndo_set_rx_mode = ionic_ndo_set_rx_mode, .ndo_set_features = ionic_set_features, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 5471c182bacc..e1839d5b7922 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -320,9 +320,13 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, buf_info = desc_info->bufs; dma_unmap_single(dev, buf_info->dma_addr, buf_info->len, DMA_TO_DEVICE); - __free_pages(buf_info->page, 0); + if (desc_info->act == XDP_TX) + __free_pages(buf_info->page, 0); buf_info->page = NULL; + if (desc_info->act == XDP_REDIRECT) + xdp_return_frame(desc_info->xdpf); + desc_info->nbufs = 0; desc_info->xdpf = NULL; desc_info->act = 0; @@ -376,6 +380,63 @@ static int ionic_xdp_post_frame(struct net_device *netdev, return 0; } +int ionic_xdp_xmit(struct net_device *netdev, int n, + struct xdp_frame **xdp_frames, u32 flags) +{ + struct ionic_lif *lif = netdev_priv(netdev); + struct ionic_queue *txq; + struct netdev_queue *nq; + int nxmit; + int space; + int cpu; + int qi; + + if (unlikely(!test_bit(IONIC_LIF_F_UP, lif->state))) + return -ENETDOWN; + + if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK)) + return -EINVAL; + + /* AdminQ is assumed on cpu 0, while we attempt to affinitize the + * TxRx queue pairs 0..n-1 on cpus 1..n. We try to keep with that + * affinitization here, but of course irqbalance and friends might + * have juggled things anyway, so we have to check for the 0 case. + */ + cpu = smp_processor_id(); + qi = cpu ? (cpu - 1) % lif->nxqs : cpu; + + txq = &lif->txqcqs[qi]->q; + nq = netdev_get_tx_queue(netdev, txq->index); + __netif_tx_lock(nq, cpu); + txq_trans_cond_update(nq); + + if (netif_tx_queue_stopped(nq) || + unlikely(ionic_maybe_stop_tx(txq, 1))) { + __netif_tx_unlock(nq); + return -EIO; + } + + space = min_t(int, n, ionic_q_space_avail(txq)); + for (nxmit = 0; nxmit < space ; nxmit++) { + if (ionic_xdp_post_frame(netdev, txq, xdp_frames[nxmit], + XDP_REDIRECT, + virt_to_page(xdp_frames[nxmit]->data), + 0, false)) { + nxmit--; + break; + } + } + + if (flags & XDP_XMIT_FLUSH) + ionic_dbell_ring(lif->kern_dbpage, txq->hw_type, + txq->dbval | txq->head_idx); + + ionic_maybe_stop_tx(txq, 4); + __netif_tx_unlock(nq); + + return nxmit; +} + static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct net_device *netdev, struct bpf_prog *xdp_prog, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h index d7cbaad8a6fb..82fc38e0f573 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h @@ -17,4 +17,5 @@ netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev); bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); +int ionic_xdp_xmit(struct net_device *netdev, int n, struct xdp_frame **xdp, u32 flags); #endif /* _IONIC_TXRX_H_ */ From patchwork Wed Feb 14 17:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13556888 X-Patchwork-Delegate: kuba@kernel.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2046.outbound.protection.outlook.com [40.107.94.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF599132498 for ; Wed, 14 Feb 2024 17:59:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.46 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933581; cv=fail; b=Hfroi8QJ9gDKSNyUK47eb8eQ301Wla3pSVRJHvKH8JVSmg/XhU+yvryp4CtYGQJFkFS67RAem3aL0dcjKVvuT3Zo2uxVkKQymtt/BuzIwluOC6kMVm9khZyuT+lruyOCxPgah6/xf5g1mf7keg1kGfeHETnWZV8RtAmrdxRXY6I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1707933581; c=relaxed/simple; bh=sYNv2BnvC0WtaGqCCzDkVlU8iOhFrfHqVdKAIaGkyRg=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NqEn7/NTFMvk1yMFHZG5352VNCWVukjPkRyG7Jx1gsruQqMX6mshhF6VQRcNSX8XouIAOGMZMRnzjECMptg2h6uZRcjH2OJ1vHj28gQZkO3MQgCaCMtnq1n2zUSqoLLA967qGCoSulq6v78TMLigKJ6vcLVjkqz/p79DHbs+TMg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ENzbwWai; arc=fail smtp.client-ip=40.107.94.46 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ENzbwWai" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BQipP81oOO+58sPRBTLGrczM9C+mIh1h37JGsIKf7/QYZTvWb4ipWkp8Z27f3KXCtJKcukiHAot63vjZzkB79tt0qkq3LTFgc2hisqFLFMYZyf4TXknSNho03ju9fte7w3Ja6R/51H+dyLXKuN4Yis0kTiDNSg4+6EW5dIiP8rmAndyjw9e/P/0WchfjBU75Z1M8jmPka7fO3pIoiKgPVDal/jE1ne4Yc5QM3eM82brjolMFeoeolXMfgA5FOsE0kFLhKlUlThs1wvpbg5D8BmDD5gR2dWnN9resOgeU8zCy/0VQgc2HyGcPxhDYQ23etYM/hqJFu3DYQPyJ38RYDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iIle3HZKovArfbhvWTBen3IWEYA3UXbAy4Im/L1m9Mc=; b=C4D6Ff/kOHalCrW9kOdBXHixAg/aJJdoskqF5deT03xx6F9yspxAkLrzBuTwTTqDxYEcD0Se2bSGKDC1BoILRTKui7dtvA+pbHgFzVCw4rqCDsXwW1cQutsWehBUvzC9QhdSSbRes2OPkjcBU253FEE+7T+BPwuWmxxks4Jd7/5bM4NYjgXVfKXbAXPpuZqyz2mV7K4S3NwxRVk4ZIRNnEoHBQBDYBvdYc4D9G+i5iPbdregbyr7Ce+nVT1uDt13cPcjvnMEgLyWZIT1BgTh9OjQZCBO/8c6smWVy4b4BU5u+UxXwrYZArO+o22oW23mciDeM796MJ05QaW21Hsu+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iIle3HZKovArfbhvWTBen3IWEYA3UXbAy4Im/L1m9Mc=; b=ENzbwWaixU4cvpyJXFKBGOqLGAFwkMghe1jnXYnS/ZTUM1YuJ0U3oNe5xl85lK2lbX00K3nBO0Frn0KnhGPldWCmfaVyGILaDkOL4C8FJWk2CDNwB4X9LUKMGQ6rARxDtG7j+dvQncHI3tS7hC0LmYzTy168Bmvl3DyBvZC2Bv8= Received: from BL1P221CA0023.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::10) by SA1PR12MB8857.namprd12.prod.outlook.com (2603:10b6:806:38d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.23; Wed, 14 Feb 2024 17:59:35 +0000 Received: from BL6PEPF0001AB71.namprd02.prod.outlook.com (2603:10b6:208:2c5:cafe::c3) by BL1P221CA0023.outlook.office365.com (2603:10b6:208:2c5::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7270.39 via Frontend Transport; Wed, 14 Feb 2024 17:59:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB71.mail.protection.outlook.com (10.167.242.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7292.25 via Frontend Transport; Wed, 14 Feb 2024 17:59:35 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 14 Feb 2024 11:59:34 -0600 From: Shannon Nelson To: , , , , CC: , , Shannon Nelson Subject: [PATCH v4 net-next 9/9] ionic: implement xdp frags support Date: Wed, 14 Feb 2024 09:59:09 -0800 Message-ID: <20240214175909.68802-10-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240214175909.68802-1-shannon.nelson@amd.com> References: <20240214175909.68802-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB71:EE_|SA1PR12MB8857:EE_ X-MS-Office365-Filtering-Correlation-Id: 3c7cedb5-2b7d-4d4f-45e7-08dc2d86b462 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: u6RCVb55V32SLSaiSR0uS5j83+RhSoez82EtTiyWQkL4HXN9dallz+2vYUTM5IcZqpJbnBPpHkS6Hp4sfeVgiXqYvhn+2tKQ2Wom+ErYv8ojoI+SXwePOGhNGxN0mM4FGk6IeAUI7UbXqkhD9fmtGCe/vyxk+FDC/bO6CPxrgwsZc4i06A8yNV2HmD6Tkppea48b/kuaSYIc2avvE6O7iCoCwAOmf2oo4l+RbdAGI8jX4Wlrnz8mzTztNmcP3FarDIc9dYo7d5FBSaJvXdx9Ca+FoAHrlHdwo+nuP/gGve0zvyoOb7nVVSlzULRx1uPLjjCCyXqmpJYwz3yN6Vvgi0bPKMT97SzY/f+INw0mpkfdlFFObCZeQlnD0euBzcxf3q8jQa1/Q0SAQpxMxCKqEgNaRgwMzPw74cyM6p1moEWVuBI/cIVDrxK6YvGfb5s/73u9IRHZK7SoEU3kOKqSHEAamEqd27ZzCqwAsb/B+/5Ovn/uVFK/FjU5gcNgBl44tzj3sQANNpEhP/CyAq/+xTdnGGjcnxivBe9eQvw5pZXELKQgrWIqYsmjtkTi4Db9Y0U/8LTe2hRQYKc4deC2eX1AXL2eagoWue1p8Xcy23Q= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(39860400002)(346002)(136003)(376002)(396003)(230922051799003)(1800799012)(451199024)(82310400011)(64100799003)(186009)(46966006)(40470700004)(36840700001)(41300700001)(82740400003)(44832011)(426003)(2616005)(336012)(6666004)(81166007)(356005)(2906002)(70586007)(70206006)(8676002)(4326008)(36756003)(8936002)(110136005)(54906003)(316002)(5660300002)(1076003)(26005)(16526019)(83380400001)(478600001)(86362001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2024 17:59:35.4662 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3c7cedb5-2b7d-4d4f-45e7-08dc2d86b462 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB71.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8857 X-Patchwork-Delegate: kuba@kernel.org Add support for using scatter-gather / frags in XDP in both Rx and Tx paths. Co-developed-by: Brett Creeley Signed-off-by: Brett Creeley Signed-off-by: Shannon Nelson Reviewed-by: Jacob Keller --- .../net/ethernet/pensando/ionic/ionic_lif.c | 12 ++- .../net/ethernet/pensando/ionic/ionic_txrx.c | 91 ++++++++++++++++++- 2 files changed, 98 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d26ea697804d..5cfc784f1227 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -881,7 +881,8 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) q->partner = &lif->txqcqs[q->index]->q; q->partner->partner = q; - if (!lif->xdp_prog) + if (!lif->xdp_prog || + (lif->xdp_prog->aux && lif->xdp_prog->aux->xdp_has_frags)) ctx.cmd.q_init.flags |= cpu_to_le16(IONIC_QINIT_F_SG); if (qcq->flags & IONIC_QCQ_F_CMB_RINGS) { @@ -1651,7 +1652,9 @@ static int ionic_init_nic_features(struct ionic_lif *lif) netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | - NETDEV_XDP_ACT_NDO_XMIT; + NETDEV_XDP_ACT_RX_SG | + NETDEV_XDP_ACT_NDO_XMIT | + NETDEV_XDP_ACT_NDO_XMIT_SG; return 0; } @@ -1799,6 +1802,9 @@ static bool ionic_xdp_is_valid_mtu(struct ionic_lif *lif, u32 mtu, if (mtu <= IONIC_XDP_MAX_LINEAR_MTU) return true; + if (xdp_prog->aux && xdp_prog->aux->xdp_has_frags) + return true; + return false; } @@ -2812,7 +2818,7 @@ static int ionic_xdp_config(struct net_device *netdev, struct netdev_bpf *bpf) } maxfs = __le32_to_cpu(lif->identity->eth.max_frame_size) - VLAN_ETH_HLEN; - if (bpf->prog) + if (bpf->prog && !(bpf->prog->aux && bpf->prog->aux->xdp_has_frags)) maxfs = min_t(u32, maxfs, IONIC_XDP_MAX_LINEAR_MTU); netdev->max_mtu = maxfs; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index e1839d5b7922..3f397049ae1d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -15,6 +15,13 @@ static int ionic_maybe_stop_tx(struct ionic_queue *q, int ndescs); static dma_addr_t ionic_tx_map_single(struct ionic_queue *q, void *data, size_t len); +static dma_addr_t ionic_tx_map_frag(struct ionic_queue *q, + const skb_frag_t *frag, + size_t offset, size_t len); + +static void ionic_tx_desc_unmap_bufs(struct ionic_queue *q, + struct ionic_desc_info *desc_info); + static void ionic_tx_clean(struct ionic_queue *q, struct ionic_desc_info *desc_info, struct ionic_cq_info *cq_info, @@ -313,6 +320,7 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, unsigned int nbufs = desc_info->nbufs; struct ionic_buf_info *buf_info; struct device *dev = q->dev; + int i; if (!nbufs) return; @@ -324,6 +332,15 @@ static void ionic_xdp_tx_desc_clean(struct ionic_queue *q, __free_pages(buf_info->page, 0); buf_info->page = NULL; + buf_info++; + for (i = 1; i < nbufs + 1 && buf_info->page; i++, buf_info++) { + dma_unmap_page(dev, buf_info->dma_addr, + buf_info->len, DMA_TO_DEVICE); + if (desc_info->act == XDP_TX) + __free_pages(buf_info->page, 0); + buf_info->page = NULL; + } + if (desc_info->act == XDP_REDIRECT) xdp_return_frame(desc_info->xdpf); @@ -364,8 +381,38 @@ static int ionic_xdp_post_frame(struct net_device *netdev, desc_info->xdpf = frame; desc_info->act = act; + if (xdp_frame_has_frags(frame)) { + struct ionic_txq_sg_elem *elem; + struct skb_shared_info *sinfo; + struct ionic_buf_info *bi; + skb_frag_t *frag; + int i; + + bi = &buf_info[1]; + sinfo = xdp_get_shared_info_from_frame(frame); + frag = sinfo->frags; + elem = desc_info->txq_sg_desc->elems; + for (i = 0; i < sinfo->nr_frags; i++, frag++, bi++) { + dma_addr = ionic_tx_map_frag(q, frag, 0, skb_frag_size(frag)); + if (dma_mapping_error(q->dev, dma_addr)) { + stats->dma_map_err++; + ionic_tx_desc_unmap_bufs(q, desc_info); + return -EIO; + } + bi->dma_addr = dma_addr; + bi->len = skb_frag_size(frag); + bi->page = skb_frag_page(frag); + + elem->addr = cpu_to_le64(bi->dma_addr); + elem->len = cpu_to_le16(bi->len); + elem++; + + desc_info->nbufs++; + } + } + cmd = encode_txq_desc_cmd(IONIC_TXQ_DESC_OPCODE_CSUM_NONE, - 0, 0, buf_info->dma_addr); + 0, (desc_info->nbufs - 1), buf_info->dma_addr); desc->cmd = cpu_to_le64(cmd); desc->len = cpu_to_le16(len); desc->csum_start = 0; @@ -449,11 +496,14 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, struct ionic_queue *txq; struct netdev_queue *nq; struct xdp_frame *xdpf; + int remain_len; + int frag_len; int err = 0; xdp_init_buff(&xdp_buf, IONIC_PAGE_SIZE, rxq->xdp_rxq_info); + frag_len = min_t(u16, len, IONIC_XDP_MAX_LINEAR_MTU + VLAN_ETH_HLEN); xdp_prepare_buff(&xdp_buf, ionic_rx_buf_va(buf_info), - XDP_PACKET_HEADROOM, len, false); + XDP_PACKET_HEADROOM, frag_len, false); dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(buf_info), XDP_PACKET_HEADROOM, len, @@ -461,6 +511,43 @@ static bool ionic_run_xdp(struct ionic_rx_stats *stats, prefetchw(&xdp_buf.data_hard_start); + /* We limit MTU size to one buffer if !xdp_has_frags, so + * if the recv len is bigger than one buffer + * then we know we have frag info to gather + */ + remain_len = len - frag_len; + if (remain_len) { + struct skb_shared_info *sinfo; + struct ionic_buf_info *bi; + skb_frag_t *frag; + + bi = buf_info; + sinfo = xdp_get_shared_info_from_buff(&xdp_buf); + sinfo->nr_frags = 0; + sinfo->xdp_frags_size = 0; + xdp_buff_set_frags_flag(&xdp_buf); + + do { + if (unlikely(sinfo->nr_frags >= MAX_SKB_FRAGS)) { + err = -ENOSPC; + goto out_xdp_abort; + } + + frag = &sinfo->frags[sinfo->nr_frags]; + sinfo->nr_frags++; + bi++; + frag_len = min_t(u16, remain_len, ionic_rx_buf_size(bi)); + dma_sync_single_range_for_cpu(rxq->dev, ionic_rx_buf_pa(bi), + 0, frag_len, DMA_FROM_DEVICE); + skb_frag_fill_page_desc(frag, bi->page, 0, frag_len); + sinfo->xdp_frags_size += frag_len; + remain_len -= frag_len; + + if (page_is_pfmemalloc(bi->page)) + xdp_buff_set_frag_pfmemalloc(&xdp_buf); + } while (remain_len > 0); + } + xdp_action = bpf_prog_run_xdp(xdp_prog, &xdp_buf); switch (xdp_action) {