From patchwork Fri Mar 4 10:22:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12768916 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B5F8C433F5 for ; Fri, 4 Mar 2022 10:22:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233070AbiCDKXk (ORCPT ); Fri, 4 Mar 2022 05:23:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232710AbiCDKXd (ORCPT ); Fri, 4 Mar 2022 05:23:33 -0500 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2098.outbound.protection.outlook.com [40.107.244.98]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2431239B81 for ; Fri, 4 Mar 2022 02:22:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VHQAiSFJsfrj3FkWGTTFHp03uXL2yRLKS7qXaxztIW02ZuFXtuKy4j4TC5aNfEPKKO1D+vnkGQPZv+qswMJQLDypqiXKJ94egAUAxW0Pcmf9Pf61wWfBf2scDMufCN05olobIUydYt9M3ioffnaXPSX/cUIjiIeH3qnhbmdaPWLlXu405s63AXck0TQLITeQt08vvqn94LqHZQTNuIHsPPPoILnxe+1sEvlP2f4pgcA9mwXxV9Nc/G/gAfpYkU5U7LvS5xTYT4hwcG6cvvnuqdgoTlqHrimRpkVeUAvKb9qAPqcIwZOkW7hapAJIqWFefi4vY6CtU1R53D82JyGA2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WGbZGa+38okN9YpcGXw8iSP6EbXWVMZs9a1GrXA1f0Y=; b=TAwr+4J6pKPf2FDZI2CXhPN7GmSoTXGKl0qJ/sLV1BL46fhjVS88QiNu+4hGWzKL8wQZaZ0ZabdeVhi2YFz2v9Bpd3UlInIeQyIJZq2cZ3V0tKAWapYtl4p6BV4pGJoEU1WN0xsvADNh0MD8AuB7P/FsBiY+17jAAN0swZr96tfoX0DPf+LD/PS9596ByDZP9GTq6LOaCgr3C17DjmeubOSl03umZ5U4GFJGR/iUkQZm8/0zKm4SzfEyo3hTbySNRSNrUiwupt32EDDtsZ+sGfIct/6cVKDBTlcvLzzYOj+P0NHPitDU+51GYbHPcciUlTaAsSo8Bsswor8CLSKPbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WGbZGa+38okN9YpcGXw8iSP6EbXWVMZs9a1GrXA1f0Y=; b=B5As7X6WrrsbPY/vF9la6E8W960orxJvkz1ZsgtpPj17aU/2WKiEM8nyEbtzTGzI3qAiyDkjVbhrhPcMJaYHYMUpa7ibc3v/Tq6jhcRFM/oWCP9rqIQ1SMKU1at2PW8T5wRm7reKI/TnhFHcI1mxmXwF3UK11el52AUK9+H/D9I= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by DM6PR13MB4539.namprd13.prod.outlook.com (2603:10b6:5:36::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5061.7; Fri, 4 Mar 2022 10:22:41 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e%2]) with mapi id 15.20.5038.017; Fri, 4 Mar 2022 10:22:41 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , =?utf-8?q?Niklas_S=C3=B6derlund?= Cc: netdev@vger.kernel.org, oss-drivers@corigine.com Subject: [PATCH net-next 1/5] nfp: expose common functions to be used for AF_XDP Date: Fri, 4 Mar 2022 11:22:10 +0100 Message-Id: <20220304102214.25903-2-simon.horman@corigine.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220304102214.25903-1-simon.horman@corigine.com> References: <20220304102214.25903-1-simon.horman@corigine.com> X-ClientProxiedBy: AM8P190CA0012.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::17) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8e984a34-910c-4419-7162-08d9fdc8ea28 X-MS-TrafficTypeDiagnostic: DM6PR13MB4539:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VLhcs9XnOTIoPU6Lduuf9CY3mobFY2u3TnwoJdR75GYVNo2r2qVMsJCZPQQLJleIUViO02FWOgHS8G5OgHnx8l0wVByzAg2c8/PTsGAU6eFxvV4kOm+ZMkloYbaDqJniYcyRhLnn27UPltz6fi9DEsFSBpeTpHDhUPx36b0++31nd237ElJAzHIjd28Y4flR+V+TZDLGBt3sxvB1X5WAD4Y6Ykx714ARTaD+Fj8Ji/afi7PeZEuvgtm5K3dvNyghV+FCi7uIU9wuuzyZy/5ufjBaO988AK0muZf5ZygvnTp2PjcfxHjOoxVffQXvDCg5/4iQGOWhgCVNY+oCJeKeQDis6V/xYZ/ohsbcPTUpaH7NWBvUntu8PueiivsKlJ37LMNd0xPkCnwyhoEM3wHyjLDTv7AEd6f6WyJ+g48Ja3DxbFQY0WUhQdybd4t3BrlhJzlioGrqEBkf+6xObEGPBq7IreT7AAg9NIXIpo7+hQLcpxAH/6xD8HbpjNjNiX0RQ4LB1rWEHouGg9R0Fejzytcj8VJXhPn7BsqEKRQ+cNMkJPDrm0l5O/vOvCVT/Pw0d+S+bJpQJtkOw2C+KVi9z8CXYZs0sAn2d3FBlDkPDUueOVB7+8nxNzjIhtC4H7fTkN+xianZBJFRyDQpEyZ2Rw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(396003)(346002)(136003)(376002)(366004)(39830400003)(6486002)(6636002)(8936002)(110136005)(5660300002)(316002)(508600001)(66946007)(66556008)(66476007)(8676002)(4326008)(6506007)(6666004)(52116002)(86362001)(38100700002)(6512007)(83380400001)(66574015)(36756003)(2616005)(2906002)(107886003)(1076003)(44832011)(186003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?gF1A/BWRm/6UbrmRtiQZzTyKLrWm?= =?utf-8?q?xK59CO7ifcUUWuQzTMqICngt0HVGEZD4e7hgj5GqMcV21I9JuOpmkwvKYT2TYii2W?= =?utf-8?q?ipRIKfmk1p4gK/C67z3Hnm7oO+URAu7dpGPTMbW/JLW2VYWG8pD790XTlsrG28rVt?= =?utf-8?q?RGqbJbGLxzAY7MEKsET4V2zuIScRepogxDSqyl8es/XPJXHwEKxIuBHifedlORCBD?= =?utf-8?q?DSJKzinA0necn0swd1FPQS2A7WPkQQWi7879oIzU14HFGFwCElwxStUxGBmPpfSny?= =?utf-8?q?5i9tw4kN5uCDcMtvcR6vD9ks7YOkBGwWO5MCjAwWl5UUCu1W0L+9H1BYVOrZ4Toa+?= =?utf-8?q?JkWKYa8CVs6qUV4tNGHDQuTmMDJECCO1tUajn+t5qk2GS8B1bRkLMy36sMTYsjK71?= =?utf-8?q?v6Stifwv224aEA8Q8JwT3JXLvc5RtlXWE3/zFhN18r2rf3eH5Pirwj8BK6aJsvADw?= =?utf-8?q?rbuylsDgFGdSu2WRca4M+5tbaOlshNDqvraDd/OKR0ZB4vuu0r6/+cPaLjhYgZNhs?= =?utf-8?q?065J9IP0scWZSQuopdKpsIoYgnQuCmtt3oxOBhFPRxOlO3tA4Q5nxmUA4g1gLnzmF?= =?utf-8?q?IfjoAMgoIjFzS2AgZScSnlZ98k3AKRiKMeaAV1VERhyOoySj3jMEIvvlXXka69CBl?= =?utf-8?q?jZg8VomgC5IyLWTjkOLeHrDwRu1KQg3V4D1S9ZEqXUiol0BzpCZqWTtobTDJnAuL+?= =?utf-8?q?+2Nim6j8ODl1K7J5d6grmpBQ0RmMhgSCTuOVSsetzjydC+XfyOS/cVnQXVJFQvqKg?= =?utf-8?q?TK0cpZD+SAECyK1qBzPyWVMapkuvtYGS2xnBA3CA0QT1CudIbzRx3zn8ZMp9xXJCz?= =?utf-8?q?FRrfdQwhzFjKTj4bWaS2xLwgfyWqBUXzJ14r2WX0/uHrSnSioMdVYmmRlNhSjJ/sH?= =?utf-8?q?U8MbZ+s0hiZT/ESUHt3AlvuH4xGqR1xuA/FXqhLfLXcMCZgbSC9W2/TSAfEEKzPuA?= =?utf-8?q?UArYJw7Jzfkyw9XqcvyulQuvZYV1HSQY4LbjWslLe24WYM5tGVHxgR6HKw2w9Bc5y?= =?utf-8?q?6M+seAR+zZBhHcKN4rQGPZttAougzsA3XMUe/Osa3SuGkOfAn5yxiDscLgWn6T50G?= =?utf-8?q?JHaoLquMgO46yf4JUn2NME6rjCjLTtGHkNJK/5Zm/6OKRQWe2THmKcYDyHenz5PJ3?= =?utf-8?q?PsmGFspkhqLma6oDgMaN5l/bQQ4noRMZKZVMBQcKU2lTgVaoYnjG6LUnPli4u8D9R?= =?utf-8?q?BGKUponCMdQQRwVQl/xDN65N5AZG7aMG8N8ri953xIYY6y8NTqLsi4nbqhk9hhCym?= =?utf-8?q?znADIeIRGb1Qlm1JI1V+gJuOmSdq8iHvh4/OsLGpsrtlNpgtdn+x0QzsopYZvO55W?= =?utf-8?q?Bo/FMSGjugL/JWZLy9vtQqkHjJYzBNc/3qTW7h8XYnSVQk/As30Z0ZoW8hWrsSpAR?= =?utf-8?q?ZhY7JDzZ+N02dL2sCF3QPckyV9FisYozWq6h9CdDYXAOfaqCWVjdMWHtXQqFWUjgj?= =?utf-8?q?gk+aPnQ3kdrpsLrmTpRj17Y45+cBEtRKqsA=3D=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8e984a34-910c-4419-7162-08d9fdc8ea28 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2022 10:22:41.5640 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ceRmXc6oZ9ugVOb4AgJN/8GZDZFJmBDdtvdbHdQ6SOj6/OT1vkYk1ErxlGjGAV1AHpOrW27Mv2r3cX+SfJIiDAFFQ0sZUqZV4yovmtg2s6s= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR13MB4539 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Niklas Söderlund There are some common functionality that can be reused in the upcoming AF_XDP support. Expose those functions in the header. While at it mark some arguments of nfp_net_rx_csum() as const. Signed-off-by: Niklas Söderlund Signed-off-by: Simon Horman --- drivers/net/ethernet/netronome/nfp/nfp_net.h | 14 ++++++++++++++ .../net/ethernet/netronome/nfp/nfp_net_common.c | 16 ++++++++-------- 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index 0b1865e9f0b5..fa40d339df8d 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -965,6 +965,7 @@ int nfp_net_mbox_reconfig_and_unlock(struct nfp_net *nn, u32 mbox_cmd); void nfp_net_mbox_reconfig_post(struct nfp_net *nn, u32 update); int nfp_net_mbox_reconfig_wait_posted(struct nfp_net *nn); +void nfp_net_irq_unmask(struct nfp_net *nn, unsigned int entry_nr); unsigned int nfp_net_irqs_alloc(struct pci_dev *pdev, struct msix_entry *irq_entries, unsigned int min_irqs, unsigned int want_irqs); @@ -973,6 +974,19 @@ void nfp_net_irqs_assign(struct nfp_net *nn, struct msix_entry *irq_entries, unsigned int n); +void nfp_net_tx_xmit_more_flush(struct nfp_net_tx_ring *tx_ring); +void nfp_net_tx_complete(struct nfp_net_tx_ring *tx_ring, int budget); + +bool +nfp_net_parse_meta(struct net_device *netdev, struct nfp_meta_parsed *meta, + void *data, void *pkt, unsigned int pkt_len, int meta_len); + +void nfp_net_rx_csum(const struct nfp_net_dp *dp, + struct nfp_net_r_vector *r_vec, + const struct nfp_net_rx_desc *rxd, + const struct nfp_meta_parsed *meta, + struct sk_buff *skb); + struct nfp_net_dp *nfp_net_clone_dp(struct nfp_net *nn); int nfp_net_ring_reconfig(struct nfp_net *nn, struct nfp_net_dp *new, struct netlink_ext_ack *extack); diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index 79257ec41987..edf7b8716a70 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -381,7 +381,7 @@ int nfp_net_mbox_reconfig_and_unlock(struct nfp_net *nn, u32 mbox_cmd) * * Clear the ICR for the IRQ entry. */ -static void nfp_net_irq_unmask(struct nfp_net *nn, unsigned int entry_nr) +void nfp_net_irq_unmask(struct nfp_net *nn, unsigned int entry_nr) { nn_writeb(nn, NFP_NET_CFG_ICR(entry_nr), NFP_NET_CFG_ICR_UNMASKED); nn_pci_flush(nn); @@ -923,7 +923,7 @@ static void nfp_net_tls_tx_undo(struct sk_buff *skb, u64 tls_handle) #endif } -static void nfp_net_tx_xmit_more_flush(struct nfp_net_tx_ring *tx_ring) +void nfp_net_tx_xmit_more_flush(struct nfp_net_tx_ring *tx_ring) { wmb(); nfp_qcp_wr_ptr_add(tx_ring->qcp_q, tx_ring->wr_ptr_add); @@ -1142,7 +1142,7 @@ static netdev_tx_t nfp_net_tx(struct sk_buff *skb, struct net_device *netdev) * @tx_ring: TX ring structure * @budget: NAPI budget (only used as bool to determine if in NAPI context) */ -static void nfp_net_tx_complete(struct nfp_net_tx_ring *tx_ring, int budget) +void nfp_net_tx_complete(struct nfp_net_tx_ring *tx_ring, int budget) { struct nfp_net_r_vector *r_vec = tx_ring->r_vec; struct nfp_net_dp *dp = &r_vec->nfp_net->dp; @@ -1587,10 +1587,10 @@ static int nfp_net_rx_csum_has_errors(u16 flags) * @meta: Parsed metadata prepend * @skb: Pointer to SKB */ -static void nfp_net_rx_csum(struct nfp_net_dp *dp, - struct nfp_net_r_vector *r_vec, - struct nfp_net_rx_desc *rxd, - struct nfp_meta_parsed *meta, struct sk_buff *skb) +void nfp_net_rx_csum(const struct nfp_net_dp *dp, + struct nfp_net_r_vector *r_vec, + const struct nfp_net_rx_desc *rxd, + const struct nfp_meta_parsed *meta, struct sk_buff *skb) { skb_checksum_none_assert(skb); @@ -1668,7 +1668,7 @@ nfp_net_set_hash_desc(struct net_device *netdev, struct nfp_meta_parsed *meta, &rx_hash->hash); } -static bool +bool nfp_net_parse_meta(struct net_device *netdev, struct nfp_meta_parsed *meta, void *data, void *pkt, unsigned int pkt_len, int meta_len) { From patchwork Fri Mar 4 10:22:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12768918 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47FE1C433FE for ; Fri, 4 Mar 2022 10:22:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233317AbiCDKXn (ORCPT ); Fri, 4 Mar 2022 05:23:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231445AbiCDKXd (ORCPT ); Fri, 4 Mar 2022 05:23:33 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2115.outbound.protection.outlook.com [40.107.243.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB0FD3982A for ; Fri, 4 Mar 2022 02:22:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KeHnqdD9ON8EEbkoFqpAerRSlF6pPmlBjsDdErOQHgVGg6W0q+Sk7xfGsbxI4HOZYcTi4RqB44IepZ9h3CNBzh1tpldaKOZaA87qxzVS/rHTp850n4sWScNH/X6CDj04BvT828VNrmz5KWcAvp1gSVYgAE9AOwF2vA4VpUOqkHi5ZD8aBcr/16OCAEYcAVOI09flGjTayBWI46ab+mlXg/b1amIdCqx272tdgLeRe3UGH6Z8VFvLjvzJz7pjY0kd9rHLvFMDTCqYsEk/CiakUaw+oY3NkIN4CJk/+TtiLg4qWxW4JX0u0ykzk58qrcfBal7La/3R6BTHr/q4eqizLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FESGjbWHNHBlPyIYx56n6E/WhnW/5Vv+TOUVWTpBUWE=; b=Xy9TlDpflEQXcuO1GrxKBMwF/CLbs5OFNg4lFGsv7f3z9LSNPaYiNESzo04MlhlfRAefKLF1jp1dARDIRXAAir2A12usTprFscO2UicI7QP1E2kXsbRY25yVGlCGgI0dlgIp1Q0I7yteZSI4udaUgSColeR8CHm1po6Kjf4/Hs8HCnHNMPYvgU5DO7M3HqWnakBvGAduTSIyxtsMkGvJvsceggmVPTmiPgB55fISgwLAABTxcoZtxpkds30JeYdalHQyCYxLGvfZljmYtJczhe6caeGPghwdFFwdlzpMlIHkipZNn2P4wzGj/EbDEWV5m8PR3C53KkOs0FG28lhyuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FESGjbWHNHBlPyIYx56n6E/WhnW/5Vv+TOUVWTpBUWE=; b=dWtj/tgPSTNXT+HwvfUrxAA36GSq10D4l30nip9uR2Vh/yZ+VFi+E60nCK57ESZTmTygksqDV8TfWFKIC3h9ao4nua1xKirDmCv6mu/51MyVjAeuAo9tFj7JqxddW1e+gQmNlc7p/xGg2RzCQLGUFvTjgZcS6kLIfkoMqej6t4I= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by BN7PR13MB2404.namprd13.prod.outlook.com (2603:10b6:406:b7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.11; Fri, 4 Mar 2022 10:22:43 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e%2]) with mapi id 15.20.5038.017; Fri, 4 Mar 2022 10:22:43 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , =?utf-8?q?Niklas_S=C3=B6derlund?= Cc: netdev@vger.kernel.org, oss-drivers@corigine.com Subject: [PATCH net-next 2/5] nfp: wrap napi add/del logic Date: Fri, 4 Mar 2022 11:22:11 +0100 Message-Id: <20220304102214.25903-3-simon.horman@corigine.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220304102214.25903-1-simon.horman@corigine.com> References: <20220304102214.25903-1-simon.horman@corigine.com> X-ClientProxiedBy: AM8P190CA0012.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::17) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 19602b54-2bea-4cea-587e-08d9fdc8eb1e X-MS-TrafficTypeDiagnostic: BN7PR13MB2404:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2CsLshtJNJUusKI7SDcf37+hlYaYJV6D3UalESD1fVo/G/DeoGpD57lv60W/pNclRcD1EUATuiXNnZXGbXcBPDLwEYJAzIJ7q3bIZgBqZ3bHOU1zPUVbva75HWJM1nYQfFOdMqODWUlz405dPKP5Wr1U8vvk7fHTOwii1g3V2Zvr60P9W2HkXXoqhE+csYeembFZ6dewARYmCzcC4iLVLbALX0h3rEGXb+fCbnbG9f3m8SV6J4G3KTTiX/9C7vhtFREnnPPFpCRfkniCGyFqjyYl6yzLE+e3tUMxAz97Gb1FumtErtQ0szFAfj2sAH2ZvdDhGuXbdr69ajaoMfdOE/JtRrcQrbQVzDzykxmiaVeSD0OGpto/Zao7TOEkqJZsoN33DRklDOuq63V/5gZTOXPRe5NSBQiPoNXgJCWkHdyMgs/cMKBPQEyoCr0S+juTzIU+eGHuIue8pNSlKAZQD0/ZDFyHXv38hpS9247bdqksVjuponnjYlUpPBuTqfKO/83XGSYWp89qXvKGkLvIdL1kpJcuW7X78M8fQkGSff3CteLbGaHLyqNVU5pf5L/p6Gd6tpmcmvOlKJ07h1jp1Ogh8OIeBM6uua/7LeY6phNmhfpz5DM8OTZ7mM7c8ADC7MPlZF8ZlrYM+29pHZzfPQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(396003)(346002)(136003)(376002)(39840400004)(110136005)(316002)(86362001)(4326008)(8676002)(38100700002)(52116002)(66476007)(66556008)(66946007)(6666004)(6512007)(6636002)(6506007)(6486002)(508600001)(8936002)(44832011)(66574015)(107886003)(5660300002)(186003)(2616005)(2906002)(1076003)(36756003)(83380400001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?O2nAzW9VrQDfOFMIHDtpgvuDtTC1?= =?utf-8?q?Z+rojkhjp0RoQlnbKUi2tx+VBkbNxSauWHszGqVHPWt66/aSkkEYiR4XY7WYb88Y0?= =?utf-8?q?ndpnqVFFmkTxMLY3+L9HykoF53iYwJoXERB/P5EBvfDd+s8un8qeQ9srQjJOP+vFZ?= =?utf-8?q?ODfQDC41gkpAUHi2dunX1sjjZIbAw7bDsLZLtQ+uPjTQYA1Bj38j35uK6miaMOlsh?= =?utf-8?q?Qj+F+RJCEaGAgsKy+f29h2FUpoDX223mluk5zp29qhILi6r5QmzjmQVDCNhTB6fjq?= =?utf-8?q?XvDiAbcIT5Cn0ieGKuaWGGxrQobw15oILNv/aiLM3kPtZkg/4g4m4WWqm6UBLp7SP?= =?utf-8?q?Q5pWJJNZ2R3khMjeotBcjdCSdxKg6uZ6pIDPBuVzE2KXxzBJFpHsIwsdfYd2mHzLx?= =?utf-8?q?d+bo5FI7rV4avjDxz1p730wwwvD9pe+o4hjBfmfKQlY6K/Fq/DAkLzcKjlofP9tCz?= =?utf-8?q?rENGhPOT3oVMvH/vRQTewlmYGQgljNn7NTR0UpFSy3WQ7aPiHjG/7lBZLGgvG2PeY?= =?utf-8?q?SVbf4TSn2aW8wLZgr3XlRjkfBYg0G14q+GJbldfNS0TbNr4IXFoKjwdT/zHt93KU9?= =?utf-8?q?hfZCGUqR0RtUFGV2RA8/3HHcjkT1vbFV89hwFtsi433qs0036Ki/YuFUgkpziHm/W?= =?utf-8?q?1fWI6JTN6uxR3rFRW9kYrlPQy99eRWnjQZHJUFR0cqBFlVcLyxJ9NEx6txExeWZyY?= =?utf-8?q?AFotae1Qy3730DXTQ8Kzs9kyksUI9eq3xclusq0Q2/MBth4JyakK3i3sDv7ZKVEWU?= =?utf-8?q?XfoTipJUjjR4pOVGzYr+4G9vvMjOKlo1hrE+Ej2gzQscMOA+kyfEGujpi3NmVg3qH?= =?utf-8?q?Ck97u1UETWxCqLmt63t0btG+Exvnp2k4ULIyuiNnr0lhWpyG0maU29rzAwI3/LAtL?= =?utf-8?q?Nf1QX1SLwqO2gTTqY0Hp399yDVayjpIJGSdZ/gu4tz43HM9op6tcO9B848s7dGDlz?= =?utf-8?q?AuTxpVpxVDcL/KvOs9+e/eq4kByia6ys9T3+N9eYo7AWAOK0m98LjUHEHBmsUvMGU?= =?utf-8?q?P5O1dXzxjAH8E4txFcK+1D7JQrdTSSvIibEGcVGgf/3GiKMH5sE2TBM3bmRqq7jny?= =?utf-8?q?ousgdymWyUrYscj/RQQL7xaTkXaBD/oSXpIwY/rWoXhUcH27xHKAOyhTCEVoK5zwS?= =?utf-8?q?6R7mXSGsCN2TPwVlxuOJCbYTNBOfETpJirXUiAbcQAjIiuuRoujatdisPBqeGq7Yv?= =?utf-8?q?XYJNjpXRQI6hR4EJeJz6rOTpHNDOcU+rK8YbYSa8l+n9Q9qsXDUOUmnd0+C5nqk0e?= =?utf-8?q?Ny3yftsAL6iY0PwpMHBVKOUsuG94i3fxpQKWSEqg7VHxQpcwxyr7rFb3Pr+l8t7F9?= =?utf-8?q?GeOIwl0KpUBSkBga0LBpSxPNr5qH3McQGXCGadtz+SHEuWTI2YM7+ii8wxvtLBhJc?= =?utf-8?q?gkgQccP1MWQpNTw4V9BfwscSj8/9m43t+ncJaM4mNInhtUEnBljkkpYiXMwY6WUqK?= =?utf-8?q?19wtlCjjOp91Q4XnIiCLW3aGe+jXVAzJe7V+GL8b/bdHzj0ZHQhL+nQnYoNhB/4Wf?= =?utf-8?q?vU2dTDBRMI8PEFayquWj0DeGgYl8p6wTTdYL0j4ONtGUy9WhSx0EaDjo6WHUGXkG5?= =?utf-8?q?+wVthBw1b2X8urb+VcWspYNjdFzd7DCynBtvIbTAGkLml3TvfrnShYUVyl1E0752Z?= =?utf-8?q?e1kkyb9MMoixDV5SNvJlMPOgfCemiiXIlMgMXpEVSsAeAoLRMF7PQ=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 19602b54-2bea-4cea-587e-08d9fdc8eb1e X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2022 10:22:43.0649 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ad8Zsc3OiXT+hieQ4rRQ/wY/09hQYD2v0F5WCa1jStD/A7GHZGVELpgWt4ryDM53rxXjdEXG0r64sjP3fCr7gkPtWzBFXGAomfnPHN5AyAc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR13MB2404 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Jakub Kicinski There will be more NAPI register logic once AF_XDP support is added, wrap our already conditional napi add/del into helpers. Signed-off-by: Jakub Kicinski Signed-off-by: Niklas Söderlund Signed-off-by: Simon Horman --- .../ethernet/netronome/nfp/nfp_net_common.c | 38 +++++++++++-------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index edf7b8716a70..e6a17af731ba 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -2638,6 +2638,25 @@ static void nfp_net_rx_rings_free(struct nfp_net_dp *dp) kfree(dp->rx_rings); } +static void +nfp_net_napi_add(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec) +{ + if (dp->netdev) + netif_napi_add(dp->netdev, &r_vec->napi, + nfp_net_poll, NAPI_POLL_WEIGHT); + else + tasklet_enable(&r_vec->tasklet); +} + +static void +nfp_net_napi_del(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec) +{ + if (dp->netdev) + netif_napi_del(&r_vec->napi); + else + tasklet_disable(&r_vec->tasklet); +} + static void nfp_net_vector_assign_rings(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec, int idx) @@ -2656,23 +2675,14 @@ nfp_net_prepare_vector(struct nfp_net *nn, struct nfp_net_r_vector *r_vec, { int err; - /* Setup NAPI */ - if (nn->dp.netdev) - netif_napi_add(nn->dp.netdev, &r_vec->napi, - nfp_net_poll, NAPI_POLL_WEIGHT); - else - tasklet_enable(&r_vec->tasklet); + nfp_net_napi_add(&nn->dp, r_vec); snprintf(r_vec->name, sizeof(r_vec->name), "%s-rxtx-%d", nfp_net_name(nn), idx); err = request_irq(r_vec->irq_vector, r_vec->handler, 0, r_vec->name, r_vec); if (err) { - if (nn->dp.netdev) - netif_napi_del(&r_vec->napi); - else - tasklet_disable(&r_vec->tasklet); - + nfp_net_napi_del(&nn->dp, r_vec); nn_err(nn, "Error requesting IRQ %d\n", r_vec->irq_vector); return err; } @@ -2690,11 +2700,7 @@ static void nfp_net_cleanup_vector(struct nfp_net *nn, struct nfp_net_r_vector *r_vec) { irq_set_affinity_hint(r_vec->irq_vector, NULL); - if (nn->dp.netdev) - netif_napi_del(&r_vec->napi); - else - tasklet_disable(&r_vec->tasklet); - + nfp_net_napi_del(&nn->dp, r_vec); free_irq(r_vec->irq_vector, r_vec); } From patchwork Fri Mar 4 10:22:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12768919 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 003E4C433EF for ; Fri, 4 Mar 2022 10:23:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233430AbiCDKXp (ORCPT ); Fri, 4 Mar 2022 05:23:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232757AbiCDKXf (ORCPT ); Fri, 4 Mar 2022 05:23:35 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2115.outbound.protection.outlook.com [40.107.243.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1DD34FC78 for ; Fri, 4 Mar 2022 02:22:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=daBWISHLG8JP7SiEc/cF6rHpJRoWEPt6IR6J/TsioUWbjWvUHHCHHYYZb7lCV6zvOlCDdnJlvf4MGKkAVJhw96cVIW9B7aNR0FEk3y9XfT8MOr8luAuPUGo3UIvbmC3pO/zG75klOCSP1bGKmiEdCndctaFPLAP2CslLTXBxcHYWNJezJDymfrP7ekvm6aYNlc4k6DQxLn61muY+vBTJrgqOf7uE8BHgqu3u44jaJXZ3Zg2zqaVO4TwLXT1qgpNq3pWAwQM0Tg7E5wDxlGdyM1pHq24A1QimHXxqrE3A4cUyp44dWCPgOtSov5EH/2ve3z8CK4ve7+vKgrY09TgAfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hU0B925EyjRBDADBs2ULn9tGrWSG7S0lYlorGoKvivs=; b=XvxeV3DjPE2Bnx0wh6IH1AdT1LWqFLXxjdg4HmlIYDeFlSdlRhAByBklG8oWO0z9KqoOHhtpPK8lQVNZU91LIlpOG8czdKzKCbSaq3fSDWjj4q2vkmOICbL2rpiGnaO8FRyNiE5p+ChD11NAJR25v385R0L3VghnjoFVH1EDGweUWLhHm6PWWDlS2nMoCeijE2N3aznJ0PZqJxyvgIov0iZLUUcTut1RwG6GuUo/wcbNoWHITQuzsh5dMopqwOO4QcXI5Zz1rJRLfo6/5tXqHnhTPDXY2zn8YXkQ9CkNL52mIL5rm+joJp4vw9el1HV0KCnQl+w9KxT5hMXRBqErBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hU0B925EyjRBDADBs2ULn9tGrWSG7S0lYlorGoKvivs=; b=D7bR4mDW/SdXM2wNdnA0xgoT4w/s0HxtDfuGnb7rJhf5kqju29CiDAcydqBUwPAl0iFGoisqw60uv8bz7mHkw07oLJZrJJMbGg7dk+GEvOnaEV3DU5VVEbh8/2qKhK9FI/EHtNizNri122F7PmLWIP150desuyimhurK2HNH6XU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by BN7PR13MB2404.namprd13.prod.outlook.com (2603:10b6:406:b7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.11; Fri, 4 Mar 2022 10:22:44 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e%2]) with mapi id 15.20.5038.017; Fri, 4 Mar 2022 10:22:44 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , =?utf-8?q?Niklas_S=C3=B6derlund?= Cc: netdev@vger.kernel.org, oss-drivers@corigine.com Subject: [PATCH net-next 3/5] nfp: xsk: add an array of xsk buffer pools to each data path Date: Fri, 4 Mar 2022 11:22:12 +0100 Message-Id: <20220304102214.25903-4-simon.horman@corigine.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220304102214.25903-1-simon.horman@corigine.com> References: <20220304102214.25903-1-simon.horman@corigine.com> X-ClientProxiedBy: AM8P190CA0012.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::17) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 633e3909-a068-4906-ee6c-08d9fdc8ebf4 X-MS-TrafficTypeDiagnostic: BN7PR13MB2404:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UZltEl/VApJy5JKWKBQw8+ViuLaWjF8mlxKQJPSrG7LeNQ1GGG2vFyGy5+HO3Me129FYvIYy4gkXVum6SQXvRgqfS4seh8ThgA1SGt1L0OrkHyvanb0w967lS0QRI4b2gDYCRoUfb1HuoTTMpTLFW1GCKqvvwO5bH4KowMxBFO5ydL/UUWPOw2Oc6wllpXW2q3XfvMTnBgjw41x5oN/nHfVhZaJzKRDHyuxR0CTJ5T7KKJe2dNRde57O3sUgtB8fLVwT+bvuhP32JYsmJXcGDF/R2JhhtLSmMPJ8w0SgIhYMku93g60mOF/l+Ei60sMFjkfstguGjjL+ey56eU3NR4obvkDXcaHLjK7PYAQsU/4JLzq0YD7Dwzh7wwxYEMttkMVOcvnmB3CcJkgvHoDWGXzdD3rfr3g8n4pUqEoYoY8dda37Z42ubngHbpENlb/n6H3t53WpcLq/xNbj7ruRPRIqhyE5voQakOUT1X2BZB1Lto6BOjCWXoJiidFgXn0drnf6NWBp7bXXfVYtIjnkCRh2K1bkCR4VLfC59OpMOP6dc1URNqfiKEQi/gQfmxtIGH/3fMvxx0pJizor2siQ4O9Nb+fYoWUYDDU7mHCZuTzUM/c3tWgkXeT2EaOxboVyC0bb9Nci+QcAZUgI3lNahA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(396003)(346002)(136003)(376002)(39840400004)(110136005)(316002)(86362001)(4326008)(8676002)(38100700002)(52116002)(66476007)(66556008)(66946007)(6666004)(6512007)(6636002)(6506007)(6486002)(508600001)(8936002)(44832011)(107886003)(5660300002)(186003)(2616005)(2906002)(1076003)(36756003)(83380400001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?5aG75+7cstYYixhawTj26JNAi1Nq?= =?utf-8?q?Rd3V1r/a4Ul4+71dPWjPDhZQzekO9BVYf3AqsBsBLVunT7AQsn9G7Wz4TKMuCqL5K?= =?utf-8?q?O7ftugR2v1y/9ftsexsr53uxOpQRrfobl335FHdqehVGsHi7M8RJ+vtRo7uSeeUrK?= =?utf-8?q?cC2epuXrIcvFqILq9rLyX0OeMWCs6G0Igosof1QMxd1pQsRDktUouiFmWptTLSLoB?= =?utf-8?q?68zfCEROoQwrwDIuu00iPes/pyXcj3N7ygV+7XZHQAs+cysZvfZGgHxHqLp1YTgQC?= =?utf-8?q?403rNX3lziVJHGZDhjalw7Lt6lsIBAOKdgIOkXty7kfCwZWj/hDjIZYxGb3RlNpYN?= =?utf-8?q?lnUtQfGgn96ek48cDbb56pzrAh2xW63GUd95l+Flm62Q42H05XwZpbE4gUn+fxZYa?= =?utf-8?q?Lk9spCF1A9DUyFhygxOGMZTZsBNKDZ2tI9boU8g32uMKDCu3029q92zKS7vDqOikZ?= =?utf-8?q?rTrYbgcTZ0aan8EPgzo0kCnJ7F3zew2SMoyTu/N57Tzb+rAYfuDhaHHsUapQy677P?= =?utf-8?q?ytckfYGGnL75xcb3JjhzLoIj+HhoRLNDgxLVEixMV5vTDALwIfZhrX6Bad2C1zqj2?= =?utf-8?q?AEM4tC4DRA6p/GQi2X/N1GLWmz0r+ThLG4HhwX7Zh0CWnVMobWp64TEokx7L61WAD?= =?utf-8?q?Q9vRqUu7CjD63oVA3L8mmh9KsNF++VxiCf/wOB8O5jBg68Fg82WdIVS2nxExknZT5?= =?utf-8?q?tDTptLZNxJpBYa14ILttz9T2/GRR3iDAC7PKK4mrm/ge5X6domHicortM9vRCxM6p?= =?utf-8?q?1ZhN8/19Db21s7/CuFBg9dPGgA2K7ylltl/bUJZgb/rbSDlR0qk09dYap/1AXCbmx?= =?utf-8?q?SdhHXK1yjQCEgyQINkzIjKaPmuuZYU4KOJtYbP4onNb22rarBZCFRouuKPGWrFeaZ?= =?utf-8?q?vaw02gf1ox2JLgFowUtBEl25/Q9CftwKbAaMW2yGPkjdZ6ryCSxV+wKrNOyjAtpEa?= =?utf-8?q?o+MWK4vq5fPdoh2NlDfZelV5QzAD6OCOLGqbDBmK5EoU/llGlcqt0y9rXiVA+6IQY?= =?utf-8?q?pkM6zzPlF4Hcbvzjv0WB3RVd3/yvfjYGPlb9U9QOWMRknZZo0mXGmycTLO4qVBBz/?= =?utf-8?q?DBx0DhIlrfJMADFxMjesAMPkbhrBb+JYSXPgZ1PDC9uI3siq8nIFwqJwVtAMoBhlM?= =?utf-8?q?ieSlXbr9LVRVVOUwP4fA+BkoZ76bRmiiXyHzin1x3nBZbLVi3Nc422xvJx+sUDK9L?= =?utf-8?q?0RncSG9FtMg2hIJYewWT/vQZyJOC7pMQu7crdfwSxePSxY2LF5KSYZFpjEwgKnaGq?= =?utf-8?q?uTXfy78QLHwt1q97MGctLzGyVzOWEJ2nUoIRBzbI1Hocli8ZmG5HjyGIDjILEOVWC?= =?utf-8?q?DsXFoqoV7PKp5AY1VsEcWts+MmMMR/0kmNVmRWfLzXkYIK59F5xNLPFTHPuDYxszE?= =?utf-8?q?dOlUUU4i/v2sMobgUnnAtJKsuRMzp4Wa086q9/5jjCl0QJ/ESzhe794ZWIagNzNxc?= =?utf-8?q?T5UXUCS6XkxoVfXnQxcvrrTQ1QoKOgkM6ytGDc18x0lORAvz9eTW3ZCN8GPsSoON3?= =?utf-8?q?+ZU+Y+ykJr1k/6oWFF2jCuLF8Sxh0PCK8r3Ew+p4vAt/9GTy8jFceQ9bM85tEAzfe?= =?utf-8?q?RpqeHKmVoIs1TSUbkEtqPgjOa8JZPLvrHN3Cy90DyMXNC2Ikt5RETvknRM8GBpPCQ?= =?utf-8?q?pVDEyVc02WgwrVcxCBhBkgJRR/xIzD6oQH1cOa2A6oDo0d6X9M/8s=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 633e3909-a068-4906-ee6c-08d9fdc8ebf4 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2022 10:22:44.5025 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZakFWGlb8BAjgttDwr6rXH1450QU63o61qFxlfm+zoLpad3OMfpuqZqJNMrvzEycyfpANWgE+Kz1rygmlVn1/nlmlB3oIUB2PQINrPtjEoU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR13MB2404 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Niklas Söderlund Each data path needs an array of xsk pools to track if an xsk socket is in use. Add this array and make sure it's handled correctly when the data path is duplicated. Signed-off-by: Niklas Söderlund Signed-off-by: Simon Horman --- drivers/net/ethernet/netronome/nfp/nfp_net.h | 4 ++++ .../ethernet/netronome/nfp/nfp_net_common.c | 19 +++++++++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index fa40d339df8d..12f403d004ee 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -109,6 +109,7 @@ struct nfp_eth_table_port; struct nfp_net; struct nfp_net_r_vector; struct nfp_port; +struct xsk_buff_pool; /* Convenience macro for wrapping descriptor index on ring size */ #define D_IDX(ring, idx) ((idx) & ((ring)->cnt - 1)) @@ -501,6 +502,7 @@ struct nfp_stat_pair { * @num_stack_tx_rings: Number of TX rings used by the stack (not XDP) * @num_rx_rings: Currently configured number of RX rings * @mtu: Device MTU + * @xsk_pools: AF_XDP UMEM table (@num_r_vecs in size) */ struct nfp_net_dp { struct device *dev; @@ -537,6 +539,8 @@ struct nfp_net_dp { unsigned int num_rx_rings; unsigned int mtu; + + struct xsk_buff_pool **xsk_pools; }; /** diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index e6a17af731ba..abfc4f3963c5 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -3302,6 +3302,15 @@ struct nfp_net_dp *nfp_net_clone_dp(struct nfp_net *nn) *new = nn->dp; + new->xsk_pools = kmemdup(new->xsk_pools, + array_size(nn->max_r_vecs, + sizeof(new->xsk_pools)), + GFP_KERNEL); + if (!new->xsk_pools) { + kfree(new); + return NULL; + } + /* Clear things which need to be recomputed */ new->fl_bufsz = 0; new->tx_rings = NULL; @@ -3312,6 +3321,12 @@ struct nfp_net_dp *nfp_net_clone_dp(struct nfp_net *nn) return new; } +static void nfp_net_free_dp(struct nfp_net_dp *dp) +{ + kfree(dp->xsk_pools); + kfree(dp); +} + static int nfp_net_check_config(struct nfp_net *nn, struct nfp_net_dp *dp, struct netlink_ext_ack *extack) @@ -3395,7 +3410,7 @@ int nfp_net_ring_reconfig(struct nfp_net *nn, struct nfp_net_dp *dp, nfp_net_open_stack(nn); exit_free_dp: - kfree(dp); + nfp_net_free_dp(dp); return err; @@ -3404,7 +3419,7 @@ int nfp_net_ring_reconfig(struct nfp_net *nn, struct nfp_net_dp *dp, err_cleanup_vecs: for (r = dp->num_r_vecs - 1; r >= nn->dp.num_r_vecs; r--) nfp_net_cleanup_vector(nn, &nn->r_vecs[r]); - kfree(dp); + nfp_net_free_dp(dp); return err; } From patchwork Fri Mar 4 10:22:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12768917 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F3A8C433EF for ; Fri, 4 Mar 2022 10:22:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233080AbiCDKXl (ORCPT ); Fri, 4 Mar 2022 05:23:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233148AbiCDKXk (ORCPT ); Fri, 4 Mar 2022 05:23:40 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2115.outbound.protection.outlook.com [40.107.243.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3F7752B36 for ; Fri, 4 Mar 2022 02:22:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XTW7UoizVqJ+NjlehBAQ0QO5zyiTJHZklNW4Bh4HMC/RQWsMVWou6oWIm4X+eDBc7sJAA+NoDj8oceEk7rxXSqa858+a14osMtq6vnp0l3ML/sCbs+K3HebQ8dNx3gQCUWlx85cv2TNFO+3fGNWlhPcvZomd0QcIjcFPqbWe4ivMkgFzFETmXxocg6l8fPivbF9hpZwKcLVX2bSV70hoKOuF/h+ygbrc5qHzc4f17hH0S4ZU6OShoblDr5ApZ7Dt8lyyf/JqK1lqCtlmN1mh/MBP4alNzUwhE70qkM/g3i4mguOFbKlWGGISYMeDknYbu5MagPlh2YxUP2AB2CJpXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LheVwKJAJHbw8rOI7CD2UWqC1OpvinT+TJjCYJC6knM=; b=BvHslYA83J9GWKqaYZ0jzQJaEFSqPMGL/RBb+Cb5cM7ny6/Eb/SO987N5U9IJfHFOEemHj5v7NTf+NEL3Rscu+M/eZbY4ePhwmajpCtsaE/UACqYpXAlIw37NBSVZSSxk/hwyiza/GRqcWvntdolMU0fOKXwjjVxdJKg2tZGIGsWvBFSIoXCpo0wKzIsypxq7SvLQXvPMZkOF0434PdYX6VhmQWPGVSPldfi1E27Ol2jPMqb01mhRVxcPHknkkQkWAbX0I2yGGv8HH7ZD++eLfjD2OYXQdTBTaBidy7INAD2MgClSd8NGN+DDcNTQbNEH+w0BXFowzBRMRyCLl9sqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LheVwKJAJHbw8rOI7CD2UWqC1OpvinT+TJjCYJC6knM=; b=Xigy+drBDp8IuvgmxlglYNAV9H6kK/ZorBUAgMWWeVXWvxuYdZT1V3YV2tjTa8nNVlUlg/uT+lqty6M0GfMVoZRP2PAcDsZ/Vwq9JuRcK38Uje+wWdUTjVj4+6OIRsNs2vp51CkWcWvoj4qfP7jNQBPnihThX7S8kcMjclUEWCA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by BN7PR13MB2404.namprd13.prod.outlook.com (2603:10b6:406:b7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.11; Fri, 4 Mar 2022 10:22:46 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e%2]) with mapi id 15.20.5038.017; Fri, 4 Mar 2022 10:22:46 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , =?utf-8?q?Niklas_S=C3=B6derlund?= Cc: netdev@vger.kernel.org, oss-drivers@corigine.com Subject: [PATCH net-next 4/5] nfp: xsk: add configuration check for XSK socket chunk size Date: Fri, 4 Mar 2022 11:22:13 +0100 Message-Id: <20220304102214.25903-5-simon.horman@corigine.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220304102214.25903-1-simon.horman@corigine.com> References: <20220304102214.25903-1-simon.horman@corigine.com> X-ClientProxiedBy: AM8P190CA0012.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::17) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2f694f45-e2ee-4d90-1d3f-08d9fdc8ecc2 X-MS-TrafficTypeDiagnostic: BN7PR13MB2404:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 80++RJOUVH++URZqJXMPPTIH5SlvyQ66qj1XDDzvyUbMmmu6pCCz3lzmbfOl1FdrQxzgn3ejU/GvKn+GPxICyYW8Duc5UDe/6it7fRHUrs/GVXulikit8ZFjuM4+rlnRjUtuQ+mST6xb3wuJR5+vBwfnjbB3L9KvJoRgV8oIZqm4rwwtZUXv0SyR2lsICnefvzv+RC1Jny984h/iAfiP5aVxsbPHMDW0D2BDnNgSuuEUn63DuESJ3EI8yA6KrO/5wO69KB1EVPEKk0qrVoNzXT0+/JaJHBALUPwrS3t50EKUW+KumzzvmcIyyWwkCThMfZ2+IEzkYBB0bZYhyMAxhzyWy19ZYYA7iex79UubGWG6fNkjHANTIroLeBQTlIkGjUoHLcS/LVjoCTIK+C+KCjXhA8oyvuzouYEZKP9NQwYsSVbGTsYRKgxvowOWmbMFwBDBuSThftS8rhYCF9Ux46qO68KSdIAHqZ8punebT3L6JF9bUysA8evFjEYGE4De9M3lsFGZRCWDg5rkj3i1jM2/spaFklDiytU6UjihWe6RkjEoCNIYUG1hV1MXBTeuW0CU2CJqpxwYAlpgtE07/ZuSP0gsOwANuml60FDcJTYRK8KMoJLadR9oWiaK8pGx0I/ozpmeE7DS6gjzEBcnMw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(396003)(346002)(136003)(376002)(39840400004)(110136005)(316002)(86362001)(4326008)(8676002)(38100700002)(52116002)(66476007)(66556008)(66946007)(6666004)(6512007)(6636002)(6506007)(6486002)(508600001)(8936002)(44832011)(107886003)(5660300002)(186003)(2616005)(2906002)(1076003)(36756003)(83380400001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?hqJdD6cnAlWKYwRzdBDMHYkUrPVE?= =?utf-8?q?2NNbQ2HZBWy/N18HnO9bdlj40Io/6382g99YUg1jk2ZwmijwXj9AxETxqhFaDAWGh?= =?utf-8?q?XtVq0ASzaYEKXTWHSMwl+uPe8f8W9ce9JAo1jypSQ8Euu2o0TI+Uc40//o5dWuIAy?= =?utf-8?q?0jK5sldqdE83/dMoGidZiv5QhpZZ0a6GxDwdfmp+n/LNBTP6jvILIq/IhGgZ9gBrt?= =?utf-8?q?fj1bj162XKHieHfjb3q4UnjdLsmSJIAPWYaTk0EI8lKU4dKb7GHd/+W507PhVzuUV?= =?utf-8?q?vhShBmlvPCkBIA7pLnVIKVIWlocxZ88qQAj64XSBzETgLm+wb/geJRSFDwB7PjBDt?= =?utf-8?q?b0j5N5KFQIqTMnW9MPdxii3ZNmJ7phH9zrDg78dcbPiuhuux6VB7A2w/4ruUDFk08?= =?utf-8?q?3dnU7uaVvIK+ME1Mx+MOlZIZMRk4yRx5AsmdRGG8ukMyVTLOnXJQDRSq1G/KCiQEB?= =?utf-8?q?RayYMEf+r3pxg0s6sExAiGiwJqLEvZo6hbqWbIcFKUPDuuxnKaTfrouNVlzJzP9VG?= =?utf-8?q?tUxaJMR5wuDVRbbRAfZ77o5jF3Q6mwBwK/RNyw4mpoW3HdurWEmO/IRDWmXFHTCdJ?= =?utf-8?q?caxK3DUQIj0ttb8U72PcWmd4ENvuAxfh2L8c7h/xsUk1ddqntFGQSbSMNgCP+nepm?= =?utf-8?q?AL0jqI9Du3sjfodSrQEL6gAtuNirky3tGLuWwt/MSY0/mDtptQFAEUJhNx8xPWu3/?= =?utf-8?q?EUokErVfRrLtRm/HG008og02W5t0W6h94IZF7pOD84JVKYRzvkvG3B3rETXXh91GD?= =?utf-8?q?70yUxyR/e5DTq62OtTeHxrFF8pSet612Yle91J7RAwE+2vZW7FxccE7F/F9d7cp3D?= =?utf-8?q?f5mHiq8N3MqhaY+a0lFKOCX5Oo3KYQjtmhd6+cSNs7yBAReixdlKVsmqWPam0yYFA?= =?utf-8?q?LD17KVBTvIzF+LxsVFVdQryopKCxKUlWcVn962b39XhmNwa0lfnwjEzHvNfZm3JQ/?= =?utf-8?q?WTrXkmtzKFlPVABMgrVM2Io9C8Mt7u70PWP7TzByDDglptAk1RwOkTesmMXYSg1Ys?= =?utf-8?q?cwjkBXtvsy5VR0PtpNLRZrkl2DaRLDUI1c+9+Kk3zLQtQ5DOBM8vBBX8FBAC1WHZA?= =?utf-8?q?6qXDrWpbnI7K1fIAPHDxj/KpvlKHd1RS+VFyenncVpHHYWJZP/mZ1eCoAmMAfo8YW?= =?utf-8?q?0KrHW6n4SMqpemNMJswBsYBECZ+Kmjwn19nu0FbeKqTfBW4oZfx9jGxC1PcaW2zKj?= =?utf-8?q?r/mRyHeEns60KtHZqFOr9aJnxVcv/YKW/hwb+0AI3dovCyMD5u8CaRBg/+eFEB+rG?= =?utf-8?q?09P+3Lugyq5NhsQVBHO9VuBy/dq5dXH8XEREo3uo0I38bYR1RL8hAFwNmKVcXuNL7?= =?utf-8?q?Xj3zH7gtveSmzeUWE9pFes0oJ7Iacxc5Z8WUSTjpuhaDdz8wOJNgfgZvXeiusmpso?= =?utf-8?q?+HQevBthMcYi/Gx86ieNE5ngoVZjy4LckDaOs67Pl/JlF56ti2lJhRHuK52TfnUd5?= =?utf-8?q?IG8QcBLL4j2yCZT5zwEzZWz2Y8SxzO/b0xv83qq59/mcsJvaahN+S2nd6HzI4o5Ui?= =?utf-8?q?Q5HE4xX9pvLO1ACPmCk+cjNtF5zGhkd83YEOb4cdOXtqR2P3xYL30l+QhlKg2REda?= =?utf-8?q?e79WEkNe/C40EPuervdpb7+VJO00BsNJUUY+KVl0JpaDnV1CPTTXHMBMcw2+GeG0R?= =?utf-8?q?MHOfNwgAtCSCFL9RaxxlW5ZiZ/zoV1iHWM5a3MGu4fvCLay7RDQLI=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f694f45-e2ee-4d90-1d3f-08d9fdc8ecc2 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2022 10:22:45.8629 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CRDvouFA5FL3CqQVDVr+JfKLj1sBi2EL1HXKerWxb0cvCBcN9VXC761ommo9Jj8qm3Z/o+dpnH9AMiwyow6n9pWy7rbiJHpAUp10QJoQ41Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR13MB2404 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Niklas Söderlund In preparation for adding AF_XDP support add a configuration check to make sure the buffer size can not be set to a larger value then the XSK socket chunk size. Signed-off-by: Niklas Söderlund Signed-off-by: Simon Horman --- .../ethernet/netronome/nfp/nfp_net_common.c | 42 +++++++++++++++++-- 1 file changed, 38 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index abfc4f3963c5..c10e977d2472 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -38,6 +38,7 @@ #include #include +#include #include "nfpcore/nfp_nsp.h" #include "ccm.h" @@ -1338,24 +1339,43 @@ static void nfp_net_tx_timeout(struct net_device *netdev, unsigned int txqueue) /* Receive processing */ static unsigned int -nfp_net_calc_fl_bufsz(struct nfp_net_dp *dp) +nfp_net_calc_fl_bufsz_data(struct nfp_net_dp *dp) { - unsigned int fl_bufsz; + unsigned int fl_bufsz = 0; - fl_bufsz = NFP_NET_RX_BUF_HEADROOM; - fl_bufsz += dp->rx_dma_off; if (dp->rx_offset == NFP_NET_CFG_RX_OFFSET_DYNAMIC) fl_bufsz += NFP_NET_MAX_PREPEND; else fl_bufsz += dp->rx_offset; fl_bufsz += ETH_HLEN + VLAN_HLEN * 2 + dp->mtu; + return fl_bufsz; +} + +static unsigned int nfp_net_calc_fl_bufsz(struct nfp_net_dp *dp) +{ + unsigned int fl_bufsz; + + fl_bufsz = NFP_NET_RX_BUF_HEADROOM; + fl_bufsz += dp->rx_dma_off; + fl_bufsz += nfp_net_calc_fl_bufsz_data(dp); + fl_bufsz = SKB_DATA_ALIGN(fl_bufsz); fl_bufsz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); return fl_bufsz; } +static unsigned int nfp_net_calc_fl_bufsz_xsk(struct nfp_net_dp *dp) +{ + unsigned int fl_bufsz; + + fl_bufsz = XDP_PACKET_HEADROOM; + fl_bufsz += nfp_net_calc_fl_bufsz_data(dp); + + return fl_bufsz; +} + static void nfp_net_free_frag(void *frag, bool xdp) { @@ -3331,6 +3351,8 @@ static int nfp_net_check_config(struct nfp_net *nn, struct nfp_net_dp *dp, struct netlink_ext_ack *extack) { + unsigned int r, xsk_min_fl_bufsz; + /* XDP-enabled tests */ if (!dp->xdp_prog) return 0; @@ -3343,6 +3365,18 @@ nfp_net_check_config(struct nfp_net *nn, struct nfp_net_dp *dp, return -EINVAL; } + xsk_min_fl_bufsz = nfp_net_calc_fl_bufsz_xsk(dp); + for (r = 0; r < nn->max_r_vecs; r++) { + if (!dp->xsk_pools[r]) + continue; + + if (xsk_pool_get_rx_frame_size(dp->xsk_pools[r]) < xsk_min_fl_bufsz) { + NL_SET_ERR_MSG_MOD(extack, + "XSK buffer pool chunk size too small\n"); + return -EINVAL; + } + } + return 0; } From patchwork Fri Mar 4 10:22:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Simon Horman X-Patchwork-Id: 12768920 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5256C433EF for ; Fri, 4 Mar 2022 10:23:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233431AbiCDKXu (ORCPT ); Fri, 4 Mar 2022 05:23:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233285AbiCDKXm (ORCPT ); Fri, 4 Mar 2022 05:23:42 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2115.outbound.protection.outlook.com [40.107.243.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CC9E4F46A for ; Fri, 4 Mar 2022 02:22:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DyRi+9VctJ8xPlgVCOFIZN7nOA3gmdMUTAeFTyHB5nZEPkDCRoGstm0k6V8vkqfgDyHVGog4YfEWoru1o8v9XgAjNBFS7sbKjc83k+kQtlepCTLuvBCFuilIMgp5+ijFLgyM8wI6WlUHG3CXGigyzvWblFkIvuGuttCWtKOxubpJW3YSiDFXVda3GrYDGkamNNfvb/F+aBIur7jyIcluwkaenc+fBHI1vV3UzHnQ7NhKEzlnmtRHrUrflaWKMwicK1z3I/qCvv2Wf333pKaLiiCpN+bttWccTD0aIoNk7sv1/cQNa40khR262HKlJYWawECNRIUgVIXbRxR+Busp9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P2pFNl6FkY9nEJ6Ok35fkZ72wDrtMIzN1FDEfNJTe3g=; b=lconc1cS4cbbOMZH8dSSuF4NkLRuhv4QhefyM4UlBJn8+g0V7+UOlO90O9D6b/rDn/vfdVCC0Swuneb7Ogv0XKC4DmOImfHr+oklJN69s49NJAOuTliUjcDqn0jZYQiwckF6Nu9RTh1l+h82e+bmkz5vR1JJasUEICHeR6Xt8zSOikNME1Sd4KBAtyeLRBaIutXP6EL4l22ycdVDJWdeeFg/ViC9gbWF1xgzQAXc5KENEZQdlp+p4syjfIn1NK81YXqd9qAnXi6ciuC/ZT6WesGrln1j4DMc1V+kDnGF05rfWiQT6EwiIAf8FtAY1AkGN+zDjwcTYmB/IKavWxkgpg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P2pFNl6FkY9nEJ6Ok35fkZ72wDrtMIzN1FDEfNJTe3g=; b=X75BQt83NAAoeuZA+UwbQgHvTDmKFWOpBVxdkocCfXAdtDmmb4SkYtWG2SxjV/c8r4x36+B5IFaIjteN6EQOJNm+dTDgtpQHp6UM7q5X7Nmfa1fCm8mV9kE2z/cYnTVx1i3NyPHaVM8PdMVRMI3pE/IvsY28aXH3RME3+fwkPis= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) by BN7PR13MB2404.namprd13.prod.outlook.com (2603:10b6:406:b7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.11; Fri, 4 Mar 2022 10:22:47 +0000 Received: from PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e]) by PH0PR13MB4842.namprd13.prod.outlook.com ([fe80::d27:c7d9:8880:a73e%2]) with mapi id 15.20.5038.017; Fri, 4 Mar 2022 10:22:47 +0000 From: Simon Horman To: David Miller , Jakub Kicinski , =?utf-8?q?Niklas_S=C3=B6derlund?= Cc: netdev@vger.kernel.org, oss-drivers@corigine.com Subject: [PATCH net-next 5/5] nfp: xsk: add AF_XDP zero-copy Rx and Tx support Date: Fri, 4 Mar 2022 11:22:14 +0100 Message-Id: <20220304102214.25903-6-simon.horman@corigine.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20220304102214.25903-1-simon.horman@corigine.com> References: <20220304102214.25903-1-simon.horman@corigine.com> X-ClientProxiedBy: AM8P190CA0012.EURP190.PROD.OUTLOOK.COM (2603:10a6:20b:219::17) To PH0PR13MB4842.namprd13.prod.outlook.com (2603:10b6:510:78::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6e802f7a-920c-4d15-805f-08d9fdc8ed91 X-MS-TrafficTypeDiagnostic: BN7PR13MB2404:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Qk52Gnq4jEvlxj7UuGc7DCadw47Ge27p+/78kygH6MwiPjEjt9okRWZuhdtDaZIbQ9l0VU4BeKyZjjjXLQygyeMW3nOp+perHxHfm8teGGTdJBFCS9j5d5sr5pSCkYCODftx89j4mKV8SgRzw2eMe+og132CdLGjobyeWXYD7EK6aPSZcq/9UzV1usOUKh58fvVmISucLWvw7DAM+YXG7V6aGpPrGiJ/yyTsJjFxCEI9ZFX8JXa+0e23X75uz1WZOB8Ir8DAJS5n8vw4mCqIeB24qAIGEPKfjbm00hsHgMyjkxnwKlaCLKlmWHFcJzhg3IDnuMKcVCMwWigvzbdorXQWKLNVs4aAj2CdJbFfk1l/EEUhNQo8R/vvARtyE0Hr4X5u2zppHwL1X5feiV9YIYhdP4DedAKphQHDHACRYWsH/saUS7Qwp7wioT2j8Z646AGSHCtgtvWLlA3SCLg+RgItITWD2pjspk1YOFOj5lYUYsa3RIqWbsfazhz6xDBRZmB+rGsrJX8WASiTmtXKNvYzM2M+/nI83S3fMGdQLZyuAtcFc5KqiC32BmepWxBdFzxHGeYXOKktMT7KbZycgQ+WLZ8+EQdMYT96gEh658IZuueCjGumA1seYCm3D7JXIcY/tgmlK2Lhl6hdNFmrIw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PH0PR13MB4842.namprd13.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(396003)(346002)(136003)(376002)(39840400004)(110136005)(316002)(86362001)(4326008)(8676002)(38100700002)(52116002)(66476007)(66556008)(66946007)(6666004)(6512007)(6636002)(6506007)(6486002)(508600001)(30864003)(8936002)(44832011)(66574015)(107886003)(5660300002)(186003)(2616005)(2906002)(1076003)(36756003)(83380400001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?EM+s0jljzbKsKvqCK7nDXdkkjk5x?= =?utf-8?q?UsJIf6AO0hhEFIgsG+2V4FYgD/JoOfzu+FD9/d51ASp5F3FC0BGgE2TSsSQlo89sS?= =?utf-8?q?Bl8JfWyA9/iL+SZHfiktMf8c4SmL04QJAZkbVdMwwYDW8J+8Pk+axxsiBjDGTxYR8?= =?utf-8?q?7M9h1nIRSkJYd8LBsgG+OV3JGNyU5KRZh9XIv56e9letvB/L5cX8D/SVdxe4180D1?= =?utf-8?q?Qn7FZrXelhecA9r8Hu9vTAPM4VvCGDDdqE7TJGD/P1KCIw7DR3lEpIlBztdb3S3l1?= =?utf-8?q?g2aqKuK5nTktpEnXf5gz4w/XBWm2B3JQIm46GNVRbZqqfWgrDoq62WFUycdWiGGUx?= =?utf-8?q?MnFQ7rmMrA20ZufjI/rg5kGBmcM4UXpMMAyh0/YSY5ncoCtV5PPjth2ImKOQ/L9nC?= =?utf-8?q?BAa/pZFrVUQVmgSlcJVfY2eU6LoW6iwB8CqTkUj1pJmD5JrliftCxlQ/CjZlkkU1Z?= =?utf-8?q?dFk5L/4R/wnLkTkaJf8+4gb2T5RbNorktO2gXrCG/S2gDh+eLmLuRBBgeqk3n4FKb?= =?utf-8?q?NG5npFR6nwrKqrCRjOYhH7GLmP27d0biA2A8ijVktW1Z1qmhHi6uYwwTVXAU0gLd5?= =?utf-8?q?PVmi0NALSdmwgzcYi4Xpk7AbILcRwVU8CjPJnW753PhX6QSjvowuUUyxxNi5eJxdy?= =?utf-8?q?J5A5nzWvgS6J/4AA5WiqBMnk9xK4FhNBCXA8MI6UT8oE3Jl0LJB9ko75558h9+TJu?= =?utf-8?q?iWHxpq0BkUTnjW/Nm66RxnXWdO5M38TH9i0BuGgrobJlaaYu9seus+4FPhxToVhvJ?= =?utf-8?q?fHFMFiUiWAA5TVQvhL8QUv3wznDSi21OQCVXSlIHSo8G2gQu8Y2aDP6C70Fdnee1R?= =?utf-8?q?JR1B1x17ECa1TRno9Vw1+/kT8jJQ+Gn9UENTH7GgnblTlxlJoy5sYErk0YSmUbdvX?= =?utf-8?q?d/tkTw4AAGNvov5phmgWEpX/eN/aZIpdXi84PYcYhzgRCLMTejwuTNAUOSOrj+5HW?= =?utf-8?q?geGWoLWfDuxu535bS4lWWfARCwK7NKt2tYfBamkknCJibdBd8c00HXR8GGzqx5nOQ?= =?utf-8?q?mtWFUiM3ia6/tpslv8iESxZn4etNApfYT3eLp/aIGBUT2YeONRrkbxKwFm/UntJGJ?= =?utf-8?q?HIBfXLaJU+WTegE80kGkAIU5FZkt8VXR8DbuaFsn6PkbZDdB1bYxMJkIfpiKT15kB?= =?utf-8?q?C3X6TflZMtSjgCWMdaRK+ljIvl56eDy479oy/+pP7i+7B94eHUtzdiewGdnTKVc5F?= =?utf-8?q?9PUrpW2kNLhHGbNe7YQiZP/cwLzRUQxl3Jr9GeXwR9qvsB+zYux02e9uBpWbTykSe?= =?utf-8?q?1hy8k/Bc1n4jyMc97S6Fm20qEIEAhVGbdrx2finaAJ+mtgzOMVqsOnwHuI+OYbZ8t?= =?utf-8?q?xptXREOfhcKdGA4g1IbEmn2BTkhV27yEU70KS31CvuMpKM4VjlpFeo86TazhZ5Uot?= =?utf-8?q?zfFTW2RRFsphPLEu9WPMhU9UHKKjZGy6+OoWbexHB2PAy8cAf70s9Ezv//8xhbEYk?= =?utf-8?q?g7o1ul4GqCkwEIsGZ/UGEeA1Wq9XGd8ge5T8emuS5OV8AeWPInM4ZGk82QCZtc8qx?= =?utf-8?q?oWTSkwI6BeroYWjAjmiJp60v9cOg0vj9cAyEP2zAtJWbOBmrBTjYhdJTcm4eX8EJZ?= =?utf-8?q?ydbt0wWQ0b9LHDw2TUQ5EQHgDf+ZAhwLUd/65WnMpX3S1R8Nn4E8PPt+z0CeOqrXf?= =?utf-8?q?meaePkgy1h5hXstEPOOm0vKCBs3eQeJTM0J3Khn0R9Q2rF333DALY=3D?= X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6e802f7a-920c-4d15-805f-08d9fdc8ed91 X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Mar 2022 10:22:47.2689 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: X8UTu88mfTtof1XWL1nbPhr5Bh6OAYrzl1SJA9aivqSzI5Tp+XcsnXU2o5cLTQYTmL9B2meT7YcleC11XmAC+HYLoAliGPYT2FSVyGaPJYM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR13MB2404 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Niklas Söderlund This patch adds zero-copy Rx and Tx support for AF_XDP sockets. It do so by adding a separate NAPI poll function that is attached to a each channel when the XSK socket is attached with XDP_SETUP_XSK_POOL, and restored when the XSK socket is terminated, this is done per channel. Support for XDP_TX is implemented and the XDP buffer can safely be moved from the Rx to the Tx queue and correctly freed and returned to the XSK pool once it's transmitted. Note that when AF_XDP zero-copy is enabled, the XDP action XDP_PASS will allocate a new buffer and copy the zero-copy frame prior passing it to the kernel stack. This patch is based on previous work by Jakub Kicinski. Signed-off-by: Niklas Söderlund Signed-off-by: Simon Horman --- drivers/net/ethernet/netronome/nfp/Makefile | 1 + drivers/net/ethernet/netronome/nfp/nfp_net.h | 31 +- .../ethernet/netronome/nfp/nfp_net_common.c | 98 ++- .../ethernet/netronome/nfp/nfp_net_debugfs.c | 33 +- .../net/ethernet/netronome/nfp/nfp_net_xsk.c | 592 ++++++++++++++++++ .../net/ethernet/netronome/nfp/nfp_net_xsk.h | 29 + 6 files changed, 756 insertions(+), 28 deletions(-) create mode 100644 drivers/net/ethernet/netronome/nfp/nfp_net_xsk.c create mode 100644 drivers/net/ethernet/netronome/nfp/nfp_net_xsk.h diff --git a/drivers/net/ethernet/netronome/nfp/Makefile b/drivers/net/ethernet/netronome/nfp/Makefile index 9cff3d48acbc..9c72b43c1581 100644 --- a/drivers/net/ethernet/netronome/nfp/Makefile +++ b/drivers/net/ethernet/netronome/nfp/Makefile @@ -31,6 +31,7 @@ nfp-objs := \ nfp_net_main.o \ nfp_net_repr.o \ nfp_net_sriov.o \ + nfp_net_xsk.o \ nfp_netvf_main.o \ nfp_port.o \ nfp_shared_buf.o \ diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net.h b/drivers/net/ethernet/netronome/nfp/nfp_net.h index 12f403d004ee..437a19722fcf 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net.h +++ b/drivers/net/ethernet/netronome/nfp/nfp_net.h @@ -171,11 +171,14 @@ struct nfp_net_tx_desc { * struct nfp_net_tx_buf - software TX buffer descriptor * @skb: normal ring, sk_buff associated with this buffer * @frag: XDP ring, page frag associated with this buffer + * @xdp: XSK buffer pool handle (for AF_XDP) * @dma_addr: DMA mapping address of the buffer * @fidx: Fragment index (-1 for the head and [0..nr_frags-1] for frags) * @pkt_cnt: Number of packets to be produced out of the skb associated * with this buffer (valid only on the head's buffer). * Will be 1 for all non-TSO packets. + * @is_xsk_tx: Flag if buffer is a RX buffer after a XDP_TX action and not a + * buffer from the TX queue (for AF_XDP). * @real_len: Number of bytes which to be produced out of the skb (valid only * on the head's buffer). Equal to skb->len for non-TSO packets. */ @@ -183,10 +186,18 @@ struct nfp_net_tx_buf { union { struct sk_buff *skb; void *frag; + struct xdp_buff *xdp; }; dma_addr_t dma_addr; - short int fidx; - u16 pkt_cnt; + union { + struct { + short int fidx; + u16 pkt_cnt; + }; + struct { + bool is_xsk_tx; + }; + }; u32 real_len; }; @@ -315,6 +326,16 @@ struct nfp_net_rx_buf { dma_addr_t dma_addr; }; +/** + * struct nfp_net_xsk_rx_buf - software RX XSK buffer descriptor + * @dma_addr: DMA mapping address of the buffer + * @xdp: XSK buffer pool handle (for AF_XDP) + */ +struct nfp_net_xsk_rx_buf { + dma_addr_t dma_addr; + struct xdp_buff *xdp; +}; + /** * struct nfp_net_rx_ring - RX ring structure * @r_vec: Back pointer to ring vector structure @@ -325,6 +346,7 @@ struct nfp_net_rx_buf { * @fl_qcidx: Queue Controller Peripheral (QCP) queue index for the freelist * @qcp_fl: Pointer to base of the QCP freelist queue * @rxbufs: Array of transmitted FL/RX buffers + * @xsk_rxbufs: Array of transmitted FL/RX buffers (for AF_XDP) * @rxds: Virtual address of FL/RX ring in host memory * @xdp_rxq: RX-ring info avail for XDP * @dma: DMA address of the FL/RX ring @@ -343,6 +365,7 @@ struct nfp_net_rx_ring { u8 __iomem *qcp_fl; struct nfp_net_rx_buf *rxbufs; + struct nfp_net_xsk_rx_buf *xsk_rxbufs; struct nfp_net_rx_desc *rxds; struct xdp_rxq_info xdp_rxq; @@ -361,6 +384,7 @@ struct nfp_net_rx_ring { * @tx_ring: Pointer to TX ring * @rx_ring: Pointer to RX ring * @xdp_ring: Pointer to an extra TX ring for XDP + * @xsk_pool: XSK buffer pool active on vector queue pair (for AF_XDP) * @irq_entry: MSI-X table entry (use for talking to the device) * @event_ctr: Number of interrupt * @rx_dim: Dynamic interrupt moderation structure for RX @@ -432,6 +456,7 @@ struct nfp_net_r_vector { u64 rx_replace_buf_alloc_fail; struct nfp_net_tx_ring *xdp_ring; + struct xsk_buff_pool *xsk_pool; struct u64_stats_sync tx_sync; u64 tx_pkts; @@ -502,7 +527,7 @@ struct nfp_stat_pair { * @num_stack_tx_rings: Number of TX rings used by the stack (not XDP) * @num_rx_rings: Currently configured number of RX rings * @mtu: Device MTU - * @xsk_pools: AF_XDP UMEM table (@num_r_vecs in size) + * @xsk_pools: XSK buffer pools, @max_r_vecs in size (for AF_XDP). */ struct nfp_net_dp { struct device *dev; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index c10e977d2472..00a09b9e0aee 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -46,6 +46,7 @@ #include "nfp_net_ctrl.h" #include "nfp_net.h" #include "nfp_net_sriov.h" +#include "nfp_net_xsk.h" #include "nfp_port.h" #include "crypto/crypto.h" #include "crypto/fw.h" @@ -1316,6 +1317,9 @@ nfp_net_tx_ring_reset(struct nfp_net_dp *dp, struct nfp_net_tx_ring *tx_ring) tx_ring->rd_p++; } + if (tx_ring->is_xdp) + nfp_net_xsk_tx_bufs_free(tx_ring); + memset(tx_ring->txds, 0, tx_ring->size); tx_ring->wr_p = 0; tx_ring->rd_p = 0; @@ -1504,10 +1508,14 @@ static void nfp_net_rx_ring_reset(struct nfp_net_rx_ring *rx_ring) /* Move the empty entry to the end of the list */ wr_idx = D_IDX(rx_ring, rx_ring->wr_p); last_idx = rx_ring->cnt - 1; - rx_ring->rxbufs[wr_idx].dma_addr = rx_ring->rxbufs[last_idx].dma_addr; - rx_ring->rxbufs[wr_idx].frag = rx_ring->rxbufs[last_idx].frag; - rx_ring->rxbufs[last_idx].dma_addr = 0; - rx_ring->rxbufs[last_idx].frag = NULL; + if (rx_ring->r_vec->xsk_pool) { + rx_ring->xsk_rxbufs[wr_idx] = rx_ring->xsk_rxbufs[last_idx]; + memset(&rx_ring->xsk_rxbufs[last_idx], 0, + sizeof(*rx_ring->xsk_rxbufs)); + } else { + rx_ring->rxbufs[wr_idx] = rx_ring->rxbufs[last_idx]; + memset(&rx_ring->rxbufs[last_idx], 0, sizeof(*rx_ring->rxbufs)); + } memset(rx_ring->rxds, 0, rx_ring->size); rx_ring->wr_p = 0; @@ -1529,6 +1537,9 @@ nfp_net_rx_ring_bufs_free(struct nfp_net_dp *dp, { unsigned int i; + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) + return; + for (i = 0; i < rx_ring->cnt - 1; i++) { /* NULL skb can only happen when initial filling of the ring * fails to allocate enough buffers and calls here to free @@ -1556,6 +1567,9 @@ nfp_net_rx_ring_bufs_alloc(struct nfp_net_dp *dp, struct nfp_net_rx_buf *rxbufs; unsigned int i; + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) + return 0; + rxbufs = rx_ring->rxbufs; for (i = 0; i < rx_ring->cnt - 1; i++) { @@ -1580,6 +1594,9 @@ nfp_net_rx_ring_fill_freelist(struct nfp_net_dp *dp, { unsigned int i; + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) + return nfp_net_xsk_rx_ring_fill_freelist(rx_ring); + for (i = 0; i < rx_ring->cnt - 1; i++) nfp_net_rx_give_one(dp, rx_ring, rx_ring->rxbufs[i].frag, rx_ring->rxbufs[i].dma_addr); @@ -2560,7 +2577,11 @@ static void nfp_net_rx_ring_free(struct nfp_net_rx_ring *rx_ring) if (dp->netdev) xdp_rxq_info_unreg(&rx_ring->xdp_rxq); - kvfree(rx_ring->rxbufs); + + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) + kvfree(rx_ring->xsk_rxbufs); + else + kvfree(rx_ring->rxbufs); if (rx_ring->rxds) dma_free_coherent(dp->dev, rx_ring->size, @@ -2568,6 +2589,7 @@ static void nfp_net_rx_ring_free(struct nfp_net_rx_ring *rx_ring) rx_ring->cnt = 0; rx_ring->rxbufs = NULL; + rx_ring->xsk_rxbufs = NULL; rx_ring->rxds = NULL; rx_ring->dma = 0; rx_ring->size = 0; @@ -2583,8 +2605,18 @@ static void nfp_net_rx_ring_free(struct nfp_net_rx_ring *rx_ring) static int nfp_net_rx_ring_alloc(struct nfp_net_dp *dp, struct nfp_net_rx_ring *rx_ring) { + enum xdp_mem_type mem_type; + size_t rxbuf_sw_desc_sz; int err; + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) { + mem_type = MEM_TYPE_XSK_BUFF_POOL; + rxbuf_sw_desc_sz = sizeof(*rx_ring->xsk_rxbufs); + } else { + mem_type = MEM_TYPE_PAGE_ORDER0; + rxbuf_sw_desc_sz = sizeof(*rx_ring->rxbufs); + } + if (dp->netdev) { err = xdp_rxq_info_reg(&rx_ring->xdp_rxq, dp->netdev, rx_ring->idx, rx_ring->r_vec->napi.napi_id); @@ -2592,6 +2624,10 @@ nfp_net_rx_ring_alloc(struct nfp_net_dp *dp, struct nfp_net_rx_ring *rx_ring) return err; } + err = xdp_rxq_info_reg_mem_model(&rx_ring->xdp_rxq, mem_type, NULL); + if (err) + goto err_alloc; + rx_ring->cnt = dp->rxd_cnt; rx_ring->size = array_size(rx_ring->cnt, sizeof(*rx_ring->rxds)); rx_ring->rxds = dma_alloc_coherent(dp->dev, rx_ring->size, @@ -2603,10 +2639,17 @@ nfp_net_rx_ring_alloc(struct nfp_net_dp *dp, struct nfp_net_rx_ring *rx_ring) goto err_alloc; } - rx_ring->rxbufs = kvcalloc(rx_ring->cnt, sizeof(*rx_ring->rxbufs), - GFP_KERNEL); - if (!rx_ring->rxbufs) - goto err_alloc; + if (nfp_net_has_xsk_pool_slow(dp, rx_ring->idx)) { + rx_ring->xsk_rxbufs = kvcalloc(rx_ring->cnt, rxbuf_sw_desc_sz, + GFP_KERNEL); + if (!rx_ring->xsk_rxbufs) + goto err_alloc; + } else { + rx_ring->rxbufs = kvcalloc(rx_ring->cnt, rxbuf_sw_desc_sz, + GFP_KERNEL); + if (!rx_ring->rxbufs) + goto err_alloc; + } return 0; @@ -2659,11 +2702,13 @@ static void nfp_net_rx_rings_free(struct nfp_net_dp *dp) } static void -nfp_net_napi_add(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec) +nfp_net_napi_add(struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec, int idx) { if (dp->netdev) netif_napi_add(dp->netdev, &r_vec->napi, - nfp_net_poll, NAPI_POLL_WEIGHT); + nfp_net_has_xsk_pool_slow(dp, idx) ? + nfp_net_xsk_poll : nfp_net_poll, + NAPI_POLL_WEIGHT); else tasklet_enable(&r_vec->tasklet); } @@ -2687,6 +2732,17 @@ nfp_net_vector_assign_rings(struct nfp_net_dp *dp, r_vec->xdp_ring = idx < dp->num_tx_rings - dp->num_stack_tx_rings ? &dp->tx_rings[dp->num_stack_tx_rings + idx] : NULL; + + if (nfp_net_has_xsk_pool_slow(dp, idx) || r_vec->xsk_pool) { + r_vec->xsk_pool = dp->xdp_prog ? dp->xsk_pools[idx] : NULL; + + if (r_vec->xsk_pool) + xsk_pool_set_rxq_info(r_vec->xsk_pool, + &r_vec->rx_ring->xdp_rxq); + + nfp_net_napi_del(dp, r_vec); + nfp_net_napi_add(dp, r_vec, idx); + } } static int @@ -2695,7 +2751,7 @@ nfp_net_prepare_vector(struct nfp_net *nn, struct nfp_net_r_vector *r_vec, { int err; - nfp_net_napi_add(&nn->dp, r_vec); + nfp_net_napi_add(&nn->dp, r_vec, idx); snprintf(r_vec->name, sizeof(r_vec->name), "%s-rxtx-%d", nfp_net_name(nn), idx); @@ -2834,8 +2890,11 @@ static void nfp_net_clear_config_and_disable(struct nfp_net *nn) if (err) nn_err(nn, "Could not disable device: %d\n", err); - for (r = 0; r < nn->dp.num_rx_rings; r++) + for (r = 0; r < nn->dp.num_rx_rings; r++) { nfp_net_rx_ring_reset(&nn->dp.rx_rings[r]); + if (nfp_net_has_xsk_pool_slow(&nn->dp, nn->dp.rx_rings[r].idx)) + nfp_net_xsk_rx_bufs_free(&nn->dp.rx_rings[r]); + } for (r = 0; r < nn->dp.num_tx_rings; r++) nfp_net_tx_ring_reset(&nn->dp, &nn->dp.tx_rings[r]); for (r = 0; r < nn->dp.num_r_vecs; r++) @@ -3771,6 +3830,9 @@ static int nfp_net_xdp(struct net_device *netdev, struct netdev_bpf *xdp) return nfp_net_xdp_setup_drv(nn, xdp); case XDP_SETUP_PROG_HW: return nfp_net_xdp_setup_hw(nn, xdp); + case XDP_SETUP_XSK_POOL: + return nfp_net_xsk_setup_pool(netdev, xdp->xsk.pool, + xdp->xsk.queue_id); default: return nfp_app_bpf(nn->app, nn, xdp); } @@ -3821,6 +3883,7 @@ const struct net_device_ops nfp_net_netdev_ops = { .ndo_features_check = nfp_net_features_check, .ndo_get_phys_port_name = nfp_net_get_phys_port_name, .ndo_bpf = nfp_net_xdp, + .ndo_xsk_wakeup = nfp_net_xsk_wakeup, .ndo_get_devlink_port = nfp_devlink_get_devlink_port, }; @@ -3948,6 +4011,14 @@ nfp_net_alloc(struct pci_dev *pdev, void __iomem *ctrl_bar, bool needs_netdev, nn->dp.num_r_vecs = max(nn->dp.num_tx_rings, nn->dp.num_rx_rings); nn->dp.num_r_vecs = min_t(unsigned int, nn->dp.num_r_vecs, num_online_cpus()); + nn->max_r_vecs = nn->dp.num_r_vecs; + + nn->dp.xsk_pools = kcalloc(nn->max_r_vecs, sizeof(nn->dp.xsk_pools), + GFP_KERNEL); + if (!nn->dp.xsk_pools) { + err = -ENOMEM; + goto err_free_nn; + } nn->dp.txd_cnt = NFP_NET_TX_DESCS_DEFAULT; nn->dp.rxd_cnt = NFP_NET_RX_DESCS_DEFAULT; @@ -3987,6 +4058,7 @@ void nfp_net_free(struct nfp_net *nn) WARN_ON(timer_pending(&nn->reconfig_timer) || nn->reconfig_posted); nfp_ccm_mbox_free(nn); + kfree(nn->dp.xsk_pools); if (nn->dp.netdev) free_netdev(nn->dp.netdev); else diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_debugfs.c b/drivers/net/ethernet/netronome/nfp/nfp_net_debugfs.c index 553c708694e8..2c74b3c5aef9 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_debugfs.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_debugfs.c @@ -42,13 +42,19 @@ static int nfp_rx_q_show(struct seq_file *file, void *data) seq_printf(file, "%04d: 0x%08x 0x%08x", i, rxd->vals[0], rxd->vals[1]); - frag = READ_ONCE(rx_ring->rxbufs[i].frag); - if (frag) - seq_printf(file, " frag=%p", frag); - - if (rx_ring->rxbufs[i].dma_addr) - seq_printf(file, " dma_addr=%pad", - &rx_ring->rxbufs[i].dma_addr); + if (!r_vec->xsk_pool) { + frag = READ_ONCE(rx_ring->rxbufs[i].frag); + if (frag) + seq_printf(file, " frag=%p", frag); + + if (rx_ring->rxbufs[i].dma_addr) + seq_printf(file, " dma_addr=%pad", + &rx_ring->rxbufs[i].dma_addr); + } else { + if (rx_ring->xsk_rxbufs[i].dma_addr) + seq_printf(file, " dma_addr=%pad", + &rx_ring->xsk_rxbufs[i].dma_addr); + } if (i == rx_ring->rd_p % rxd_cnt) seq_puts(file, " H_RD "); @@ -103,20 +109,23 @@ static int nfp_tx_q_show(struct seq_file *file, void *data) tx_ring->rd_p, tx_ring->wr_p, d_rd_p, d_wr_p); for (i = 0; i < txd_cnt; i++) { + struct xdp_buff *xdp; + struct sk_buff *skb; + txd = &tx_ring->txds[i]; seq_printf(file, "%04d: 0x%08x 0x%08x 0x%08x 0x%08x", i, txd->vals[0], txd->vals[1], txd->vals[2], txd->vals[3]); - if (tx_ring == r_vec->tx_ring) { - struct sk_buff *skb = READ_ONCE(tx_ring->txbufs[i].skb); - + if (!tx_ring->is_xdp) { + skb = READ_ONCE(tx_ring->txbufs[i].skb); if (skb) seq_printf(file, " skb->head=%p skb->data=%p", skb->head, skb->data); } else { - seq_printf(file, " frag=%p", - READ_ONCE(tx_ring->txbufs[i].frag)); + xdp = READ_ONCE(tx_ring->txbufs[i].xdp); + if (xdp) + seq_printf(file, " xdp->data=%p", xdp->data); } if (tx_ring->txbufs[i].dma_addr) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.c b/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.c new file mode 100644 index 000000000000..ab7243277efa --- /dev/null +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.c @@ -0,0 +1,592 @@ +// SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +/* Copyright (C) 2018 Netronome Systems, Inc */ +/* Copyright (C) 2021 Corigine, Inc */ + +#include +#include +#include +#include +#include + +#include "nfp_app.h" +#include "nfp_net.h" +#include "nfp_net_xsk.h" + +static int nfp_net_tx_space(struct nfp_net_tx_ring *tx_ring) +{ + return tx_ring->cnt - tx_ring->wr_p + tx_ring->rd_p - 1; +} + +static void nfp_net_xsk_tx_free(struct nfp_net_tx_buf *txbuf) +{ + xsk_buff_free(txbuf->xdp); + + txbuf->dma_addr = 0; + txbuf->xdp = NULL; +} + +void nfp_net_xsk_tx_bufs_free(struct nfp_net_tx_ring *tx_ring) +{ + struct nfp_net_tx_buf *txbuf; + unsigned int idx; + + while (tx_ring->rd_p != tx_ring->wr_p) { + idx = D_IDX(tx_ring, tx_ring->rd_p); + txbuf = &tx_ring->txbufs[idx]; + + txbuf->real_len = 0; + + tx_ring->qcp_rd_p++; + tx_ring->rd_p++; + + if (tx_ring->r_vec->xsk_pool) { + if (txbuf->is_xsk_tx) + nfp_net_xsk_tx_free(txbuf); + + xsk_tx_completed(tx_ring->r_vec->xsk_pool, 1); + } + } +} + +static bool nfp_net_xsk_complete(struct nfp_net_tx_ring *tx_ring) +{ + struct nfp_net_r_vector *r_vec = tx_ring->r_vec; + u32 done_pkts = 0, done_bytes = 0, reused = 0; + bool done_all; + int idx, todo; + u32 qcp_rd_p; + + if (tx_ring->wr_p == tx_ring->rd_p) + return true; + + /* Work out how many descriptors have been transmitted. */ + qcp_rd_p = nfp_qcp_rd_ptr_read(tx_ring->qcp_q); + + if (qcp_rd_p == tx_ring->qcp_rd_p) + return true; + + todo = D_IDX(tx_ring, qcp_rd_p - tx_ring->qcp_rd_p); + + done_all = todo <= NFP_NET_XDP_MAX_COMPLETE; + todo = min(todo, NFP_NET_XDP_MAX_COMPLETE); + + tx_ring->qcp_rd_p = D_IDX(tx_ring, tx_ring->qcp_rd_p + todo); + + done_pkts = todo; + while (todo--) { + struct nfp_net_tx_buf *txbuf; + + idx = D_IDX(tx_ring, tx_ring->rd_p); + tx_ring->rd_p++; + + txbuf = &tx_ring->txbufs[idx]; + if (unlikely(!txbuf->real_len)) + continue; + + done_bytes += txbuf->real_len; + txbuf->real_len = 0; + + if (txbuf->is_xsk_tx) { + nfp_net_xsk_tx_free(txbuf); + reused++; + } + } + + u64_stats_update_begin(&r_vec->tx_sync); + r_vec->tx_bytes += done_bytes; + r_vec->tx_pkts += done_pkts; + u64_stats_update_end(&r_vec->tx_sync); + + xsk_tx_completed(r_vec->xsk_pool, done_pkts - reused); + + WARN_ONCE(tx_ring->wr_p - tx_ring->rd_p > tx_ring->cnt, + "XDP TX ring corruption rd_p=%u wr_p=%u cnt=%u\n", + tx_ring->rd_p, tx_ring->wr_p, tx_ring->cnt); + + return done_all; +} + +static void nfp_net_xsk_tx(struct nfp_net_tx_ring *tx_ring) +{ + struct nfp_net_r_vector *r_vec = tx_ring->r_vec; + struct xdp_desc desc[NFP_NET_XSK_TX_BATCH]; + struct xsk_buff_pool *xsk_pool; + struct nfp_net_tx_desc *txd; + u32 pkts = 0, wr_idx; + u32 i, got; + + xsk_pool = r_vec->xsk_pool; + + while (nfp_net_tx_space(tx_ring) >= NFP_NET_XSK_TX_BATCH) { + for (i = 0; i < NFP_NET_XSK_TX_BATCH; i++) + if (!xsk_tx_peek_desc(xsk_pool, &desc[i])) + break; + got = i; + if (!got) + break; + + wr_idx = D_IDX(tx_ring, tx_ring->wr_p + i); + prefetchw(&tx_ring->txds[wr_idx]); + + for (i = 0; i < got; i++) + xsk_buff_raw_dma_sync_for_device(xsk_pool, desc[i].addr, + desc[i].len); + + for (i = 0; i < got; i++) { + wr_idx = D_IDX(tx_ring, tx_ring->wr_p + i); + + tx_ring->txbufs[wr_idx].real_len = desc[i].len; + tx_ring->txbufs[wr_idx].is_xsk_tx = false; + + /* Build TX descriptor. */ + txd = &tx_ring->txds[wr_idx]; + nfp_desc_set_dma_addr(txd, + xsk_buff_raw_get_dma(xsk_pool, + desc[i].addr + )); + txd->offset_eop = PCIE_DESC_TX_EOP; + txd->dma_len = cpu_to_le16(desc[i].len); + txd->data_len = cpu_to_le16(desc[i].len); + } + + tx_ring->wr_p += got; + pkts += got; + } + + if (!pkts) + return; + + xsk_tx_release(xsk_pool); + /* Ensure all records are visible before incrementing write counter. */ + wmb(); + nfp_qcp_wr_ptr_add(tx_ring->qcp_q, pkts); +} + +static bool +nfp_net_xsk_tx_xdp(const struct nfp_net_dp *dp, struct nfp_net_r_vector *r_vec, + struct nfp_net_rx_ring *rx_ring, + struct nfp_net_tx_ring *tx_ring, + struct nfp_net_xsk_rx_buf *xrxbuf, unsigned int pkt_len, + int pkt_off) +{ + struct xsk_buff_pool *pool = r_vec->xsk_pool; + struct nfp_net_tx_buf *txbuf; + struct nfp_net_tx_desc *txd; + unsigned int wr_idx; + + if (nfp_net_tx_space(tx_ring) < 1) + return false; + + xsk_buff_raw_dma_sync_for_device(pool, xrxbuf->dma_addr + pkt_off, pkt_len); + + wr_idx = D_IDX(tx_ring, tx_ring->wr_p); + + txbuf = &tx_ring->txbufs[wr_idx]; + txbuf->xdp = xrxbuf->xdp; + txbuf->real_len = pkt_len; + txbuf->is_xsk_tx = true; + + /* Build TX descriptor */ + txd = &tx_ring->txds[wr_idx]; + txd->offset_eop = PCIE_DESC_TX_EOP; + txd->dma_len = cpu_to_le16(pkt_len); + nfp_desc_set_dma_addr(txd, xrxbuf->dma_addr + pkt_off); + txd->data_len = cpu_to_le16(pkt_len); + + txd->flags = 0; + txd->mss = 0; + txd->lso_hdrlen = 0; + + tx_ring->wr_ptr_add++; + tx_ring->wr_p++; + + return true; +} + +static int nfp_net_rx_space(struct nfp_net_rx_ring *rx_ring) +{ + return rx_ring->cnt - rx_ring->wr_p + rx_ring->rd_p - 1; +} + +static void +nfp_net_xsk_rx_bufs_stash(struct nfp_net_rx_ring *rx_ring, unsigned int idx, + struct xdp_buff *xdp) +{ + unsigned int headroom; + + headroom = xsk_pool_get_headroom(rx_ring->r_vec->xsk_pool); + + rx_ring->rxds[idx].fld.reserved = 0; + rx_ring->rxds[idx].fld.meta_len_dd = 0; + + rx_ring->xsk_rxbufs[idx].xdp = xdp; + rx_ring->xsk_rxbufs[idx].dma_addr = + xsk_buff_xdp_get_frame_dma(xdp) + headroom; +} + +static void nfp_net_xsk_rx_unstash(struct nfp_net_xsk_rx_buf *rxbuf) +{ + rxbuf->dma_addr = 0; + rxbuf->xdp = NULL; +} + +static void nfp_net_xsk_rx_free(struct nfp_net_xsk_rx_buf *rxbuf) +{ + if (rxbuf->xdp) + xsk_buff_free(rxbuf->xdp); + + nfp_net_xsk_rx_unstash(rxbuf); +} + +void nfp_net_xsk_rx_bufs_free(struct nfp_net_rx_ring *rx_ring) +{ + unsigned int i; + + if (!rx_ring->cnt) + return; + + for (i = 0; i < rx_ring->cnt - 1; i++) + nfp_net_xsk_rx_free(&rx_ring->xsk_rxbufs[i]); +} + +void nfp_net_xsk_rx_ring_fill_freelist(struct nfp_net_rx_ring *rx_ring) +{ + struct nfp_net_r_vector *r_vec = rx_ring->r_vec; + struct xsk_buff_pool *pool = r_vec->xsk_pool; + unsigned int wr_idx, wr_ptr_add = 0; + struct xdp_buff *xdp; + + while (nfp_net_rx_space(rx_ring)) { + wr_idx = D_IDX(rx_ring, rx_ring->wr_p); + + xdp = xsk_buff_alloc(pool); + if (!xdp) + break; + + nfp_net_xsk_rx_bufs_stash(rx_ring, wr_idx, xdp); + + nfp_desc_set_dma_addr(&rx_ring->rxds[wr_idx].fld, + rx_ring->xsk_rxbufs[wr_idx].dma_addr); + + rx_ring->wr_p++; + wr_ptr_add++; + } + + /* Ensure all records are visible before incrementing write counter. */ + wmb(); + nfp_qcp_wr_ptr_add(rx_ring->qcp_fl, wr_ptr_add); +} + +static void nfp_net_xsk_rx_drop(struct nfp_net_r_vector *r_vec, + struct nfp_net_xsk_rx_buf *xrxbuf) +{ + u64_stats_update_begin(&r_vec->rx_sync); + r_vec->rx_drops++; + u64_stats_update_end(&r_vec->rx_sync); + + nfp_net_xsk_rx_free(xrxbuf); +} + +static void nfp_net_xsk_rx_skb(struct nfp_net_rx_ring *rx_ring, + const struct nfp_net_rx_desc *rxd, + struct nfp_net_xsk_rx_buf *xrxbuf, + const struct nfp_meta_parsed *meta, + unsigned int pkt_len, + bool meta_xdp, + unsigned int *skbs_polled) +{ + struct nfp_net_r_vector *r_vec = rx_ring->r_vec; + struct nfp_net_dp *dp = &r_vec->nfp_net->dp; + struct net_device *netdev; + struct sk_buff *skb; + + if (likely(!meta->portid)) { + netdev = dp->netdev; + } else { + struct nfp_net *nn = netdev_priv(dp->netdev); + + netdev = nfp_app_dev_get(nn->app, meta->portid, NULL); + if (unlikely(!netdev)) { + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + return; + } + nfp_repr_inc_rx_stats(netdev, pkt_len); + } + + skb = napi_alloc_skb(&r_vec->napi, pkt_len); + if (!skb) { + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + return; + } + memcpy(skb_put(skb, pkt_len), xrxbuf->xdp->data, pkt_len); + + skb->mark = meta->mark; + skb_set_hash(skb, meta->hash, meta->hash_type); + + skb_record_rx_queue(skb, rx_ring->idx); + skb->protocol = eth_type_trans(skb, netdev); + + nfp_net_rx_csum(dp, r_vec, rxd, meta, skb); + + if (rxd->rxd.flags & PCIE_DESC_RX_VLAN) + __vlan_hwaccel_put_tag(skb, htons(ETH_P_8021Q), + le16_to_cpu(rxd->rxd.vlan)); + if (meta_xdp) + skb_metadata_set(skb, + xrxbuf->xdp->data - xrxbuf->xdp->data_meta); + + napi_gro_receive(&rx_ring->r_vec->napi, skb); + + nfp_net_xsk_rx_free(xrxbuf); + + (*skbs_polled)++; +} + +static unsigned int +nfp_net_xsk_rx(struct nfp_net_rx_ring *rx_ring, int budget, + unsigned int *skbs_polled) +{ + struct nfp_net_r_vector *r_vec = rx_ring->r_vec; + struct nfp_net_dp *dp = &r_vec->nfp_net->dp; + struct nfp_net_tx_ring *tx_ring; + struct bpf_prog *xdp_prog; + bool xdp_redir = false; + int pkts_polled = 0; + + xdp_prog = READ_ONCE(dp->xdp_prog); + tx_ring = r_vec->xdp_ring; + + while (pkts_polled < budget) { + unsigned int meta_len, data_len, pkt_len, pkt_off; + struct nfp_net_xsk_rx_buf *xrxbuf; + struct nfp_net_rx_desc *rxd; + struct nfp_meta_parsed meta; + int idx, act; + + idx = D_IDX(rx_ring, rx_ring->rd_p); + + rxd = &rx_ring->rxds[idx]; + if (!(rxd->rxd.meta_len_dd & PCIE_DESC_RX_DD)) + break; + + rx_ring->rd_p++; + pkts_polled++; + + xrxbuf = &rx_ring->xsk_rxbufs[idx]; + + /* If starved of buffers "drop" it and scream. */ + if (rx_ring->rd_p >= rx_ring->wr_p) { + nn_dp_warn(dp, "Starved of RX buffers\n"); + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + break; + } + + /* Memory barrier to ensure that we won't do other reads + * before the DD bit. + */ + dma_rmb(); + + memset(&meta, 0, sizeof(meta)); + + /* Only supporting AF_XDP with dynamic metadata so buffer layout + * is always: + * + * --------------------------------------------------------- + * | off | metadata | packet | XXXX | + * --------------------------------------------------------- + */ + meta_len = rxd->rxd.meta_len_dd & PCIE_DESC_RX_META_LEN_MASK; + data_len = le16_to_cpu(rxd->rxd.data_len); + pkt_len = data_len - meta_len; + + if (unlikely(meta_len > NFP_NET_MAX_PREPEND)) { + nn_dp_warn(dp, "Oversized RX packet metadata %u\n", + meta_len); + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + continue; + } + + /* Stats update. */ + u64_stats_update_begin(&r_vec->rx_sync); + r_vec->rx_pkts++; + r_vec->rx_bytes += pkt_len; + u64_stats_update_end(&r_vec->rx_sync); + + xrxbuf->xdp->data += meta_len; + xrxbuf->xdp->data_end = xrxbuf->xdp->data + pkt_len; + xdp_set_data_meta_invalid(xrxbuf->xdp); + xsk_buff_dma_sync_for_cpu(xrxbuf->xdp, r_vec->xsk_pool); + net_prefetch(xrxbuf->xdp->data); + + if (meta_len) { + if (unlikely(nfp_net_parse_meta(dp->netdev, &meta, + xrxbuf->xdp->data - + meta_len, + xrxbuf->xdp->data, + pkt_len, meta_len))) { + nn_dp_warn(dp, "Invalid RX packet metadata\n"); + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + continue; + } + + if (unlikely(meta.portid)) { + struct nfp_net *nn = netdev_priv(dp->netdev); + + if (meta.portid != NFP_META_PORT_ID_CTRL) { + nfp_net_xsk_rx_skb(rx_ring, rxd, xrxbuf, + &meta, pkt_len, + false, skbs_polled); + continue; + } + + nfp_app_ctrl_rx_raw(nn->app, xrxbuf->xdp->data, + pkt_len); + nfp_net_xsk_rx_free(xrxbuf); + continue; + } + } + + act = bpf_prog_run_xdp(xdp_prog, xrxbuf->xdp); + + pkt_len = xrxbuf->xdp->data_end - xrxbuf->xdp->data; + pkt_off = xrxbuf->xdp->data - xrxbuf->xdp->data_hard_start; + + switch (act) { + case XDP_PASS: + nfp_net_xsk_rx_skb(rx_ring, rxd, xrxbuf, &meta, pkt_len, + true, skbs_polled); + break; + case XDP_TX: + if (!nfp_net_xsk_tx_xdp(dp, r_vec, rx_ring, tx_ring, + xrxbuf, pkt_len, pkt_off)) + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + else + nfp_net_xsk_rx_unstash(xrxbuf); + break; + case XDP_REDIRECT: + if (xdp_do_redirect(dp->netdev, xrxbuf->xdp, xdp_prog)) { + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + } else { + nfp_net_xsk_rx_unstash(xrxbuf); + xdp_redir = true; + } + break; + default: + bpf_warn_invalid_xdp_action(dp->netdev, xdp_prog, act); + fallthrough; + case XDP_ABORTED: + trace_xdp_exception(dp->netdev, xdp_prog, act); + fallthrough; + case XDP_DROP: + nfp_net_xsk_rx_drop(r_vec, xrxbuf); + break; + } + } + + nfp_net_xsk_rx_ring_fill_freelist(r_vec->rx_ring); + + if (xdp_redir) + xdp_do_flush_map(); + + if (tx_ring->wr_ptr_add) + nfp_net_tx_xmit_more_flush(tx_ring); + + return pkts_polled; +} + +static void nfp_net_xsk_pool_unmap(struct device *dev, + struct xsk_buff_pool *pool) +{ + return xsk_pool_dma_unmap(pool, 0); +} + +static int nfp_net_xsk_pool_map(struct device *dev, struct xsk_buff_pool *pool) +{ + return xsk_pool_dma_map(pool, dev, 0); +} + +int nfp_net_xsk_setup_pool(struct net_device *netdev, + struct xsk_buff_pool *pool, u16 queue_id) +{ + struct nfp_net *nn = netdev_priv(netdev); + + struct xsk_buff_pool *prev_pool; + struct nfp_net_dp *dp; + int err; + + /* Reject on old FWs so we can drop some checks on datapath. */ + if (nn->dp.rx_offset != NFP_NET_CFG_RX_OFFSET_DYNAMIC) + return -EOPNOTSUPP; + if (!nn->dp.chained_metadata_format) + return -EOPNOTSUPP; + + /* Install */ + if (pool) { + err = nfp_net_xsk_pool_map(nn->dp.dev, pool); + if (err) + return err; + } + + /* Reconfig/swap */ + dp = nfp_net_clone_dp(nn); + if (!dp) { + err = -ENOMEM; + goto err_unmap; + } + + prev_pool = dp->xsk_pools[queue_id]; + dp->xsk_pools[queue_id] = pool; + + err = nfp_net_ring_reconfig(nn, dp, NULL); + if (err) + goto err_unmap; + + /* Uninstall */ + if (prev_pool) + nfp_net_xsk_pool_unmap(nn->dp.dev, prev_pool); + + return 0; +err_unmap: + if (pool) + nfp_net_xsk_pool_unmap(nn->dp.dev, pool); + + return err; +} + +int nfp_net_xsk_wakeup(struct net_device *netdev, u32 queue_id, u32 flags) +{ + struct nfp_net *nn = netdev_priv(netdev); + + /* queue_id comes from a zero-copy socket, installed with XDP_SETUP_XSK_POOL, + * so it must be within our vector range. Moreover, our napi structs + * are statically allocated, so we can always kick them without worrying + * if reconfig is in progress or interface down. + */ + napi_schedule(&nn->r_vecs[queue_id].napi); + + return 0; +} + +int nfp_net_xsk_poll(struct napi_struct *napi, int budget) +{ + struct nfp_net_r_vector *r_vec = + container_of(napi, struct nfp_net_r_vector, napi); + unsigned int pkts_polled, skbs = 0; + + pkts_polled = nfp_net_xsk_rx(r_vec->rx_ring, budget, &skbs); + + if (pkts_polled < budget) { + if (r_vec->tx_ring) + nfp_net_tx_complete(r_vec->tx_ring, budget); + + if (!nfp_net_xsk_complete(r_vec->xdp_ring)) + pkts_polled = budget; + + nfp_net_xsk_tx(r_vec->xdp_ring); + + if (pkts_polled < budget && napi_complete_done(napi, skbs)) + nfp_net_irq_unmask(r_vec->nfp_net, r_vec->irq_entry); + } + + return pkts_polled; +} diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.h b/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.h new file mode 100644 index 000000000000..5c8549cb3543 --- /dev/null +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_xsk.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) */ +/* Copyright (C) 2018 Netronome Systems, Inc */ +/* Copyright (C) 2021 Corigine, Inc */ + +#ifndef _NFP_XSK_H_ +#define _NFP_XSK_H_ + +#include + +#define NFP_NET_XSK_TX_BATCH 16 /* XSK TX transmission batch size. */ + +static inline bool nfp_net_has_xsk_pool_slow(struct nfp_net_dp *dp, + unsigned int qid) +{ + return dp->xdp_prog && dp->xsk_pools[qid]; +} + +int nfp_net_xsk_setup_pool(struct net_device *netdev, struct xsk_buff_pool *pool, + u16 queue_id); + +void nfp_net_xsk_tx_bufs_free(struct nfp_net_tx_ring *tx_ring); +void nfp_net_xsk_rx_bufs_free(struct nfp_net_rx_ring *rx_ring); + +void nfp_net_xsk_rx_ring_fill_freelist(struct nfp_net_rx_ring *rx_ring); + +int nfp_net_xsk_wakeup(struct net_device *netdev, u32 queue_id, u32 flags); +int nfp_net_xsk_poll(struct napi_struct *napi, int budget); + +#endif /* _NFP_XSK_H_ */