From patchwork Mon Oct 21 10:20:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843832 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 801BC16F27E for ; Mon, 21 Oct 2024 10:24:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506283; cv=fail; b=X5pkFojZxBsAyCzWk1YBbZFUa3AS9BEa0pr4NjyzGLj0gadZNlW6EX+KoT5DKOB32HDgaKczASCUKtH9Bv4EmGJmNIhva16zqD2sQ+I3WkuHkR0Zgz8qRaMkQs3R9Qz4TdKRb8snCzsUhmuAfaJKtC/V2SalLmCi3iOmnQtMlmA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506283; c=relaxed/simple; bh=T+HLM7kYBiNhj32OMoMaS2Ud/Ymy2vZaxhbfZbD09JI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=qRgwC/nGoZT1u6FLZ9OaWbTilNE9AcLoRMVC0lc//nZixtXhjlCz9X4y3rIRtC6UpEVj8AUPh/SbOAxy6LC7iDxs5OSWV9SarmeJLAPzoN7fkHUaWQmFN+vFbwMbPRugyLpp7JIU+HDEeqtV3iZ8Udhwwhnr86bp2u2wpJ8+d4U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=WgCWGnEL; arc=fail smtp.client-ip=185.132.181.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="WgCWGnEL" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2109.outbound.protection.outlook.com [104.47.18.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 86F1FC005A; Mon, 21 Oct 2024 10:24:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lKIoQYgffWAtapp/tBAeWaD7KJR4bmF8GWYX319+9NVqp8dwCJF3G59h1PpCaXPqgMYU5NKXYDUQIjayiV9FR3UY4UIqFn5Gdl4KftQU7d1K1FdSWm7o/6NEDA/Ta/2QbZeMfq0wheoaBRBmFWBD82NONJ/ADMgmx1EDulQCZebCgSkRQH/e9VEwuHdDKotUmg9QiffxtCVauO72tDJMUT3DfK+G7KgCwTdmNjjok4ZjyixSK5I4IGJGJhNu6rijQvev1sDjqUhFmg4m8+MrbToG07j5ikw/uiZQGI1Z813iBWNWzfY3ANtIiFRgbU0GIExs9O5smPZ88oVZ8xJkow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XIfnBpyQ3DQM/afkECK6Vaqlj2HbGYLTwLExlShUpBg=; b=NDfTZmyGUZ/GdsWzHh0AycaXfsi3ywDlVuXwTV5Dq0YknpJcjxJM9b7cwCVYOQfLmP82u91MvkI4JH1rR2gewgDiL2Ma3Dud159lI+yDC5TPyTAjIS8MZDRLDB5RHMzE0v0yCSATPfA3mCgF+VldJlyP5T3x9EeLKu5sPwun/cyqpcR6+NguRlT3GggDmzDgvRkadHyMpRxMaBNhJjMvclXI5Bi5d0hsphxhlFeqcEokTKYb0jWGJd0Hz1qnIvrvvLMy79ZV/8M3YlMC0r/D+XA+LZimP2jRhF5iVTuAmMISk55s5ej6k/5ndQxlyDYuB3TmS1r0yydWdYkgmzx12g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XIfnBpyQ3DQM/afkECK6Vaqlj2HbGYLTwLExlShUpBg=; b=WgCWGnEL1PM6HbyM86Lwxk4QVQ2MePkzlWczECyMCuRI99PqNx/ZrynumltXwbpu+vzd6JMD81pOBSJpZ3sQfXifPwwJdxqYNgNYfe/jK/x5YEzbaxTumzdhC4wjscKMlVYYZgOoZnqYK3rlJ/4WkDCAg3TPgEy3mtTrueMkOlA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by DU0PR08MB9051.eurprd08.prod.outlook.com (2603:10a6:10:470::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Mon, 21 Oct 2024 10:24:32 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:32 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 1/6] neighbour: Add hlist_node to struct neighbour Date: Mon, 21 Oct 2024 10:20:53 +0000 Message-ID: <20241021102102.2560279-2-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|DU0PR08MB9051:EE_ X-MS-Office365-Filtering-Correlation-Id: 3fffea36-50d4-43e9-803f-08dcf1ba8d71 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|52116014|376014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: OKzoiVjN5cHY+VAaf/XLIsn1h+7Cw/g2JYP3Lz6tLOWBjUOJjxffu9zrXwnQVUzQ0v9yxu1VtpxgfoPJccvTyV8jVF+7M3OdSXrSiOXppYEjszQnQvWTFB0hjXeQYlaY7fNntgf5nkUF5PMZfQb1pEZ+e3yt8cQNRxO0QmsPlOtH2tiXYJgor+QsIXL3ogkqp2FEF4F9gzLZw8eOgrNiGM+21fyVZKZfMD1K3tPoZEh6K2Lz1GqHewOvj3RktkxfkQbONEFgPlYeF4gHxSnlIaE1kI7me9+tc7Tz5iYoTjQkncV6mPqU08amWheCywNBTSV1fGQ1tEMu7WUKmijL4VC2B+q6PQgQus/JTAs7DFIetWQnUj5oV3qoi62nvFNr9ETWY1ORQdGfE3l7I032XE6ozyJBe8vYP1jtszc/PYz4vF73iZdLwqPncGqC4/6BQNSVGWue/vW+R0c+gh/ujYb8n3aM2pQSmZojvK7KKLnIt98AYASGmsi5NB6vTgkOLOlqbO8rI0jQMWsR9XNR7D6GERsoBKB36Rz9KMsKRFvUby/Ly02LT/cNCB5ymrWr4fYDpd4IwmQeXGtNheptdMW38W1f9VagDuNjNa/dpKuC/qSf6FTzoiMbNIx0CXS9ceExTD3mp0kS3s+J4BkmE8mY5BRu1zjZUaQChnuVNfI0eIk7pEm2zsO2InQLtoPu7R1ZK2RhjvZuvW27ziIWApol4AG/9D/YdfJDKKgM1a48UqUnd+x6BvST5wYyaB43L6iGLwopHGBAhdY1S6e/VDn8bchpbd4Vc5qV3QbQMuwojpqScfPpEhrp+PT7m032HHV5tGdEVl1oaDGBeggNaNjKWmwhk6k4QrfTFmtMKzFkb81e9dHLsvkoeWowBwNHwbInSQUgwJUxAoC11p32dd4uHJlPlYzYRChsPL/kMpC4danmhvFxnh+klbeMrwtbA6CtyPkBfUr+vTV3bfIzv4apB1BKt/s+1HeFGV79olIh5A1UxQZVdyn0tPYGDAh0AbJifvrcqNwUgPjehgB5YCltj3gz+1uzvjNPfVQyT3IpIjkcKNfI56EOUYJOkRCuCtNJTtxrbT2jhmgUBsaZqEfS2vGkHBuQV4LrgQj3Iieu8ZFM3P36dF9C/SY/7Cr7b+1xi652LJiDU2jwEwO5jp8M73BW2u2aVQ1JFggvCeQsR+cSWc6pg0kUrVfjMBiJJP6Teq4u7Ddcg84dhJTJREAMc+NoVCO7r/jujAB8KUgkMWXpMxKwwlqrwTUiECy0MVqnNsNplzJCPFThe8rbWIEvpHeLPZLe+yZg7tDbc1dYhh9YFr3MiYFyCt8aYSVrKUXKrKaat1/6fm4MVxLz4PbtBEhVnvlO0+iivY4k1kc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(52116014)(376014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A3pejwYp1uJC5CP112Ga/+IA7ctCC0haylJs2VoVEWRJ5JwjigyaKXUot9HquRy15o+WOyECtvaLuWpb4lgOOHV8uAkbm7lcnwKpeTPHw19mYN3u5yExQcd37ZsUOrOqJ4va2ETaIYmmgw5OIxjpiysHzs6GSQfyUEW8pmVc/cAKbaW6o8ZwtY/QHTpVEZVyyK/gSbJH0yt3l07Dm+yaxQv5o5Yw3PkMWxERdJ2xj8C98lQS9GujPT6P0UDRB5+TfsBNSk96iGmNfjwctGeBD4zYlA1K8ITqi7rVMyGpzAT8akopazVKlycGHIeYzJZ4XEguqxqXO1Ft2g8jDXJ5yZrEQE/lZu479OSIo1Ob3NrL/3ZFYZUk3iYo1SF/UvEu++VQa7agpdeJSuH5T8o05Tl2p28pg5l+/bYfn7W41prDUYYeUr9BKbmgqk1z1PACGmdhvuDOItG7X+LtsiZMqFStUmQ1PqMCIa9HQ3fV5MDwmw0EEIdes9F9WnMMUTs6nhHx22nNYeqTCLe32d+g/RjasylceVrHEJgUEJpm1VbN2BW3k/frwVuDWv/i+Y6gI7bilYoJPA0HvhEXmnVLOqLd4hniyyEzymZO40ermOStjeRpit66L6fmMTkWUZoslKVoQHib56ygc+KJ+wSFTcSqs00OeOyl+lUxv4ut5MCcs0qjg53bz7P15b9SUvNv60ZNZtE0kTMOfKyH4W1W8XD96/ACAsPavmpFqOjbORjoW5fchkxhXn2gbaWOVrKWh46mWi7wlTcZ4U5/3M9Z9sRZuNz9TIQ4X6VUGFIIurjcVeGEhjQq4BB3eeFz82Ld9OrHZDTCeKLohNin2eYO1eh6wTTyiy60a7LilOHCDrlPy4FOgUIKYzRdQnxSD+MNJgDsBPKUy/yvhgq8qGwbQMndJEHM9M76iVT9q766dD+z6Jo9L1Jv5ID4X4OX6c33MODvf7olGeAlVNoZOedrxor5dtfXB71s7WKg4YVDwGkxXh+780HauZsoXFmPuQBacVsdRsqna2FO99L/llT5mBDpUYSSAp7n6YfXNgGl8OUrNRbarJNUrTQK9oC65OfjJ5ggc/GayvVdIJUU+/BYu9l45JxTLnNscLFs53tBk8McntsvCxF6yCEjaDX5FepEYGOxmNA3TpatmnPLIRo+4SaRLGiF/aRgJXq1AHfdppuwCSBXblTg/yHeRN7n+FPSNEuZakqB/Spql90pYML1Js5+T89YSuwF/cnShyVWQz4Krf1YrO5PP6G/bU3ZdKfgn3iQRpBL7BNKqU3q0KlK13rF9E5L4QkWKSlL3bo2PIGbIFW4Cq1JI+FQ+T/jmYkLb6fDZVXiJDNTrEMbZpiwqbin7LTauGkmeWOm6moJqEN8YZ/J/Qdop6KUz5clnN1pBNCX6iaM+3ehU6MAEZJHbaGPbjJ8/VYn8hmlL/saZv4hOIostmB1/X3Yg46M7yxFaxFEfH+/vSTGvgVqHZdOQXQxpvwdLZu55l8XPm3f6r6jdi3l8Q04C2FXWohpBOqTImXt4eyon34CWuCMoXL+843RTWMrHYoFcmtGdJYDUwe3FnfiGzoCsXXy+MNLrVYmZl1WlOVJP3DaB1tdBDzmjQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 93YTaa8R0iUCgBKG0SQkVdVVf4CClukH6UfpkIwJIzQ+GMGc0yCDX06G7iaKUBv/FvyYQrjDy5YPqtAK5YffUzwiq7boHgywZLQh7o/ky1m3K3s2IRHKgcyj2DxMgH2ceqX+Tv3DCvDcXsbHISwiSve61yqAOOBF9IyLUSXwG0xYetVtsuFS2fblOspRzlMmKiw9GE9TWMt39DKXfuOhQObbyLm4K7NhLcLg2FRTlEcIVGIxYtFakOTbP//MtoqolDhhLKOOHDyXcLkWm5TI/crx5c+90YpYLTziUSQcr2riz6lcSLHYJCSS62MqDxyZb59FTVgWayGl3zUkNXVPlfuc1slRtwH5z/huF16BJOytsETPMIs25ljAgv2nikgHEJJNNBMc/CL77Orhz9ngKHOHUwZA9SlXborVL7IqoG2ZqhxcYRLknAOf73p/24Huhl3EZKF+0R2l8TkNXkF8JFDjomfWWJ9/0Yy7tqNVtRSAizsYk9IsQdT4YzD+bFvG1S7wavjUGIv6ItrvNPeglCB2eH87V3nE+LsWH9WizlVTZzqelJXBoZ0TZYKGrHmHrAv6gAbjdq/1eqbSCzxNwX+WMPDJP90emwQyX0fvaqjsIjmEomo2cVTwLQI1Wgt2 X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3fffea36-50d4-43e9-803f-08dcf1ba8d71 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:32.0516 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d1opCJiPH802TyeTqn15y/jGNeA5ppjVTf37SQrSjsJjY05CKo0D0B1g88MREHew3jjXpoDAXsJfAMLV3G0XoQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9051 X-MDID: 1729506274-C2XmVRV_hHaj X-MDID-O: eu1;fra;1729506274;C2XmVRV_hHaj;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Add a doubly-linked node to neighbours, so that they can be deleted without iterating the entire bucket they're in. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 ++ net/core/neighbour.c | 40 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 2 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 3887ed9e5026..0402447854c7 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -136,6 +136,7 @@ struct neigh_statistics { struct neighbour { struct neighbour __rcu *next; + struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -191,6 +192,7 @@ struct pneigh_entry { struct neigh_hash_table { struct neighbour __rcu **hash_buckets; + struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; struct rcu_head rcu; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 395ae1626eef..45c8df801dfb 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -217,6 +217,7 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, neigh = rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock)); rcu_assign_pointer(*np, neigh); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; } @@ -403,6 +404,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); neigh_mark_dead(n); @@ -530,27 +532,47 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { + size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neigh_hash_table *ret; struct neighbour __rcu **buckets; + struct hlist_head *hash_heads; + struct neigh_hash_table *ret; int i; + hash_heads = NULL; + ret = kmalloc(sizeof(*ret), GFP_ATOMIC); if (!ret) return NULL; if (size <= PAGE_SIZE) { buckets = kzalloc(size, GFP_ATOMIC); + + if (buckets) { + hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); + if (!hash_heads) + kfree(buckets); + } } else { buckets = (struct neighbour __rcu **) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); + + if (buckets) { + hash_heads = (struct hlist_head *) + __get_free_pages(GFP_ATOMIC | __GFP_ZERO, + get_order(hash_heads_size)); + kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); + if (!hash_heads) + free_pages((unsigned long)buckets, get_order(size)); + } } - if (!buckets) { + if (!buckets || !hash_heads) { kfree(ret); return NULL; } ret->hash_buckets = buckets; + ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) neigh_get_hash_rnd(&ret->hash_rnd[i]); @@ -564,6 +586,8 @@ static void neigh_hash_free_rcu(struct rcu_head *head) rcu); size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); struct neighbour __rcu **buckets = nht->hash_buckets; + size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + struct hlist_head *hash_heads = nht->hash_heads; if (size <= PAGE_SIZE) { kfree(buckets); @@ -571,6 +595,13 @@ static void neigh_hash_free_rcu(struct rcu_head *head) kmemleak_free(buckets); free_pages((unsigned long)buckets, get_order(size)); } + + if (hash_heads_size < PAGE_SIZE) { + kfree(hash_heads); + } else { + kmemleak_free(hash_heads); + free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + } kfree(nht); } @@ -607,6 +638,8 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_buckets[hash], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(new_nht->hash_buckets[hash], n); + hlist_del_rcu(&n->hash); + hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } } @@ -717,6 +750,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, rcu_dereference_protected(nht->hash_buckets[hash_val], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(nht->hash_buckets[hash_val], n); + hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -1002,6 +1036,7 @@ static void neigh_periodic_work(struct work_struct *work) rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3131,6 +3166,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); } else np = &n->next; From patchwork Mon Oct 21 10:20:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843833 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7228A1E2840 for ; Mon, 21 Oct 2024 10:24:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506284; cv=fail; b=HxrdevX4ta4l7zM3Vou/ytrKEA6LqC/rpMullh+zg/kwlqgpfPE2f7e0t2rvykndUFRWmdk0z7CKbIf7ZE1N0sB5q9ShDk9ax5+NjKRmggYWLpsObOEEJmsyEUP16eZPoV/B8LUFB/IiJco0LT59d/84+4fZrKPx3Tax5xDtCHc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506284; c=relaxed/simple; bh=A+5WLTVTgKnbJgd5p+xzqrEFTvxsdewGTqgillMnlVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jZ9yR9OwsW+IeX8DEf+oGV+vow1cA06wF87t6hsR1kj9Hv4G0fh3Zusp9X/dJNZL2EpvfDhseyiF99lWuQoXhnJzx7rOLtIuuooyCf+ePlpCmLv6x5aCuIpJX8SNf82RAA23k7FLDSLTDBbfecmNvBMq0Po4LIsXFsE1BzEA6w4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=lUQl7m3N; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="lUQl7m3N" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id EF7B26C0075; Mon, 21 Oct 2024 10:24:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XxuVbmYbiQWHYJjVTcpTdr6Lwf3D6wvTyOJT76750issRqwF/har3E+ENJlZjjgQDMr+oA1qsrZ1am2Zbbj7ZNEwWkWAW6xXw8QxRYqM8h2DPSP8YLR+Sj36OKNraVgw64JFbJT45DJyM0JgX7Asn8Nkprt6z0J/qFg4I6U0K/AoV70JzHhH9hnjIQ9GJCssqk/SaGf7Rtg0iZr5PWlHFQvklfygABceKrGh+1l8ugr36I5bq2CNc2GeV7EYeU1ZsPc0jFAdIQrsoOqIePHKKoyNsqwPZSh7+x3GY0B19YSPCtLlxAzP1PzI0kmiHaYRSbs7h5pYLs9onIOyXRx4dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S82OqHsPVGbQfGIM4S/NdJl8U8lhgnpRIP+IpeCwvVw=; b=Uu3GDpNPI+RQ+p1wfSCK6y8ZjiNNVxGT4FDhWv3g8uV1x5fYk64tAU2PXnxrUFlyaVGKhf2YNZdcCgfdx1kiNsa0qTXOpCmepw4sB3byIl6Q+J+7hH/0TPQfhudqLsVjfSd1UelrJr89vzDTuoerQBJiKOt5lNN3/vlKTBlC2fgL+A5fdw0Ol9z2ye/rFARJt6UFdsUPKaYh/3b38PaQQkX/Q5XIyygVseYJMb33KUi3bciHDOTIXyxP3LSDNbQuBRSq5fF7HbVLluWMseBHA3hJQUOQXeCqR8ocgeVQwZsah1z/VEbJhLM1rIHtogxM8xJNLaknYc9IzVtarlk0GA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S82OqHsPVGbQfGIM4S/NdJl8U8lhgnpRIP+IpeCwvVw=; b=lUQl7m3Npvre3/inhx6EEhI9fY4NSRt3WuyVJMFA0N0B4AcN/1Bna2PGBh3ZOUjdzw5rUU6ineTExZ7hLS0XNpeWW6uwgQzSCNZ8YLqrMYU2sEXGvYG1h7gw+JFiUzaIrfvEqPRsypuA6mPN54zIcUpd3SJ19ePaDPl5ASWkp88= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by DU0PR08MB9051.eurprd08.prod.outlook.com (2603:10a6:10:470::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Mon, 21 Oct 2024 10:24:33 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:33 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 2/6] neighbour: Define neigh_for_each_in_bucket Date: Mon, 21 Oct 2024 10:20:54 +0000 Message-ID: <20241021102102.2560279-3-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|DU0PR08MB9051:EE_ X-MS-Office365-Filtering-Correlation-Id: d4aa08a3-95c4-40c5-4aa8-08dcf1ba8e0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|52116014|376014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: u1oPARCKL9tUK1X+eZdfLV4F3xK3VRdlOvvzKEXuLlgr+4XsGM/FnrDoqoRBUT63/aA1EQ6gOZ9YTTd/HLPiv2J3N7wwrtdtalVMj3U+lDqEvJhhzVnDQ4SF8p/KFQdWaOaU3IXkLsYRV+560jJJTPmB41HXzyZQUcqVKpyKAhqqQdmi+MPwafcqQld5nGxAmuQ2V0RMuV9FEJq2WHT3XbMDE0C8uYLNhsLcr7rwntlBF5f13CAkunBYpf/iac7BpteFfNnmL6ChSJNswiht9q5Zgs1WlBp4fQGg/iMspRbT98DDP3YH6OldEPRyP/2q6P1hixMT24ABh9zHqlJ2FKP6x91alWRnP3cWu/+BzCYqk4EFW1h92JPI3zr0m8qdtHYK7R9iiFcjAcgS8Ahl2mHqkQtjW1iAYWJIysQMoTu3J80Rx7Yg/xCaPN1kthvhwofR66lYGB1PTs7Y5uENtWL5sLp0xj0LeQX6Zm0vu6fRkF9UBrxXXDzwvSVbht/E3603qlFrF8ByulKbskdnFZeGLsyVr7Viag5MN1eBZZvPmIz4adux+0zvedHPjMnXerQ0BZNValht5giaSYk8FP7LVQdzcbh81HESxIuphxMvcBl2yG3DQUX5DS6fpCY5tYSVobcPy3b6bqhV9zvIZSYz3/9LLKl0aijlE37XrXOkXTRooyg5qa9bveJ26xCPmguDa7NG9u9wBGVn5+nIKxvf3UkymKw0/GGWkzXzptqDXD3ValJgtRD/DCsAV+voOw1QO/9AaaEnLxoqAjoRoEi/6CtamNTidkRBuK6VX0d+FHVcuyvv7obO6zZaNMVrsziW4pxmSEXHUmpm2FPl9/OfX3thS477h8Csq7Kg7SIC6STd72xia3skVJOIMg3Fpkm5aMj+1mxqv7z/qwSrMLrEK3knE0JNfd667WWRXK6umj8vRpHqvXJMFSBqRl6Q0wQaL/aoZpDAs9zTgX9L8Y10gpfFJO8rPApZqGKW6wfE/hzan5s/RI6pImCKIogCxiYrFarcKSgpzf8shli20QhHXaHVAV6RR0B8kzhs3DKWZoizoAzAYLoJ6CCipmKUdwustMO4ULg/k/hcP5K47rawTALGpu4ywk0XaokcLVG0CcZNLERIpBKlW9o1QTHBfAYsZ7RL35RyVlMy9AaDWOfsjDRHCqkBuhThud3la+qipknSoY3XvN6qlFMdxmgYZTFarJ57jH3w3WoGhELQqKONzaOXRMjejSz3loZELfuLQu6wPf10BKUz3bw4mTI8wpaQ8Z9gWblvs/ifGWzUeC4ZyDkJPUkCuVW1QBgJySRRvD7emruBDIj/UGnSrlRRDHsSeOSk3FLxMqD9iaDNLdLsE9ZMfjQN9FqPrp+hp+8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(52116014)(376014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DH9ct8UT0snPEyuFSLmNEufSRKYqo0nm/+tTV2MyPG0jAPDKZPy9EZpzvzuY4U3DdjMOtWc2jElprTVVyB/1djNbOwytazBdtGof8vNLVbJclA6FwdPF/B6L+fJz77mhlZotKI6OqA3wb5CYa8avqa+ab++WP0IDUMTlwj4rrJ91Hi6HR5psb53GbXIq9QA3Eo8qRbg/Irh7VrGIHt20cbfvIrk/mJVo9iadGsty5JzI7/4RBK2o7gblxfjMtDpvKrsPEYwpEdH9t3T1GSqQ2f1eEobOJK1PPJfAwN5uDRvXeMkOfI2fY/9Bwc9fJowkuQkla4w3QyFGf+moZmJ9uIt5RWfSpbCgUcNFzkfwZUFVBXWTajj9RhuabIDrg1pCRiqNGz1rTGKtCckZ5mYbTtdXibpEk0EHv1I/6+TA9k1FNcNgbTGlTBPHLQuRpXEygME4suzxJ8dNzu3citQS2LgAvyOalFuvjBOE5YuykWqu3hevlmTRsm5Z1pCyifUl5iwPz6hLRhrlpUEeEaEOonDwITrXHg0OnMnRH1CKyK7uq7fTLYg/OTpRwrDFbr+aYt5gdw6DVKwip16/RLdyxUQK+eJJ7up7VtSJJ5QjktK9J3ZoM95Hp8y9Xo9x0SocIvw6LZRf31Ov5vHUaAwfwdI++kTb3uI7I3tsb3EWDuwtrt94oyO1q2lee1+vTtlaaIYEYvd2VflBPttOugsUqBbb7HAF0CHjCPLkDLUPMcFaj0AEhZM8OYjDL+IJEkmWsRozZ/672EzV+e7B2fSsvBEehV/dwkJzDFkm3oFKpST6WGTIY5YzmpY3ZvLrfcPl8TgKwsxi0tkVpsxaLVPAhrctUcYgRkmu5L0dpIEyLAlHLDeMfe//Lesmn1Mk+3NTGaSx8X0XfnrBFJ6VGXhN8GxV0Gu6KHogqrViJHsCGbJwYGlIdwG337LrxBGCLwKQ+Fk5PhZzPV/I8Gjw8s/bgqbhjI/N08W6+1Mdu6HK097YmkFWANF4lxm9KX7Hhr4pe6wseGtUt0uXnpAu4DOYJm2OJlVipjQpePNjIVWkzX0TiJfQBsS3dgwEAUPQypPACd3U/hY9PYbDjSP6AqXbBp8x2d3rGWxqi6C85c11d/s2hgN9Qp0osFSQoxeFqPws+iRACYCiZomkg56uDA2/D9QdnbSAyYgWZUMKGYEObad9IfiobbkI0fa3uSyVm5DtRYuKmrF0LOH0ZvarkM2ZmI+HhzmaRtqJnroiT4IOXhbRIRKhqsLR+3/3gso7t6yNGT1MRaRcP7uu8hv1xudsij3rF/H1r1LHuyqkWIjrtubG/CyYQNBz4gPMjpInb4ZqsqLOGJarNDKQn1mZZ58kIMFt1YjiIodkI6niOQ5vNOkTzRCYzVfIBS7Kt0a1LLk/WYUWTg2A1VTHsdUxr5ntD0qz11LC55YSVmqOSpCmMZkKIbUwJZ0deAnJ3HRHswLHYYTKHprT141nai/Y0wmVGWtMKP74r1590o8/dISDe6pFJHdAsgYPOj8Q6xx2nt1Xt1lCuasPZ0bcJLBz8uXAydubbtSksZbkF52wZgv/jxBQbfAMyjL6J7BuLiLTNyn6LI8l2jYdUqO0WWiUoHVeSA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: bucDxoom6Z15vcaPfAbIe1Z1dOMpgr37PD6YG+ZhhdSBFh5SIGJi5wRTD97sNb4fyWQDAdirPm4ks4QUOgkbHqR358LnG6t9KrDEdGE4OLbY2hZ7AA1fLoy8K/YGLl9IkBWHFx17zghhyd/7Sxj/tkj+aO/T8LlUZNMpQWxJjuZ68SGg9hzeX4XfvXYIlnx3TUcRTlcDV56mfg+dMhhFwknvAIwq7gjpDrcILtdAAn1rX30Dmpoi1v4reuYiqfmwtLUGTMAuNgp6t0CRWuWUj1Oir4KFkQjLC1BB7zjJguHW6Yt7BjGjga2NrDsDn5EBgTIvrK0K6Uq0B2MjuXcZg8FK2Dps16T2uE+6AZKa5B6InhqiCK7SdeQuEH0OZ/qHggGnNDJyqiZ68im3M5LTT56LwKTj47L/bSKxUmlwtIyU5O2XAY3787kr5ZzsJg2dTmtanuEi3rS3X+A+9v1hR02ipxG+VwXEM6dHJ6nShkVbcWfz5+4NFeaQcoukrQ1cVqmevd35U5ZNioYHj1QfgSAKYluX2RUk6rmiq27fwQwUmLbZ6GJODfXAJYM5mCWcNODJEdUsN8l8FGYcJ7IkXvMDkBPTU1jKxkG+agBxAlrm1HpDJs5oVGT4TR7xtHtR X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4aa08a3-95c4-40c5-4aa8-08dcf1ba8e0c X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:33.0443 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RMmG549HmD8X+Q9S0DGLfDKMsztAkD10t1ELA6YD4crH9zcJnErjpyt0QQSYWHu97wYdrgi+NstM4TNHzX048A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9051 X-MDID: 1729506274-pFf_MJ8A2d53 X-MDID-O: eu1;fra;1729506274;pFf_MJ8A2d53;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Introduce neigh_for_each_in_bucket in neighbour.h, to help iterate over the neighbour table more succinctly. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 0402447854c7..69aaacd1419f 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -277,6 +277,10 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; +#define neigh_for_each_in_bucket(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_for_each_in_bucket_safe(pos, tmp, head) \ + hlist_for_each_entry_safe(pos, tmp, head, hash) + static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { return *(const u32 *)n->primary_key == *(const u32 *)pkey; From patchwork Mon Oct 21 10:20:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843836 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57E9F1E3776 for ; Mon, 21 Oct 2024 10:24:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; cv=fail; b=pm5CrXTsxeXc5CTFbPzI1IuVS9xhrg7QEWaMw2iLNNFWC4pA3Tmcr0pvGKqQyqA2ydLMkwlrxebS9IS6ViH7EI741dhJCtIvjwihRTn8WtPmxbFKGy7Ad+6i89FK0AZ9geO7q9cjVpl98V0g9bn3szCKA9zu738y43dcQQdECdo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; c=relaxed/simple; bh=0iT3+N5VaA0ZAaGrdpKMFu1FPkSiEZCcUdbu0jjJPBc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=mIpY7ymVhn0tQMkDtoSh19iIhrY9xLAuLPHr7BtIAXqx8v3wAXM9hljQrVtEBbYEKF901Efu9s7v6TDQsszIzZ4TjJOV1dWLOpTb2PVrEhTsnGcsQ/FdQ9+ZvVmtywC7V7+gB+EoruoTht1Su3QPQupM9VklQ2m7JyzGBjx4n0w= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=TuKUbFOm; arc=fail smtp.client-ip=185.132.181.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="TuKUbFOm" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 24FF374005C; Mon, 21 Oct 2024 10:24:35 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=u3nhexkwuvqAjm6TAh9tQU4b5zPyi0V/zsW6HJvfBFurnT+hYp+ScPZVdxfZ8djuKlwNbaNODQ+7Bdr92FnTMbLyY+qTUViWXnqEpprCMb+rXJLevEkvPh7VK3u+7zsPH6H+S+necdwFMc/kq+IWr+TG3Xi/GYvObSdLBtcn/DLKPwGr9Gt9Q5ZKJFSswBPgw7YAaekrVRnU8tM4iX7wLbMb40L79Ez/tkg5YbYGEgr407WF5zHticUjnUpJwq55JXnMiM5X5NMzQKObY2+YqATulsDpjxd7+VK3gRsPQC7diqxxUPVCCubIN4y8HXSXyGWG6EnWXjUl++KlQDe3pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w0h10CUPRqmSVe4x9KonpneOcIGQy36PK4ZRYFfr49E=; b=afzVApa8rEwsbP7odgRzx3MrZiaTpI/b8f1EnVb1EQaXR6mMF9SCXw0RSMdVKX/V5Ox0KzXIB+TuA3V3U6qKspXR/xPJJVQ14gVczeIVLpLbIHs2pnYtFIBNzYctvZIEpUCXDKTt5s+Z2tSxZMNRI9ACqZMV8jSMxJa/IvcmQ/UrQ1nZp8o2vmy3BabQ+V9/Axw6UuVF3VgI2vjYV63gnTlcZfDz+6dmAtDoEhtKbUegVt8xUe9x+hLuhncwyQYTbbUrNJmq0WCq3PiSdr2cFiY2dh/gyjeFQ3KuZHSiOCmpe93XDY3Vcg1JjZYP51IN7+3kb33nmyRRvGhUQDxm3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w0h10CUPRqmSVe4x9KonpneOcIGQy36PK4ZRYFfr49E=; b=TuKUbFOmD17pZDx2L2ZNMzkZ+PTN4Z6HRSNOpnisjBjYYCTSS7IJ7Td4bCa+uEErFawm4xWKZv/5XGlrP20DyUMPRCWpbHAJLWbAKnBo1mCHFsIiBLWeJWfp8yMX9FKJcVNFTaU6gqfJI4yFmGJPLNMlKXaN1fXKw61B3rxoetg= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by DU0PR08MB9051.eurprd08.prod.outlook.com (2603:10a6:10:470::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.28; Mon, 21 Oct 2024 10:24:34 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:34 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 3/6] neighbour: Convert seq_file functions to use hlist Date: Mon, 21 Oct 2024 10:20:55 +0000 Message-ID: <20241021102102.2560279-4-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|DU0PR08MB9051:EE_ X-MS-Office365-Filtering-Correlation-Id: 3b5af68c-1bfe-432b-54db-08dcf1ba8ea0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|52116014|376014|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: Nz0dfXXihYFG1uU7Eg2+V92HAzibUFUVLInF/0VyBQczCVdwr4M4VSojAjgGN7e0cHv7EPaBZ4QsNIgJ5efnDdssRQg46JlafRp4zprT1WO5Lz5F8egtMPhAQDpYW9WKiWsVGtVh4iAQljnu/qNPsvQl6X3SOhia3qaU8rgBqqdDlHEn2JBBOQGn20zDnrKj7f4Rm8P7mE/eNrfTxOy2lEIGyTmbtZ/kQvUrJJnhHBdkH6ksLZqHRLXDxxxiVrkhAuqsYszJOBvDgOFoTvNwCUk9pOsoT1nUl1Wt05tuqtbTw+mB07OsdWy1Um5cMm3NDltfE/OkWGGMHkpG6rWITEznhyr+W9TOvhbVM2HIxoDWp+6KTjCB5ZrRdzu085RO5I5i925zC6fPZ9OpkMAE5dMU1rSSqisELmqBpI+LoDln8onWLokoPphGAOj7vnTxDwCeDlOnnAZ+NcHRCYt5JevBafRPY9JRMw4p2Q0jlyLCZPpMQnPtUnXTD8LXs6VyQcTw7Eli92XEvEehu6qTDGQeg3p3wwst/+nHRJqK602PAFP8lyFPt7etN7+jTJ1eWAkrSstICfdQvVa17hYoh4ubp/SazUIFjnwfF9nBo6GsfyXjhgNaAnC+NEnU6w1V2dgpPyt3EVTJPTkdeSPimisdC6f/kl8Qex7OcIeBE4XCwkTwevIvTN8NFiX7cXr4h0ASRlr/KNydDkjBij1tbO7pEbX9qhAJ45JcwSAGb/HUUlkXK4Za7Lvq+9moZRYLpTGTxRWFtIO4DLOxIB8U8aS8SOTVfOqqCXZZNwRHgvTSMwp5PduhiH8RXS5HVQNiOmNh2vks64T+def2almHYBxGfTe0P14Z0w8T54fK6YoqNkKUk8umNcVBpMIpApIvgxUxud4FWXTxkrQ63sjVMAUzXUJDSlxXtMqOegg6O9kCzsZHMJiq6mtufaxq1lt6L4uIHxxGwZdetORRcG1Li7efoVHwDwWoe4DCqzZtZAqTb6vld+jQXcnh+KF2FIkNOqBR0sdHWyuNf8BBibM83rPV6OpNCM7du/iqBjuhTf+4rCkBUZUPCCbpHE4Yw1pplHVc+ahiLNCUs3w91Oc4a+wU4T/wrJHsIHZ7ao6lL3q1LpfxVw9EMIg6hNom+59mgN7Dv7F+Bp4VdTrBhX5CZ6PU8bo98jwIRyAlu8mXzutGVG40UL3x2GUZ9TxgX9P5o2+I2mUzWh+9F0dJRv/9vGxqI8TT2iEm4tirAu2Jn+d0P9HgLDF9jPzVJhsLETHnGEkXJ9NZTPrySyl49yEHHc3WRSeWrrWgEU3GL6DrBtE1/QL543qfimgmFFuIIcY+y4/pOuVTnWBedwLqE0jWDmbkR+wjI2t83/thu0p0XYU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(52116014)(376014)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Xfond1wY7O5Skd9Hf8XoBywFooXIaC9lymANQQTsykYpqKP8WuCPLW0j5ESVyrXfCOMXChf8OaTRNrv4okkDt2pIrTa2nXhbGSjigjAtpNWypTdhj3CqB4rGhOWITJb/bdjmATt77mXT9dKR/A3ytXHQ62o1wqqSwTG9igmxYYwfAbMdVufQkuz53YSgtQv7cBFxwfewd3WCWBYuCPD0h/4pNUJLq5bVadxwWZD1/7XPai61Ip/d/hE87eb2Pzd059ZWvo/uuimqjJbO1nNutJXaJpLoxLprpBsA6hVqXOCfnG9zRdIyrvJUnyN4Idb5+uiMJsQ92ftuRR/ucIo3DJTRuPelY+mvKuJ8FQYNcgPIJbMD2EQ27Oh235IhWgILGmu1lwnDOuxtCOSJb92/V6EIVyHqYy+jiMNz2VL09b+GwNsLuyXzLAosvjW0Nd9GWVSm7aGuiFww3PL7eOHOy4uGkDhCHDDba2Hu6PebHbYehHm++JnQLxYuBnORcku7IKmtjYYJL2prFMZSqEGTZTAVfqvMGtk+GLPcDQKnEnZHB8BKGCh09YOiRBE28TEpCux6uhaQ/WYFBYBaJOFFQGOxJdRfsrlyhzm5WNfJIlxLP5iR9iCTMbykbecUCUH4+B5EYeXBZV37sWutAZwg7CCCfc7v+n8dqVPObfhTg/8Pb2dH4R1Heb/xEwcAe+biG47bNO7TnuPIVx9oOk/4hNDaD6X0Rw07qetMBkKpqhWKxy73pCWxLhpdy8MJvfiP4q5D/C+4+iRXMSb01FCDc1sFF59DxZHboOJcOCC05HrzKDGAl3su3dqR9nn3UsNSwUuWS4GLszuGivrE/42rM3AmpvJLXMqJHBbkRfguDB3pxljjXs3a4zWXYilp/6H6V0J+K0h/Qr86Xyq82KLlAT9Hn2AhhQE3r2jHibOzpdzLlOPumgCKgfepBYgmInQskmK7pKMUfknRcUGYuLrK8ObAOatI1rQ4V5CbOTMYs681G3PXZWrvZNmrsfEf0rM2bkPOpinUvoRFuxaWzp3gT7/7pU/RpeJ7cqpDukK3+MzqCkzZgJs+w5riX/X9tvmWAsnTPOrL6cyeiUFGdM7//ZIGOA0g85CtZ1k7wiMy+BMRlEXVARviyxn7B1hvsJNToyQ7pfSPZmTbiu1DGSoMQqtPElgFuvL7q2SWTfZp5M5u3YEi6x2aOyAKh54Vyw2oebLGwQ8CC8IpxUvi2ey27Cju9GF316dCO94HIqxc7lCVtJGhY861GEvSlhFTRxJHTOzAyntfdibvuaWW0LGnYFxLU2AKs+FRJyAwytcXYRvONsP4TPTLiMh5Ldn8pAvoeDu4joK/RFQULQ7ljIgw5U99KaOUAn7Z6T2LEasnqbRdligQfTZGw7KL3pVQGVgqKEwCzvz7z9iEzK8bj80gMK/B4LH85Ib+FsQlbnh6rtW2fBDHySOmZEabF/0xYsThuJqAIaDzyHf765nRbRDKMSUJBB2zWe7IvoF2KxFpw7F0xlaiUf/K8WmFPISaGSwXE8sYMMdlAoJ5hNr5xPoUUBG1Jz4+8DrxAenZBRO05/rHK/98d4lEXKWuZHtwmIc5ssxmn24kd8Cyc/wAZJbB3g== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: BrG8p3bEGfWN53Vwipv8hqW+IyNhlHs5WlFM6KfNt/ao3WZsDTETE+zl4Yf8oa2ymmYj7OynHzQPvOOJY1QbImRazhfrn3UyAjRy/wNRXGeOzFRrPag+k9VJ1b62VGYzw62ZmZlPjS5ycPXJFVx4bZTvVGXeDZlU4kp7UTWIb4h3IfB8O2wCatN4DbeG6UcaHw4IOrPLY+ZR+UiUqCUhyUyaIvC8UXvG1hfXdBTm9uAy7/BGI/J1S0QY6yK6uHLrtqdfz1EmTrVKeihatvdRCuc5mY0eby2hJSmwMiuAs2xDebZMi4Ko1/CaZtcm7bMt/hoZ7PBFt3fC9QBX4knF4Gw9a8d/14Z0GX9Y0oi/fCpGDsSX8EbHSFc3frfA1Q7haoKeZqIguQ+weqAKeXtgRjBzdUxOFsugg5ll6mCQasa2uwOiYr1NdH+CSSmgLSr2vcXg6CdnAj+t61Jz84zVGNRa20XwAWQ9WZOf5DwlRpVsHkzHR6Py5NlYGVvWMCXY7cLhQpJCH8DXJV+ie0T9jkp2wQaYYJ8WG9dw3RaIfgZCuWQ8GHsAbwfHuw37D7GzEPKbz2nAfAfYhMQrg6SW4wtBEWndDPpv2HtNccMYS2BgN4PJuRFPsHbgwUCjJqXj X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b5af68c-1bfe-432b-54db-08dcf1ba8ea0 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:34.0067 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: D+7Hujx/VqYBWCrXO9RE17fTEekwBj8bmnaOKsxp+JbKYW6wwlCqVQ7FZKXuj3Y5QZd7dwAQRR3bmGobAArDgA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9051 X-MDID: 1729506275-g4RnBup4Cl9v X-MDID-O: eu1;fra;1729506275;g4RnBup4Cl9v;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Convert seq_file-related neighbour functionality to use neighbour::hash and the related for_each macro. Signed-off-by: Gilad Naaman --- net/core/neighbour.c | 104 ++++++++++++++++++++----------------------- 1 file changed, 48 insertions(+), 56 deletions(-) diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 45c8df801dfb..e70693643d04 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3226,43 +3226,53 @@ EXPORT_SYMBOL(neigh_xmit); #ifdef CONFIG_PROC_FS -static struct neighbour *neigh_get_first(struct seq_file *seq) +static struct neighbour *neigh_get_valid(struct seq_file *seq, + struct neighbour *n, + loff_t *pos) { struct neigh_seq_state *state = seq->private; struct net *net = seq_file_net(seq); + + if (!net_eq(dev_net(n->dev), net)) + return NULL; + + if (state->neigh_sub_iter) { + loff_t fakep = 0; + void *v; + + v = state->neigh_sub_iter(state, n, pos ? pos : &fakep); + if (!v) + return NULL; + if (pos) + return v; + } + + if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) + return n; + + if (READ_ONCE(n->nud_state) & ~NUD_NOARP) + return n; + + return NULL; +} + +static struct neighbour *neigh_get_first(struct seq_file *seq) +{ + struct neigh_seq_state *state = seq->private; struct neigh_hash_table *nht = state->nht; - struct neighbour *n = NULL; - int bucket; + struct neighbour *n, *tmp; state->flags &= ~NEIGH_SEQ_IS_PNEIGH; - for (bucket = 0; bucket < (1 << nht->hash_shift); bucket++) { - n = rcu_dereference(nht->hash_buckets[bucket]); - - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - loff_t fakep = 0; - void *v; - v = state->neigh_sub_iter(state, n, &fakep); - if (!v) - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + while (++state->bucket < (1 << nht->hash_shift)) { + neigh_for_each_in_bucket(n, &nht->hash_heads[state->bucket]) { + tmp = neigh_get_valid(seq, n, NULL); + if (tmp) + return tmp; } - - if (n) - break; } - state->bucket = bucket; - return n; + return NULL; } static struct neighbour *neigh_get_next(struct seq_file *seq, @@ -3270,46 +3280,28 @@ static struct neighbour *neigh_get_next(struct seq_file *seq, loff_t *pos) { struct neigh_seq_state *state = seq->private; - struct net *net = seq_file_net(seq); - struct neigh_hash_table *nht = state->nht; + struct neighbour *tmp; if (state->neigh_sub_iter) { void *v = state->neigh_sub_iter(state, n, pos); + if (v) return n; } - n = rcu_dereference(n->next); - - while (1) { - while (n) { - if (!net_eq(dev_net(n->dev), net)) - goto next; - if (state->neigh_sub_iter) { - void *v = state->neigh_sub_iter(state, n, pos); - if (v) - return n; - goto next; - } - if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) - break; - if (READ_ONCE(n->nud_state) & ~NUD_NOARP) - break; -next: - n = rcu_dereference(n->next); + hlist_for_each_entry_continue(n, hash) { + tmp = neigh_get_valid(seq, n, pos); + if (tmp) { + n = tmp; + goto out; } - - if (n) - break; - - if (++state->bucket >= (1 << nht->hash_shift)) - break; - - n = rcu_dereference(nht->hash_buckets[state->bucket]); } + n = neigh_get_first(seq); +out: if (n && pos) --(*pos); + return n; } @@ -3412,7 +3404,7 @@ void *neigh_seq_start(struct seq_file *seq, loff_t *pos, struct neigh_table *tbl struct neigh_seq_state *state = seq->private; state->tbl = tbl; - state->bucket = 0; + state->bucket = -1; state->flags = (neigh_seq_flags & ~NEIGH_SEQ_IS_PNEIGH); rcu_read_lock(); From patchwork Mon Oct 21 10:20:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843837 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5A70B1E3DF3 for ; Mon, 21 Oct 2024 10:24:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.7 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; cv=fail; b=Hq/kf1SdAI6uxLKXqQk48wBbx7Yt8L57uKSZXQ4iSk6pJyjdUaT2knDW1VVg7lr04kWkajf4F8TZ5zpT6LJasFfYmkgpciHFXNH2dXKorNsWGR96ENc4/01HcsW9wpXTLscm3faUXMq/XA7xuenJ3mxqDl8KZpuWWGnbZuG9EC8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; c=relaxed/simple; bh=E0Q+txzHoHcyPDio2XkdOD9ksxQytXX3Bdj9z/kjmhc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=XjY75/dtp2IrTTCBUNl0KpCFzGhiiZaxb9haowTsLLrKY50gJUzQAUFcruQ96+19g3ESGAQwUq4nUXxlSxNVvDm0tDBW8BAR+HfThzxJz9qD+ZKB2oQjuKTK071jQaxC//6pAbl+enozmGqCbKCRUsPic7EWH3R6Pp4nmkR5atA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=PNx7AaRA; arc=fail smtp.client-ip=185.132.181.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="PNx7AaRA" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02lp2105.outbound.protection.outlook.com [104.47.11.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 4D9BBC0062; Mon, 21 Oct 2024 10:24:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sDRcJG5v9SfDCpQjjR2S53Ezf4b/didaiU/zbC6amFBZZlcNbmSMOryapaivO/rY9sjnDpUnmLcO/E7ga5g+5gpOflt+HgrhH+mTVvhhlw83vj57eKSKBPSGx8xKpwUUf4X/SZYKRay5r7rALmOnYt3+kVreQVXQ0021CGTxSnQ6KCRQ3lP7j4khKQzy1o6cNv3oo+rhscf4qzUeG3TeoyQM9x5a9d7v1A9B5qKz4QaIgACbGFXtoAshEYIfbwaqtRZ9HykU5dyr6E6HTqok13HZwIO6Xby4eDRXQz8dY5eBZsDlvj+0RJST/y7alBefDWgasVrOnYV5rg9xnArfAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CV938W5MlSLYy2/R85NGtVzjF4XUd41Xv4F2LHIOI3o=; b=VqiEz6lqL0F4k5nO5iFD6IxmOedFSihAL8tePE0YVvliZLjrJEi3HMpWuyLfGzUZ0nuhsHWvhZOfkBGrRI4AvyX5lMQkhdX6S+ij9EFkuc2S1u7VugNWo2QSub5DiU2+nIjNHwlCSsUPfPr3JJujhCl12uAtIPfLZSudGc2HiQ1zuWN5whRAsxmMD7q+vr357RneEvJFEXFV7NoUIawO6ewD0ikFBXhgtr8sRJ6EZPdKxCtcQVpCK9RQUV7bGZw5GgnInD0o46M3MuktJOhP6v/v5z4HrQIhXD5D4qa82TxfaMZ7h/28ixCN6U9weTsGjUiljR3sP4E4d9ojuQPA7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CV938W5MlSLYy2/R85NGtVzjF4XUd41Xv4F2LHIOI3o=; b=PNx7AaRAihRUEgFLclp/7p122Xm3mKO3FMP/GYJnWbD4H5JCTIVSBdYJlP+rezuITHhEcpeVV8DjXqi2Xw9W3MFQAi3ijwE/ew9xUpiucPGJLitMpeF2uXmv6nIQ17yZUkjihs3FlNc317BlyhVB2d0CYTVDYBsoY2TSfoZb56U= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV1PR08MB7707.eurprd08.prod.outlook.com (2603:10a6:150:52::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.18; Mon, 21 Oct 2024 10:24:35 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:35 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 4/6] neighbour: Convert iteration to use hlist+macro Date: Mon, 21 Oct 2024 10:20:56 +0000 Message-ID: <20241021102102.2560279-5-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV1PR08MB7707:EE_ X-MS-Office365-Filtering-Correlation-Id: 66746bed-addf-4ef0-bad6-08dcf1ba8f33 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|52116014|366016|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: PKZPpVw8B+M/tyYi8BByYlSHMvWWHwIEXo8q29lV4VjVnLRwHh2pK2rgQP2AxN6DSQDfmrejw3VWslI3N6+G9auyOy9R5jJu2Ojx6YV/Jghvw0qILnK/93/QIx9ayWw4o/Wuy9NauH4dYb83cBf4govWBAC0ho3057aM9jsKyi1o2nBf1ojr1X50sm9ivrbkqJYa2hmprNau3NfJHrk7+9FK5wMbx1o3tatZX4e3XKzl+hjcgmCQH20y+JDjjbdV4Y4awsAHU6K7CCvujI14IoGDqMLLmmotNf0UiQhyv0kzAGTNRA41UQVabrsLNLmlNQkBcI2fHWb5ZLplU95ghPF3OL60mohyuqo2NUZagbMTlGkkx1hCsY+5IR8M59CiadYiKLRBgY1cU5ONxPx5/xawTeBdDb1NE8S8Beqx6qS0Ng2TCRlBYjPfnGEhLlNTKQ36/EWpqxDQy53L6g1W55fs5azhi/Iq5gNM136ZYPgK0BQ9H7Ci/L3geuSCbENKt3c2fIo0kb7BiDrLP9OgxpPPQNUGxkF0Y4v7o3JBU1Gy7NMamiw2u2oHg8wLL3CF+VUt/EAnRRznv6agqjeQ3ApZFA20c/tAFIcJiwJluiK1Ga6DkIS+lJS/d63JSn0aDEpJPMNbhb1Y3DWJ+1t046AZogwdIpYIm3Qw6GJ00HKrS4TQ0SkgK56EctivFD2YA308ymvB/eSiHbf0d3Wv3sNfFBX3Af9bJIr2ctoYupU+3iLLEUSD1ywwtGt57R2/oLVmirnbye0IWiJaE+UWrZc46vplN21isYtJi05sPxscAnUE1FJPQi2Q7gi7XA+3BJcpeUeTNt2F6a+EBi7wAm2cbwxz9LVTLk0y75Fobk5mLXMuuFGY0+YSydnsZOrXnn3LpUu8rGKb5JAvar8KajAnplziFNVydpO7Ti0YkvPGU+yE6CfmU8PN/0WSd6bH6uP9N4MGWUqI7ndFPLXQQssW3P+VeLPbFKl9splY3HnQjoLW6YirmJIcZGzpSGL/saB+lJM9f0aJW9jmJNrl1gLoUPifDHmfvN3+2gvsncv8rE6lxmv8Wbl0fP21/JEITobNi+xtbIN7HnqIp0A/aEihHHbgkdJERXbh7/9bnMhS1YTAmNsl+9j1gkn7EgKKT99oLenSZvA+U94ij4DQz/wfxpu0uPkQOGdW2d/1luAju1bTW6KwKT02UuigmOBLkjwAr2W1bnR35E5tQ1iPZvuoqB5/GS0mXfxoVlnF4znTnlxorAZRkyiEuDI1eqeSOkag1qGz+iM0D0BCQfJ8MtuYt2BDmayYENKCvVm0HTzR6VaRKIYS+oicvDRrLRG6zTGAz898JCs/mX26SfUjEYN2vkP3WiatFtsMZba+PPA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(52116014)(366016)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KfyRKmZBZ88X2kQxvrI0FaL9kjduqIzotOeCP/AnS1PchBpngAx0ojN9Z3KO60jnYa/Y1QOIdCpdvN3SnlFsMyQcNDfAbIvVw1Ty9WPipBQwyafnq114ogmkNtEm2yLJES8l1Mbs0DNZfhF8cjHcdZYjyu9ppSEv3bdYFTbepZVinTzVBcOvkHpIwjD2VzUoMc21NtaRLiXjsnQSWfgQT2/lSmVp7MW4bVfPbnSgcoqmbC350ga3JONU3N44JI4UaOfjXhN3Ub1Vs9bP0vru5v0CTnzttuS+QJ0SP0GkqWDEZkOtJ63rVLocjHOiKY0awecNYLXLgeBXqcLuwae7ShuqDkKDOCWquiMq8WEJxyMilrVDdDx++4UAMqCaXs6Yv307dq1mZtFj7BkUD1x3avyQd5yOMHIADMQzpm04pAzT0W+BtdVnn8taiqlKGf1X2HIS51kNEL8k2Xr7SVChEJ9rNR/2T4/4xN9DapJn5/U9I0Pj3Ga9bI6Vb0/KeC4l2Srm9WBl+ftX0Ee3/Mf1pqT+jFo8RDbwX4AaVeNOf1RXRbgsr2gtNVylZF8XWGdgfvSwuAAcHlhr65lbw1YUvYWi79ibX+QMtKIcskHEyuwPPW6vpaHeq4/UkJls3yq9cxoydxzuWrfKJnuIuzKyNLpqTToB+Fi0wlFU3sHStp+yhxep6GSE65TaqQ0JfVAGD3mCE08BPu7UfFNyr9Lnu9hOOpNuwyh2/czpPcVOEjuAlRAVG4Kf1CoEXNwAedrkw8uRzoXulzcjrRouuNbtEjAncqe6bxc0xYcYYJsSlQ7c1G6OaDxIY4RTiYOv4CJJFVG6BbGorebr/+muNc3R2tx9cDvu+1bzvNHhBEcsGC8pEbTynhtLe6C2u7GYFknDmPesRajaidGS6WdPDGPqQnTneEPagBz8aWTzovqz2S03XV96Xo8/nZpenX0RmQ45js6XfxuPCRSuQKdwTSCxMDQtSyKqgbrKFAD9+GYDGj83II+yWCWi5/i/YbKBcHjkvexiyVxJyqi6EBPYu/aF7w1uQ7zdfKn3oSGCsLg9iBlfA9H9RkH4N1EFbaLtvb3yfT5X805lKbxIKgADLrOwKR2nPhKpJa28TjvsOlTyDNkhYz+5GbDRYHlrgFFpcQTKhe+vK6xemTzuDS63eTulDqXRoHVjDAyWdSKHW0k2TPyidHxs+oPr8Fs8owG3DfDX2V30dX/UB6MjHHoDXFvrgTrCNKe9JDcvIifA0Tbw6oMnY4wCmXzXon2/QwM3UoseOszPiGrc++cYSvDo+PoFXLaRmvHi3XU4E9K7NWsza77tIgCL0k/7wURuRM5bMQibg+652kAX7uve5qL7CobW4puGUoNHRtkj0xBch2NpUbqSzFAHmNRuRejJMVBA+9drfIEwUKFlzrg/VTrDxHwIdozj/TdjUG4msMNkrD6DoVBLdObwubYI66s4lqx7beNfWXkexplyXo7mdE7POIWmfjefTJEvG9r4gZmp56HmE5bKd4H4bBk/yCBicK1Y7nE0c9z4DVNV4Zg+pA2kw3JiABxgwhCUxpvjR4oXl4tD1YUU9LDSqoljwrirVquzgPG7yDPf7ojfGPcMBLZqbtTdDg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Zg88QyFId3r4SBo9d+jJ1OzGOp9XwFZoGvI8knWrLfulcX3W81yZl4lPFKdZPtI7RYIh/N65TrwweYUehT6/ttwmT6se7C8Fx4gDfSo/afaoXhWYOZddmfipPzAt7HsrxlSGxdDYk5PYXN/qhAXE4oacnEywK2N+FVuy7qjDcuWvN9lG064M/y6tz0oqiqijORgSJvKWZRAvECvXXLS3sCYWBp79N7OfMWwoPzpN1U0vureOCLae4Ja1y7VX/WsykYTZgzcj3Ed5/M9ESbvsUAXRiLbe0Pa7gMeBUWTNT82gGQFodSxeXQ/R4jO494D5Z1jNHeuT/bghKtyitnuvLQPhDs9fXsFZ3E+PJbEi4GuGYdPYBe3ZBnyptpSoON7DfWQjH/0sfYEp3YTqApLG0B1BJx8IpZl915Ibf7Kp+xG/OsDFo3426o5eOL0dpGj2w05nIkBo/d+X/tmyJPIOWi3jTNdmyiosXVGle5faKljVGQFngXc4iJZiH3pIWXj3JyUHPFtpkndYCdkerFEIrDGyqQB+w75FdfucdPo72qsjJRCpd+zkSqZ62vJ3i1PzsMrkcj1WzPzgGGFNQ1ZVxekPShw5joKOV1VUMiaaP17oYwrHRdfjHYF5GSJZVzo/ X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 66746bed-addf-4ef0-bad6-08dcf1ba8f33 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:34.9731 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /pAKl55YXoiSaZxDwlZZCBObkvdAFVBXr0fbt8WQQFyRVhlhQazVmT8r3Q80HHMad3BLFvR/wBp0hVMb+9lAyg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7707 X-MDID: 1729506281-trmW8L5bgJG5 X-MDID-O: eu1;fra;1729506281;trmW8L5bgJG5;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove all usage of the bare neighbour::next pointer, replacing them with neighbour::hash and its for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 5 +---- net/core/neighbour.c | 45 ++++++++++++++++------------------------- 2 files changed, 18 insertions(+), 32 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 69aaacd1419f..68b1970d9045 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -309,12 +309,9 @@ static inline struct neighbour *___neigh_lookup_noref( u32 hash_val; hash_val = hash(pkey, dev, nht->hash_rnd) >> (32 - nht->hash_shift); - for (n = rcu_dereference(nht->hash_buckets[hash_val]); - n != NULL; - n = rcu_dereference(n->next)) { + neigh_for_each_in_bucket(n, &nht->hash_heads[hash_val]) if (n->dev == dev && key_eq(n, pkey)) return n; - } return NULL; } diff --git a/net/core/neighbour.c b/net/core/neighbour.c index e70693643d04..395078d8b226 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -388,11 +388,11 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, lockdep_is_held(&tbl->lock)); for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour *n; struct neighbour __rcu **np = &nht->hash_buckets[i]; + struct hlist_node *tmp; + struct neighbour *n; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { if (dev && n->dev != dev) { np = &n->next; continue; @@ -620,18 +620,14 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, return old_nht; for (i = 0; i < (1 << old_nht->hash_shift); i++) { - struct neighbour *n, *next; + struct hlist_node *tmp; + struct neighbour *n; - for (n = rcu_dereference_protected(old_nht->hash_buckets[i], - lockdep_is_held(&tbl->lock)); - n != NULL; - n = next) { + neigh_for_each_in_bucket_safe(n, tmp, &old_nht->hash_heads[i]) { hash = tbl->hash(n->primary_key, n->dev, new_nht->hash_rnd); hash >>= (32 - new_nht->hash_shift); - next = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); rcu_assign_pointer(n->next, rcu_dereference_protected( @@ -726,11 +722,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, goto out_tbl_unlock; } - for (n1 = rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock)); - n1 != NULL; - n1 = rcu_dereference_protected(n1->next, - lockdep_is_held(&tbl->lock))) { + neigh_for_each_in_bucket(n1, &nht->hash_heads[hash_val]) { if (dev == n1->dev && !memcmp(n1->primary_key, n->primary_key, key_len)) { if (want_ref) neigh_hold(n1); @@ -982,10 +974,11 @@ static void neigh_connect(struct neighbour *neigh) static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); - struct neighbour *n; + struct neigh_hash_table *nht; struct neighbour __rcu **np; + struct hlist_node *tmp; + struct neighbour *n; unsigned int i; - struct neigh_hash_table *nht; NEIGH_CACHE_STAT_INC(tbl, periodic_gc_runs); @@ -1012,8 +1005,7 @@ static void neigh_periodic_work(struct work_struct *work) for (i = 0 ; i < (1 << nht->hash_shift); i++) { np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; write_lock(&n->lock); @@ -2763,9 +2755,8 @@ static int neigh_dump_table(struct neigh_table *tbl, struct sk_buff *skb, for (h = s_h; h < (1 << nht->hash_shift); h++) { if (h > s_h) s_idx = 0; - for (n = rcu_dereference(nht->hash_buckets[h]), idx = 0; - n != NULL; - n = rcu_dereference(n->next)) { + idx = 0; + neigh_for_each_in_bucket(n, &nht->hash_heads[h]) { if (idx < s_idx || !net_eq(dev_net(n->dev), net)) goto next; if (neigh_ifindex_filtered(n->dev, filter->dev_idx) || @@ -3132,9 +3123,7 @@ void neigh_for_each(struct neigh_table *tbl, void (*cb)(struct neighbour *, void for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; - for (n = rcu_dereference(nht->hash_buckets[chain]); - n != NULL; - n = rcu_dereference(n->next)) + neigh_for_each_in_bucket(n, &nht->hash_heads[chain]) cb(n, cookie); } read_unlock_bh(&tbl->lock); @@ -3146,18 +3135,18 @@ EXPORT_SYMBOL(neigh_for_each); void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)) { - int chain; struct neigh_hash_table *nht; + int chain; nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; + struct hlist_node *tmp; struct neighbour __rcu **np; np = &nht->hash_buckets[chain]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); From patchwork Mon Oct 21 10:20:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843834 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA6291E376D for ; Mon, 21 Oct 2024 10:24:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.6 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; cv=fail; b=DWuCAQ0HHWWyEFNtQ9Dp+238V+nVrm4ambthexirscshbzi8gbMW2l08FwFsLA27lut94rsKRdeVO2kJpSTzySB6ymYbgRxU9vsAcVRdqd+T0K1nw6f63jc2wo4dvMRexW1IkEg1dHBQJ5qqhMGzUAIa2cmn+QAzprn8D3kMNVY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; c=relaxed/simple; bh=3R3Y9NdEOqtW5nZrMNTGClJCrALUqZ+TDHsDb8lDYSM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=hXjl5rnNekSv5LnUHUT8uLT660+Dhx9jds7oeTXcrm2HLNslHrA0pnLMptDXEWqBKBUK5db+8t0ffz4f28Bo+btAz1y+sdmyAQ4KnwEsnDABjOZMCEBQ2LDtaCQyfVf4eLDV1x1IcO3AkiFyjA9kcNADYsjfKKeQLGkGth9ijYg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=iRpMCWrG; arc=fail smtp.client-ip=185.132.181.6 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="iRpMCWrG" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02lp2105.outbound.protection.outlook.com [104.47.11.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 16BE0C0061; Mon, 21 Oct 2024 10:24:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tdNBdIOgRMtgGehH5DdgaJ21cYAUn1Jt/T06G+/Q4seKol2pJ4N/kjB9DCTsuYvU0MfBU1Ks9GNXQFmh0n1qId35G1Ne1v7iM3731xkoFNCdM6lCuqMEiEQCPx7LQJ/1Mxrj47+od/iubyPKFvoYeYPO3WcLB+SJ5aW3sV374fUPCEnjxYdK0kLaGjFX2GR7H+yEsJkImHS4/iBwXUOmGttl2sP+RnvZk6u10P+mIBKAgAfQFml6nUmfPcrVN9yZo3NiRK6R2cyfIclTr6DPFteFTRpWYbRj2YfpbcrCaaFAjpCWkHhnUvgoGuJ3M1glS13FlKtRUV0x4BADh8VwPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9vaIc93buxMZWlnuWFwbsZHQkrBZeYShSWJoJaAvaxg=; b=uxr0lP23UyBjqSZyJoXyMX0XFkxCAqi972jgqDK9JZV2brJK/H3ShV5uWgyn2pWRm6o6L8i1AsfyZkQ4N//sXfn/X/khlqqAM9Y+/SWwz0eIOTpsT7za2rm2mnN/14UevgxFl1pJFIOlDJYsgXksNvXPCxXmjqe1tdrvsyvqM3ysZmbbh1BQi0Z29E+UesK1FbslxeBt12s21+KuhdWppAVElXVKH+RE9IYYd0cmmt+fVApEhBQJFjzaFLUGnw5zniL5NfdIUfv9FwdUV5mYvqDYaOIqiW35+ECtWuEELdFkLoJrLlSRN0eh4umKdzSZw3ZDUvQK4wfBbsXJnd6NvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9vaIc93buxMZWlnuWFwbsZHQkrBZeYShSWJoJaAvaxg=; b=iRpMCWrGt4bGnn4sKy2yUDB5HMR5cca7dIJqstdvvhA01RO52jAN2evuDXTgDdUjP+XCmmiBZ8mAMxof7PHA12DDtXxiaa2te0vxtXp/nmD+gVE0k2kVUvWDGgPauSu7DuYBV1dpEFz38ChF8xIRK3jVHmxMiWAP/45CO/1UF2k= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV1PR08MB7707.eurprd08.prod.outlook.com (2603:10a6:150:52::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.18; Mon, 21 Oct 2024 10:24:36 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:36 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 5/6] neighbour: Remove bare neighbour::next pointer Date: Mon, 21 Oct 2024 10:20:57 +0000 Message-ID: <20241021102102.2560279-6-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV1PR08MB7707:EE_ X-MS-Office365-Filtering-Correlation-Id: fa6c973d-d5ea-4419-a568-08dcf1ba8fc6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|52116014|366016|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: YGhLw8yeGEM8rSS3R5x1UUeyYohrdoehmlUhF/oOfj62GsBzQmTwgkyhUC3q/3kgzsxo3GuruZYe6GAPdKyMRjEgqt+lNMFcAgg/XoeI2wfGMhASnx3Nc6xX+Y/wppIVDNcv8BuG5Bv7IHNMjNPHFe18Qhnq8+KWxszZGAyBs+BP+R6sssdseSdFG/aF4Qg4Vyc/gttrPR/H4Ym5mKI3a0gTUZ6Rj67Sy0wfYBRVGe4UicPZhcwh1XeE5xHzPnaPsCFZFLsI4F3rA4+MnS6A89gFSlGYvbSBa+LvDKSzUU3vtzhI3jbeYK04vUVge7Xj0b+Wz/M5pE2HYitfC4sGIQEsHclRL53koDy07uR0xjriTaTPr0KqU3RxhBvRBPruZ0uDmfKRAHwFnNdFeKJ+UNv2TUzCE/AY89cGrNbAc+LB2SA80N0p//ZVtMtoANuXnC2W9x3avVSeO4FNd6o96jQeJfbaesq0HJHRMwy6ZBFaancgCuqb5EnJxsxQt/+gNVNhBhfgjtt+eRz1GaGE9JTV5p1cgY4kpdfLYxsjK0idFWY/IkTrmSx3yJdmYrlRNN2FsCVveeDKWryn5Y9oX8bJt3XhF5yrI2veR8ZWgy2Wnva8zSlh+B3LXev9tfk7nimoD2K5EvXu217k1EZGJ9nTrfEHNces9wi1eJoI7Ma4LAOGHKXlyT14ExRXx3dfGvDYDJom5591Y1cCJq+SdMNOyos2grnDZ4TBKpWgERACKSgB5W/mxBOfSCV/YiZ1pyy9XwUOQIiQxrElbD3RIVxnMrcLbh53ySe3td/DdDhN7YWcoQMbJbw4hknRzPXNLVTTrtMveo2kSMl/AkzpaZe3CAvpHMSWJ/LQLtIcqn5TFPeEFlW1e72QLuHgcLBLpSTbKAf6sr1JWvajtWtk2gqv+EDj5SBvQw21irM1jn0C6RMmFSDhJn8gHAmSP5R+MXXVIrMWrHgHOhKwiILdtrGnu74MURfq4j5Vdtm+ZeLuiU2QpSBm+pSosUsVNN+gWJD5GyDQasftsBgQyowLzq3Nh4kBsYgcbXFKNqa7mZmqJIGnDqSx+7DsntRtTIKeFAVCK8p0rn93+oNtT2GyYZYN2N9OMahgNLkDTTHidWSjQgqX9h7GkVSJrRQA5kyV82HKfK1bNQjJn33JrVa7luzGHLyRaCKxk5PL9Yx3GggMFe4Ms4VTph6nexeVHxTCfg8gCdjNvKQ5reyPyi+7eVGtLdf5IAjgoQS/V6gikhsLGjtkwnbxX8EqvudYWVOqgcxIQT2qmNc9fWENRrQZLrsCswa3Cqjk2A68lChmp7PDggChvD0WDgg4ekPubQvLMcSO+4f0zPA6opCHiG72Z4vgF1Vgc6vLsS9BABeu8as= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(52116014)(366016)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ESqPrcwJzCA9gMSmUBI3ZXHyFfaEFrLlenuTbx0lpNiP578j3katU5UmfHQZu6Z+74ilDRHeXy2fcI/GguxzhfL1m98VVMGmVrUI0ikbsuTaqEjZu4EqOv/ge0oDgWyUOsMO2SJZu2PQ5yCYquTnvSJQqhJz96/GPzP0XleQFb5k/kjbq/x94FcIaia4sz/Jo2A5mj99xxPNcCxesjbrN3XEgSU3kV3rU+vDeV1kZNlqbm3ZEP3tH8PYwqDjAjognJkqMWYC8xw8kWRNauIy0LxpIx9Rw09bB3mt6rrXzVu9ajdCJ8m47MOHmSaMTVawetx+qpZ5ByU4bRwHTfXNvozlm/KAMkcKqmMcwz7bbmBkAboQN14Rg9zWOgu0LIWUdiSZH2qNyxT7QDjPRRwzk8vZdeS0M29jsIXFFFC/uDydvYwY1sO7TTypd1nsw5CPqETWgkBOk6Y82O8y8VgRaSNDFAFH53bh07bphs2t2sWUnK899oH1U8kWu86YXmFk0WrJkJjg2PHR92c1k4XPJ4yrwyRRMxf55k5u43J2pq/vLBOCINda9wz6KMqdUSszfBWpZPjKJ/OerpRrWj+ZZOUID+6aff1JMd8qhCNLsJC7AsWCNji4Fe3qQBOGb1XLDaNWlhOE1ws3t33j7XTRGqEpetXv4jP1+/malB4eAuIpCsaU49RTHi2eXdMatXmfbU4loA/g/TodNnFa+AhD6qXKi0Fo8nna2VsflP+NNRLSkwZuF/ZMuOfocf+o5uyqUFCWU29sJqA1LhpF82gpIs7yWcBP1q2mkCQ+0uci8vDoddDOGkug+kMy++72R5mQlQNE34EPMqbPa3M/jkX8wjSAXD693sqC4ccT4nM09NzVoqJnpbWjz9TWH4cVfplB/rPyBp8ngtfva0Wzn49ic9OYNrnSplf/7konKldmP6/wDZucBEQCTIKlMooE+KENpaK3Npr0aORKfHByMqpzNNHtJYRydeUfjW2HMEgS0NANBUn6gty9X0yyo5R2ib/hMa16oYcx9KaWCWcmnShqqrgatTwVPYD2bWES59Y9cJMDpqyvrxR0NVFfVaG+p6y0jSrCqxL75lD6E0Nekg9O2K0HQaHQZHFZVQ79xm47L2KorAt3tgnKomnLqT+MKdw8g1mKePcGbaHk4lDjRZ3h78uQNg9FieVbDbw5BDIm3ATDC9xUpbPLppG1mZBpQibhWgiHg1kpc1D70NQNXHnz0mMQo/JmxZgWMOQbVAfTkGLH4vxl54dH/8gPTrlb5fFEU98v6LvMaEqVDRCBrMZIZXY3LY+hixl6HBPNU1lcUvtG+gI8KR8FIZE5Dn+hstsYTyoqhJIFvmyHhAj1BG6BbZifhS3OORkDP0NypAOYUSd9TDFpC2/jKOolMrpQiSuZNXav2HKoH/3QWE1Oh8mpINZ0DYQvpAy1H5Qr+LhNMnct41auRGgcAD+9Bn84iTlxrRfQzqJZ+ZCPgf1qVGjXEdrC/+mAAH4VATTa+Xi2IcD+NOIj/o+vD7ghCncxjFOHq17sQNzzKG5wGqKSyj7o+WsAKv2ebJM/tUTyfHrU+FCv2LFE9Xq2i9tY2HXii/5no4T02IWXwb7jsHOCJnV9kw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: bDZQ9UAkChqx2scmb7+IORCgi7KRjMjaByDMSpY7uo2G9YaLxzOVFuOeS7wMXNd9plet97JsKKA4cCujUBmSL5YIHG+79Uu0LPVfr4eWFTrhzy8XqaQpN4SBdoTVsXwL1E6pz4L2WIrj1LUAFf7wnmjh+G7AA1TMnUcyJbDJFt4DbXeBFUcmaDgrTf9iQG8Agg5su4ILs8ENd9H6ESmOKKvNwJJ5/qoQ3fTlF8hAwaSAcsiphzNVgC7OKq3dYUVfRgh2ATta1O7JftAotv8rGkw/anhIvFNFoC/wqhyUM2nV+vpoYEStr7vK5r40eyTq+bTvGUCmSCwtilhIt/1Yg04FQr8kMTnsBYWHtmSPeFntPEKXBnPXpuoLJt/crokTGOdacmmzAMgBgCKpZaLqetv3a4ECMHA9jWUjxPD9VPWSa4lX8/Em+T61VyylRKKyx4fK/H/lNpGAfr/NA6cYL5l/ZiZeA75mt8IcDVL0J/VDYjuWSyfZrZD7bWyeagVgz5rlsfuJCMBHmzBvhcY/27C2pCjBkIkxGW/381MJnp2lBL2YcJVXBg3EnwDadXda7Df/xJ8gPU1QWPmNYOSXPYXIlCCaGWVNEQiAy9+sbyQJgoa/z2yRGEA7w6lVwPY9 X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: fa6c973d-d5ea-4419-a568-08dcf1ba8fc6 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:35.9425 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TGhvGBigTtpYNsd2Vk1nUAo6QzDTXMCC0NH9vDw03YZb5AH/DathDdI9brv6FgW7wQJrLGp4qaoO4ZfoJjqJvw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7707 X-MDID: 1729506280-EG50qw8tYK7f X-MDID-O: eu1;fra;1729506280;EG50qw8tYK7f;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove the now-unused neighbour::next pointer, leaving struct neighbour solely with the hlist_node implementation. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 4 +- net/core/neighbour.c | 118 ++++++---------------------------------- net/ipv4/arp.c | 2 +- 3 files changed, 18 insertions(+), 106 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 68b1970d9045..0244fbd22a1f 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -135,7 +135,6 @@ struct neigh_statistics { #define NEIGH_CACHE_STAT_INC(tbl, field) this_cpu_inc((tbl)->stats->field) struct neighbour { - struct neighbour __rcu *next; struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; @@ -191,7 +190,6 @@ struct pneigh_entry { #define NEIGH_NUM_HASH_RND 4 struct neigh_hash_table { - struct neighbour __rcu **hash_buckets; struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; @@ -352,7 +350,7 @@ int __neigh_event_send(struct neighbour *neigh, struct sk_buff *skb, int neigh_update(struct neighbour *neigh, const u8 *lladdr, u8 new, u32 flags, u32 nlmsg_pid); void __neigh_set_probe_once(struct neighbour *neigh); -bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl); +bool neigh_remove_one(struct neighbour *ndel); void neigh_changeaddr(struct neigh_table *tbl, struct net_device *dev); int neigh_ifdown(struct neigh_table *tbl, struct net_device *dev); int neigh_carrier_down(struct neigh_table *tbl, struct net_device *dev); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 395078d8b226..47eadf1b2881 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -205,18 +205,12 @@ static void neigh_update_flags(struct neighbour *neigh, u32 flags, int *notify, } } -static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, - struct neigh_table *tbl) +bool neigh_remove_one(struct neighbour *n) { bool retval = false; write_lock(&n->lock); if (refcount_read(&n->refcnt) == 1) { - struct neighbour *neigh; - - neigh = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(*np, neigh); hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; @@ -227,29 +221,6 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, return retval; } -bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) -{ - struct neigh_hash_table *nht; - void *pkey = ndel->primary_key; - u32 hash_val; - struct neighbour *n; - struct neighbour __rcu **np; - - nht = rcu_dereference_protected(tbl->nht, - lockdep_is_held(&tbl->lock)); - hash_val = tbl->hash(pkey, ndel->dev, nht->hash_rnd); - hash_val = hash_val >> (32 - nht->hash_shift); - - np = &nht->hash_buckets[hash_val]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock)))) { - if (n == ndel) - return neigh_del(n, np, tbl); - np = &n->next; - } - return false; -} - static int neigh_forced_gc(struct neigh_table *tbl) { int max_clean = atomic_read(&tbl->gc_entries) - @@ -277,7 +248,7 @@ static int neigh_forced_gc(struct neigh_table *tbl) remove = true; write_unlock(&n->lock); - if (remove && neigh_remove_one(n, tbl)) + if (remove && neigh_remove_one(n)) shrunk++; if (shrunk >= max_clean) break; @@ -388,22 +359,15 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, lockdep_is_held(&tbl->lock)); for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour __rcu **np = &nht->hash_buckets[i]; struct hlist_node *tmp; struct neighbour *n; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { - if (dev && n->dev != dev) { - np = &n->next; + if (dev && n->dev != dev) continue; - } - if (skip_perm && n->nud_state & NUD_PERMANENT) { - np = &n->next; + if (skip_perm && n->nud_state & NUD_PERMANENT) continue; - } - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); @@ -532,9 +496,7 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { - size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); - size_t size = (1 << shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets; + size_t size = (1 << shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads; struct neigh_hash_table *ret; int i; @@ -545,33 +507,17 @@ static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) if (!ret) return NULL; if (size <= PAGE_SIZE) { - buckets = kzalloc(size, GFP_ATOMIC); - - if (buckets) { - hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); - if (!hash_heads) - kfree(buckets); - } + hash_heads = kzalloc(size, GFP_ATOMIC); } else { - buckets = (struct neighbour __rcu **) + hash_heads = (struct hlist_head *) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); - kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); - - if (buckets) { - hash_heads = (struct hlist_head *) - __get_free_pages(GFP_ATOMIC | __GFP_ZERO, - get_order(hash_heads_size)); - kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); - if (!hash_heads) - free_pages((unsigned long)buckets, get_order(size)); - } + kmemleak_alloc(hash_heads, size, 1, GFP_ATOMIC); } - if (!buckets || !hash_heads) { + if (!hash_heads) { kfree(ret); return NULL; } - ret->hash_buckets = buckets; ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) @@ -584,23 +530,14 @@ static void neigh_hash_free_rcu(struct rcu_head *head) struct neigh_hash_table *nht = container_of(head, struct neigh_hash_table, rcu); - size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets = nht->hash_buckets; - size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + size_t size = (1 << nht->hash_shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads = nht->hash_heads; - if (size <= PAGE_SIZE) { - kfree(buckets); - } else { - kmemleak_free(buckets); - free_pages((unsigned long)buckets, get_order(size)); - } - - if (hash_heads_size < PAGE_SIZE) { + if (size < PAGE_SIZE) { kfree(hash_heads); } else { kmemleak_free(hash_heads); - free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + free_pages((unsigned long)hash_heads, get_order(size)); } kfree(nht); } @@ -629,11 +566,6 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, hash >>= (32 - new_nht->hash_shift); - rcu_assign_pointer(n->next, - rcu_dereference_protected( - new_nht->hash_buckets[hash], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(new_nht->hash_buckets[hash], n); hlist_del_rcu(&n->hash); hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } @@ -738,10 +670,6 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, list_add_tail(&n->managed_list, &n->tbl->managed_list); if (want_ref) neigh_hold(n); - rcu_assign_pointer(n->next, - rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(nht->hash_buckets[hash_val], n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); @@ -975,7 +903,6 @@ static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); struct neigh_hash_table *nht; - struct neighbour __rcu **np; struct hlist_node *tmp; struct neighbour *n; unsigned int i; @@ -1003,7 +930,6 @@ static void neigh_periodic_work(struct work_struct *work) goto out; for (i = 0 ; i < (1 << nht->hash_shift); i++) { - np = &nht->hash_buckets[i]; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; @@ -1014,7 +940,7 @@ static void neigh_periodic_work(struct work_struct *work) if ((state & (NUD_PERMANENT | NUD_IN_TIMER)) || (n->flags & NTF_EXT_LEARNED)) { write_unlock(&n->lock); - goto next_elt; + continue; } if (time_before(n->used, n->confirmed) && @@ -1025,9 +951,6 @@ static void neigh_periodic_work(struct work_struct *work) (state == NUD_FAILED || !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); @@ -1035,9 +958,6 @@ static void neigh_periodic_work(struct work_struct *work) continue; } write_unlock(&n->lock); - -next_elt: - np = &n->next; } /* * It's fine to release lock here, even if hash table @@ -1984,7 +1904,7 @@ static int neigh_delete(struct sk_buff *skb, struct nlmsghdr *nlh, NETLINK_CB(skb).portid, extack); write_lock_bh(&tbl->lock); neigh_release(neigh); - neigh_remove_one(neigh, tbl); + neigh_remove_one(neigh); write_unlock_bh(&tbl->lock); out: @@ -3143,22 +3063,16 @@ void __neigh_for_each_release(struct neigh_table *tbl, for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; struct hlist_node *tmp; - struct neighbour __rcu **np; - np = &nht->hash_buckets[chain]; neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); release = cb(n); if (release) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); - } else - np = &n->next; + } write_unlock(&n->lock); if (release) neigh_cleanup_and_release(n); diff --git a/net/ipv4/arp.c b/net/ipv4/arp.c index 11c1519b3699..cb9a7ed8abd3 100644 --- a/net/ipv4/arp.c +++ b/net/ipv4/arp.c @@ -1215,7 +1215,7 @@ int arp_invalidate(struct net_device *dev, __be32 ip, bool force) NEIGH_UPDATE_F_ADMIN, 0); write_lock_bh(&tbl->lock); neigh_release(neigh); - neigh_remove_one(neigh, tbl); + neigh_remove_one(neigh); write_unlock_bh(&tbl->lock); } From patchwork Mon Oct 21 10:20:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13843835 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.132.181.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 136071E376F for ; Mon, 21 Oct 2024 10:24:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.132.181.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; cv=fail; b=jlgjFhrqGpwUssPwsNrs+aN3fbdsUAtaQKQx2L9Kltkr4wS8IqtBkHw3QjKmWkCb6ShvutL4YXihAYZS6fUWbzUFZnbZ2jCtpbiUF/2MFDDINrROlVNHf3gcs4Hm+NXyHCNy8vccfKLxi0ObUs7aeldi3L0gfTBTlljPUdLQt4Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729506285; c=relaxed/simple; bh=deDKQYDIYSLQUSmzxOwY9tjDxARHkMwfVIVlyEVrzOg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=RCyg4ELIQz7tCwIH9opa+4JiTw7YDP1PhUlu2HOGKrJKHtwr3rY54JWe6e0/KS/w37t5L7eGBI3rvZmMU4QOeMdiY9khJBp9DLKVfa4WHVGY4uDRUc6Oo6zoBQi+L+mLWYPrnhhvzNcxoEziWRa5UFJhy+cgpzzyNJ5kSFtc2ts= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=DQa/it87; arc=fail smtp.client-ip=185.132.181.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="DQa/it87" Received: from eu1-mdac22-5.fra.proofpoint.com (unknown [10.70.45.132]) by dispatch1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 6B9AD200C1; Mon, 21 Oct 2024 10:24:41 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02lp2105.outbound.protection.outlook.com [104.47.11.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 8F3B4C0061; Mon, 21 Oct 2024 10:24:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=U4FqCvGs3zqV4ozZcbhAIWbvEtIav8IeBviTZJtfDybV1Blx50LkY6owKtL/aGZNz+ezQkHvZxlP5LxW0TGkG4fAy+stJ9CT4iZZLV/pHe8BjRJFaViat67Vky93T4P5hzXTul32t7v1bdpc7aTFkPzYD2c5oi1+/wxgj5sCKf2yLvJ1oatH8W/revRXuy7GVDlbFRnYrI5ZmDPofqft/dld5nt55uD0Spk9echfLX7DPCtfOIhwfsiyEZmxWoaCldQje84Ruf40Pm3pn14gic089e1BWvxFStuhE9mUkVMxv0sB0En85oosRUwydjkpC2xNzDZdsqGvIDt6LHWRkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ah2ZnSH0F8oXkIftLqxT1Ne/AAu07vzhaYPbiIt3vIk=; b=gnnIlu9jgMoxQDn+J0+Lwchf5a80EP/rub3JNRSG/81Hkmsixq7hhRLd4a2YXNreiGFGwdupZ9c1v+7ieg/tivNmd7yRVAIPpxmK4oChKes4Nt/qVGeaK25QnJqHxTPVi5e2RqtXxcgb7HjPfuN1hofHuKORn06lu1z5BusAPO7X/msIzdqwmGz3EKiLCeB+NilMdH/ZyWRBo/pOKBu++h0JreA2NQJt+xQljQJacx820SL1NlCrWhq5+bBFPDsAPAsVfMs8NCblV7JSGA53J8ItzAOYzT16LNPUsSNWcp66/gMqrYySKr4DEmG8rSvrcnzCgc7d6dsfsJheZmGsFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ah2ZnSH0F8oXkIftLqxT1Ne/AAu07vzhaYPbiIt3vIk=; b=DQa/it87GDIyQDekK1OttgDyRwc+Fz8lTNqhh+kFFuVjUrqOoIAVSOpyTH4LV2nbjgDcj5+6Dbp1Q1MjUlsvJHBnowTe4T7cqv7kdpZVjyIMSElHXatzzYXNVEWVX52nXYNs8rGap6tH8sP0uwdEDgDRcqXW/wcAPA/RJJ2F8OM= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by GV1PR08MB7707.eurprd08.prod.outlook.com (2603:10a6:150:52::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.18; Mon, 21 Oct 2024 10:24:37 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.027; Mon, 21 Oct 2024 10:24:37 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v6 6/6] neighbour: Create netdev->neighbour association Date: Mon, 21 Oct 2024 10:20:58 +0000 Message-ID: <20241021102102.2560279-7-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241021102102.2560279-1-gnaaman@drivenets.com> References: <20241021102102.2560279-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO4P302CA0010.GBRP302.PROD.OUTLOOK.COM (2603:10a6:600:2c2::18) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|GV1PR08MB7707:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d4defa4-9f81-4255-2fe8-08dcf1ba905e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|52116014|366016|1800799024|38350700014; X-Microsoft-Antispam-Message-Info: i8zbawtcEkvoHL41Gxdgq7hgPjgNyrx8HZAinofnjuvxa+lGgAtVUuaXlQ37EJLDc3l3qFz+/08nc8ivx/37eCEX2as7ydtaMh8jWZ7WNsTMg6FqNmmc3QAwiu3TY6a4GB6MFYcQ1iPgsCjP4s7MlDbSJfoZnVi5IG+LoR9RzE4dkG3iOazEArCkPtJKdMJsPMDuo6XQUOXvwlslkHQexnuCZkOfVwUd5yKpv+2saa/C8C8tkpz32jtGbmgQKQBG8wZAI6p+pKgbEXm2xNSHqeL0tNd+KD6UBGf/Sco7uWa97ezHA/vERAyCi+MoVKIp+ZU0+dla2+WKiWgNipYkQy6yZYdxGQkioRNTmGbznQWkbWgMUdcZUvyHUIbJYea0hXRriO9seV4Sj9fRIt4l4o842delDGOeiR5tjCeRWFN22nVEDdss9g8kZDMmlNa0VGAKLvk3Sd7IJqFM0UaeIV5r1NOcin+0+QWiDHP99eayRJnXwzq9bf1CTJpob8OkBxWSIVKZ6RDSqWo2cBHTvEiaHs4cNv/2W+3H95NaLluojrihmFukJaPLbC4QOWfhftpu8l2oIDIVqd5iMFqGu/mk05MLvRdaumCWEbVlIiE5jhS/4svqMC8HW3YEUhROqv406vDDKkh5pfyKd07Q4zarumS8QsLV2zWmifQyBNzoGkucIi+qxz4vlRC1VeJu3vNPHGTiLqxb7DENqfFy8UjGPO7Vak/ehmuK/huNtZWbrMMJRH+HOY//4/IDYb2cXDgy5wZILVFgiAGESM5Osi57nCgxPW8L8bej1tI0t1xD8v3ClamCXQ8Go2M6bPcv7Niu7t0tbuRrDBfI3stGusmkRKabF/hzF0+/Dlf24BUhJsEp0MJ7PpOTP+6axzJaPMkfcZmeDWFnCLZ6m+4UGXhN8N13lI9rbNl3T3uTGzvc6h9Mg7Y3T3D86N4fCw+PkBoFs7BvhwWWYv+BJwlCqttyrHhVfOxLkaGBz9FaDXgO567QDeIfSov0zyJ+ekwXmNSElonbDKbCRni1gAH49NWBfWQpq7gKbf7cRwOQ2mR5H9tp7xWKSTawJfriVv+JWHFtsIc07WWALgg20HGCTL1Cxl7XG4Jn2YsHbgzHcdJ8aKwcnvuI8Rh7Il9H59N9MyA97UaC9DnyoEIkoRcy14E6oc+A0m/S1jnq1HYKRlEMM6R9oMmiuFj1fHxVwunAA2BCL1AE5RDyifX5aBf/WSzrgKz0D1QyVMKkE/+ot3yF/i5wqnGUhlBa/GNgCYtXZJx7C+ZnZIiifrrWfJSaV7MW2Q6rigFBQY0MRvCR7NBnjzOCpmn3GRo5JIibg4aUZ+JADVeUSw+Jn+Le70TxHyJCQO7oJZgHt6DQ8M5k+Zk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(52116014)(366016)(1800799024)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QZIqUaeIniEIutsoasagJawHjuazmN3tFFlq0xtIee4ibeLhzfmFYVJ1VKhcNeLYUOHc1/QUorCxtBlA8iZ2++pY4a2LfkV5Zucq6uahyA8DKI4ryXMGPv4lFlNhHhzUR/jrtAPwTFiM5g36wU3I3l4+dMmHO/o+y8rxMoGpULCBXPqBGvzCk8sldweDuuc9kxQ3FG6DtWIFdWNfUjhPsWKpsxo5E0eZf+T4RR4UCrek/1jjSACg+f0s8DAqBcy6uVIPqhzCur8hlBe3c1qKy2xpdtClXSA2sMAO93sk7pbllJkW6WqbOelhRC+PJT59wwuVIfe4dJ3n1KivGrbEybD2zVPuYl/24wtkF+u2fwK2Ijl95p29SlhSFQFh2unQjee/4vGet/f2a0fvImF/txXntIdeYzwdYnVqQ/Y0E1oXsuOF+FfJBjTSnaYVfz1AYWpy7bU6JR20iztxUfBH/ZiKMj1EIbnkGwjRsnfQ6VUoQb63KM/tTeJmST7AZaWZZOc5muHOEBYjAAN8TsyXG671RZJS4HxofP2/vZW9SySU+D+KndGx9tpcpQNPh/X7W83k/Ftqu5Kjo9sk8bWuMwCfG74n4+eZwfRXchRiPxkwmCiP06lPgQmuoiYpqELvgX6tTu9JZbbt8vOkC5Oj8MUq5tZnjxccgVQJ5LXHtb+wj2lcfhSKpEfld1p/R5Ksf2lnuLyd/30t0/NMB4fgns/6rMjOgIGWiOYmehKugessMueEtD2NO23jkV7ubdXtclKDLiUcoE8Ti4EHROa5NC9t1ARSax/M0v/jP9QN9AAqT7PN/07E07QicyfvEz9VnxzEMO6oXi9Cbr3/HZRT+fhljDEexB9/4M3bbQQMHSct4AztousJIv8Q6XXkSVHZzeTOmPZqtM6WnYykui1Ly/fJ/0uzNUrgRJ2ZfnjEvd0IzC6N0fsWx0bW7Y75FXaDvxacWOIcid20LCDNY0kSitU6d5aB+OXKnSeembja1fP/D/X4HIz5kmNqxgEpjPg9qlDFGsFsSVi/j9tzWHn1IoUXu6zvIpccx5moeCaMTMuRUrFUpEBtAVwRvpfhh6SB04ldv26wm2WD1ymz6rXvsDt+L0EFasyWk+suUv34QT0YJFWq4jyaUM6TTHweeo4ONWjyvBuM9TJi8DU+DYQc+EKTDEoa9runsiw1C6mgFG0sd5thdu8Nl5r8HB04JgE7wWRbtg1SrZ3uQslx4FYfoHlOtjQmP1uv/pJe5Gb+Bv4vPBDrahxViRsARCwWTf3LIkpXA1dIvLD3s3mpKdvZ6ZmjmRVdWf9xScDxuLnSnFLMQe1xin9KtSE0e87UmQcPw4iC7RL+8m632UDpVc0j1ub48V9WvMNNEy0GJcqh+XwJ19voqp1Y/rwPyDwmrSupVPFWdSdr7SaPWfnloO0V9Ist49KQaLh+PefDDxuGVERRAQ7bZxBSk5RTm+yTdfAFyPO6E/1qf4pK6uOxxDUuqvW4qvCL3jMkepgtefBrfM143dLLe3RuNS+ddFHufnOtFKw2SgZHKI0en5p2S95cYsI5l4WqgpCBLkR3uADF7sTWHDDIR571qPx0sAP+I90NewFrejOx5t+sFYHKZTs7WA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: xRqyAPlnGPOCtxGu0Si4eBUqaeQ//2tc0Ogl9MLDEkhmI6/FpCxiBrZVJDvbCwSTq9jbqOaWHL4YxS8v/8+6fyXgjQtBCnF6Iy0bd14oTy23t6sH7o7JjaIQ6Ae2rPBIg5JG6MAPwB8KZhZPW+ZyrgtKkl+estCynYMcCLWxfEeNJs3uCoXygJC9S3U0fCUqg77/9weCLPKIT0KynhJp/Xp76aww5KhA1Mqyp8Gwn263c9z2gez89QsGg6oZ4eOfc8gjC8cID2wAsAahL+6K2P9Ozb9faMWzkQEUNtChxnzSfDPzaOqCC5Rk9DNHAicbUPKlFGY1oCBbQ+Q3pOEI+g8EiN8OzHSYFNSQ5d8ibK/oZ+DPYtV5TsD3QiH+4I4vnbxAeJA9Te3IQgAr0IySATwYEj/ZWSSQuGCBMPSWenJPkC/B+GOspoBXKt6rtXt570VWRiz7b/jnG7fgHjDWOIBTRAJKISDiQ4PGmi4VQ9lyJIXJiysMOZgwyKTBzLYyY1GnyDXqQMLMMDfiENM0gETl9wigDuaph3PyVwBIaWbcJA95WZooHhEIS4te9GJlBWYdDTl7h4ytLnqyrwM0DqnRSa4aSnGN9t5kR/6O/n+yHGBol54Qfv/xRBnRDsPf X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0d4defa4-9f81-4255-2fe8-08dcf1ba905e X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2024 10:24:36.9803 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jEjanCManvCAHJWpfkPsDoiS97B8RnrNja2/FgV+sYXhkZI+laWzYwNReaCDWWub5YuV5S5F9199cQulxYfknQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB7707 X-MDID: 1729506281-n6WWp9Isinzv X-MDID-O: eu1;fra;1729506281;n6WWp9Isinzv;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Create a mapping between a netdev and its neighoburs, allowing for much cheaper flushes. Signed-off-by: Gilad Naaman --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 7 ++ include/net/neighbour.h | 9 +- include/net/neighbour_tables.h | 12 +++ net/core/neighbour.c | 95 +++++++++++-------- 5 files changed, 79 insertions(+), 45 deletions(-) create mode 100644 include/net/neighbour_tables.h diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index db6192b2bb50..2edb6ac1cab4 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -189,4 +189,5 @@ u64 max_pacing_offload_horizon struct_napi_config* napi_config unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs +struct hlist_head neighbours[2] =================================== =========================== =================== =================== =================================================================================== diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 8feaca12655e..80bde95cc302 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -52,6 +52,7 @@ #include #include #include +#include struct netpoll_info; struct device; @@ -2034,6 +2035,9 @@ enum netdev_reg_state { * @napi_defer_hard_irqs: If not zero, provides a counter that would * allow to avoid NIC hard IRQ, on busy queues. * + * @neighbours: List heads pointing to this device's neighbours' + * dev_list, one per address-family. + * * FIXME: cleanup struct net_device such that network protocol info * moves out. */ @@ -2443,6 +2447,9 @@ struct net_device { */ struct net_shaper_hierarchy *net_shaper_hierarchy; #endif + + struct hlist_head neighbours[NEIGH_NR_TABLES]; + u8 priv[] ____cacheline_aligned __counted_by(priv_len); } ____cacheline_aligned; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 0244fbd22a1f..bb345ce8bbf8 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -29,6 +29,7 @@ #include #include #include +#include /* * NUD stands for "neighbor unreachability detection" @@ -136,6 +137,7 @@ struct neigh_statistics { struct neighbour { struct hlist_node hash; + struct hlist_node dev_list; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -236,13 +238,6 @@ struct neigh_table { struct pneigh_entry **phash_buckets; }; -enum { - NEIGH_ARP_TABLE = 0, - NEIGH_ND_TABLE = 1, - NEIGH_NR_TABLES, - NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ -}; - static inline int neigh_parms_family(struct neigh_parms *p) { return p->tbl->family; diff --git a/include/net/neighbour_tables.h b/include/net/neighbour_tables.h new file mode 100644 index 000000000000..bcffbe8f7601 --- /dev/null +++ b/include/net/neighbour_tables.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _NET_NEIGHBOUR_TABLES_H +#define _NET_NEIGHBOUR_TABLES_H + +enum { + NEIGH_ARP_TABLE = 0, + NEIGH_ND_TABLE = 1, + NEIGH_NR_TABLES, + NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ +}; + +#endif diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 47eadf1b2881..69c570c2a919 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -61,6 +61,25 @@ static int pneigh_ifdown_and_unlock(struct neigh_table *tbl, static const struct seq_operations neigh_stat_seq_ops; #endif +static struct hlist_head *neigh_get_dev_table(struct net_device *dev, int family) +{ + int i; + + switch (family) { + default: + DEBUG_NET_WARN_ON_ONCE(1); + fallthrough; /* to avoid panic by null-ptr-deref */ + case AF_INET: + i = NEIGH_ARP_TABLE; + break; + case AF_INET6: + i = NEIGH_ND_TABLE; + break; + } + + return &dev->neighbours[i]; +} + /* Neighbour hash table buckets are protected with rwlock tbl->lock. @@ -352,48 +371,42 @@ static void pneigh_queue_purge(struct sk_buff_head *list, struct net *net, static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, bool skip_perm) { - int i; - struct neigh_hash_table *nht; - - nht = rcu_dereference_protected(tbl->nht, - lockdep_is_held(&tbl->lock)); + struct hlist_head *dev_head; + struct hlist_node *tmp; + struct neighbour *n; - for (i = 0; i < (1 << nht->hash_shift); i++) { - struct hlist_node *tmp; - struct neighbour *n; + dev_head = neigh_get_dev_table(dev, tbl->family); - neigh_for_each_in_bucket_safe(n, tmp, &nht->hash_heads[i]) { - if (dev && n->dev != dev) - continue; - if (skip_perm && n->nud_state & NUD_PERMANENT) - continue; + hlist_for_each_entry_safe(n, tmp, dev_head, dev_list) { + if (skip_perm && n->nud_state & NUD_PERMANENT) + continue; - hlist_del_rcu(&n->hash); - write_lock(&n->lock); - neigh_del_timer(n); - neigh_mark_dead(n); - if (refcount_read(&n->refcnt) != 1) { - /* The most unpleasant situation. - We must destroy neighbour entry, - but someone still uses it. - - The destroy will be delayed until - the last user releases us, but - we must kill timers etc. and move - it to safe state. - */ - __skb_queue_purge(&n->arp_queue); - n->arp_queue_len_bytes = 0; - WRITE_ONCE(n->output, neigh_blackhole); - if (n->nud_state & NUD_VALID) - n->nud_state = NUD_NOARP; - else - n->nud_state = NUD_NONE; - neigh_dbg(2, "neigh %p is stray\n", n); - } - write_unlock(&n->lock); - neigh_cleanup_and_release(n); + hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); + write_lock(&n->lock); + neigh_del_timer(n); + neigh_mark_dead(n); + if (refcount_read(&n->refcnt) != 1) { + /* The most unpleasant situation. + * We must destroy neighbour entry, + * but someone still uses it. + * + * The destroy will be delayed until + * the last user releases us, but + * we must kill timers etc. and move + * it to safe state. + */ + __skb_queue_purge(&n->arp_queue); + n->arp_queue_len_bytes = 0; + WRITE_ONCE(n->output, neigh_blackhole); + if (n->nud_state & NUD_VALID) + n->nud_state = NUD_NOARP; + else + n->nud_state = NUD_NONE; + neigh_dbg(2, "neigh %p is stray\n", n); } + write_unlock(&n->lock); + neigh_cleanup_and_release(n); } } @@ -671,6 +684,10 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, if (want_ref) neigh_hold(n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); + + hlist_add_head_rcu(&n->dev_list, + neigh_get_dev_table(dev, tbl->family)); + write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -952,6 +969,7 @@ static void neigh_periodic_work(struct work_struct *work) !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3071,6 +3089,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, release = cb(n); if (release) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); } write_unlock(&n->lock);