From patchwork Tue Oct 15 16:59:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836749 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 526001F667D for ; Tue, 15 Oct 2024 16:59:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011593; cv=fail; b=fgziS9VvY5/Gk7sP3tN5ioGHjZTLsFVqg7B6noHCiemGNXJcaZ/W37JJ0dcDZ3GKjw5MYc8NaYnkkTeJfLeQF/KNAZ2aFdXAkHy+IXr78WzQL/r/yj+wu3Sm9lAndPyyATw/GV6KVbeWQSyTkEp+qRieb6tpZuM++e9Zm27fke4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011593; c=relaxed/simple; bh=KLcviuSR7qOU6Iqfm4dXaNmdM6pWgpDZBAub0TQeAzk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=U6AZ5Swf5JY+XELO+sQKYvM5O8YlJ4nAfVP3SMh1UTJSdpBrDgxmMQetfwRJWh6rtKDn59/2WdZnLf7RGL7Bsb1eSu2fGEkAVbvudfGJVsHJdVbGoGksFsFFdEovJp0ONu0LRHGainzP3sSO5hNBfSnM9OjmgmMdny5xjlkKyDo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=FVoUfgdE; arc=fail smtp.client-ip=185.183.29.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="FVoUfgdE" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2109.outbound.protection.outlook.com [104.47.17.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id AE2001C005B; Tue, 15 Oct 2024 16:59:43 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lG7G60D21D/98NdHvbIhhCh+5fPROhlkWVvi1KpqRDGerd2iEc6ATC0GbOHjz4KJCpHymJfkAGcFHJ1RN2r58B4cnevTFrLg8NQLHUCrDLxHZagQLO+Lo24zZIAAAHxnGlDAeE9CmdHIbLhDz3/1CIB/lzkVUoF2/8I8xfVOmY6xdhn7x8BHO38irkzZTBQKj/shkEKUXasxYksmWlUWkBc+t58PIA77q/TiOxUuiNNTasTMFZEWC3F6QnamSjmYuXMc56xkL7uXnqus9ouj6ezIDLryPpRq9+72etX5Fv0qWyKVpnSSJwjYq0EHdaKLMxZT4nJaGIAnIuQoJ6WHaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m/LbQSB2dkWbqfmG31+kNjxECzuQ7GZ3Ffhkit2kWdk=; b=LaaraZB2//Hdkp+xEVd5s704mvpHQ/wtq1oMnWXO5iHMwJCgaHJrMtTYBSoC6rrBvvlL/MU8Ti2acqXpR6VlGW3GCxXOLQ0BWRDJ4DeHpAwgq8eXZYYWwIywUq7teIlmN2OJjHB/mkuA/BBiQf44xFekF0WXw8ud2dRxF1pUTuwXUllCXyf/mLPQ/hd1gpADH6ggLSmeW6Qsegzm0/Z4PqOXiKfrhzjhnyB/KQYbKtT0lFibdnk4+qGSFAFmBkFRfBx/Mg8HtB1OsWXuZXvaQFeu6FUnTBwf4MHge1VyWWD5q/zKGmtoAoD+Y74f3It4QkTLytDftpkXuTyn8BS30g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m/LbQSB2dkWbqfmG31+kNjxECzuQ7GZ3Ffhkit2kWdk=; b=FVoUfgdEsVIzw/sKcKVuu07BdQe4B7Wj3k0JB9qOVFploNqzkX70/32o3Sj107V5jG/iX9/e8N91a4BUi3rq31vXXmX8CsvWZC4uX8MNPKJqXbrYz3eRmsNpLufedAz1gylHMmNY9gfZ+9KVUhJXxkqefF3Zu/9Q2p/k7qUQWCk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:42 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:42 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 1/6] Add hlist_node to struct neighbour Date: Tue, 15 Oct 2024 16:59:21 +0000 Message-ID: <20241015165929.3203216-2-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: 2db612de-30b8-46e0-f4df-08dced3ac344 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: B8/9A/JutDtmHQAGFwflAh2u4fh+U7X7LyocGCahAlVkZUksfkMmQEf5emjhBp99u6tVp6jQGCiDNGSiphuthu8EPscK4tchuWFT0BK3ceYaFepy2mgs2OkT+l2OjBgCYShkApGfJOEZuz7oh6r0HEJ6r0JOnymjuNHgIkTSGIWcXERX+AO3V+gGE1iszY/sL5+Tytx4/MjgjDSmuqjaNOOlLLEdGI//EHb+b5+Aahu/NVLjqfbTjGdK3v1N+K22yARuFrCRmxsGjR1ibyiikiLzP3kBsYaJXFtserfA2dQkSuL3IiRql6BrWxb44fzvi7r6pWsKry2S3adYu22+ssYfgNGLsrKcJfp7BbmyXKwk2ZYvSiIIZUGztljVwNbtR07OXkqsnIiFegQfanDR636LpaKgMlZoWaarOTnY5f7p275ba5B+mIFqnzi7euOyGEIoglhkUxIeW4XQOrPNKqvD6kkRTJebwkHoVW9rJZl5Le4POESghBwcRXH8ShJsjX/49M2g1kuLryeRuAgHgoUnhUhKiWBF6U/QOc6qH7mKMVtqRvoazM/5ZhbDIeLOn1jfQs76lOEXXscpiy8b+JK9m9DgxxYDF0bsyCMurCsjbywh7TNcbQMJ29sChjuZ3C8dgkXXl0U5RdI1WgArn/h/ZmEO2EzLJ4CQ/JPbzrGXxIinuWu89MMz44YHMVLYswJoP/wOdLyd8VmIXhkf0miEOPmuyTgzB+67V2eh08tR9DwvAvoZvufpmnnGGdHIrie0it66vQgzQeEtb2fou05IoBW/Pq45YryRbBsNG0hHV/a9wuvdZ0LS3B/7CXT/zQpB0RPZW11rSpv8p+bfUczQntCP+ID77uzYA6s4J80t3we2JF9ZjVAMoeeT19JYNSa9SGCZmvle/wrBGyEsCFEn6+tuKJ/fBtwaCwCaTjUEemWVqWeFMIrUEJxaRJD6OggTI+djEKKWh91nfyK8wEYRsHJ3K3TYHV1/Ud5Jnr4lXrKKQYC3p5I7H8X+u2CrjyQabglWOgI3KZGJ2dao0IkJ3BTcn4igeb0ltzSdpn+pYWcNmncI6etcSgv0oT054gDyzz5o0FElo4/WtmtmIfRIHn/KrETCXMooqpyQv9Hu40Ie+tq+4EVTAfRB+i0nAIFSdF8FqJaaqtL51PgS+kWdRi3LOnqcUXCBBPIreDsFWQAZbZQeCL6YWkFwQkchyr1Rwi3c/Uig6eQbyUjEjBIlCCYm+YA9XXNd8a4CbdyB7EdQq+FS/19ETwtBRgcTeID9SqtExAp4J5RlE+4mDP9pNMTJzwu+CpETcetoahy7aE0CnDLyVnKtJc7D2cnq2FN8lB2LH2FKUHyTTVUSOQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tT/Tzny6EqrIlx/eJ3DCPPQRjbHwnG+StNFdjBvvIY0xZ0b5BJ+927ByfMdTiDcvZIjlAFTqEp0q1n40J8yXZ4e09TyiJpLOd3Cr5DybIqTLGsQymSbyjZL2k3CngwCbS5it0aXvN5LASlCjLCx6q0xEx+rggwzfcz8TR+NQ5Ojb8mRqpwwVkLoPqBhwVSQKTcNipa/7aAi0Dz2V/IicUQWBEiOGy4EONIxWGUo5qvtin38lHBB+qPJbAjaQQcxvVthD034M7yVyW3p0UtvJgJPmmwdduNulaWUSz6ArhXPj9qWYxBMRDFVoReFqPg8svLC+j/IbSZXwhErH51p6M9nxW973d/cOteoDb2L5CJsCZbckuCkk/28EEoBcJ+UY0Ido06p/0rPwA9uzGzDTPIg9gGIY6UiE7Eh9nShCNG2HWM5h7f0yrpGDS/QpmfYX0L70pqseM7pyh9+VWUTbGp7+D0r1+pObXJheEawX9VjvT/eGHab9cZPgI3jqspWkBucVoZzSEAehVVn12huhG5YK/saEUa05wxow/gGZ80KZFe3iKKUv2npSIwhGaLXwxCeoMQFOGUQKjIGsIlWgXaYH4Xwflu6mn1h/nLG7ITZbjS69IH6e7y7ZJ0w4+kOLNJg0a6t27eKCh3EgxFVdVc+uAE8qEp98aSPE0NFDwvgWJZoJK8cPD2xAFRUvKDxV1d3bddsIYok1OlMU6hj4gcXOqRi5WHjAoAqJXLLkRaDRFX8nA9IvtMncD+TyCFYuN+j6j6Eg5bhk38C+Dg/dS7GNauDZYFErEN5+swNwm4bRFIdVOhwCsXFR47+YLoZ3PxtCdUrAv3mGWVcwOVh3wPdwYWllZOaH2SryDwqaqU5St3dtPH77ZyADtRA0E7cGELmujV4GHzk1HA+2v0Sl1x18CEb3RgJ9OmT/Cr6kDyz8kKE8UbCGAZK1JJQmeyIfQXXff3JNd6xFSa1/cYhd9dY9O39QsLza0uuVXkjvxfxCxsJxP+vQBfr1Tirbel0Be1ctLZJ960Ac9PlfaqLOYTfnI2kCWD6mvw+oPZapNkhMh1kT9Kips9x3NZvfQEGDoa9TFg3Fe5F3eseyuVuXulLLDJThZWupX1jJRBcjkA5RTvuIsL+c25gOHut14ky/pcwufFB4GApUdAPGUAXuwn8oOI7BgJ84kfK4wsap76onJOEuXWO4PokqywjHkhKLxpmLWNfvuqOsJlf+j2X05yEZ9/MiycyE5c5tdsK1qMjRhSQ/MutBVzZxiLt52jXU5HhDSambQlrBs3l4tYcNplbLNHb3+fhIjEcRCUP1iZGamz+fu7MAZxltUVdkSgwMvGtF7BxEnQGizmFUQlnmk3xS3X/bT6H0pZk2b+Pbe0vDK7augfZbzthdzo2N7yNNqHKanZDG/Z/JDlCD6U2qAvPDpNQ+BrdKb/mbWN84rGUl/Yh8YhnY3+qPfyM1dU5U6SGYHMMmVIy5VO7B1fB0sCEYX/P3Xbw31QPzSC1uRe3A96OE1R8xjvAj3hCpfJGjnESaa1N6jRSXpj5OEyFXwGt7OEIYuOYnbWC6iwxcosDhsNK2SJDplFmQsv6Ef9IjSTVXV5gK6vUOGkp+t5n/8g== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: K5PXdWS/zaNe3sbTEiQDBmfkqEZRHLg6cx5wUBdk/bG/GGYHqiMnYkJPlxfWGu1V73VomhpYDeV6QzKyjhlxtxeQup/K9RO65Hs1cankkCOxMGjPMQOnz3+hsuHr+WpspHLIi+hw9AhKDUif/y9JXhguKdLSG0EaYGD3Aoo6JyqiAcdvAYUqhr5eRblK7rqnYjgnPkf6ql9aplbfzotbvm35QlWiZA19RSHaR9cPcWjP4lkXilKZVr8WKLrbAKWROBOozMR7xnCmjBRa8PQ2JnAcsGd6HjwCYL4WKK0k/WUpre8jTEMzC43CjotRH6uDjXVyDUB5JvNdDyyZewNh6heQSoCixeRXCQApoHnwxnDEzWLDk4vvT25rFSzCRnxwWCl73TawQegOdNCiu39KnGbDXg0MaWiAwuIvHkJsAzN7Ln6nwoFgzsoGDxJwvynC6AExKMHa70+e+gyEGEliFAU/ptUH9BJTWGbXkjKupctLvh3J3rSFRLgbno2vgSNS7jPRpPKR2s5c1/qFZGiYJXa7VEZc0cdMgeu6YryCCU+x7CAQjNCtFvUZMhdDzFCWzVX1bi+nCmwVFwsvDp/kkfZVGheJUuDd4cCL1oN6pIPARxCQdH1eSBfzzERy2e55 X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2db612de-30b8-46e0-f4df-08dced3ac344 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:42.1115 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t4tMtwC/vDtRqk1O13vpIDjvN72YOps5U5kqTJeXQPyr232S2RMPPM7dDFO4ZzRHYbM7fAgYneUgpi9GEBVVsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011584-zERSm_HpY-JT X-MDID-O: eu1;ams;1729011584;zERSm_HpY-JT;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Add a doubly-linked node to neighbours, so that they can be deleted without iterating the entire bucket they're in. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 ++ net/core/neighbour.c | 31 ++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index a44f262a7384..5f2b7249ba02 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -136,6 +136,7 @@ struct neigh_statistics { struct neighbour { struct neighbour __rcu *next; + struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -191,6 +192,7 @@ struct pneigh_entry { struct neigh_hash_table { struct neighbour __rcu **hash_buckets; + struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; struct rcu_head rcu; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 77b819cd995b..01987368b6c5 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -217,6 +217,7 @@ static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, neigh = rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock)); rcu_assign_pointer(*np, neigh); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); retval = true; } @@ -403,6 +404,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); neigh_mark_dead(n); @@ -531,7 +533,9 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { size_t size = (1 << shift) * sizeof(struct neighbour *); + size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); struct neigh_hash_table *ret; + struct hlist_head *hash_heads; struct neighbour __rcu **buckets; int i; @@ -540,17 +544,28 @@ static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) return NULL; if (size <= PAGE_SIZE) { buckets = kzalloc(size, GFP_ATOMIC); + hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); + if (!hash_heads) + kfree(buckets); } else { buckets = (struct neighbour __rcu **) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, get_order(size)); kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); + + hash_heads = (struct hlist_head *) + __get_free_pages(GFP_ATOMIC | __GFP_ZERO, + get_order(hash_heads_size)); + kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); + if (!hash_heads) + free_pages((unsigned long)buckets, get_order(size)); } - if (!buckets) { + if (!buckets || !hash_heads) { kfree(ret); return NULL; } ret->hash_buckets = buckets; + ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) neigh_get_hash_rnd(&ret->hash_rnd[i]); @@ -564,6 +579,8 @@ static void neigh_hash_free_rcu(struct rcu_head *head) rcu); size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); struct neighbour __rcu **buckets = nht->hash_buckets; + size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + struct hlist_head *hash_heads = nht->hash_heads; if (size <= PAGE_SIZE) { kfree(buckets); @@ -571,6 +588,13 @@ static void neigh_hash_free_rcu(struct rcu_head *head) kmemleak_free(buckets); free_pages((unsigned long)buckets, get_order(size)); } + + if (hash_heads_size < PAGE_SIZE) { + kfree(hash_heads); + } else { + kmemleak_free(hash_heads); + free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + } kfree(nht); } @@ -607,6 +631,8 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_buckets[hash], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(new_nht->hash_buckets[hash], n); + hlist_del_rcu(&n->hash); + hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } } @@ -717,6 +743,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, rcu_dereference_protected(nht->hash_buckets[hash_val], lockdep_is_held(&tbl->lock))); rcu_assign_pointer(nht->hash_buckets[hash_val], n); + hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -1002,6 +1029,7 @@ static void neigh_periodic_work(struct work_struct *work) rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3131,6 +3159,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, rcu_assign_pointer(*np, rcu_dereference_protected(n->next, lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); neigh_mark_dead(n); } else np = &n->next; From patchwork Tue Oct 15 16:59:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836750 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E25D1F76AF for ; Tue, 15 Oct 2024 16:59:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011594; cv=fail; b=Yww5xqE6CIUmKfWkT/sm7YLveLcoVJCPfoYfphquSmBB9EHcDTWqnv/cS73lRj4EfwUBI0Fx74X8k942qO6T7wWdM5kBMPTK/Y0TVxUB0mXe7LABZnhIS7AKPfWp3xKtlXLmrfaf/nFtWiu/MYCV7s/maGzhLl5oSmIPcFBGJlI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011594; c=relaxed/simple; bh=mqow4GwpKVCb4uKJl3IHqy1FHjQQ4/GCj9ypx3nUp74=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=PufkklraZhB3d1KyUNlf/HK9KE2DR4NsUt52IfWtNWfVNw3q1pQJUN4AZjA9Y3Tr13BQVs6/2iZB0eYKE2TX1b0aKagz6MMFNlsPXx5ITr7a+Pl2PjEj33Ln8PpwWeuBQoLBx9vykrohiFtENKyHqitpTc6h9/EFCtJK0mIUurQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=IF48Xqbl; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="IF48Xqbl" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 387E0200058; Tue, 15 Oct 2024 16:59:44 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xSQVkTM7Xi9BChhjZbR41xjhdVKevSDVelCd7lDxAgd/nrsi4t53kxUfV5mzFqJ5+LL9d5BD3DlEFfGtG1Ks8cZoExPKPM3GOGM6FfuCNKoJKcH3Y5mgFiP+zgI+pMAySxlxuhO0NLcnW3T8jZMshoLacDnaIdnC1bTUQQIrKqRpPEuYRkEkEVdGBoOpwCcRYjbImJn+GzdixZGp3CRlaV6EM9r7eIioJvoh+RRWpIUCbJ85cJXlgQ9DVpJxtuLlwI7T1dxCRQSl66kxwnp+/5lUcGp4qyrWcz6ShNxMdSWO0X9d0CuFm7x8b9/RuQMPoioVXlWPX3E5gLT0Hvy1fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FH0kTpDubKw9a5AmpY/kCXXEW3xh2mammEcnEQ9jseA=; b=cd3vUwRIgvRC2OdpVl7nh8mww5LcgKd111A1BFwjwWDtHXYDhjsA4C4EAVXCRDP6fBeLVuFMm40SmvimjivVEK7BSKXkzNrlFfBXzsKq7TWBoslqb4vnXm58LxQ3i/0ZxZyfOJe+u+5YIWZ6jtZZ73Ss9gmRMAdMNjKu5Un0ByhwbQykGWpTZHCVs7i6Q8OZSqOE896aT3dfko6ouxwG/POJ6y7Q4rueNsBXbOZZHu2dFU8tT5xILnQ6gFaySdSbMPCxWo6Fuw17sZthySMLyYcWHYbgWRRefA0vq4CO5aJ9kP2C/bAypaUy5INORzRbD/wgz+zVz+zYk/L2qbHoTw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FH0kTpDubKw9a5AmpY/kCXXEW3xh2mammEcnEQ9jseA=; b=IF48XqblTuXPrhhpyYXdbxLMq60OmnuWAqsk1OkQrtXKA1+tc03S14UyHCa5486AvscH02EP8BgCMMcc3HPyhXezGUxDmtXL2A9gwL51kHRUBCV73y8coQmYD4q4c+VjQ3Nsbja7P2E6OnYthmb+Vgc2bqaystaqQMG5aoGlYrY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:43 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:43 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 2/6] Define neigh_for_each Date: Tue, 15 Oct 2024 16:59:22 +0000 Message-ID: <20241015165929.3203216-3-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: 5ec6ffc8-ffaf-44c6-4480-08dced3ac3ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: smCb+PvY4BMxTXTzDQomVB1sq2vjxkFdUolO/nnJEfA249U1jMm/680ojtxh2EIs7PjRjf6p1QVsZvhhtG1NQmSg5L9ig6TjDUMcWtydk4p+9C/vIOXzaH62hH/yX8jnqtr/XE1arvT+o4hnOi5iShY43x2s7rJfsEJuZagGFBEqWjtiv/7pkGWX9bOXGxHyR/Qinwsm3cGWpWgvGywXUx0U728EL6FG604WO5RjSfi28FHig7Nsb13wLZ2lP8Ur5Cgy+8ZB5QZvCemfrU9uVF4osoQTwIhbk2kbMwoyOk4P/paUtyEAs1oczLPCZ690wi2wdzQ5j1GWMGMc92TQ/3qQGinGEAZUpX/OAgYBessufJ36IsoAaSRZEKtt7jTEZdPQJLSEWUKYaKgle8fxcbKTaJSqdaMPVUArbpi15Q8u9UK1nAq1s3meb6WMOfMaypm5Qwo5haAkiHBsYkQpuIvPOsUoDay6F+vn0KmmzuX0YJs9jhl6WSE2evRYZ1Us635sld5eZ/jEY9EIop4+6/RZSJNmKAfffA2PmQxogbL1imyDjq22sOL1P9S4WFwW9WG/xpEL+sRUEG3CaYitm3cKw4vLYxuAscpypw3TqC5OpzahLyPP5gTDep65uwRXxKS/vpJzbFhKLG+2UU4vmtNe13AkGaP5m8zVhAqMe0bNpWefieHcC+1yHnbtLo84A2DeiqUOjz1NNMEl6fjfEwDwGAip4LIgRBE7c7s8t34xqEcafGpvRzaFHYmUCxIvat8VX7jTpGsRsEQkDIQv9ne7skKQACKkctx4MVTpG7WZuJTZVuqUAdJ+tHhVrY5SATMuXpbQXMIpfRzlQqoCme1M6YcQ4Vtn+Rn3000qhL5rW5K2tNdr5HfamZNJqYa0vdvoXPplcHt4i/eSAvdeCvkx6FKDCO19xEZ3EfR4BC72K2wRufjg24MPx3nokbEp1HkX5VdZJeuwGY5tuGrMSXKIKW7F8tTMS9pJeDODteC3NKo42t5pgwdye14VLefZzvdg9l+bGGA/DhuKizmfmqfTlREDxKwOb/E+qYembbP1NXviIrdm/TfgeBpCJvOz4+7Lz0jl/WBXjhkt0JUpveBG8x8iw/X9W7UC/PruYHZec8TIxBRk5HpSGJ2DQdGFpHk3y23qsRNyCjmRzOU9u+ZYFCUjMT73E6oSzm3N49dExBF3KACliFoUg6jYWe/5wIn5XwPoLhv52SaLTgQ7i5xyUflRPlGY8uCwZqjUYf2x+yn3RKzrYvveSvkRzwbKQp8ZrU9U+1gbPqKi4aT7oy9k8zLpPhhjDzOERNJ+o22TOumOEn+hQZShedHCAp0ceO6C8avF5yOXzmOQ5bkfrg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: DxsVnk//JV5+KBbDgAeBDzbNV83NzBjMOfgoftiC/1G7P7h4MYhaaKrqAkfiF4rpcOElWUwtneghYJ+6eAERQVGDckVCUm5GYCjsPWlD2O6EKpbS3Ky4vi853BREbZvdIYYQvU0Xfpah6522X3YXjE5D5SwD9mgeZjVDtkcvYl52TI44EjbU0fgLPAxX45uaLhAcXfHxBUh7pc4xTgtncdmnG0q6bfjqHNgYF9vd2fKEA3izQqNrzJbP8bAbdx0CiR+mIZ8N4gABmSAbo+/Fa7EsdfU4hGflTLWKQrg8f/2oFEMp/HpKJTVInTaOj8pcdNRcUExx8jYEhP0Yz3tLng8hjaPiCyTgUn2HtInT+IxCDMZguB0jMlUhcPOWYQmH4qFS/136MqZiRTnLVyAdxS8gDz9j7Yi+YPxSOBoX8h2u4sjG7v23xuMQAQ97OKdM7OUzx84PeAGgWcRqLB16NOKEA2UkJdQOaTcd0N23HoTdK5H2Ik6zJN2lGdkXDsX9PJRzRBZUeNrVpjGPr2EGWj59iWdWPdpLYinDjK0pBS/HxeoYYVb6gveb8mtHimogqf5LWvG8MRaipq7uwK0LuNYrZByICf6rdQGMaJ7jIDxAF2wmI12DKiHOplBXVSt0noHr+pbxZC8zBFWTK0I1rmDuYezklVvIUbbCYP/rly6gWpV11U5GtL6OOrb/jQ0mK4S4JIL9HUKBjeuidiaQncJ8ZNcy4Kz0eozU+NrAfIATuumZ9jwldOb4CQBo70+/9eyFZaJ/cbPDJt45ikGfrNOrhMAb61KgYbX12G9Ymv5bPKEGZ6R9Cqkwq+tF9XThYANmAeV4COshBHaLltHTKCMFD2v6BejSrx1gHgYti5T4+mC0upYwxVEUcjoPTk+fN7V0+PKoiVoJGPdFeFseH00AkKqWdz+PkghuXXql8ptMo8IUZB8BbVynZOs8aLilHQYhmt/oGv7947nfaN6UKXGZJktFXZ+3hgdYu4fA9GmmFol+bnDHHYGpOC/ryoKYqcUQJ8w+Y+04StPjhiLHr/7m3MhyJTYSgw46MT8Kg1hXbr8zhNlFIgOE8+QkWJD/WmKBuSPQFRyuzlHN55cjOAfTe2r9TR6gf9pB8C8nC/HuzPUcAON0ZAGJAkVsyv9AbI6STYXEyiarJOHc2U/zY7a1Xw4v+nRqNVIdoLDId1M1+X6yQY8TOa5z3iJjiGqvuRQS8WZXbLU/1yCx7n9sQhDiqC82MZ75llNeVezCZnMV+gUAKzF9L6qGmM0GXFBAqq7cQ1jw5esVqFCrSS4sHtCGVYAWde4aPdB926NYJPKHZqMSK1cF5oITu64d8WOeDc+FswlCiVMwvvZxm4RcnLWawENc8ijpuOtm05J5nJJun9CK15l95ns1HdFnD+ovyKq5gaQYOeh9grn403WFdyo8ccFA/TmeMXVoDNZng6kG0cBKHkOVyz2BQlFm0Qg4bEWYFNV5A3K9X1whF/tYJhrrFWqhV75hh1YECVCRpBjNQLd45ApCvyZfIzCLySifUyPSD+itqXiF1NyPV1+awBUb1F57QgQqzJ43+8cYrtrh+LD9OzXvVT/n2LBEzQpBK/2WbzTR2oWRszfAj8PyCw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: k3yylNO+18vzCEEEO9Ymafnmw9saSf+n85gYQ2z8w6KJg4CaaNfaM678ovtExyuUQYjeHTshj29Vt80CfJQWbxsG21s9FvBypplvDbcwVWncBQQ+PojLTAJU9LnLLrCQrdeEzdI/kLwTnOE4fE/vOv92Y8WC0MbfiZruaHdN3G+3OPL79Gvd55G9w2EAL+O9vWdl+Siz19O+iP3qNtWW3MJCt7AG5PT7/X0MDGuQ5o58xFRQPlTemWSoOWVB3EjNDGOwArN0zt0Qn0WxZBAb5a0LrQLfCNmQ+SiPZa8CkhM5G1lK2laVMgELOCG7gD9d4ErN/+2E4Y+ZT/6KsSfrSYiTSXty0J2PwdDornYVyy/T1BQgdQHN9+2Bm5be5gIae+v71OS5XxQdUb6QKq73TWlCX43Hgc1YuwrnU7EEwY3MdUjRzqjzbAbbDVVVseoPbQMCmO9w/w4gkT+4FBNKkGD3OwAdf5BVZPtuR/su05Bnthr2KGEd840zP25CL+hmUAcx0Nj1/2gpC9FD2jLUHMKQ4wqBHCWZ4+ypSK2/+V7VWy+SJEiHtgNwEkg4wuksHjBG5ikH32cWvIbvylecJmGnJm8YbQoZajRCD7uTe3YJg1klXebdo+DX/kAv2EqB X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ec6ffc8-ffaf-44c6-4480-08dced3ac3ec X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:43.1946 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gYERo2tG7pyH7BJFEvpUp9GuwwmN2S+cO8R6QqfFwfsp//ZKtT+9xthPIxCCHJ5TPgqdCAMCzK+F13nJHL+tgQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011585-yiIKYJ0fJwp4 X-MDID-O: eu1;ams;1729011585;yiIKYJ0fJwp4;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Define neigh_for_each in neighbour.h and move old definition to its only point of usage within the mlxsw driver. Signed-off-by: Gilad Naaman --- .../ethernet/mellanox/mlxsw/spectrum_router.c | 25 +++++++++++++++++-- include/net/neighbour.h | 4 +-- net/core/neighbour.c | 22 ---------------- 3 files changed, 25 insertions(+), 26 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c index 800dfb64ec83..0bb46aba2502 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c @@ -3006,6 +3006,27 @@ static void mlxsw_sp_neigh_rif_made_sync_each(struct neighbour *n, void *data) rms->err = -ENOMEM; } +static void mlxsw_sp_neigh_for_each(struct neigh_table *tbl, + void (*cb)(struct neighbour *, void *), + void *cookie) +{ + int chain; + struct neigh_hash_table *nht; + + rcu_read_lock(); + nht = rcu_dereference(tbl->nht); + + read_lock_bh(&tbl->lock); /* avoid resizes */ + for (chain = 0; chain < (1 << nht->hash_shift); chain++) { + struct neighbour *n; + + neigh_for_each(n, &nht->hash_heads[chain]) + cb(n, cookie); + } + read_unlock_bh(&tbl->lock); + rcu_read_unlock(); +} + static int mlxsw_sp_neigh_rif_made_sync(struct mlxsw_sp *mlxsw_sp, struct mlxsw_sp_rif *rif) { @@ -3014,12 +3035,12 @@ static int mlxsw_sp_neigh_rif_made_sync(struct mlxsw_sp *mlxsw_sp, .rif = rif, }; - neigh_for_each(&arp_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); + mlxsw_sp_neigh_for_each(&arp_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); if (rms.err) goto err_arp; #if IS_ENABLED(CONFIG_IPV6) - neigh_for_each(&nd_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); + mlxsw_sp_neigh_for_each(&nd_tbl, mlxsw_sp_neigh_rif_made_sync_each, &rms); #endif if (rms.err) goto err_nd; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 5f2b7249ba02..2f4cb9e51364 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -278,6 +278,8 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; +#define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) + static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { return *(const u32 *)n->primary_key == *(const u32 *)pkey; @@ -391,8 +393,6 @@ static inline struct net *pneigh_net(const struct pneigh_entry *pneigh) } void neigh_app_ns(struct neighbour *n); -void neigh_for_each(struct neigh_table *tbl, - void (*cb)(struct neighbour *, void *), void *cookie); void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)); int neigh_xmit(int fam, struct net_device *, const void *, struct sk_buff *); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 01987368b6c5..e91105a4c5ee 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3113,28 +3113,6 @@ static int neigh_get(struct sk_buff *in_skb, struct nlmsghdr *nlh, return err; } -void neigh_for_each(struct neigh_table *tbl, void (*cb)(struct neighbour *, void *), void *cookie) -{ - int chain; - struct neigh_hash_table *nht; - - rcu_read_lock(); - nht = rcu_dereference(tbl->nht); - - read_lock_bh(&tbl->lock); /* avoid resizes */ - for (chain = 0; chain < (1 << nht->hash_shift); chain++) { - struct neighbour *n; - - for (n = rcu_dereference(nht->hash_buckets[chain]); - n != NULL; - n = rcu_dereference(n->next)) - cb(n, cookie); - } - read_unlock_bh(&tbl->lock); - rcu_read_unlock(); -} -EXPORT_SYMBOL(neigh_for_each); - /* The tbl->lock must be held as a writer and BH disabled. */ void __neigh_for_each_release(struct neigh_table *tbl, int (*cb)(struct neighbour *)) From patchwork Tue Oct 15 16:59:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836752 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40A5F1F76C6 for ; Tue, 15 Oct 2024 16:59:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011595; cv=fail; b=NFuNr+DaHONrCHUULCalxY2dCoYB9ll/1OPLhTuae0suHz/9+oo60aYLbdnvD/jq5clJR5kk4OrVmCtAbBPAhIJaUyTDKQP6fc7A/8lZzTmQ/vLIrFHPfFk+uS2Sh3/hy33UrHSI6W8MR0+vLdm0Mw+/G31kHVs4jkir7E7a/Cg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011595; c=relaxed/simple; bh=cZ0xkR96fkOBNvwl2CPTUoEDacUmIp5i3LzzsO9AEcE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=kobR1cHbcfd+HZA8vzQRMMU9P6LjPFGEzHSW4gMU3AjjEUaV8AcfAbMDuRCN8q89ILonXIGISPRLuu9L6axeaIW+7uHd8ijMR5nqK2c+5EAzMFwz6nAoGrbFpH3PVm9C21JvIUn2AQ4KvkaTXepql68rXWCAw8SYY9SaAcbfbqg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=Np0BgQaH; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="Np0BgQaH" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 6057920005A; Tue, 15 Oct 2024 16:59:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OLbxjO5OsJXJWneTs6pkS+yNVgQ4AVlz4ggJSJqCm2gww8+T17hHho2bdlEcgj3A0QNmkI2vByqN499JOu0ZyUPz4asaRuTFdKeCS3WSkqfivHT6IuAYQSHC/5p02w3F49QqbthT34HQ1e5dq0HWl5Yz91rtrYDGVZc53PUfGcY986C8HEb9m7I8Of2Cp7HBSgdFUDtZfGxnOz1ungwEVDwN8NoCKsC8TJQ8cvMgZRrYJKy83hnmSEMNsx5+d7BFIxesAmW/PDMbxdUDXzNv0jKHv8PLz6irF2dIqEeNq0F7miabDgGEgtwiUPwwavHuxPaPYE42dFaO/kYu7H8oRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J9ssrDYMpF4+bD825oiW+rpMkk5jMJRW3sohGoIXtvg=; b=GRaXy4p9wEGdfgn4dwlBmLHB0n0YYlTFPH+AhAyQ8n4hbJ16ei4ttmhpv/ceoUGmd1aTktiGK9TbXrMBCNM9OcEtdNepqG9X65EGyFUCo/HycVgelqIMEu4zlnR0HRHDtjaQbN1Jc8vpWNM2i3uJzByzvo60nW4t5ownRNhEwT0AbId62RUeHxuT8JENj+Cq0kPg3SyIWQf4eHHhIXrJJ7h3KgvkTFgrLS1Yb3i01QjZIi3TblNwoXW4o71hErehRSF79aRAjV6bM6aT5NXUmMmH84PFLwbETKg9tJ3Z52SIGQvzt7CRwFsbSyk35oDhwvHlA4FqexRn+hvB8NyqYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J9ssrDYMpF4+bD825oiW+rpMkk5jMJRW3sohGoIXtvg=; b=Np0BgQaHNXgIPVbmTJEmSnYlADLgz2Chnctz+lf+oJpGF+f8XddFKTgbPyasJJfdyBg9vqk5PdCKzqSwZlHEn4xqw1I+c8wHcQ/WVbJl1pKVm1Jt+iPMyXSsSBJsJiTntnAg8de8fvibJ8QBIyJtg3htZ+UWbMMBKnz48MN6qW8= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:44 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:44 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 3/6] Convert neigh_* seq_file functions to use hlist Date: Tue, 15 Oct 2024 16:59:23 +0000 Message-ID: <20241015165929.3203216-4-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: b283ec86-412e-4413-4761-08dced3ac48d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: uSF17XTBGYU7RuqO5FQIplCUADHQxq8Vmdjpwzw+dryCrz1GBbYLZlCEGteTk7ge1oM8pG1AGRnBauJ7yOx8faR9y0nWiywrajewgCkXxuu1MXGgdCs1n7st8PpxerNd/jQeJJY/Uyl/arSyr4ZkLyS6UOzQmV0yKgL0p+veUr6SDG+Tfdc8POFg6Jx5vkiompoRzlscb6NvTd1+fGVGRySHxM3HXCeFR4toxl+rl1FTKjK76TSbQ5gBlvTClCQpXlrEXJtY2ajClHzxKtNLvJR/Ry/dMrzTWryC5H7Q1gK9cxS6vOOviCaAK0y8RNbVgoQuJ3bTqeN3gOZ7HZiMVwTIFacrDAQaHeBPr2WNlW1BjVKwIc/Tpq3Wk2eHBiqjDRQKSxvjRAS82S8sho/AQRgeFfj4CELuO0MaW746d//jduWcOuE9SVKoBVBxSoats3Lwnbp5EMj9wgAOzI2dsocJ79975oBtL5pxpMiuYODKPyQgueBovbRK6ERQSnqYlbeImdv5kIWv2UhNA9RtRyb/9YF92FxvXCUKbKTaxd8iF3Fnn5XkxVyqVTbP3ybLoX/LGjpTsdlM6UD5sFGGUWoMQDoU61xQhA4LojpSVyR4Zkjwp9dSShFaO+QHb628qK3V1Qok8ULqP+RM6gthBsOWwRMuPCXYEsjiF9Dzuj6dXP/CnispDPxSizBUKejipqixi7PExR1SxTfCbW0fR9FbQcyrwfKSmQM9eqBHxt0O6teaxp1cwBfNNs1y2NqWU5igdjIBod5DcDMsKiqjbk84huN1OgqcCVc4B+6jh4GTPMqpuXWCXgTPPZ0s68wz6ia8/ETSonBsW5EgiW3Jp6LRL3Zx/ctaHb6P+uwFjTDOl7FICFS1BYcox/cG5WKYpBIojFUX9Afvj8J9nh1/kPoXPgPHracM7s3VKeo6u53d0IB3pW4PN/P+HYxAGPkr5/7kO08w5oE7tY2K3cblbFwSXRX2MbJHjBYkl0PkR7i1XpvtzO5DTpb6EKlu2E775RiGLSnLZwH+1ChELx9+Ga1b4fYAvB5QhC4lvmwDbyAZo0Z+KJyyo52yM8uHIqf/11ui9ZDkwTwcgfESz3CiwfN+63I3LD3/orj2rSJtykO0J8KpkZCWbBWDvY6tIyMdTj2fQYZ891eBSy54u6mtjQ3O1kV+bj0idmbCeiwj08hivlfvDONYx94UzXA/di12G+3y0rKluan/nD+zNmf9IcPjmTRSDqWeDkVzlGqFRipQreL3hjDmrFkvFh/Kn8jEYDtScQbH/ukLcWvLrLJD02UezZ+y9nilQxpyl3iGpSD/16ySwvL8AxXJE2lb1mex5zTh4mxhAjf+E2fwyVcEMQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RCgfI3XkwlLTT/3a84nFsdOHVMrRG4Fh98ZGDiBGvoVG8c4GTWugPQdWNikTi2zWn8mnB03JFf2saPjKQc/5uCOfKhNs8FyfK0GIIB/XuTQAqpJsMFcdd5lPVRbSlTPsvgGvI1jaPnKCgE75dMH7G1PXgWfNilxIl6T2OFsQsVumGHJXqlHm+BpTuRnlb4oHOKaxi+GYUBqeoSMiRPrbTTOoJM9BnjA8H4j+9NNVVzewBjYAx0HRzoImJJLAzfS4OHJLuDstFoareUXOVfZPdG4QA+NAYxc2TqGe6dNcgbxthMlmW1YVgsF4Y0kYUwCpB4tphh4vp8Qj0lkFYd4yZRs10pPAQrivl0y1NNVD89Sl91cXdRp57KqQ6zZ6xKMJg99vmpBQxALWJsdi3EI7KVE0bbZU6GjaVriq7OXdolAuyW88U4KIZARafOAYDT9pYe6wt5l9UL37steAHqxXamZbFzlXCSpD095syZjG+PjZaP11Wm2MRxEl6Pa+yBf6ff3TdzZ7Yypi1ynApDbukAuhA+T6FAclAL31p8dj3jSRxnJ+n0vNk4Kb+kANnBedn+ufX396iH56hXUp8gR3JRj9xn/Yl/FxezL9RnNcdHEr8NHdSFh1Jtlyk4aNiMz6Jy5qFWAfr+Y+IbPAISVgeKkPlsCtOOjgFFRtgBs72GlJDWeW1FuIfx48+z7d6j//TTKC0oXQMeIESdmxXd8Ts6SkWuCrkRMNQDV4b/6kopjSC/o+Xzp8cFIXF8K+sWQeNbq7raDPkB1s8ioMzBuwGJ76h+4JSpF15ChCoGFHdIT58VGwcMZpbnf0gXEyQUkMyp1cgYucK20dX40ARLgP3gSbPpztS9IC/ut9wgD7HM6ICxnOY1M8+SmrlEPriBGdxe6f8HMBkpHDqJAUm8YeaWlE+AfTRMU7XDmduYPhT+6RBoU9phleNCIH0i/HOzNcE2evWGGyChJ3o/hgmzVqSWSpPkme29RDwqtBSGLlPAESqCCjlXtkgCrjxyYp7joqiwyLq+zQz/7phn1iS8WUQTr1BmcLQ6dbF+V/jtU+BClJ7jgwuDmC8gT4UB7c9rW21b9u5SoZpC8lv/h6cEy24+fHr7dS6bbmS7lxT+c9j08OUscxM6ucLz3SJcTWenWujyL9/vgWqsG8VFB1DbHZBqZk7MBw7xY3aNMD98m6Pqygtw7fZ5+gUYtyQzsaigCpeCMlg2yFIRn1uCGfRTmzJYmPabeY++XMHXdgdeKIP0VkORkUxec8ySr94JRPBpCGFAwpp0JzRm6PYO/2F+9uvlNPtd/145W+q6qsc4g+BPE2FCelAxKfXUydh+R+gnElQeVL89ktBaa4CdORxHa6BysNQKAGAlr4wut75H9Ew1KZAuCr5sQ1W/CsFbXkKPCub0kSGkIwP9HzlrXOkgMqo/S0fbEz70JMvJml784eN8VvyMmiSToB69R7bM6zBd1FAZnMHEVOSGghxlWOnnrTRf3KijxBcd2TTEJMj3LkQSTwe38fHJ6lCU0bmWho90Djkuzl3MLA+658B2+CJbhRkjrrtvmwykRp600uFDp1kfp89lr0MLERbTKtfovBZpGZAcI83DBkVhyOYyJSL3pReQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: jy9H+DFbzbghi4Eejwam5z8tGvkUOhD3AhWko0VL9ipffuwNTslfeG//wTlpk9kptANSW7OSjGyqW23GE/zq9y/wZuRcT9D6wpuZe39T8xnN3jHlQq5QsieqNiC2UXtpdetd/bGuIr6TLoql96H3uxF64fGGKac9uadeNCQfFO5aKjXKz0lP7iCbh5YPyGfimp+6rcMd+19Tw+VsSTi81jOhxl7gT4OKB0zX43FjdcXtxtWnmIzUg1oh9AL2b4QfiqG4bvPRI93bT68b5n8WHRfHmJ/GkE5Vwh7/MdhF2rUfxv7I9tUTLf2StK94KLBh3k0fjhQYv6HYXwx+J/Kv1Foj6kRLNEFS3Pexa1Hv/f8AMOoPbBBSMYYu2ujav2ReLMrpQGel8nHQ00wu5Dngg9JPdPTHm00977XyX7Cawza5tWMimLDyxSm6DFL1VVwpV0pV4c86W5ICGnWEkM+gfiZNV8OhUHwME3aeFG3kZMjGcRue5+y5PcHzkUowfhaHlE7BbEVggJUmrU2Ueuv8UQiAesJiVDCUdPvnoln8UWTkGuQMBB69Je3sewga/udfUlg2GU5x05wJIHrI5Nky4ibU87Jc1NE3P6/sKjiExHwgnsbnMXFbUfdCadBOjWv7 X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: b283ec86-412e-4413-4761-08dced3ac48d X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:44.2502 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: leRPBA09SCSRKE9YK8PdQ50psrqvdzziT9ISO+GjfiWlgvfkJOykowhFV2jMODWb9pI8djGkLzmbJt96JDjacg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011586-a7Gou46vFKQv X-MDID-O: eu1;ams;1729011586;a7Gou46vFKQv;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Convert seq_file-related neighbour functionality to use neighbour::hash and the related for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 4 ++++ net/core/neighbour.c | 26 ++++++++++---------------- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 2f4cb9e51364..7dc0d4d6a4a8 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -279,6 +279,10 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; #define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_hlist_entry(n) hlist_entry_safe(n, struct neighbour, hash) +#define neigh_first_rcu(bucket) \ + neigh_hlist_entry(rcu_dereference(hlist_first_rcu(bucket))) + static inline bool neigh_key_eq32(const struct neighbour *n, const void *pkey) { diff --git a/net/core/neighbour.c b/net/core/neighbour.c index e91105a4c5ee..4bdf7649ca57 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -3207,25 +3207,21 @@ static struct neighbour *neigh_get_first(struct seq_file *seq) state->flags &= ~NEIGH_SEQ_IS_PNEIGH; for (bucket = 0; bucket < (1 << nht->hash_shift); bucket++) { - n = rcu_dereference(nht->hash_buckets[bucket]); - - while (n) { + neigh_for_each(n, &nht->hash_heads[bucket]) { if (!net_eq(dev_net(n->dev), net)) - goto next; + continue; if (state->neigh_sub_iter) { loff_t fakep = 0; void *v; v = state->neigh_sub_iter(state, n, &fakep); if (!v) - goto next; + continue; } if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) break; if (READ_ONCE(n->nud_state) & ~NUD_NOARP) break; -next: - n = rcu_dereference(n->next); } if (n) @@ -3249,34 +3245,32 @@ static struct neighbour *neigh_get_next(struct seq_file *seq, if (v) return n; } - n = rcu_dereference(n->next); while (1) { - while (n) { + hlist_for_each_entry_continue_rcu(n, hash) { if (!net_eq(dev_net(n->dev), net)) - goto next; + continue; if (state->neigh_sub_iter) { void *v = state->neigh_sub_iter(state, n, pos); if (v) return n; - goto next; + continue; } if (!(state->flags & NEIGH_SEQ_SKIP_NOARP)) break; if (READ_ONCE(n->nud_state) & ~NUD_NOARP) break; -next: - n = rcu_dereference(n->next); } if (n) break; - if (++state->bucket >= (1 << nht->hash_shift)) - break; + while (!n && ++state->bucket < (1 << nht->hash_shift)) + n = neigh_first_rcu(&nht->hash_heads[state->bucket]); - n = rcu_dereference(nht->hash_buckets[state->bucket]); + if (!n) + break; } if (n && pos) From patchwork Tue Oct 15 16:59:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836753 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36BCE1F9EA5 for ; Tue, 15 Oct 2024 16:59:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.34 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011596; cv=fail; b=JJH1SXqpKlkzfPn58IvLq7tveOK7iffl8YQW///JjbcljiGtY+OKVKTfFx/OYtTPBk1PuZcLP+0ZyC9kjNu+NLwLNqsWxtzMUxyQCbVo4hx4sGoJJZaEa2HwH/JppHt/kapuZYUj15CmEmFe2NG6qYpN6JFyC8SgKJu+/ymocAc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011596; c=relaxed/simple; bh=i+pMkxN1KqyAnoh1DUu0wgK4Zq7pKRwaXA1pRHVPbGU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=BSltgYsDfKAkwVGnDtenztK3k7HN0NKU6R+fBE2I+xWpaxPwBkg9bu2RIiYm8hgFNL6GuVFfSEyGcq3+U9sWT0xJ4vDzl3B0XXGeN77vHWZxzxTKrDbuDKLs/aEPUrKkDQ+jt00/dtP+sAeL3vpO45jQfUpapdsMmEpnV/lW7og= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=lparX1mQ; arc=fail smtp.client-ip=185.183.29.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="lparX1mQ" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2169.outbound.protection.outlook.com [104.47.17.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 75414480058; Tue, 15 Oct 2024 16:59:46 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=v5mdsQUpRiDpe+iT173oEr4MqIiklFSXees49rqAX08E6Cp3Nbol+JB8GSIbyEZkCH1q1DPtJiXEoZpSs+m/9xSgk/IU8pT+9CKgUipqH0RBGYTMuir/A0me9xhdbaffOd+hhRMLjWJvYes30+oAMvdpPb6/pwjQO8nsDYuZ/zBacOGWeRHCs0cjRdp6n69joBNYqqHAbEJDq9ogNsKEposD+C8prs7xOV5fa5xrR4iuEhohA4kMSNIoYbXlDOqbItOUZsQwPi4H4wWs0w/CukMAjRilZZ9z8gnIbmlxJ8v4XM8UHyX1lPwDzHtUTBfI40s8hnQIRbPro3+o5P8rmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P2w68Frqn5CmZVxMYA8yP84xvbUNkoXIyBSn5v34jX4=; b=HcNCg0x2vnM78l5U8BkIG5HYRbtZhWxlNPmUDrhaWjLSWY7voY1b+zGXunsm35rCGL5AAMSzoizaONnPmrm8+vnst6sid4hq1qiLTqYeU2ZkT3E4pkvx+gjSREGbQN5q1LE68QF+Tq0nWZpjfGcu4ZfuVkh1MHnWMv3h2XyyJD2AFcMQutgLDIDWy4T0jiaaidi6GzBoVwtykC4ad+yMgTiHWK+f2T3sBKLYNsOJdS2F3rSxsRlp5nrguDWJv1JKKnhEATpjcBkkbfoMDt1OOtZiuCt3otCYP03oiuIuev8Pn96YBDDd+wbI/Cth8yLkhUlOTI0wF5HnSo58G+cnSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P2w68Frqn5CmZVxMYA8yP84xvbUNkoXIyBSn5v34jX4=; b=lparX1mQV8Y26kUzMM3pI9+L+YL0mVtAZGiO/0yDk6AVVQjTWgAkCTfzZhww6sXcHPOWL6FYWYCcTSRpy5ImCxnzSev94hiYb7OYcRICysWZQzNMGS68rMx0dUSz27dsXu+tP/pXxrza+d5KEASLRtdhCdMbzAKhVzhWDjbRwFs= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:45 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:45 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 4/6] Convert neighbour iteration to use hlist+macro Date: Tue, 15 Oct 2024 16:59:24 +0000 Message-ID: <20241015165929.3203216-5-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: bc086242-b336-4992-e6e1-08dced3ac528 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: 7Vvy4SdeEtQWLAJd3K3XnJDscDSz7dY8fskz8mkLlhwafc6lovlLbcaZqWL+zjqmCwmi1bzrAplEo9hxuQ5dv16Xckniu5QfSNyLNgnJ7CMV+2H+akcPsG7Xk04hrP8iZ5K7934aLqlC6P3Ez75x0Nw4wdvrskSMo6MlYu4/W4aVJerznvylC3VHjNxVdlm24ARCycLBPX5nq1aNinfY3+flUC7Aa7G7jqWHB39s0LwwukzTgBXvqp29tcROM056CI8/8g97wSoIMblrZ5Q+l4PuIvtXiUTfdJ58tr5lSQ1O3rojCmoc4R0nYHetbUyrcutWpY05nli3nsogSSJIYSY5IDDdGkobn2Zcf/6beVPFlWa2SIOtciLhRqtv+1+GlvLlpfXhyEZv0yI1SZAmnWFwD/4HILBxBxlvve1kKNIPgJnKpN8V58l+GZWxl6WRLdgyuIwTCGDRLs3+j6hqx0IDzdTqzYCOWfPpOyoEWD3z/1Cj9Tmo4JortHKr/zSG7LJLHhYwhzbid01sIWU64c05u2+fUnHWXNnJ6x+ArIem+w2sBkmEYP9BD/cb9IlBnefs6/1oyJ6RqZe6m9F7lL7JuS/L4HsgdDr2mMecvr642byw130wNMx8Pw8qyQSFkkS6JE2WT4WjkWuo1AOWNlVMKKNWk1I8UMapZv2Ow0pfOfZAzfiTYEn4m+wLKcFBRIHa3ffxOhXDVKelAiTz0Dxhbhoh2ImAwjVzWQEsLu7Fr9cQRWFTxEkqN17k1CO7FqhqYlEHXJetQgdSgSscRr3ipbyC51nHZhAbNMEbj0vH6sD1Deb9xLujSRHDS1rE9nuENV5DAI7GFiSsvTz4hvSVtC/m95aNMktsqb6Vjv1qobDF73Ypw6zA/d2ugIp0XotvWpYMLYgAFSNWAU02F9/K1JUE8SXY38BDXg+FZJqZTSXLRQVtJr9l1qWZ4WrdAYNlPTA3rlpVqjWYgkssyloIkGHJpviilc63Wpnqkej9U0fAduBOI08ASAHbwjr6yHsMl0El1sekg2d2wOoUf07xn1p2syalxo1uGl3/sexhT1dCSoKW32wStEmr0KA8Jm+f11XYtkvFUDk09zR2WOiI1jpSwfBGCCbNCcmsZKmwl4CB9fKAPh2+db0VbQoBR5u1rW+GrBFeqqNiA6DDYrT5r87Vdio6gI5RifJK1a8f9Lo8HujU88EosGPvFekhW6IYAajjwPSt2DbYdSdLWlQdaMsrdAgro+Y8q+p+EUBEY55EG70DVFVmRSO91SeXoJkWytgA0ORA13tUM9icOB+mZlWSL0qAxjNKMes/wIRlVsgTy7mKFjb7wC7pGGbsUNQAEak9tfQGa3RUmz9Wgg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZUoib/CEhs4jXnsxgqbdsq0H8JTPX1IoRTUswHIRTd/NyhYDNaiurx8Zf7piDNyStRksL+NerCSqYPhzNWMSNxiRXVXy3/o22FBVUfZkUqWPOjUp/1qXvMbZm95/fXY5uPzpkGoWK3tt0y/PS5Wqw96DcarO+q/EmyHFy86PA9rg4dYKDtpPJiX7BcvSLkl77jfZzW2YyAxoAmGb84FeQGJKZijLqsHRUHLmnSI/R1cBGszqdqU35YJeiBFKgW6WPovOXn/GoJTzp23dGTwcmNYhDa2J8CiL8bianKtZN6FjhIrHGFbQyhZmUaeUzYoPFBLLfHpA3tTPdc5H1soSJBlyxHzEbsFsf7/Oj2XMiCX/4VXwVrGKkld/ZJZYLoFaQu4bxJ6dWGQODqdZmK4Svd5AXtF+STYZr8DO2IEajPR7UMvz8TJT4Bsz702HDsmXuJANE7ixP2b6kp3Et+8UMWah3yHoj+0Et7pV4KNIGRwvr/Hawbu5sNNA3NEEHe2aSc501JiQVFI1G36MeJorun2Im1qPV5Lwsr9Ny7FxcqVqCkTLtkNVVm9vkw4oIFsNamNToJrfnYWtvXvDsafAy+FQ7xAtXuOSjT818EyBWlKll5fI+YhpDmGVlE58HQWumhIP9+1Mww2GWi/232x+ntvF3LAXPKVUW9hSTNaOcOoXaxuoomJi98+eyODNw1KlQn+Lr0wWzFwBGB8qcmkEio3j72D04sX61nZxGbOpyDzPr34/3PA3bK3n+MFASdq9wgNGVjKMCGbdrgw1fIDcaGZFfHi742dDiDV7Nbtst6oJV1ivmtboads3Tc48nRTmL+jvolKlVZJZ4e2Blft7dT32W6XQGwnihmBNyBJ0ikiOJBLttgfbqpsLfiGvSLTRhNWUVQD0cu32qkvaPw8oaf7CgKbTAsdxChMMh7/r6TgrFfpxpO2lwvz3Gt5gH94m8V9zflz/M+paRU9n7/1Y9IcPYhl4TKkGIBnvFl2qLxN0rSyYTqn/Na6IMAIMw0OIo0oVwR5YNpY6p7Xspdy2WcYE8bb044/CcP6Nd+MoyWQU2UAiiNBcCD/ScbeHnCxn2wTtMOU05kezqS2NzUm0fTiFAD7zVZzSnJLhFY81r9geZ2eiY+vjet+2UK6FRb8oweK+jz423LwrLoFAaGftqk1+TtKGU990UkrhTAGraHAyZl70vNkifHGdN6Ma+zAyXuwcfwMI2+vCFaGIM9pa6HML6emXZjI0fjcpQQIvOMVV47kzGhYBILCu4gMjpmc4xpUY3gbv+lZxI0FAm50qUFQlmicYaMSxHjMr4pDVSWjx+1h9JoApgbi37KauP1y7r2vY8WMya30RckrnsvecNAH24SzoZu5vdSBJ+4BocqS4ERmvmpuXSQWN/CERK6d6LOU1Jx8rpMU5bOarAdOMB2jFu9Eikx2EWANUid0hcNgO3yCc0RutnbcgOLvkK46scviQT6caX0bDxjWZsWaYct/FxTWtWkH8M8M8URbljOR09YpGNOupwRr1Gp19HphTpDfCkOf1MCk2+56GPj1eY87N847P2FS3G6RUK2p3voLSVamnS01UwQvE3R8XokR28qrALefpKo8f5m7nQi5YBQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: JDjLSCMFxrvxGatQ6XwpDtcGwR8c1RRXbacNlznF66lPHxBCuzM+MsCEb92hGPRisr9lDDeL8ZUUlXb51R6isCVTiVCF1NcixmbuF1Sux2Tv8HKWoBQRfMYWkMuJQPxPda6fSnzkzY++gLOWlHLKnrZSt1xp80USJPwnUWpcUDcYTPw9dYYCfA9sgQtYKMS4bblIfX9/62hI1pelw+HoGYYkqHu5tPT6uv47xdA0oauSgPJLpPhhgQKNmT6jaQ2HcS06v6IT3t8Ev7cXveTIZS99WmBeaMBRgL+9VRn2bkMppzzPUj9ZH3rhO68Uzvt4/0PbDK2GJlKHOse+Z7ACPPKUJLvDHol3csqQVuGsJtQjJ0ynzWfmVZSNOOI/fLLs/HBZaJN45gDdKftjDMMZFUaGJ6K5PgvP1K3zLJNTRlD1CytGBdjiyKG5tYoFDxg2lxew+PIqF/PBL1ZMAO28hOsiZTVQobctkO88/vPS9364sFzXsPGnzLADp4D0sXn0IwOh/bTgWAYXawq48y7683rryZiRuq1j7mAtIh1H0zPD2PCfgFfSMYcVau7JZoyj+NGshd13wyzc0VYQqu9PpSEyWy7piUbpxZcvNs8Cy86L/PBUoGKR1/RQkFCkxnKd X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: bc086242-b336-4992-e6e1-08dced3ac528 X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:45.2618 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7z5Oaut/yCmR7IRBrwaInPoX+X5dFI/28fGNnBXcUHZA0VjoolRfFqspsQ5yLyk81WyUjOFswjDw6tV1qq1Yig== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011587-xRkNbYue6_Gh X-MDID-O: eu1;ams;1729011587;xRkNbYue6_Gh;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove all usage of the bare neighbour::next pointer, replacing them with neighbour::hash and its for_each macro. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 7 +++---- net/core/neighbour.c | 29 +++++++++++------------------ 2 files changed, 14 insertions(+), 22 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 7dc0d4d6a4a8..c0c35a15d2ad 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -279,6 +279,8 @@ static inline void *neighbour_priv(const struct neighbour *n) extern const struct nla_policy nda_policy[]; #define neigh_for_each(pos, head) hlist_for_each_entry(pos, head, hash) +#define neigh_for_each_safe(pos, tmp, head) hlist_for_each_entry_safe(pos, tmp, head, hash) + #define neigh_hlist_entry(n) hlist_entry_safe(n, struct neighbour, hash) #define neigh_first_rcu(bucket) \ neigh_hlist_entry(rcu_dereference(hlist_first_rcu(bucket))) @@ -312,12 +314,9 @@ static inline struct neighbour *___neigh_lookup_noref( u32 hash_val; hash_val = hash(pkey, dev, nht->hash_rnd) >> (32 - nht->hash_shift); - for (n = rcu_dereference(nht->hash_buckets[hash_val]); - n != NULL; - n = rcu_dereference(n->next)) { + neigh_for_each(n, &nht->hash_heads[hash_val]) if (n->dev == dev && key_eq(n, pkey)) return n; - } return NULL; } diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 4bdf7649ca57..cca524a55c97 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -391,8 +391,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, struct neighbour *n; struct neighbour __rcu **np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each(n, &nht->hash_heads[i]) { if (dev && n->dev != dev) { np = &n->next; continue; @@ -427,6 +426,7 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, n->nud_state = NUD_NONE; neigh_dbg(2, "neigh %p is stray\n", n); } + np = &n->next; write_unlock(&n->lock); neigh_cleanup_and_release(n); } @@ -614,11 +614,9 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, for (i = 0; i < (1 << old_nht->hash_shift); i++) { struct neighbour *n, *next; + struct hlist_node *tmp; - for (n = rcu_dereference_protected(old_nht->hash_buckets[i], - lockdep_is_held(&tbl->lock)); - n != NULL; - n = next) { + neigh_for_each_safe(n, tmp, &old_nht->hash_heads[i]) { hash = tbl->hash(n->primary_key, n->dev, new_nht->hash_rnd); @@ -719,11 +717,7 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, goto out_tbl_unlock; } - for (n1 = rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock)); - n1 != NULL; - n1 = rcu_dereference_protected(n1->next, - lockdep_is_held(&tbl->lock))) { + neigh_for_each(n1, &nht->hash_heads[hash_val]) { if (dev == n1->dev && !memcmp(n1->primary_key, n->primary_key, key_len)) { if (want_ref) neigh_hold(n1); @@ -976,6 +970,7 @@ static void neigh_periodic_work(struct work_struct *work) { struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); struct neighbour *n; + struct hlist_node *tmp; struct neighbour __rcu **np; unsigned int i; struct neigh_hash_table *nht; @@ -1005,8 +1000,7 @@ static void neigh_periodic_work(struct work_struct *work) for (i = 0 ; i < (1 << nht->hash_shift); i++) { np = &nht->hash_buckets[i]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; write_lock(&n->lock); @@ -2756,9 +2750,8 @@ static int neigh_dump_table(struct neigh_table *tbl, struct sk_buff *skb, for (h = s_h; h < (1 << nht->hash_shift); h++) { if (h > s_h) s_idx = 0; - for (n = rcu_dereference(nht->hash_buckets[h]), idx = 0; - n != NULL; - n = rcu_dereference(n->next)) { + idx = 0; + neigh_for_each(n, &nht->hash_heads[h]) { if (idx < s_idx || !net_eq(dev_net(n->dev), net)) goto next; if (neigh_ifindex_filtered(n->dev, filter->dev_idx) || @@ -3124,11 +3117,11 @@ void __neigh_for_each_release(struct neigh_table *tbl, lockdep_is_held(&tbl->lock)); for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; + struct hlist_node *tmp; struct neighbour __rcu **np; np = &nht->hash_buckets[chain]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock))) != NULL) { + neigh_for_each_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); From patchwork Tue Oct 15 16:59:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836754 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1548C1FAEEA for ; Tue, 15 Oct 2024 16:59:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011598; cv=fail; b=Ak84bjTk4+cn+MUPhRNW9AwcvnEor8/kLFB6IxTUJ/nsB5z9CLZsfjrwb2AZwssYAr4RlKxqfehxb/DS162gd3sN2rMZAarLUV6TFQEQABvpnU9qiOj3YRgTRxRkQ/Z6ocDxUhC+0i/3UbN8/Ezvl+iEGcWsvvEOqn8N4Fhimjo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011598; c=relaxed/simple; bh=TovbSJ0os06xxq2VkZmFY7wFC3lAKfN7L/IpdaJg9MQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=bdyz3sWYJ6217yO0lCWFCrA8ZJ3o+A73Nm6hUndDWmreR0MwBZw0X7VNHiG65WJu3m/BglT3KsE2XIljtNy9pfoNHmI3p85jnOpdHGv67UshD5PN/hU0gVP7D6K883soGA4zlTgKJXxKAR9P/P4VZRUBM+quN+ThojHejNp4LZc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=tUNFX+Uz; arc=fail smtp.client-ip=185.183.29.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="tUNFX+Uz" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 9C1D3200051; Tue, 15 Oct 2024 16:59:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PSbEhLCrrC5abPcgMOx9CRIoxwVi0gpmKgpZKUybcnErXTF70anjPsLAjrcZ8b73DiNQwacskMhA+bEu48lCLEKYjTvgo8OIehrNhWkJoFFytY+XkTyWaAejfR1DqeAQWNTlCslOUM8PyFT/deLq1qrGvyOU6dSI88oluKiElHMRl8WlFDHDDhEpUlRJqUJqRdZgRWFPz0SRuCxyY3O/OGum95c1O+iokpbjj9jXMLVCFY4ggkreGrF08BdDpa4E1GcRLp620eanmIz5eQYj3V4v6wTQQYHQ8imCKWHtjRVbbIyMYHylMxGgGPdCPeyn8WtQpMe6DjpJwdsl43fNAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hmEJauJotV5YE4waAL3AhBonmkwobPKGVGWn3lk9ePI=; b=RmUvVJF+VaVef3lmnumo8hKuJa7Oy82vRSsX4Xr1wXlh9e2WemOG0tF6yQQE76CyOAsQB2hLXaA2wWsFSvV2TutL5ufnrZgrTyu2knwmpWaDNV4Dt/JnVoXk3htNUU359gp2iprlGyb/CQ7LKc84a4hA8W5LQVGId7Ig/XQeohYfDIF+FoztZJX5k9xiOBpNkZ9OrMjN0EP9ZMG2XJYKyvoB0cQvyNMv0PcpqbiWHjEZ85bfS88I/dZlP+GTgNx6HuvMmkMzRSRLKkSH1dAFcFDOoAUzyqxIgzmB6TAJTS5kgQlopKG7hHYlRFY5vK6qEYI6Iap9Aweij+B8VERp6Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hmEJauJotV5YE4waAL3AhBonmkwobPKGVGWn3lk9ePI=; b=tUNFX+UzqWIskV1uti+PmH8K3Q5bTPKdoHESHWCYLP9HtTLaIrww20g1NML2cxSAua76UoJHqQUsJF9tXzJoE4nENOXbb51/wCpZhK+TlhDFHkfin9t9LVdrsc0n7kJp81NnSwgvXXfWVoatwzkuIVjxPwFOgZFMTklV3RTGNSQ= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:46 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:46 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 5/6] Remove bare neighbour::next pointer Date: Tue, 15 Oct 2024 16:59:25 +0000 Message-ID: <20241015165929.3203216-6-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: e1afdfcc-5a0b-4ad1-ae91-08dced3ac5cc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: pjyaYtCvOZqkAC2k1aEFAFUXbNnbawjIbzuNZzb//Ge1sOyfcXGoYoRQmrjWXXHhJmyg82P+0vWFOc0yli0l76DH1X6M8ZrMyjAGWnYwlts26geHVttY4YTEatdKQeTbVJ2Qv10hBw5bsT5j8TFieMQ41mN2ZYfzUozqlZSgi46ENxmf3A7iWPsCi7MjQRMi+1DqUAF2Ch0FgS0zR8YLgQZoEAIETbhRr6UjztCp4Zd3KCGRxSnWQAaIu/VmUHktzUu9zuz1AydOgizZpYvVl97MVf7x5cWQ1A9MZqVslLqogOKPj+luoeG99o7+J4zmbf7yiG3hXZl3SKkTf/6lF0AN0VG0Cjf+1sNGUy1hXXzQ5uubM+fMMqg7suZ6JDDcGV4iSOpDYSmQdTt5ncs0AMnfflFAOBmhTJmFBOCVFkqH9Q5oeGvkAL5Tx6jRtiwuA0x9gzRlUPauo6f4UMN2vx4hujkLrETP3EKLLw9XtPFDr0SP0BI+qV42RHj9rzPLXz30HUTO3oT0+q+4hPh6upBK0cyxoY/7RMTgBme5A/DIep97PSZdS3RvRsqVHirhuPWt80tlfqb35y+H7BxOkw92f5VA1k3LcegVcYkU01r7Op6L9zinVtN2maRG3TgekktLPej1qPtFZo92n0+Jr+yYQch0ydqLRaGhzWF6emZrzzxbTclWHhYZVdyV1fZ/br9Qvi8/ItJr8Fp7b78odRXc00wlovySRlt1t47/PFOAQX99aL5X6h7JUqECXyvsmWKsR2ks7Jzvil7v8FzTjOS2Z4kLWtP1/hx7a870Z8BjNEHHzdJqOhXnT5XG88sTCswRX5Zr+Bx84fNx6uG0EMgO3veCv86mktsIEHt+WxV8STQfTXRFt6hwXs1DkggAy/Fxp3nmBVvzQAH0gsLBpH0XL+l2OmqwlIObRKWFJNhtbdmENxy7t2wSVMyRAR7b5TAXWblGkS/o+/3PAmsvqAIaPsuszmAch5/4jrDoc3kfv2hNTD3CYXkU9WzenL/mPvQhTYxHqB2WATFEhwuKzQAJ6NCFGm65Xf4OPPiXxyLQ+mgzKARuGk0y9ZD1pg7DkfRBf0brUNQPfyglPMAIvU5GIX+HfWmRi9hhzxcMogZOxhdLFxVCwOYJRlxMTBBc6KsbCqGV5IDB9gAmpaOzbpAlZqFXLsheGNDqgnLPzdAwR8v8gUvJE7rvNDh84Zk4fYuARBEPql3/GUSl6kfow4zqczoBcinP0dueViXUVeSDPJMkWoTOYyogyxcu+UINJ+VpXNoupAQXo+EHjwfyUNoaFXngizMiqn7f+Msi4qtcvvXB/4JMK/yx9g0ntInK9InA3sgohfOpRG6Lrn6vIw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: sNxvqeKwhHQ5AMIPKHZQq9l1WdU5pjbFmC2Doa6T3PbG0CuojXy1k6KsHZZlx13SNJxqSlGBsVlrDNROFJX4MkeIp7M1ODWZi5yEl+vbu78K74wGEbIVitssolbaS7fYZU3xKcKAKhB4+TD4g2zrYLt0smQnRkoxcBkh8Am0TaTccr1/6GCwZIeMIIGXUCZ2mIPsgDNthlmVEznL7YpQn/+PWUt9lc60aJzPpL4u+2PXLPbBVGob8CgBJAkBn6kES+LQrjnmMVGJmX6DXjW2h2bQeLc+6kZ3IN0fFZWXyvbRygCEWatXja3vgkgfIRPE8fZlrnJ91MmaK2ohTpm/J0XvT/qzJeHla5Rh+jxQkDx1haflyFgEXOYL1kx68WbJrVwG+fSkrlIABoVzLXCnx9ekAW3P0AMF6cwTSBd2ve1PNG8qrpbtKbzKLu4O+0cTsEb+V/6Dv3nu0pFVXGlztQlJW4AACjStzknFiUjnPIs1Q7ex9S+drUtszSolmXuD6S3b+Da2MbMyB0N3jiWd0O9RiHy4awv3Og5+i/UUP71HviGjJo/ec6S//f8e1h/4Kpm6VeE8OYbYfleh9gii5+xkVsoQOK7cDSD1gyeGQ2m4iFEp6ra/KN57VKXOYrbAvvmR/8yalqR8JheykC7abx+X6atXehgZAX1KCggkPsxJtoUFmrjZj3hJM7+bkc/6rhfM6rjf2pbHmb92CmXTNdXZOL/GGA5RYwZuG6v5T1/V3h8f6IGF327yeTok9zf+j4zNFPVipG+67eAj+VjxFWvyWA8gZ8Q0vedV8Y5wmD97lbo0gdO+F1ha4dASBwv2PMskqIErzhG7RPRpImQwJPviXn3WUh90DLB1wB3mFsuh6Gl2DZF73R0fh977PDCDu9e7Z6q/GHBKuN/ppIXUL3Yl0/Pwh7aOVZOnsME9Fp6vFUC13Z0ofbNZYh5RlK1rs3dBnjuxRKeZJB0MbDFEc8zQLV8/UGOSOXcrRBkluXjoMLSFTz/8HS9lDsfzvK9GUnAPvEdVSxYyTp/rczL/3ujz2RIFyDAxp9t0fYT774mAN2qp3emKaI+4RKSUksv7e/+dXwR+ZrkPxA/cwy7NjcjqQEzqbftcwmd9ZK76wl31UzonkpjfW3UmJgUaiQ/u8rUbJGmC/sH6mnad6AkKZFWhYFUOxCXmGjj+iKjg7xllPyPK7CmmdNZA78sownIzITVkfMdm0W/vzvNDcJnFehQI3nAv4sPiNUvfBUwmslUk7pWtxHznluYT4u2S0ADOdRi3D7awB+kmWwVw8/ILPYYpp6BaGCUjYJf3Z9nVfbpr9dUTq00L9tnDsZIJl0VTRoBGtOh364N+8iqhZ+414Ltb0A3Tler+0/cVJStVq2XYnEjyQ2iBLyoQbKZu2XX+8rfrTsrlwGhavy9cHGZPDdbIZWKuunFU/ELaM+aiBUgRHd75CFw9ficvVuxAQsVVpMy17/p7lmpdtbxBLMIyUwQZcGX2UL4qdOYC2JgbziFbPWViho7qw1TJ6ex+42E79pDIyMcqynmJCB29ML7gDj1BQklhRieKQAcAJLy5njS3NB+PCfLjxZKP+rnwYjcqi14voJc4EXO7GgtkxJztpw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: /uH0IEP2P9UOx897GSYCW6oLkPl2B4rVnEEllpfRggi9vqPqSghjJiMdgJxrM0lLT8HsHD8XUsa7SwjvSH0jCS47UZyNhO47ow+AUFVHDUdyV1+tu8XT6hBDn0mcznSMXV1Xq9R9Z+QumC3yo/pFtYLqFbXqYCOURtXk8RHuw2wQIgEHLa+8OQHiOrJt9JHgSUKqaIZpHWrkfGzDbxDS9yJ22wGxg2GrEYim0vLBFImDpZAR8XZc/GHByq/P7l74j7subUodetNSQlZRsNWApOt3cR6qEicsJMjrf8jG2KbfZoB1zY3/C1Bo6u/foHJgpT+b1mN3uV4Q8k+23C47DO1/odnAdB5sQMrpiQd3ADHRl2FQgq6PVzhQCwNHXL68KWjhagxqUHdtjm39dh1oXnr6kUqJu7xwxa4TIsPMTBtuSJW88j4Mrdj5AITr/xtlXnlHl89OXlr2Fp+nO0A2g4O4Zh273F5GsI3Wm27j2jdllpcMPyAUDx2zuH9x9UipIP8jXRYyKqmi2IU9X8V5bR5bkVLaVABZY78JHtKfK9Nb0uV0kcySbBAjf+fteoSQDop2CEHH7cziHA/xA4dU4H15CymMeND9DwyEe8Bbxoka1KiwbHJ/PSjH2v01Aq3J X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: e1afdfcc-5a0b-4ad1-ae91-08dced3ac5cc X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:46.3665 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aGvlClagIpW8DV6cDTbsWmKkkm3Ox4E7qQUCzVPnCamvb3hAX2x+ScVdvKSPHnxDRiAgeBQjSMzG49G6JXkpRw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011588-V-0D41zhNx9V X-MDID-O: eu1;ams;1729011588;V-0D41zhNx9V;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Remove the now-unused neighbour::next pointer, leaving struct neighbour solely with the hlist_node implementation. Signed-off-by: Gilad Naaman --- include/net/neighbour.h | 2 - net/core/neighbour.c | 128 ++++++++-------------------------------- 2 files changed, 24 insertions(+), 106 deletions(-) diff --git a/include/net/neighbour.h b/include/net/neighbour.h index c0c35a15d2ad..21c0c20a0ed5 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -135,7 +135,6 @@ struct neigh_statistics { #define NEIGH_CACHE_STAT_INC(tbl, field) this_cpu_inc((tbl)->stats->field) struct neighbour { - struct neighbour __rcu *next; struct hlist_node hash; struct neigh_table *tbl; struct neigh_parms *parms; @@ -191,7 +190,6 @@ struct pneigh_entry { #define NEIGH_NUM_HASH_RND 4 struct neigh_hash_table { - struct neighbour __rcu **hash_buckets; struct hlist_head *hash_heads; unsigned int hash_shift; __u32 hash_rnd[NEIGH_NUM_HASH_RND]; diff --git a/net/core/neighbour.c b/net/core/neighbour.c index cca524a55c97..61b5f0d4896a 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -205,49 +205,24 @@ static void neigh_update_flags(struct neighbour *neigh, u32 flags, int *notify, } } -static bool neigh_del(struct neighbour *n, struct neighbour __rcu **np, - struct neigh_table *tbl) -{ - bool retval = false; - - write_lock(&n->lock); - if (refcount_read(&n->refcnt) == 1) { - struct neighbour *neigh; - - neigh = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(*np, neigh); - hlist_del_rcu(&n->hash); - neigh_mark_dead(n); - retval = true; - } - write_unlock(&n->lock); - if (retval) - neigh_cleanup_and_release(n); - return retval; -} - bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) { struct neigh_hash_table *nht; - void *pkey = ndel->primary_key; - u32 hash_val; - struct neighbour *n; - struct neighbour __rcu **np; + bool retval = false; nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); - hash_val = tbl->hash(pkey, ndel->dev, nht->hash_rnd); - hash_val = hash_val >> (32 - nht->hash_shift); - np = &nht->hash_buckets[hash_val]; - while ((n = rcu_dereference_protected(*np, - lockdep_is_held(&tbl->lock)))) { - if (n == ndel) - return neigh_del(n, np, tbl); - np = &n->next; + write_lock(&ndel->lock); + if (refcount_read(&ndel->refcnt) == 1) { + hlist_del_rcu(&ndel->hash); + neigh_mark_dead(ndel); + retval = true; } - return false; + write_unlock(&ndel->lock); + if (retval) + neigh_cleanup_and_release(ndel); + return retval; } static int neigh_forced_gc(struct neigh_table *tbl) @@ -389,20 +364,13 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, for (i = 0; i < (1 << nht->hash_shift); i++) { struct neighbour *n; - struct neighbour __rcu **np = &nht->hash_buckets[i]; neigh_for_each(n, &nht->hash_heads[i]) { - if (dev && n->dev != dev) { - np = &n->next; + if (dev && n->dev != dev) continue; - } - if (skip_perm && n->nud_state & NUD_PERMANENT) { - np = &n->next; + if (skip_perm && n->nud_state & NUD_PERMANENT) continue; - } - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); + hlist_del_rcu(&n->hash); write_lock(&n->lock); neigh_del_timer(n); @@ -426,7 +394,6 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, n->nud_state = NUD_NONE; neigh_dbg(2, "neigh %p is stray\n", n); } - np = &n->next; write_unlock(&n->lock); neigh_cleanup_and_release(n); } @@ -532,39 +499,26 @@ static void neigh_get_hash_rnd(u32 *x) static struct neigh_hash_table *neigh_hash_alloc(unsigned int shift) { - size_t size = (1 << shift) * sizeof(struct neighbour *); - size_t hash_heads_size = (1 << shift) * sizeof(struct hlist_head); + size_t size = (1 << shift) * sizeof(struct hlist_head); struct neigh_hash_table *ret; struct hlist_head *hash_heads; - struct neighbour __rcu **buckets; int i; ret = kmalloc(sizeof(*ret), GFP_ATOMIC); if (!ret) return NULL; if (size <= PAGE_SIZE) { - buckets = kzalloc(size, GFP_ATOMIC); - hash_heads = kzalloc(hash_heads_size, GFP_ATOMIC); - if (!hash_heads) - kfree(buckets); + hash_heads = kzalloc(size, GFP_ATOMIC); } else { - buckets = (struct neighbour __rcu **) - __get_free_pages(GFP_ATOMIC | __GFP_ZERO, - get_order(size)); - kmemleak_alloc(buckets, size, 1, GFP_ATOMIC); - hash_heads = (struct hlist_head *) __get_free_pages(GFP_ATOMIC | __GFP_ZERO, - get_order(hash_heads_size)); - kmemleak_alloc(hash_heads, hash_heads_size, 1, GFP_ATOMIC); - if (!hash_heads) - free_pages((unsigned long)buckets, get_order(size)); + get_order(size)); + kmemleak_alloc(hash_heads, size, 1, GFP_ATOMIC); } - if (!buckets || !hash_heads) { + if (!hash_heads) { kfree(ret); return NULL; } - ret->hash_buckets = buckets; ret->hash_heads = hash_heads; ret->hash_shift = shift; for (i = 0; i < NEIGH_NUM_HASH_RND; i++) @@ -577,23 +531,14 @@ static void neigh_hash_free_rcu(struct rcu_head *head) struct neigh_hash_table *nht = container_of(head, struct neigh_hash_table, rcu); - size_t size = (1 << nht->hash_shift) * sizeof(struct neighbour *); - struct neighbour __rcu **buckets = nht->hash_buckets; - size_t hash_heads_size = (1 << nht->hash_shift) * sizeof(struct hlist_head); + size_t size = (1 << nht->hash_shift) * sizeof(struct hlist_head); struct hlist_head *hash_heads = nht->hash_heads; - if (size <= PAGE_SIZE) { - kfree(buckets); - } else { - kmemleak_free(buckets); - free_pages((unsigned long)buckets, get_order(size)); - } - - if (hash_heads_size < PAGE_SIZE) { + if (size < PAGE_SIZE) { kfree(hash_heads); } else { kmemleak_free(hash_heads); - free_pages((unsigned long)hash_heads, get_order(hash_heads_size)); + free_pages((unsigned long)hash_heads, get_order(size)); } kfree(nht); } @@ -613,7 +558,7 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, return old_nht; for (i = 0; i < (1 << old_nht->hash_shift); i++) { - struct neighbour *n, *next; + struct neighbour *n; struct hlist_node *tmp; neigh_for_each_safe(n, tmp, &old_nht->hash_heads[i]) { @@ -621,14 +566,7 @@ static struct neigh_hash_table *neigh_hash_grow(struct neigh_table *tbl, new_nht->hash_rnd); hash >>= (32 - new_nht->hash_shift); - next = rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock)); - rcu_assign_pointer(n->next, - rcu_dereference_protected( - new_nht->hash_buckets[hash], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(new_nht->hash_buckets[hash], n); hlist_del_rcu(&n->hash); hlist_add_head_rcu(&n->hash, &new_nht->hash_heads[hash]); } @@ -733,10 +671,6 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, list_add_tail(&n->managed_list, &n->tbl->managed_list); if (want_ref) neigh_hold(n); - rcu_assign_pointer(n->next, - rcu_dereference_protected(nht->hash_buckets[hash_val], - lockdep_is_held(&tbl->lock))); - rcu_assign_pointer(nht->hash_buckets[hash_val], n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); @@ -971,7 +905,6 @@ static void neigh_periodic_work(struct work_struct *work) struct neigh_table *tbl = container_of(work, struct neigh_table, gc_work.work); struct neighbour *n; struct hlist_node *tmp; - struct neighbour __rcu **np; unsigned int i; struct neigh_hash_table *nht; @@ -998,7 +931,6 @@ static void neigh_periodic_work(struct work_struct *work) goto out; for (i = 0 ; i < (1 << nht->hash_shift); i++) { - np = &nht->hash_buckets[i]; neigh_for_each_safe(n, tmp, &nht->hash_heads[i]) { unsigned int state; @@ -1009,7 +941,7 @@ static void neigh_periodic_work(struct work_struct *work) if ((state & (NUD_PERMANENT | NUD_IN_TIMER)) || (n->flags & NTF_EXT_LEARNED)) { write_unlock(&n->lock); - goto next_elt; + continue; } if (time_before(n->used, n->confirmed) && @@ -1020,9 +952,6 @@ static void neigh_periodic_work(struct work_struct *work) (state == NUD_FAILED || !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); write_unlock(&n->lock); @@ -1030,9 +959,6 @@ static void neigh_periodic_work(struct work_struct *work) continue; } write_unlock(&n->lock); - -next_elt: - np = &n->next; } /* * It's fine to release lock here, even if hash table @@ -3118,22 +3044,16 @@ void __neigh_for_each_release(struct neigh_table *tbl, for (chain = 0; chain < (1 << nht->hash_shift); chain++) { struct neighbour *n; struct hlist_node *tmp; - struct neighbour __rcu **np; - np = &nht->hash_buckets[chain]; neigh_for_each_safe(n, tmp, &nht->hash_heads[chain]) { int release; write_lock(&n->lock); release = cb(n); if (release) { - rcu_assign_pointer(*np, - rcu_dereference_protected(n->next, - lockdep_is_held(&tbl->lock))); hlist_del_rcu(&n->hash); neigh_mark_dead(n); - } else - np = &n->next; + } write_unlock(&n->lock); if (release) neigh_cleanup_and_release(n); From patchwork Tue Oct 15 16:59:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gilad Naaman X-Patchwork-Id: 13836751 X-Patchwork-Delegate: kuba@kernel.org Received: from dispatch1-eu1.ppe-hosted.com (dispatch1-eu1.ppe-hosted.com [185.183.29.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3433C1F4FBB for ; Tue, 15 Oct 2024 16:59:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=185.183.29.33 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011594; cv=fail; b=ti3qWvgeVOxZ5nUvjAbp7unu+RekEYIOTSt6aL6JP788dlxctKBSUomnUaBOvCClXzE3V7hR/QcTry00ecGMFZRXd+BexKEHCDedJo9jBF8Xt4r/iaVDYA7QqBoYlQVSb6gWiocZiaVQUUjyvVmx9zfT1xh9JFq02Ro9ARuxkG8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729011594; c=relaxed/simple; bh=07A2N/b28n/u59nTYk0xtg0N7IDeA+zlBQ0+dtfqAZU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=K0ca4rpm6rIcT/CB9NtrkGz4i0CCtSey5BQl4j6UVcDHf4nka9taMWaT1+NshJZFmeFrITbALTdV2euMNEEPyLINO9a0Q1963CYipqrD5BTZRy5V9lehXJ06pHl7ujQHQkbK5WM4LuZ/QpYlNPj5gXaUJUEzonN7kA9yAoL/IDQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com; spf=pass smtp.mailfrom=drivenets.com; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b=HU+dXS9H; arc=fail smtp.client-ip=185.183.29.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=drivenets.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=drivenets.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=drivenets.onmicrosoft.com header.i=@drivenets.onmicrosoft.com header.b="HU+dXS9H" X-Virus-Scanned: Proofpoint Essentials engine Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05lp2172.outbound.protection.outlook.com [104.47.17.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1-eu1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 9C5C434005A; Tue, 15 Oct 2024 16:59:48 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=r20CxdOnnCXtluUNo2xSs2ZHZbcYU3EdDfNr+LAtNBJsSQ0MSVHnkbcbEA7//R1ltWcPoFPQF73qJ/z5PaWioe+rSfgOjsZX54kphndlJEYlqLCGzo/E7oKaAlxq87f/fYO5GzPHM5kJ4xCJVwqt89YDNgDpr4rzK83kx6J4WsA+vpemwWeKHf2PGcinNC1F7dge3uojl1S+qKY0e+hCNGAgobYFbVc4xbNDZ6dgagUrL3u9qHphcSfTmv8kPekrxXnsijxuWdQkqP8jCwhJKC0ZP1LupnyZT4GQ65q1OkhVvMCrS4NEHQ+noqTbL6xkcp7fi7LcrTsD4Rag5Ev1Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4I5Uh7JSsfgHtT+ozN6AbXrFUZizsi7J7DIyroK6YQs=; b=mNqf+KBQfd7Cd8rsQhyDFMcEqvZ5DWMTcxrAeWKVxjQ0e951DSBq3OzrUGq4p70yhcVj8Zn9aGPhyez8xxGhXeSZ7EME9hAGcU2ddDmOFBQ6QlL3uMcMpxaY6uBpfkL9q4RO+qVd03xvDuVLxKgyRqEq+8fEbJ9oRv+ekiLyQsfu4C5I+FmCbtNgA9Sq0ZcS6cbSjYzvJBMBBYvnvFSBo67F1MPFPadQ/gnkvjZGUXFGdigIuysx9LMeM9bauD4VkhUk2S1BqHBmVHiq+g5JzVceQJtY6dJf1TLv2LXWlEYoj74hAbicscZAJziJQsnuBSUiscesmlvN4d/OcGu9fA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=drivenets.com; dmarc=pass action=none header.from=drivenets.com; dkim=pass header.d=drivenets.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=drivenets.onmicrosoft.com; s=selector2-drivenets-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4I5Uh7JSsfgHtT+ozN6AbXrFUZizsi7J7DIyroK6YQs=; b=HU+dXS9HNBAFv0P1C4NNnl8PClZ0D61ocvSzRzGi2W7173vcQ2+Oeo8AFcQM0kPn3J1zKcq6Wq7GpKgPu5pZombEJ2Jk6BHv5yHzBfLy19BwRlDgm/a/sBPd+vrktG0suKaafB3yKUMim18QglYqFzIeUSL+XRECuO/9JQjVLWE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=drivenets.com; Received: from DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) by AS8PR08MB8249.eurprd08.prod.outlook.com (2603:10a6:20b:53f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.26; Tue, 15 Oct 2024 16:59:47 +0000 Received: from DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e]) by DB8PR08MB5388.eurprd08.prod.outlook.com ([fe80::29dd:6773:4977:dc4e%5]) with mapi id 15.20.8069.016; Tue, 15 Oct 2024 16:59:47 +0000 From: Gilad Naaman To: netdev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Cc: Gilad Naaman , Kuniyuki Iwashima Subject: [PATCH net-next v4 6/6] Create netdev->neighbour association Date: Tue, 15 Oct 2024 16:59:26 +0000 Message-ID: <20241015165929.3203216-7-gnaaman@drivenets.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241015165929.3203216-1-gnaaman@drivenets.com> References: <20241015165929.3203216-1-gnaaman@drivenets.com> X-ClientProxiedBy: LO2P265CA0059.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:60::23) To DB8PR08MB5388.eurprd08.prod.outlook.com (2603:10a6:10:11c::7) Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB8PR08MB5388:EE_|AS8PR08MB8249:EE_ X-MS-Office365-Filtering-Correlation-Id: 014f4181-37ff-4ee2-115d-08dced3ac66e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|38350700014; X-Microsoft-Antispam-Message-Info: mQOYwzZEtaPYRMwWkMTM9EIxvE343Lb0Z0SMF1gZRDmeYk56x2uvFESupenhBJVB//B/g3+aLABVN2jApLVUYQut3vC1kYJ3YrzHNIsqdo216KGcDs9Z7nq1EnEajEFdXL1/QTfLFO0Rfjhfa2DchpBTYSjXJYMDXE0rQzWEkI/0o/DjjqIgs62G+M/zP9pKf7JKXc8nCXs0oOOWSRsocEAC2oifrXZ8LTNBF2uMsGrlIJzitgwmFxMZIdeLEKEo2DCBWCn8NALLl8OjfAXzt1+oXdy258x7cuzD6GDfFAtroZiUkV1xUH2As+EazNU5PEysyY7gOOF9UgWXjtuFV/noxhr9My6E43k5nq+Lr7U8wRs1nh/8LN3Bc3x6X5xp56VrxaD50ykBWyuoBBfrQgsdQZth935qUaL4hjZc0rtsOtmkeBxoG/alVth8TYJGbzk0ik6FGqxRgLCn3sF7Egn4YX+l3Eqq7ARo/hoTTjkq1VF+1FjPEebliERHj1VVy0wAOLA0tjFHPXfPUkj1v+wdUwKiCZxToJZ8eYscvtu4GSI3mlgKfPOvLumBx1U9LUPGpmkIj0HAWVVPl8dV43N8v/9MUUef6Z72bCRHNpytUp+wjM+IRWx1dDPgKTmd3pe8tp75a4aDQvvWMHdGfpWLyuh4eeqL2W/AoQcKnY5pDUiJ2Nn+Iohz5Is19CTf8ZxWF1K5vrC4oXV0fnXehHW5+tOz15eyapKrp6Ccdie2Ws0R97n2jgIhirZiRVeS9SnEHuHiC548YCdHG9vFqSyPDpZGe5MIrFD5zXZQt36obOagYymnTdzKDcvGI9ROHOWjARLrbrAciyyLhfdOHfY/mphyVlFCol73Mg4IvjJRBkeDgFBej1UQ2qvvEzwC6HVjrQ9uah70uAXOpndLs1D/8emiEfIB7QvOvSHxAN+myorpS6Dqh1jPz950y4c2be5W5TMrXb1DMyW3b1oDc2kFwSt7/nb0T3y0eV+VkK2lxGHolBRl6ysbN4PQ8ASczGbXibWUi775VQLqOBcqkXRPTf2+CWFjv1KS30Pj1tycpGhd7TI0JJ5VmllP6z3lZOgPSAXEN/OnnooTBAYdoOVvQzDjvE1PovX/KTeKmcgYlKFXzzQHhWfAbuOH487Mwrpr4APQhmHiBWRfQ7cVgEM6hITudN8PhLcZdm2Zdgm13Tubn8teCmxTjm3eZ2q+Xz4w7hCi8qBJTZZTwz7FYo3zFt9ipyogVuldF+UKMDDuBkejjADMRvRuSxqjauAr7wVJji9nLfaopUawCN1rET9o83dIjT1eDhyjwlWSqYxP2w2wnaz6Yt2IEUgLftL5C2D6BhyxXtw8C+qyl2Z7Xw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB8PR08MB5388.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bQZ8Mc+GBzH2xKrrzTrdJERJ2s7Mc5wNCl6oAkpQuqaHQUro858HnbphyeDUe3GQ8b6ea+WNTh6cxfFj1+hp/rPFGed3hR6CxZg3VIsKgmCeoWJe1+JDXgjuhOV71SMTT1U+xCfF3fH3Pndct/l4Oj0XTHsVTp5VTKegLVcGzQZlGLqcQ+kFuJrD2Jgo74L79iiobLHsmPdzuRiiiMaSogS8bNI0vepozWGa3s4Ih/MXfFWXy9NexMfas7iSFE+Cob2LWnG+z4HamiyfAlUvUVaoZ1H4oI9e+ITS2qXrpZMjm0En6MINz4ymXqJ+aHa1scy6eUShRx0/l45Bo5btBHJR7AOettYkpWCdNZhRRv2NRn+lkUv93Rd8gnvhbeosiy697lAxE3Bn2uVi9pUEuNMOR93Bu2oIVmCmSu3yWYirFnuwH0wvDTZUE8oRk4UFAMFdgkaC+QNEYgMc0znwMeTt1e6gK1AP+18UkHJM9nMWRI7JawyXDIC9C9wcUWdGJwZ4uq/DPud3kpRyOaSJNBN/ldEDXksVssZ3aZZC23YwtBO8yOqLmhc2odfB5kV7LtM2gJspM0bna0YXFDTtUReFti6uu1ZEwmaZOEofSTLRnNl69bann0OxNHnxOpvQKlbNp4L/22d6tE3mUigSrLq2W6Vp0pfg5/TDBFV8dGIkM0wUe6wvqlGf/xJU4A4apMjQchUKyVRmOdMC9b+QOMQf0RhAkECT6aZRVGP+53RFzmZYz+VAgyG4wil5eTqAn/F7OvmOxIXWCKyTTSWyi/kpKqtZsE3iQ5MCLYK/bhJqGEYIZjJFLA54zbgfmlmZYsq7rh6g5+tXJzONFMPku+krqqiVjN5bV4gZI1Hb/UE9p0R4VLVelRFaOfb87gC9o1PHwlIRNJQDzJ9wUD71D0W5E6WPI8hNkqWXWyiAVHRxU8r6fhDArNA+75XTCI3mXk/HCaALSAGiwjVA06A7uHbuayYhEcVAbOKz+W4bSWyw76fa4OvTFwBpw4vrPC8bKfDhwPQrB/pu4WNBQpKMwIgD8zBsElLQgUtN1nJJ3FsfUn9w+rXwvuAS0IwAymdrWVT1NJZqc3UnWk2WDXHyTGkDJ11HCaqeMIerZs65B9m+hbCJiHYYnth6NRWv/WzgLfcAVJ+j+W27UuLXKMhUTuQ72Y0AupbRVgt2fKUrhFCUunIIiSgtUpHHcIHQ1cAxlr7Lt88zhqep/sBz0dKFuzelu0i3jlan9+tiD2dhcyvobmOzvTgY3QgS+ncK8uS/2utqvFgggJwbjBxu3iaP/RirBvhVNRnc9L4NzgpnhSP/CvYt2VXIYJ45MZ8/VDv1Ajd3n6dXx0x3eW4O6lhZFljWp4AHG+b7DRH1dTTN+8Ue5CBdHXxd5ze6XOrVJ0e25ziUOLLUN6lEl9jDNW6wSFCPfL5NH/4b5ro0RnKEJUV1G/v1HQ7KjICJRoycOzNcaOAAwzPjrJkDkGfW7Ne1tTB0rfMTjR+X6QTzwGqXFgSN/h1uGQ41E5uarlBbNzXZOi3A0f2IGcFobNmcoeuZ/nAtKW3+xNFc6jdGYpYLMJvVGbTXPhRQ7VPcP+25F8iOP/sqXLdEBcnNYX/onN5zkw== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: v3AC8znCDi+eDinrhwY8Sw06mB6sjAFM3LlBehhyfqjDqBCtMnpGRF1TGWh+ba3NqrYvlN+MRf8du4WsyIy8nVVa1tsGQgDG9lgnemPzLHrmzU/RnM/AXCN/PqwZMBzGSsI+Oy9B69RbVw0erW2Rb9vqmQrN7TzxQzgh9nxfI04D3yeoRy6Su9Zh3uxQxl3RiX1nWXvIrFihVF4IMgN0bUc/9KHQViuj6fXhh3BugG4B3Jm+02bwg8rT33etgEksP5Po0WeLIi7ZTr/zy4r/RJ9zgLAU2q0T12qsQApNBZx5ZSEPseANZgF3BdV5gWcTKPvNp7l0XMm7nxX7rXRtSHH81SS4PcW3baYkH8e/Ai+YkK2woGr2hXThqoud6OJOgyaRTPz6NGiFEGyzoVDksTDYbrXN9lc0kGVYZSd7G0Jo2y2Hv8uaPEjbwBBO5FPE0IpDucuJ8gPP8YCYGEt7Do8yzNsnjJuN/EBrDYGFbvOLRJiNYRJ1yHi7mHVoFmcIS9Gk9vgc3kLZkuU5PUZBFSyV4pVSXjkoG1BiaZqMLqXer0fRSfOEp82Y7QfBW+YyzPQMntlR6XW2NatNec0id/SHqMzDLpP49caR/BcghP5vDQfhnaITqHBiEnmwoHzy X-OriginatorOrg: drivenets.com X-MS-Exchange-CrossTenant-Network-Message-Id: 014f4181-37ff-4ee2-115d-08dced3ac66e X-MS-Exchange-CrossTenant-AuthSource: DB8PR08MB5388.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 16:59:47.4057 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 662f82da-cf45-4bdf-b295-33b083f5d229 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j74C+QPI40cJwc8O2mX+1dJViXpDqiBC44NwDND8o1a8P2SGZnnN+Pa12coguUlvnqIco1RrjbB07dDiOUkypQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8249 X-MDID: 1729011589-EiczDBLQw0FE X-MDID-O: eu1;ams;1729011589;EiczDBLQw0FE;;495c1e7a27a6c3e35a5fabc922783896 X-PPE-TRUSTED: V=1;DIR=OUT; X-Patchwork-Delegate: kuba@kernel.org Create a mapping between a netdev and its neighoburs, allowing for much cheaper flushes. Signed-off-by: Gilad Naaman --- .../networking/net_cachelines/net_device.rst | 1 + include/linux/netdevice.h | 7 ++ include/net/neighbour.h | 10 +-- include/net/neighbour_tables.h | 12 +++ net/core/neighbour.c | 85 ++++++++++++------- net/mpls/af_mpls.c | 2 +- 6 files changed, 75 insertions(+), 42 deletions(-) create mode 100644 include/net/neighbour_tables.h diff --git a/Documentation/networking/net_cachelines/net_device.rst b/Documentation/networking/net_cachelines/net_device.rst index db6192b2bb50..2edb6ac1cab4 100644 --- a/Documentation/networking/net_cachelines/net_device.rst +++ b/Documentation/networking/net_cachelines/net_device.rst @@ -189,4 +189,5 @@ u64 max_pacing_offload_horizon struct_napi_config* napi_config unsigned_long gro_flush_timeout u32 napi_defer_hard_irqs +struct hlist_head neighbours[2] =================================== =========================== =================== =================== =================================================================================== diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 8feaca12655e..80bde95cc302 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -52,6 +52,7 @@ #include #include #include +#include struct netpoll_info; struct device; @@ -2034,6 +2035,9 @@ enum netdev_reg_state { * @napi_defer_hard_irqs: If not zero, provides a counter that would * allow to avoid NIC hard IRQ, on busy queues. * + * @neighbours: List heads pointing to this device's neighbours' + * dev_list, one per address-family. + * * FIXME: cleanup struct net_device such that network protocol info * moves out. */ @@ -2443,6 +2447,9 @@ struct net_device { */ struct net_shaper_hierarchy *net_shaper_hierarchy; #endif + + struct hlist_head neighbours[NEIGH_NR_TABLES]; + u8 priv[] ____cacheline_aligned __counted_by(priv_len); } ____cacheline_aligned; diff --git a/include/net/neighbour.h b/include/net/neighbour.h index 21c0c20a0ed5..d0bc618d4fe2 100644 --- a/include/net/neighbour.h +++ b/include/net/neighbour.h @@ -29,6 +29,7 @@ #include #include #include +#include /* * NUD stands for "neighbor unreachability detection" @@ -136,6 +137,7 @@ struct neigh_statistics { struct neighbour { struct hlist_node hash; + struct hlist_node dev_list; struct neigh_table *tbl; struct neigh_parms *parms; unsigned long confirmed; @@ -236,14 +238,6 @@ struct neigh_table { struct pneigh_entry **phash_buckets; }; -enum { - NEIGH_ARP_TABLE = 0, - NEIGH_ND_TABLE = 1, - NEIGH_DN_TABLE = 2, - NEIGH_NR_TABLES, - NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ -}; - static inline int neigh_parms_family(struct neigh_parms *p) { return p->tbl->family; diff --git a/include/net/neighbour_tables.h b/include/net/neighbour_tables.h new file mode 100644 index 000000000000..bcffbe8f7601 --- /dev/null +++ b/include/net/neighbour_tables.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _NET_NEIGHBOUR_TABLES_H +#define _NET_NEIGHBOUR_TABLES_H + +enum { + NEIGH_ARP_TABLE = 0, + NEIGH_ND_TABLE = 1, + NEIGH_NR_TABLES, + NEIGH_LINK_TABLE = NEIGH_NR_TABLES /* Pseudo table for neigh_xmit */ +}; + +#endif diff --git a/net/core/neighbour.c b/net/core/neighbour.c index 61b5f0d4896a..dbfd27f79bb8 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -61,6 +61,19 @@ static int pneigh_ifdown_and_unlock(struct neigh_table *tbl, static const struct seq_operations neigh_stat_seq_ops; #endif +static int family_to_neightbl_index(int family) +{ + switch (family) { + case AF_INET: + return NEIGH_ARP_TABLE; + case AF_INET6: + return NEIGH_ND_TABLE; + default: + DEBUG_NET_WARN_ON_ONCE(1); + return 0; /* to avoid panic by null-ptr-deref */ + } +} + /* Neighbour hash table buckets are protected with rwlock tbl->lock. @@ -216,6 +229,7 @@ bool neigh_remove_one(struct neighbour *ndel, struct neigh_table *tbl) write_lock(&ndel->lock); if (refcount_read(&ndel->refcnt) == 1) { hlist_del_rcu(&ndel->hash); + hlist_del_rcu(&ndel->dev_list); neigh_mark_dead(ndel); retval = true; } @@ -357,46 +371,45 @@ static void neigh_flush_dev(struct neigh_table *tbl, struct net_device *dev, bool skip_perm) { int i; + struct neighbour *n; + struct hlist_node *tmp; struct neigh_hash_table *nht; + i = family_to_neightbl_index(tbl->family); + nht = rcu_dereference_protected(tbl->nht, lockdep_is_held(&tbl->lock)); - for (i = 0; i < (1 << nht->hash_shift); i++) { - struct neighbour *n; - - neigh_for_each(n, &nht->hash_heads[i]) { - if (dev && n->dev != dev) - continue; - if (skip_perm && n->nud_state & NUD_PERMANENT) - continue; + hlist_for_each_entry_safe(n, tmp, &dev->neighbours[i], dev_list) { + if (skip_perm && n->nud_state & NUD_PERMANENT) + continue; - hlist_del_rcu(&n->hash); - write_lock(&n->lock); - neigh_del_timer(n); - neigh_mark_dead(n); - if (refcount_read(&n->refcnt) != 1) { - /* The most unpleasant situation. - We must destroy neighbour entry, - but someone still uses it. - - The destroy will be delayed until - the last user releases us, but - we must kill timers etc. and move - it to safe state. - */ - __skb_queue_purge(&n->arp_queue); - n->arp_queue_len_bytes = 0; - WRITE_ONCE(n->output, neigh_blackhole); - if (n->nud_state & NUD_VALID) - n->nud_state = NUD_NOARP; - else - n->nud_state = NUD_NONE; - neigh_dbg(2, "neigh %p is stray\n", n); - } - write_unlock(&n->lock); - neigh_cleanup_and_release(n); + hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); + write_lock(&n->lock); + neigh_del_timer(n); + neigh_mark_dead(n); + if (refcount_read(&n->refcnt) != 1) { + /* The most unpleasant situation. + * We must destroy neighbour entry, + * but someone still uses it. + * + * The destroy will be delayed until + * the last user releases us, but + * we must kill timers etc. and move + * it to safe state. + */ + __skb_queue_purge(&n->arp_queue); + n->arp_queue_len_bytes = 0; + WRITE_ONCE(n->output, neigh_blackhole); + if (n->nud_state & NUD_VALID) + n->nud_state = NUD_NOARP; + else + n->nud_state = NUD_NONE; + neigh_dbg(2, "neigh %p is stray\n", n); } + write_unlock(&n->lock); + neigh_cleanup_and_release(n); } } @@ -672,6 +685,10 @@ ___neigh_create(struct neigh_table *tbl, const void *pkey, if (want_ref) neigh_hold(n); hlist_add_head_rcu(&n->hash, &nht->hash_heads[hash_val]); + + error = family_to_neightbl_index(tbl->family); + hlist_add_head_rcu(&n->dev_list, &dev->neighbours[error]); + write_unlock_bh(&tbl->lock); neigh_dbg(2, "neigh %p is created\n", n); rc = n; @@ -953,6 +970,7 @@ static void neigh_periodic_work(struct work_struct *work) !time_in_range_open(jiffies, n->used, n->used + NEIGH_VAR(n->parms, GC_STALETIME)))) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); write_unlock(&n->lock); neigh_cleanup_and_release(n); @@ -3052,6 +3070,7 @@ void __neigh_for_each_release(struct neigh_table *tbl, release = cb(n); if (release) { hlist_del_rcu(&n->hash); + hlist_del_rcu(&n->dev_list); neigh_mark_dead(n); } write_unlock(&n->lock); diff --git a/net/mpls/af_mpls.c b/net/mpls/af_mpls.c index df62638b6498..a0573847bc55 100644 --- a/net/mpls/af_mpls.c +++ b/net/mpls/af_mpls.c @@ -1664,7 +1664,7 @@ static int nla_put_via(struct sk_buff *skb, u8 table, const void *addr, int alen) { static const int table_to_family[NEIGH_NR_TABLES + 1] = { - AF_INET, AF_INET6, AF_DECnet, AF_PACKET, + AF_INET, AF_INET6, AF_PACKET, }; struct nlattr *nla; struct rtvia *via;