| Message ID | 20230809140234.3879929-3-liuhangbin@gmail.com (mailing list archive) |
|---|---|
| State | Deferred |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | send notify when delete source address routes | expand |
| Context | Check | Description |
|---|---|---|
| netdev/series_format | success | Posting correctly formatted |
| netdev/tree_selection | success | Clearly marked for net-next |
| netdev/fixes_present | success | Fixes tag not required for -next series |
| netdev/header_inline | success | No static functions without inline keyword in header files |
| netdev/build_32bit | success | Errors and warnings before: 3131 this patch: 3131 |
| netdev/cc_maintainers | success | CCed 6 of 6 maintainers |
| netdev/build_clang | success | Errors and warnings before: 1575 this patch: 1575 |
| netdev/verify_signedoff | success | Signed-off-by tag matches author and committer |
| netdev/deprecated_api | success | None detected |
| netdev/check_selftest | success | No net selftest shell script |
| netdev/verify_fixes | success | No Fixes tag |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 3294 this patch: 3294 |
| netdev/checkpatch | warning | WARNING: line length of 88 exceeds 80 columns |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/source_inline | success | Was 0 now: 0 |
On Wed, Aug 09, 2023 at 10:02:34PM +0800, Hangbin Liu wrote: > After deleting an interface address in fib_del_ifaddr(), the function > scans the fib_info list for stray entries and calls fib_flush() and > fib_table_flush(). Then the stray entries will be deleted silently and no > RTM_DELROUTE notification will be sent. > > This lack of notification can make routing daemons like NetworkManager, > miss the routing changes. e.g. [...] > To fix this issue, let's add a new bit in "struct fib_info" to track the > deleted prefer source address routes, and only send notify for them. In the other thread Thomas mentioned that NM already requests a route dump following address deletion [1]. If so, can Thomas or you please explain how this patch is going to help NM? Is the intention to optimize things and avoid the dump request (which can only work on new kernels)? [1] https://lore.kernel.org/netdev/07fcfd504148b3c721fda716ad0a549662708407.camel@redhat.com/
On Thu, Aug 10, 2023 at 06:08:28PM +0300, Ido Schimmel wrote: > On Wed, Aug 09, 2023 at 10:02:34PM +0800, Hangbin Liu wrote: > > After deleting an interface address in fib_del_ifaddr(), the function > > scans the fib_info list for stray entries and calls fib_flush() and > > fib_table_flush(). Then the stray entries will be deleted silently and no > > RTM_DELROUTE notification will be sent. > > > > This lack of notification can make routing daemons like NetworkManager, > > miss the routing changes. e.g. > > [...] > > > To fix this issue, let's add a new bit in "struct fib_info" to track the > > deleted prefer source address routes, and only send notify for them. > > In the other thread Thomas mentioned that NM already requests a route > dump following address deletion [1]. If so, can Thomas or you please > explain how this patch is going to help NM? Is the intention to optimize > things and avoid the dump request (which can only work on new kernels)? > > [1] https://lore.kernel.org/netdev/07fcfd504148b3c721fda716ad0a549662708407.camel@redhat.com/ In my understanding, After deleting an address, deal with the delete notify is more efficient to maintain the route cache than dump all the routes. Hi Thomas ,do you have any comments? Thanks Hangbin
Hi Thomas, Any comments? Thanks Hangbin On Tue, Aug 15, 2023 at 08:55:35PM +0800, Hangbin Liu wrote: > On Thu, Aug 10, 2023 at 06:08:28PM +0300, Ido Schimmel wrote: > > On Wed, Aug 09, 2023 at 10:02:34PM +0800, Hangbin Liu wrote: > > > After deleting an interface address in fib_del_ifaddr(), the function > > > scans the fib_info list for stray entries and calls fib_flush() and > > > fib_table_flush(). Then the stray entries will be deleted silently and no > > > RTM_DELROUTE notification will be sent. > > > > > > This lack of notification can make routing daemons like NetworkManager, > > > miss the routing changes. e.g. > > > > [...] > > > > > To fix this issue, let's add a new bit in "struct fib_info" to track the > > > deleted prefer source address routes, and only send notify for them. > > > > In the other thread Thomas mentioned that NM already requests a route > > dump following address deletion [1]. If so, can Thomas or you please > > explain how this patch is going to help NM? Is the intention to optimize > > things and avoid the dump request (which can only work on new kernels)? > > > > [1] https://lore.kernel.org/netdev/07fcfd504148b3c721fda716ad0a549662708407.camel@redhat.com/ > > In my understanding, After deleting an address, deal with the delete notify is > more efficient to maintain the route cache than dump all the routes. > > Hi Thomas ,do you have any comments? > > Thanks > Hangbin
On Mon, 2023-08-28 at 14:14 +0800, Hangbin Liu wrote: > > > > > > In the other thread Thomas mentioned that NM already requests a > > > route > > > dump following address deletion [1]. If so, can Thomas or you > > > please > > > explain how this patch is going to help NM? Is the intention to > > > optimize > > > things and avoid the dump request (which can only work on new > > > kernels)? > > > > > > [1] > > > https://lore.kernel.org/netdev/07fcfd504148b3c721fda716ad0a549662708407.camel@redhat.com/ > > > > In my understanding, After deleting an address, deal with the > > delete notify is > > more efficient to maintain the route cache than dump all the > > routes. Hi, NetworkManager does so out of necessity, as there is no notification. Overall, it seems a pretty bad thing to do, because it's expensive if you have many routes/addresses. Unfortunately, it's hard to ever drop a workaround, because we never know when the workaround can be dropped. Also, it's simply a notification missing, and not tied to NetworkManager or to maintaining a cache. If you run `ip route monitor`, you also don't see the notification that kernel drops a route? The effort that NetworkManager takes to maintain correct information is not something that most programs would do. Thomas
Hi Ido, Do you think if I should modify the patch description and re-post it? Thanks Hangbin On Mon, Sep 11, 2023 at 11:35:05AM +0200, Thomas Haller wrote: > On Mon, 2023-08-28 at 14:14 +0800, Hangbin Liu wrote: > > > > > > > > In the other thread Thomas mentioned that NM already requests a > > > > route > > > > dump following address deletion [1]. If so, can Thomas or you > > > > please > > > > explain how this patch is going to help NM? Is the intention to > > > > optimize > > > > things and avoid the dump request (which can only work on new > > > > kernels)? > > > > > > > > [1] > > > > https://lore.kernel.org/netdev/07fcfd504148b3c721fda716ad0a549662708407.camel@redhat.com/ > > > > > > In my understanding, After deleting an address, deal with the > > > delete notify is > > > more efficient to maintain the route cache than dump all the > > > routes. > > > Hi, > > > NetworkManager does so out of necessity, as there is no notification. > Overall, it seems a pretty bad thing to do, because it's expensive if > you have many routes/addresses. > > Unfortunately, it's hard to ever drop a workaround, because we never > know when the workaround can be dropped. > > Also, it's simply a notification missing, and not tied to > NetworkManager or to maintaining a cache. If you run `ip route > monitor`, you also don't see the notification that kernel drops a > route? The effort that NetworkManager takes to maintain correct > information is not something that most programs would do. > > > Thomas >
Le 12/09/2023 à 04:30, Hangbin Liu a écrit : > Hi Ido, > > Do you think if I should modify the patch description and re-post it? At least, this patch seems to be targeted for net, not for net-next. Regards, Nicolas
diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h index a91f8a28689a..12dedfa4c9fd 100644 --- a/include/net/ip_fib.h +++ b/include/net/ip_fib.h @@ -153,7 +153,8 @@ struct fib_info { #define fib_advmss fib_metrics->metrics[RTAX_ADVMSS-1] int fib_nhs; u8 fib_nh_is_v6:1, - nh_updated:1; + nh_updated:1, + pfsrc_removed:1; struct nexthop *nh; struct rcu_head rcu; struct fib_nh fib_nh[]; diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index ce1c10e408cf..6cd919b442a7 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -1884,6 +1884,7 @@ int fib_sync_down_addr(struct net_device *dev, __be32 local) continue; if (fi->fib_prefsrc == local) { fi->fib_flags |= RTNH_F_DEAD; + fi->pfsrc_removed = 1; ret++; } } diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c index 74d403dbd2b4..9cadeeaaa93a 100644 --- a/net/ipv4/fib_trie.c +++ b/net/ipv4/fib_trie.c @@ -2026,6 +2026,7 @@ void fib_table_flush_external(struct fib_table *tb) int fib_table_flush(struct net *net, struct fib_table *tb, bool flush_all) { struct trie *t = (struct trie *)tb->tb_data; + struct nl_info info = { .nl_net = net }; struct key_vector *pn = t->kv; unsigned long cindex = 1; struct hlist_node *tmp; @@ -2088,6 +2089,9 @@ int fib_table_flush(struct net *net, struct fib_table *tb, bool flush_all) fib_notify_alias_delete(net, n->key, &n->leaf, fa, NULL); + if (fi->pfsrc_removed) + rtmsg_fib(RTM_DELROUTE, htonl(n->key), fa, + KEYLENGTH - fa->fa_slen, tb->tb_id, &info, 0); hlist_del_rcu(&fa->fa_list); fib_release_info(fa->fa_info); alias_free_mem_rcu(fa);
After deleting an interface address in fib_del_ifaddr(), the function scans the fib_info list for stray entries and calls fib_flush() and fib_table_flush(). Then the stray entries will be deleted silently and no RTM_DELROUTE notification will be sent. This lack of notification can make routing daemons like NetworkManager, miss the routing changes. e.g. + ip link add dummy1 type dummy + ip link add dummy2 type dummy + ip link set dummy1 up + ip link set dummy2 up + ip addr add 192.168.5.5/24 dev dummy1 + ip route add 7.7.7.0/24 dev dummy2 src 192.168.5.5 + ip -4 route 7.7.7.0/24 dev dummy2 scope link src 192.168.5.5 192.168.5.0/24 dev dummy1 proto kernel scope link src 192.168.5.5 + ip monitor route + ip addr del 192.168.5.5/24 dev dummy1 Deleted 192.168.5.0/24 dev dummy1 proto kernel scope link src 192.168.5.5 Deleted broadcast 192.168.5.255 dev dummy1 table local proto kernel scope link src 192.168.5.5 Deleted local 192.168.5.5 dev dummy1 table local proto kernel scope host src 192.168.5.5 As Ido reminded, fib_table_flush() isn't only called when an address is deleted, but also when an interface is deleted or put down. The lack of notification in these cases is deliberate. And commit 7c6bb7d2faaf ("net/ipv6: Add knob to skip DELROUTE message on device down") introduced a sysctl to make IPv6 behave like IPv4 in this regard. So we can't send the route delete notify blindly in fib_table_flush(). To fix this issue, let's add a new bit in "struct fib_info" to track the deleted prefer source address routes, and only send notify for them. After update: + ip monitor route + ip addr del 192.168.5.5/24 dev dummy1 Deleted 192.168.5.0/24 dev dummy1 proto kernel scope link src 192.168.5.5 Deleted broadcast 192.168.5.255 dev dummy1 table local proto kernel scope link src 192.168.5.5 Deleted local 192.168.5.5 dev dummy1 table local proto kernel scope host src 192.168.5.5 Deleted 7.7.7.0/24 dev dummy2 scope link src 192.168.5.5 Suggested-by: Thomas Haller <thaller@redhat.com> Signed-off-by: Hangbin Liu <liuhangbin@gmail.com> --- v2: Add a bit in fib_info to mark the deleted src route. --- include/net/ip_fib.h | 3 ++- net/ipv4/fib_semantics.c | 1 + net/ipv4/fib_trie.c | 4 ++++ 3 files changed, 7 insertions(+), 1 deletion(-)