| Message ID | 20210920153454.433252-1-atenart@kernel.org (mailing list archive) |
|---|---|
| State | Rejected |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next] openvswitch: allow linking a VRF to an OVS bridge | expand |
| Context | Check | Description |
|---|---|---|
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Clearly marked for net-next |
| netdev/subject_prefix | success | Link |
| netdev/cc_maintainers | success | CCed 5 of 5 maintainers |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 24 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
| netdev/header_inline | success | Link |
On 9/20/21 9:34 AM, Antoine Tenart wrote: > There might be questions about the setup in which a VRF is linked to an > OVS bridge; I cc'ed Luis Tomás who wrote the article. My head just exploded. You want to make an L3 device a port of an L2 device. Can someone explain how this is supposed to work and why it is even needed? ie., given how an OVS bridge handles packets and the point of a VRF device (to direct lookups to a table), the 2 are at odds in my head.
On Mon, Sep 20, 2021 at 5:45 PM David Ahern <dsahern@gmail.com> wrote: > > On 9/20/21 9:34 AM, Antoine Tenart wrote: > > There might be questions about the setup in which a VRF is linked to an > > OVS bridge; I cc'ed Luis Tomás who wrote the article. > > My head just exploded. You want to make an L3 device a port of an L2 > device. > > Can someone explain how this is supposed to work and why it is even > needed? ie., given how an OVS bridge handles packets and the point of a > VRF device (to direct lookups to a table), the 2 are at odds in my head. > Hi David, Thanks for your comment. And yes you are right, this probably is a bit of an odd setup. That said, OVS is not pure L2 as it knows about IPs and it is doing virtual routing too (we can say it is 2.5 xD) What we want to achieve is something similar to what is shown in slide 100 here http://schd.ws/hosted_files/ossna2017/fe/vrf-tutorial-oss.pdf, but instead of connecting the VRF bridge directly to containers, we have a single ovs bridge (where the OpenStack VMs are connected to) where we connect the vrfs in different (ovs) ports (so that the traffic in the way out of OVS can be redirected to the right VRF). The initial part is pretty much the same as in the slide 100: 1) creating the vrf - ip link add vrf-1001 type vrf table 1001 2) vxlan device, in our case associated to the loopback, for ECMP (instead of associate both nics/vlan devices to the VRF) - ip link add vxlan-1001 type vxlan id 1001 dstport 4789 local L_IP nolearning 3) create the linux bridge device - ip link add name br-1001 type bridge stp_state 0 4) link the 3 above - ip link set br-1001 master vrf-1001 (bridge to vrf) - ip link set vxlan-1001 master br-1001 (vxlan to bridge) Then, I'm attaching the vrf device also as an ovs bridge port, so that traffic (together with some ip routes in the vrf routing table) can be redirected to OVS, and the (OpenStack) virtual networking happens there (br-ex is the ovs bridge) - ovs-vsctl add-port br-ex vrf-1001 - ip route show vrf vrf-1001 10.0.0.0/26 via 172.24.4.146 dev br-ex (redirect traffic to OpenStack subnet 10.0.0.0/26 to br-ex) 172.24.4.146 dev br-ex scope link Perhaps there is a better way of connecting this? I tried (without success) to create a veth device and set one end on the linux bridge and the other on the OVS bridge. Best regards, Luis
On Tue, Sep 21, 2021 at 10:12 AM Luis Tomas Bolivar <ltomasbo@redhat.com> wrote: > > On Mon, Sep 20, 2021 at 5:45 PM David Ahern <dsahern@gmail.com> wrote: > > > > On 9/20/21 9:34 AM, Antoine Tenart wrote: > > > There might be questions about the setup in which a VRF is linked to an > > > OVS bridge; I cc'ed Luis Tomás who wrote the article. > > > > My head just exploded. You want to make an L3 device a port of an L2 > > device. > > > > Can someone explain how this is supposed to work and why it is even > > needed? ie., given how an OVS bridge handles packets and the point of a > > VRF device (to direct lookups to a table), the 2 are at odds in my head. > > > > Hi David, > > Thanks for your comment. And yes you are right, this probably is a bit > of an odd setup. That said, OVS is not pure L2 as it knows about IPs > and it is doing virtual routing too (we can say it is 2.5 xD) > > What we want to achieve is something similar to what is shown in slide 100 > here http://schd.ws/hosted_files/ossna2017/fe/vrf-tutorial-oss.pdf, but instead > of connecting the VRF bridge directly to containers, we have a single ovs > bridge (where the OpenStack VMs are connected to) where we connect the > vrfs in different (ovs) ports (so that the traffic in the way out of OVS can be > redirected to the right VRF). > > The initial part is pretty much the same as in the slide 100: > 1) creating the vrf > - ip link add vrf-1001 type vrf table 1001 > 2) vxlan device, in our case associated to the loopback, > for ECMP (instead of associate both nics/vlan devices to the VRF) > - ip link add vxlan-1001 type vxlan id 1001 dstport 4789 local L_IP > nolearning > 3) create the linux bridge device > - ip link add name br-1001 type bridge stp_state 0 > 4) link the 3 above > - ip link set br-1001 master vrf-1001 (bridge to vrf) > - ip link set vxlan-1001 master br-1001 (vxlan to bridge) > > Then, I'm attaching the vrf device also as an ovs bridge port, so that > traffic (together with some ip routes in the vrf routing table) can be > redirected > to OVS, and the (OpenStack) virtual networking happens there > (br-ex is the ovs bridge) > - ovs-vsctl add-port br-ex vrf-1001 > - ip route show vrf vrf-1001 > 10.0.0.0/26 via 172.24.4.146 dev br-ex > (redirect traffic to OpenStack subnet 10.0.0.0/26 to br-ex) > 172.24.4.146 dev br-ex scope link > > Perhaps there is a better way of connecting this? > I tried (without success) to create a veth device and set one end on > the linux bridge and the other on the OVS bridge. Follow up on this. I found the mistake I was making on the veth-pair addition configuration (ovs flow was setting the wrong mac address before sending the traffic through the veth device to the vrf). And it indeed works connecting the VRF to the OVS bridge by using a veth pair instead of directly plugin the VRF device as an OVS port. > > Best regards, > Luis
Hello, Quoting Luis Tomas Bolivar (2021-09-21 13:20:08) > > Follow up on this. I found the mistake I was making on the veth-pair > addition configuration (ovs flow was setting the wrong mac address > before sending the traffic through the veth device to the vrf). And it > indeed works connecting the VRF to the OVS bridge by using a veth pair > instead of directly plugin the VRF device as an OVS port. Great! This means there is no need for this patch I believe, OVS bridge is not different in the end. Thanks, Antoine
diff --git a/net/openvswitch/vport-netdev.c b/net/openvswitch/vport-netdev.c index 8e1a88f13622..e76b2477d384 100644 --- a/net/openvswitch/vport-netdev.c +++ b/net/openvswitch/vport-netdev.c @@ -75,6 +75,7 @@ static struct net_device *get_dpdev(const struct datapath *dp) struct vport *ovs_netdev_link(struct vport *vport, const char *name) { + unsigned int saved_flags; int err; vport->dev = dev_get_by_name(ovs_dp_get_net(vport->dp), name); @@ -98,8 +99,17 @@ struct vport *ovs_netdev_link(struct vport *vport, const char *name) if (err) goto error_unlock; + /* While IFF_NO_RX_HANDLER is rightly set for l3 masters (VRF) as they + * don't work with upper devices, they can be attached to OVS bridges. + */ + saved_flags = vport->dev->priv_flags; + if (netif_is_l3_master(vport->dev)) + vport->dev->priv_flags &= ~IFF_NO_RX_HANDLER; + err = netdev_rx_handler_register(vport->dev, netdev_frame_hook, vport); + vport->dev->priv_flags = saved_flags; + if (err) goto error_master_upper_dev_unlink;
VRF devices are prevented from being added to upper devices since commit 1017e0987117 ("vrf: prevent adding upper devices") as they set the IFF_NO_RX_HANDLER flag. However attaching a VRF to an OVS bridge is a valid use case[1]. Allow a VRF device to be attached to an OVS bridge by having an OVS specific tweak. This approach allows not to change a valid logic elsewhere and the IFF_NO_RX_HANDLER limitation still applies for non-OVS upper devices, even after a VRF was unlinked from an OVS bridge. (Patch not sent as a fix as the commit introducing the limitation is not recent). [1] https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/ Signed-off-by: Antoine Tenart <atenart@kernel.org> --- Hi all, I thought about other ways to fix this but did not want to add yet another flag, nor to add specific logic outside of net/openvswitch/. A custom netdev_rx_handler_register having priv_flags as a parameter could also have been added, but again that seemed a bit invasive. There might be questions about the setup in which a VRF is linked to an OVS bridge; I cc'ed Luis Tomás who wrote the article. Thanks, Antoine net/openvswitch/vport-netdev.c | 10 ++++++++++ 1 file changed, 10 insertions(+)