mbox series

[0/7] Add vmnet.framework based network backend

Message ID 20210617143246.55336-1-yaroshchuk2000@gmail.com (mailing list archive)
Headers show
Series Add vmnet.framework based network backend | expand

Message

Vladislav Yaroshchuk June 17, 2021, 2:32 p.m. UTC
macOS provides networking API for VMs called vmnet.framework.
I tried to add it as a network backend. All three modes are supported:

-shared:
  allows the guest to comminicate with other guests in shared mode and
  also with external network (Internet) via NAT

-host:
  allows the guest to communicate with other guests in host mode

-bridged:
  bridges the guest with a physical network interface

Separate netdev for each vmnet mode was created because they use quite
different settings, especially since macOS 11.0 when vmnet.framework
gets a lot of updates.

Not sure that I use qemu_mutex_lock_iothread() and
qemu_mutex_unlock_iothread() in correct way while sending packet
from vmnet interface to QEMU. I'll be happy to receive
recomendations how to make this thing better if I done sth wrong.

Also vmnet.framework requires com.apple.vm.networking entitlement to
run without root priveledges. Ad-hoc signing does not fit there,
so I didn't touch anything related to signing. As a result we should
run qemu-system by a priviledged user:
`$ sudo qemu-system-x86_64 -nic vmnet-shared`
otherwise vmnet fails with 'general failure'.

But in any way it seems working now,
I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
models:
- e1000-82545em
- virtio-net-pci

and having such guests:
- macOS 10.15.7
- Ubuntu Bionic (server cloudimg) 

Vladislav Yaroshchuk (7):
  net/vmnet: dependencies setup, initial preparations
  net/vmnet: add new netdevs to qapi/net
  net/vmnet: create common netdev state structure
  net/vmnet: implement shared mode (vmnet-shared)
  net/vmnet: implement host mode (vmnet-host)
  net/vmnet: implement bridged mode (vmnet-bridged)
  net/vmnet: update qemu-options.hx

 configure           |  31 +++++
 meson.build         |   5 +
 net/clients.h       |  11 ++
 net/meson.build     |   7 ++
 net/net.c           |  10 ++
 net/vmnet-bridged.m | 123 ++++++++++++++++++
 net/vmnet-common.m  | 294 ++++++++++++++++++++++++++++++++++++++++++++
 net/vmnet-host.c    |  93 ++++++++++++++
 net/vmnet-shared.c  |  94 ++++++++++++++
 net/vmnet_int.h     |  48 ++++++++
 qapi/net.json       |  99 ++++++++++++++-
 qemu-options.hx     |  17 +++
 12 files changed, 830 insertions(+), 2 deletions(-)
 create mode 100644 net/vmnet-bridged.m
 create mode 100644 net/vmnet-common.m
 create mode 100644 net/vmnet-host.c
 create mode 100644 net/vmnet-shared.c
 create mode 100644 net/vmnet_int.h

Comments

Vladislav Yaroshchuk Aug. 6, 2021, 7:03 p.m. UTC | #1
ping
https://patchew.org/QEMU/20210617143246.55336-1-yaroshchuk2000@gmail.com/

чт, 17 июн. 2021 г. в 17:33, Vladislav Yaroshchuk <yaroshchuk2000@gmail.com
>:

> macOS provides networking API for VMs called vmnet.framework.
> I tried to add it as a network backend. All three modes are supported:
>
> -shared:
>   allows the guest to comminicate with other guests in shared mode and
>   also with external network (Internet) via NAT
>
> -host:
>   allows the guest to communicate with other guests in host mode
>
> -bridged:
>   bridges the guest with a physical network interface
>
> Separate netdev for each vmnet mode was created because they use quite
> different settings, especially since macOS 11.0 when vmnet.framework
> gets a lot of updates.
>
> Not sure that I use qemu_mutex_lock_iothread() and
> qemu_mutex_unlock_iothread() in correct way while sending packet
> from vmnet interface to QEMU. I'll be happy to receive
> recomendations how to make this thing better if I done sth wrong.
>
> Also vmnet.framework requires com.apple.vm.networking entitlement to
> run without root priveledges. Ad-hoc signing does not fit there,
> so I didn't touch anything related to signing. As a result we should
> run qemu-system by a priviledged user:
> `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> otherwise vmnet fails with 'general failure'.
>
> But in any way it seems working now,
> I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
> models:
> - e1000-82545em
> - virtio-net-pci
>
> and having such guests:
> - macOS 10.15.7
> - Ubuntu Bionic (server cloudimg)
>
> Vladislav Yaroshchuk (7):
>   net/vmnet: dependencies setup, initial preparations
>   net/vmnet: add new netdevs to qapi/net
>   net/vmnet: create common netdev state structure
>   net/vmnet: implement shared mode (vmnet-shared)
>   net/vmnet: implement host mode (vmnet-host)
>   net/vmnet: implement bridged mode (vmnet-bridged)
>   net/vmnet: update qemu-options.hx
>
>  configure           |  31 +++++
>  meson.build         |   5 +
>  net/clients.h       |  11 ++
>  net/meson.build     |   7 ++
>  net/net.c           |  10 ++
>  net/vmnet-bridged.m | 123 ++++++++++++++++++
>  net/vmnet-common.m  | 294 ++++++++++++++++++++++++++++++++++++++++++++
>  net/vmnet-host.c    |  93 ++++++++++++++
>  net/vmnet-shared.c  |  94 ++++++++++++++
>  net/vmnet_int.h     |  48 ++++++++
>  qapi/net.json       |  99 ++++++++++++++-
>  qemu-options.hx     |  17 +++
>  12 files changed, 830 insertions(+), 2 deletions(-)
>  create mode 100644 net/vmnet-bridged.m
>  create mode 100644 net/vmnet-common.m
>  create mode 100644 net/vmnet-host.c
>  create mode 100644 net/vmnet-shared.c
>  create mode 100644 net/vmnet_int.h
>
> --
> 2.23.0
>
>
Jason Wang Aug. 9, 2021, 3:23 a.m. UTC | #2
在 2021/8/7 上午3:03, Vladislav Yaroshchuk 写道:
> ping
> https://patchew.org/QEMU/20210617143246.55336-1-yaroshchuk2000@gmail.com/ 
> <https://patchew.org/QEMU/20210617143246.55336-1-yaroshchuk2000@gmail.com/>


Will review this week.

Thanks


>
> чт, 17 июн. 2021 г. в 17:33, Vladislav Yaroshchuk 
> <yaroshchuk2000@gmail.com <mailto:yaroshchuk2000@gmail.com>>:
>
>     macOS provides networking API for VMs called vmnet.framework.
>     I tried to add it as a network backend. All three modes are supported:
>
>     -shared:
>       allows the guest to comminicate with other guests in shared mode and
>       also with external network (Internet) via NAT
>
>     -host:
>       allows the guest to communicate with other guests in host mode
>
>     -bridged:
>       bridges the guest with a physical network interface
>
>     Separate netdev for each vmnet mode was created because they use quite
>     different settings, especially since macOS 11.0 when vmnet.framework
>     gets a lot of updates.
>
>     Not sure that I use qemu_mutex_lock_iothread() and
>     qemu_mutex_unlock_iothread() in correct way while sending packet
>     from vmnet interface to QEMU. I'll be happy to receive
>     recomendations how to make this thing better if I done sth wrong.
>
>     Also vmnet.framework requires com.apple.vm.networking entitlement to
>     run without root priveledges. Ad-hoc signing does not fit there,
>     so I didn't touch anything related to signing. As a result we should
>     run qemu-system by a priviledged user:
>     `$ sudo qemu-system-x86_64 -nic vmnet-shared`
>     otherwise vmnet fails with 'general failure'.
>
>     But in any way it seems working now,
>     I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
>     models:
>     - e1000-82545em
>     - virtio-net-pci
>
>     and having such guests:
>     - macOS 10.15.7
>     - Ubuntu Bionic (server cloudimg)
>
>     Vladislav Yaroshchuk (7):
>       net/vmnet: dependencies setup, initial preparations
>       net/vmnet: add new netdevs to qapi/net
>       net/vmnet: create common netdev state structure
>       net/vmnet: implement shared mode (vmnet-shared)
>       net/vmnet: implement host mode (vmnet-host)
>       net/vmnet: implement bridged mode (vmnet-bridged)
>       net/vmnet: update qemu-options.hx
>
>      configure           |  31 +++++
>      meson.build         |   5 +
>      net/clients.h       |  11 ++
>      net/meson.build     |   7 ++
>      net/net.c           |  10 ++
>      net/vmnet-bridged.m | 123 ++++++++++++++++++
>      net/vmnet-common.m  | 294
>     ++++++++++++++++++++++++++++++++++++++++++++
>      net/vmnet-host.c    |  93 ++++++++++++++
>      net/vmnet-shared.c  |  94 ++++++++++++++
>      net/vmnet_int.h     |  48 ++++++++
>      qapi/net.json       |  99 ++++++++++++++-
>      qemu-options.hx     |  17 +++
>      12 files changed, 830 insertions(+), 2 deletions(-)
>      create mode 100644 net/vmnet-bridged.m
>      create mode 100644 net/vmnet-common.m
>      create mode 100644 net/vmnet-host.c
>      create mode 100644 net/vmnet-shared.c
>      create mode 100644 net/vmnet_int.h
>
>     -- 
>     2.23.0
>
Roman Bolshakov Aug. 12, 2021, 6 a.m. UTC | #3
On Thu, Jun 17, 2021 at 05:32:39PM +0300, Vladislav Yaroshchuk wrote:
> macOS provides networking API for VMs called vmnet.framework.
> I tried to add it as a network backend. All three modes are supported:
> 
> -shared:
>   allows the guest to comminicate with other guests in shared mode and
>   also with external network (Internet) via NAT
> 
> -host:
>   allows the guest to communicate with other guests in host mode
> 
> -bridged:
>   bridges the guest with a physical network interface
> 
> Separate netdev for each vmnet mode was created because they use quite
> different settings, especially since macOS 11.0 when vmnet.framework
> gets a lot of updates.
> 
> Not sure that I use qemu_mutex_lock_iothread() and
> qemu_mutex_unlock_iothread() in correct way while sending packet
> from vmnet interface to QEMU. I'll be happy to receive
> recomendations how to make this thing better if I done sth wrong.
> 
> Also vmnet.framework requires com.apple.vm.networking entitlement to
> run without root priveledges. Ad-hoc signing does not fit there,
> so I didn't touch anything related to signing. As a result we should
> run qemu-system by a priviledged user:
> `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> otherwise vmnet fails with 'general failure'.
> 
> But in any way it seems working now,
> I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
> models:
> - e1000-82545em
> - virtio-net-pci
> 
> and having such guests:
> - macOS 10.15.7
> - Ubuntu Bionic (server cloudimg) 
> 

Hi Vladislav,

I appreciate the efforts and I'm sorry I didn't look into it yet, lack
of time :(

To all: earlier this year another series was sent by Phillip Tennen to
add vmnet.framework and some comments were provided:
https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg05874.html

I'm not sure how to proceed with arbitration which of the series is
preferred. FIFO or LIFO?

Regards,
Roman

> Vladislav Yaroshchuk (7):
>   net/vmnet: dependencies setup, initial preparations
>   net/vmnet: add new netdevs to qapi/net
>   net/vmnet: create common netdev state structure
>   net/vmnet: implement shared mode (vmnet-shared)
>   net/vmnet: implement host mode (vmnet-host)
>   net/vmnet: implement bridged mode (vmnet-bridged)
>   net/vmnet: update qemu-options.hx
> 
>  configure           |  31 +++++
>  meson.build         |   5 +
>  net/clients.h       |  11 ++
>  net/meson.build     |   7 ++
>  net/net.c           |  10 ++
>  net/vmnet-bridged.m | 123 ++++++++++++++++++
>  net/vmnet-common.m  | 294 ++++++++++++++++++++++++++++++++++++++++++++
>  net/vmnet-host.c    |  93 ++++++++++++++
>  net/vmnet-shared.c  |  94 ++++++++++++++
>  net/vmnet_int.h     |  48 ++++++++
>  qapi/net.json       |  99 ++++++++++++++-
>  qemu-options.hx     |  17 +++
>  12 files changed, 830 insertions(+), 2 deletions(-)
>  create mode 100644 net/vmnet-bridged.m
>  create mode 100644 net/vmnet-common.m
>  create mode 100644 net/vmnet-host.c
>  create mode 100644 net/vmnet-shared.c
>  create mode 100644 net/vmnet_int.h
> 
> -- 
> 2.23.0
>
Vladislav Yaroshchuk Aug. 17, 2021, 9:10 a.m. UTC | #4
Hi Roman,

Sorry for the late reply. In my series of patches I partially reuse Phillip
Tennen's work that you have mentioned
https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg05874.html. The
main idea and provided feature is the same, so please select the series on
your choice. I just can say that I'm ready to update/improve my patches
asap, make them ready to be merged. Can't say anything about Phillip, is he
ready too or not.

Regards,
Vladislav

чт, 12 авг. 2021 г. в 09:01, Roman Bolshakov <r.bolshakov@yadro.com>:

> On Thu, Jun 17, 2021 at 05:32:39PM +0300, Vladislav Yaroshchuk wrote:
> > macOS provides networking API for VMs called vmnet.framework.
> > I tried to add it as a network backend. All three modes are supported:
> >
> > -shared:
> >   allows the guest to comminicate with other guests in shared mode and
> >   also with external network (Internet) via NAT
> >
> > -host:
> >   allows the guest to communicate with other guests in host mode
> >
> > -bridged:
> >   bridges the guest with a physical network interface
> >
> > Separate netdev for each vmnet mode was created because they use quite
> > different settings, especially since macOS 11.0 when vmnet.framework
> > gets a lot of updates.
> >
> > Not sure that I use qemu_mutex_lock_iothread() and
> > qemu_mutex_unlock_iothread() in correct way while sending packet
> > from vmnet interface to QEMU. I'll be happy to receive
> > recomendations how to make this thing better if I done sth wrong.
> >
> > Also vmnet.framework requires com.apple.vm.networking entitlement to
> > run without root priveledges. Ad-hoc signing does not fit there,
> > so I didn't touch anything related to signing. As a result we should
> > run qemu-system by a priviledged user:
> > `$ sudo qemu-system-x86_64 -nic vmnet-shared`
> > otherwise vmnet fails with 'general failure'.
> >
> > But in any way it seems working now,
> > I tested it within qemu-system-x86-64 on macOS 10.15.7 host, with nic
> > models:
> > - e1000-82545em
> > - virtio-net-pci
> >
> > and having such guests:
> > - macOS 10.15.7
> > - Ubuntu Bionic (server cloudimg)
> >
>
> Hi Vladislav,
>
> I appreciate the efforts and I'm sorry I didn't look into it yet, lack
> of time :(
>
> To all: earlier this year another series was sent by Phillip Tennen to
> add vmnet.framework and some comments were provided:
> https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg05874.html
>
> I'm not sure how to proceed with arbitration which of the series is
> preferred. FIFO or LIFO?
>
> Regards,
> Roman
>
> > Vladislav Yaroshchuk (7):
> >   net/vmnet: dependencies setup, initial preparations
> >   net/vmnet: add new netdevs to qapi/net
> >   net/vmnet: create common netdev state structure
> >   net/vmnet: implement shared mode (vmnet-shared)
> >   net/vmnet: implement host mode (vmnet-host)
> >   net/vmnet: implement bridged mode (vmnet-bridged)
> >   net/vmnet: update qemu-options.hx
> >
> >  configure           |  31 +++++
> >  meson.build         |   5 +
> >  net/clients.h       |  11 ++
> >  net/meson.build     |   7 ++
> >  net/net.c           |  10 ++
> >  net/vmnet-bridged.m | 123 ++++++++++++++++++
> >  net/vmnet-common.m  | 294 ++++++++++++++++++++++++++++++++++++++++++++
> >  net/vmnet-host.c    |  93 ++++++++++++++
> >  net/vmnet-shared.c  |  94 ++++++++++++++
> >  net/vmnet_int.h     |  48 ++++++++
> >  qapi/net.json       |  99 ++++++++++++++-
> >  qemu-options.hx     |  17 +++
> >  12 files changed, 830 insertions(+), 2 deletions(-)
> >  create mode 100644 net/vmnet-bridged.m
> >  create mode 100644 net/vmnet-common.m
> >  create mode 100644 net/vmnet-host.c
> >  create mode 100644 net/vmnet-shared.c
> >  create mode 100644 net/vmnet_int.h
> >
> > --
> > 2.23.0
> >
>