mbox series

[RFC,v3,0/5] Introduce MMIO/PIO dispatch file descriptors (ioregionfd)

Message ID cover.1613828726.git.eafanasova@gmail.com (mailing list archive)
Headers show
Series Introduce MMIO/PIO dispatch file descriptors (ioregionfd) | expand

Message

Elena Afanasova Feb. 21, 2021, 12:04 p.m. UTC
This patchset introduces a KVM dispatch mechanism which can be used 
for handling MMIO/PIO accesses over file descriptors without returning 
from ioctl(KVM_RUN). This allows device emulation to run in another task 
separate from the vCPU task.

This is achieved through KVM vm ioctl for registering MMIO/PIO regions and 
a wire protocol that KVM uses to communicate with a task handling an 
MMIO/PIO access.

TODOs:
* Implement KVM_EXIT_IOREGIONFD_FAILURE
* Add non-x86 arch support
* Add kvm-unittests
* Flush waiters if ioregion is deleted

v3:
 - add FAST_MMIO bus support
 - add KVM_IOREGION_DEASSIGN flag
 - rename kvm_ioregion read/write file descriptors
 - split ioregionfd signal handling support into two patches
 - move ioregion_interrupted flag to ioregion_ctx
 - reorder ioregion_ctx fields
 - rework complete_ioregion operations 
 - add signal handling support for crossing a page boundary case
 - change wire protocol license
 - fix ioregionfd state machine
 - remove ioregionfd_cmd info and drop appropriate macros
 - add comment on ioregionfd cmds/replies serialization
 - drop kvm_io_bus_finish/prepare()

Elena Afanasova (5):
  KVM: add initial support for KVM_SET_IOREGION
  KVM: x86: add support for ioregionfd signal handling
  KVM: implement wire protocol
  KVM: add ioregionfd context
  KVM: enforce NR_IOBUS_DEVS limit if kmemcg is disabled

 arch/x86/kvm/Kconfig          |   1 +
 arch/x86/kvm/Makefile         |   1 +
 arch/x86/kvm/vmx/vmx.c        |  40 ++-
 arch/x86/kvm/x86.c            | 273 +++++++++++++++++-
 include/linux/kvm_host.h      |  28 ++
 include/uapi/linux/ioregion.h |  30 ++
 include/uapi/linux/kvm.h      |  25 ++
 virt/kvm/Kconfig              |   3 +
 virt/kvm/eventfd.c            |  25 ++
 virt/kvm/eventfd.h            |  14 +
 virt/kvm/ioregion.c           | 529 ++++++++++++++++++++++++++++++++++
 virt/kvm/ioregion.h           |  15 +
 virt/kvm/kvm_main.c           |  36 ++-
 13 files changed, 996 insertions(+), 24 deletions(-)
 create mode 100644 include/uapi/linux/ioregion.h
 create mode 100644 virt/kvm/eventfd.h
 create mode 100644 virt/kvm/ioregion.c
 create mode 100644 virt/kvm/ioregion.h

Comments

Paolo Bonzini Feb. 21, 2021, 5:06 p.m. UTC | #1
On 21/02/21 13:04, Elena Afanasova wrote:
> This patchset introduces a KVM dispatch mechanism which can be used
> for handling MMIO/PIO accesses over file descriptors without returning
> from ioctl(KVM_RUN). This allows device emulation to run in another task
> separate from the vCPU task.
> 
> This is achieved through KVM vm ioctl for registering MMIO/PIO regions and
> a wire protocol that KVM uses to communicate with a task handling an
> MMIO/PIO access.
> 
> TODOs:
> * Implement KVM_EXIT_IOREGIONFD_FAILURE
> * Add non-x86 arch support
> * Add kvm-unittests
> * Flush waiters if ioregion is deleted

Hi ELena,

as a quick thing that jumped at me before starting the review, you 
should add a test for the new API in tools/testing/selftests/kvm, as 
well as documentation.  Ideally, patch 4 would also add a testcase that 
fails before and passes afterwards.

Also, does this work already with io_uring?

Paolo

> v3:
>   - add FAST_MMIO bus support
>   - add KVM_IOREGION_DEASSIGN flag
>   - rename kvm_ioregion read/write file descriptors
>   - split ioregionfd signal handling support into two patches
>   - move ioregion_interrupted flag to ioregion_ctx
>   - reorder ioregion_ctx fields
>   - rework complete_ioregion operations
>   - add signal handling support for crossing a page boundary case
>   - change wire protocol license
>   - fix ioregionfd state machine
>   - remove ioregionfd_cmd info and drop appropriate macros
>   - add comment on ioregionfd cmds/replies serialization
>   - drop kvm_io_bus_finish/prepare()
> 
> Elena Afanasova (5):
>    KVM: add initial support for KVM_SET_IOREGION
>    KVM: x86: add support for ioregionfd signal handling
>    KVM: implement wire protocol
>    KVM: add ioregionfd context
>    KVM: enforce NR_IOBUS_DEVS limit if kmemcg is disabled
> 
>   arch/x86/kvm/Kconfig          |   1 +
>   arch/x86/kvm/Makefile         |   1 +
>   arch/x86/kvm/vmx/vmx.c        |  40 ++-
>   arch/x86/kvm/x86.c            | 273 +++++++++++++++++-
>   include/linux/kvm_host.h      |  28 ++
>   include/uapi/linux/ioregion.h |  30 ++
>   include/uapi/linux/kvm.h      |  25 ++
>   virt/kvm/Kconfig              |   3 +
>   virt/kvm/eventfd.c            |  25 ++
>   virt/kvm/eventfd.h            |  14 +
>   virt/kvm/ioregion.c           | 529 ++++++++++++++++++++++++++++++++++
>   virt/kvm/ioregion.h           |  15 +
>   virt/kvm/kvm_main.c           |  36 ++-
>   13 files changed, 996 insertions(+), 24 deletions(-)
>   create mode 100644 include/uapi/linux/ioregion.h
>   create mode 100644 virt/kvm/eventfd.h
>   create mode 100644 virt/kvm/ioregion.c
>   create mode 100644 virt/kvm/ioregion.h
>
Elena Afanasova Feb. 22, 2021, 4:40 p.m. UTC | #2
On Sun, 2021-02-21 at 18:06 +0100, Paolo Bonzini wrote:
> On 21/02/21 13:04, Elena Afanasova wrote:
> > This patchset introduces a KVM dispatch mechanism which can be used
> > for handling MMIO/PIO accesses over file descriptors without
> > returning
> > from ioctl(KVM_RUN). This allows device emulation to run in another
> > task
> > separate from the vCPU task.
> > 
> > This is achieved through KVM vm ioctl for registering MMIO/PIO
> > regions and
> > a wire protocol that KVM uses to communicate with a task handling
> > an
> > MMIO/PIO access.
> > 
> > TODOs:
> > * Implement KVM_EXIT_IOREGIONFD_FAILURE
> > * Add non-x86 arch support
> > * Add kvm-unittests
> > * Flush waiters if ioregion is deleted
> 
> Hi ELena,
> 

Hi Paolo,

Thank you for your answer.

> as a quick thing that jumped at me before starting the review, you 
> should add a test for the new API in tools/testing/selftests/kvm, as 
> well as documentation.  Ideally, patch 4 would also add a testcase
> that 
> fails before and passes afterwards.
> 
Ok

> Also, does this work already with io_uring?
> 
I have a few kvm-unittests and QEMU testdev patch for testing base
functionality. I haven't tried to run them with io_uring (only run with
socket and pipes). Will do.

> Paolo
> 
> > v3:
> >   - add FAST_MMIO bus support
> >   - add KVM_IOREGION_DEASSIGN flag
> >   - rename kvm_ioregion read/write file descriptors
> >   - split ioregionfd signal handling support into two patches
> >   - move ioregion_interrupted flag to ioregion_ctx
> >   - reorder ioregion_ctx fields
> >   - rework complete_ioregion operations
> >   - add signal handling support for crossing a page boundary case
> >   - change wire protocol license
> >   - fix ioregionfd state machine
> >   - remove ioregionfd_cmd info and drop appropriate macros
> >   - add comment on ioregionfd cmds/replies serialization
> >   - drop kvm_io_bus_finish/prepare()
> > 
> > Elena Afanasova (5):
> >    KVM: add initial support for KVM_SET_IOREGION
> >    KVM: x86: add support for ioregionfd signal handling
> >    KVM: implement wire protocol
> >    KVM: add ioregionfd context
> >    KVM: enforce NR_IOBUS_DEVS limit if kmemcg is disabled
> > 
> >   arch/x86/kvm/Kconfig          |   1 +
> >   arch/x86/kvm/Makefile         |   1 +
> >   arch/x86/kvm/vmx/vmx.c        |  40 ++-
> >   arch/x86/kvm/x86.c            | 273 +++++++++++++++++-
> >   include/linux/kvm_host.h      |  28 ++
> >   include/uapi/linux/ioregion.h |  30 ++
> >   include/uapi/linux/kvm.h      |  25 ++
> >   virt/kvm/Kconfig              |   3 +
> >   virt/kvm/eventfd.c            |  25 ++
> >   virt/kvm/eventfd.h            |  14 +
> >   virt/kvm/ioregion.c           | 529
> > ++++++++++++++++++++++++++++++++++
> >   virt/kvm/ioregion.h           |  15 +
> >   virt/kvm/kvm_main.c           |  36 ++-
> >   13 files changed, 996 insertions(+), 24 deletions(-)
> >   create mode 100644 include/uapi/linux/ioregion.h
> >   create mode 100644 virt/kvm/eventfd.h
> >   create mode 100644 virt/kvm/ioregion.c
> >   create mode 100644 virt/kvm/ioregion.h
> >
Stefan Hajnoczi Feb. 24, 2021, 11:34 a.m. UTC | #3
On Sun, Feb 21, 2021 at 03:04:36PM +0300, Elena Afanasova wrote:
> This patchset introduces a KVM dispatch mechanism which can be used 
> for handling MMIO/PIO accesses over file descriptors without returning 
> from ioctl(KVM_RUN). This allows device emulation to run in another task 
> separate from the vCPU task.
> 
> This is achieved through KVM vm ioctl for registering MMIO/PIO regions and 
> a wire protocol that KVM uses to communicate with a task handling an 
> MMIO/PIO access.
> 
> TODOs:
> * Implement KVM_EXIT_IOREGIONFD_FAILURE
> * Add non-x86 arch support
> * Add kvm-unittests
> * Flush waiters if ioregion is deleted
 * Add ioctl docs to api.rst
 * Add wire protocol docs to <linux/ioregionfd.h>

Great, looks like userspace can really start trying out ioregionfd now -
most features are implemented. I will do a deeper review of the state
machine when you send the next revision.

Stefan