| Message ID | 20240718193543.624039-4-ilstam@amazon.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: Improve MMIO Coalescing API | expand |
Hi Ilias, kernel test robot noticed the following build warnings: [auto build test WARNING on kvm/queue] [also build test WARNING on next-20240719] [cannot apply to mst-vhost/linux-next linus/master kvm/linux-next v6.10] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Ilias-Stamatis/KVM-Fix-coalesced_mmio_has_room/20240719-034316 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20240718193543.624039-4-ilstam%40amazon.com patch subject: [PATCH v2 3/6] KVM: Support poll() on coalesced mmio buffer fds config: x86_64-randconfig-122-20240719 (https://download.01.org/0day-ci/archive/20240720/202407200922.pJAJMVRk-lkp@intel.com/config) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240720/202407200922.pJAJMVRk-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202407200922.pJAJMVRk-lkp@intel.com/ sparse warnings: (new ones prefixed by >>) >> arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: sparse: incorrect type in assignment (different base types) @@ expected restricted __poll_t [usertype] mask @@ got int @@ arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: expected restricted __poll_t [usertype] mask arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: got int arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c: note: in included file (through include/linux/mmzone.h, include/linux/gfp.h, include/linux/mm.h, ...): include/linux/page-flags.h:240:46: sparse: sparse: self-comparison always evaluates to false include/linux/page-flags.h:240:46: sparse: sparse: self-comparison always evaluates to false vim +241 arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c 231 232 static __poll_t coalesced_mmio_buffer_poll(struct file *file, struct poll_table_struct *wait) 233 { 234 struct kvm_coalesced_mmio_buffer_dev *dev = file->private_data; 235 __poll_t mask = 0; 236 237 poll_wait(file, &dev->wait_queue, wait); 238 239 spin_lock(&dev->ring_lock); 240 if (dev->ring && (READ_ONCE(dev->ring->first) != READ_ONCE(dev->ring->last))) > 241 mask = POLLIN | POLLRDNORM; 242 spin_unlock(&dev->ring_lock); 243 244 return mask; 245 } 246
Hi Ilias, kernel test robot noticed the following build warnings: [auto build test WARNING on kvm/queue] [also build test WARNING on next-20240719] [cannot apply to mst-vhost/linux-next linus/master kvm/linux-next v6.10] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch#_base_tree_information] url: https://github.com/intel-lab-lkp/linux/commits/Ilias-Stamatis/KVM-Fix-coalesced_mmio_has_room/20240719-034316 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue patch link: https://lore.kernel.org/r/20240718193543.624039-4-ilstam%40amazon.com patch subject: [PATCH v2 3/6] KVM: Support poll() on coalesced mmio buffer fds config: riscv-randconfig-r121-20240719 (https://download.01.org/0day-ci/archive/20240721/202407210447.D2HLgqLd-lkp@intel.com/config) compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1) reproduce: (https://download.01.org/0day-ci/archive/20240721/202407210447.D2HLgqLd-lkp@intel.com/reproduce) If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <lkp@intel.com> | Closes: https://lore.kernel.org/oe-kbuild-all/202407210447.D2HLgqLd-lkp@intel.com/ sparse warnings: (new ones prefixed by >>) >> arch/riscv/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: sparse: incorrect type in assignment (different base types) @@ expected restricted __poll_t [usertype] mask @@ got int @@ arch/riscv/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: expected restricted __poll_t [usertype] mask arch/riscv/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: got int vim +241 arch/riscv/kvm/../../../virt/kvm/coalesced_mmio.c 231 232 static __poll_t coalesced_mmio_buffer_poll(struct file *file, struct poll_table_struct *wait) 233 { 234 struct kvm_coalesced_mmio_buffer_dev *dev = file->private_data; 235 __poll_t mask = 0; 236 237 poll_wait(file, &dev->wait_queue, wait); 238 239 spin_lock(&dev->ring_lock); 240 if (dev->ring && (READ_ONCE(dev->ring->first) != READ_ONCE(dev->ring->last))) > 241 mask = POLLIN | POLLRDNORM; 242 spin_unlock(&dev->ring_lock); 243 244 return mask; 245 } 246
On Sat, 2024-07-20 at 09:35 +0800, kernel test robot wrote: > Hi Ilias, > > kernel test robot noticed the following build warnings: > > [auto build test WARNING on kvm/queue] > [also build test WARNING on next-20240719] > [cannot apply to mst-vhost/linux-next linus/master kvm/linux-next v6.10] > [If your patch is applied to the wrong git tree, kindly drop us a note. > And when submitting patch, we suggest to use '--base' as documented in > https://git-scm.com/docs/git-format-patch#_base_tree_information] > > url: https://github.com/intel-lab-lkp/linux/commits/Ilias-Stamatis/KVM-Fix-coalesced_mmio_has_room/20240719-034316 > base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git queue > patch link: https://lore.kernel.org/r/20240718193543.624039-4-ilstam%40amazon.com > patch subject: [PATCH v2 3/6] KVM: Support poll() on coalesced mmio buffer fds > config: x86_64-randconfig-122-20240719 (https://download.01.org/0day-ci/archive/20240720/202407200922.pJAJMVRk-lkp@intel.com/config) > compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) > reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240720/202407200922.pJAJMVRk-lkp@intel.com/reproduce) > > If you fix the issue in a separate patch/commit (i.e. not just a new version of > the same patch/commit), kindly add following tags > > Reported-by: kernel test robot <lkp@intel.com> > > Closes: https://lore.kernel.org/oe-kbuild-all/202407200922.pJAJMVRk-lkp@intel.com/ > > sparse warnings: (new ones prefixed by >>) > > > arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: sparse: incorrect type in assignment (different base types) @@ expected restricted __poll_t [usertype] mask @@ got int @@ > arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: expected restricted __poll_t [usertype] mask > arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c:241:22: sparse: got int > arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c: note: in included file (through include/linux/mmzone.h, include/linux/gfp.h, include/linux/mm.h, ...): > include/linux/page-flags.h:240:46: sparse: sparse: self-comparison always evaluates to false > include/linux/page-flags.h:240:46: sparse: sparse: self-comparison always evaluates to false > > vim +241 arch/x86/kvm/../../../virt/kvm/coalesced_mmio.c > > 231 > 232 static __poll_t coalesced_mmio_buffer_poll(struct file *file, struct poll_table_struct *wait) > 233 { > 234 struct kvm_coalesced_mmio_buffer_dev *dev = file->private_data; > 235 __poll_t mask = 0; > 236 > 237 poll_wait(file, &dev->wait_queue, wait); > 238 > 239 spin_lock(&dev->ring_lock); > 240 if (dev->ring && (READ_ONCE(dev->ring->first) != READ_ONCE(dev->ring->last))) > > 241 mask = POLLIN | POLLRDNORM; This must be EPOLLIN | EPOLLRDNORM, but I will wait for more feedback / reviews before sending another version. Ilias
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c index 6443d4b62548..64428b0a4aad 100644 --- a/virt/kvm/coalesced_mmio.c +++ b/virt/kvm/coalesced_mmio.c @@ -16,6 +16,7 @@ #include <linux/slab.h> #include <linux/kvm.h> #include <linux/anon_inodes.h> +#include <linux/poll.h> #include "coalesced_mmio.h" @@ -98,6 +99,10 @@ static int coalesced_mmio_write(struct kvm_vcpu *vcpu, smp_wmb(); ring->last = (insert + 1) % KVM_COALESCED_MMIO_MAX; spin_unlock(lock); + + if (dev->buffer_dev) + wake_up_interruptible(&dev->buffer_dev->wait_queue); + return 0; } @@ -224,9 +229,25 @@ static int coalesced_mmio_buffer_release(struct inode *inode, struct file *file) return 0; } +static __poll_t coalesced_mmio_buffer_poll(struct file *file, struct poll_table_struct *wait) +{ + struct kvm_coalesced_mmio_buffer_dev *dev = file->private_data; + __poll_t mask = 0; + + poll_wait(file, &dev->wait_queue, wait); + + spin_lock(&dev->ring_lock); + if (dev->ring && (READ_ONCE(dev->ring->first) != READ_ONCE(dev->ring->last))) + mask = POLLIN | POLLRDNORM; + spin_unlock(&dev->ring_lock); + + return mask; +} + static const struct file_operations coalesced_mmio_buffer_ops = { .mmap = coalesced_mmio_buffer_mmap, .release = coalesced_mmio_buffer_release, + .poll = coalesced_mmio_buffer_poll, }; int kvm_vm_ioctl_create_coalesced_mmio_buffer(struct kvm *kvm) @@ -240,6 +261,7 @@ int kvm_vm_ioctl_create_coalesced_mmio_buffer(struct kvm *kvm) return -ENOMEM; dev->kvm = kvm; + init_waitqueue_head(&dev->wait_queue); spin_lock_init(&dev->ring_lock); ret = anon_inode_getfd("coalesced_mmio_buf", &coalesced_mmio_buffer_ops, diff --git a/virt/kvm/coalesced_mmio.h b/virt/kvm/coalesced_mmio.h index 37d9d8f325bb..d1807ce26464 100644 --- a/virt/kvm/coalesced_mmio.h +++ b/virt/kvm/coalesced_mmio.h @@ -26,6 +26,7 @@ struct kvm_coalesced_mmio_dev { struct kvm_coalesced_mmio_buffer_dev { struct list_head list; struct kvm *kvm; + wait_queue_head_t wait_queue; spinlock_t ring_lock; struct kvm_coalesced_mmio_ring *ring; };
There is no direct way for userspace to be notified about coalesced MMIO writes when using KVM_REGISTER_COALESCED_MMIO. If the next MMIO exit is when the ring buffer has filled then a substantial (and unbounded) amount of time may have passed since the first coalesced MMIO. To improve this, make it possible for userspace to use poll() and select() on the fd returned by the KVM_CREATE_COALESCED_MMIO_BUFFER ioctl. This way a userspace VMM could have dedicated threads that deal with writes to specific MMIO zones. For example, a common use of MMIO, particularly in the realm of network devices, is as a doorbell. A write to a doorbell register will trigger the device to initiate a DMA transfer. When a network device is emulated by userspace a write to a doorbell register would typically result in an MMIO exit so that userspace can emulate the DMA transfer in a timely manner. No further processing can be done until userspace performs the necessary emulation and re-invokes KVM_RUN. Even if userspace makes use of another thread to emulate the DMA transfer such MMIO exits are disruptive to the vCPU and they may also be quite frequent if, for example, the vCPU is sending a sequence of short packets to the network device. By supporting poll() on coalesced buffer fds, userspace can have dedicated threads wait for new doorbell writes and avoid the performance hit of userspace exits on the main vCPU threads. Signed-off-by: Ilias Stamatis <ilstam@amazon.com> --- v1->v2: - Added a NULL-check for dev->ring in coalesced_mmio_buffer_poll() virt/kvm/coalesced_mmio.c | 22 ++++++++++++++++++++++ virt/kvm/coalesced_mmio.h | 1 + 2 files changed, 23 insertions(+)