| Message ID | 20230214061743.114257-1-lulu@redhat.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | vp_vdpa: fix the crash in hot unplug with vp_vdpa | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Not a local patch |
On Tue, Feb 14, 2023 at 2:17 PM Cindy Lu <lulu@redhat.com> wrote: > > While unplugging the vp_vdpa device, the kernel will crash > The root cause is the function vp_modern_get_status() called following the > vp_modern_remove(). This needs some tweaking, maybe it's better to say vdpa_mgmtdev_unregister() will access modern devices which will cause a use after free. >So need to change the sequence in vp_vdpa_remove > > [ 195.016001] Call Trace: Let's paste the full log with the reason for the calltrace (e.g general protection fault or whatever else). > [ 195.016233] <TASK> > [ 195.016434] vp_modern_get_status+0x12/0x20 > [ 195.016823] vp_vdpa_reset+0x1b/0x50 [vp_vdpa] > [ 195.017238] virtio_vdpa_reset+0x3c/0x48 [virtio_vdpa] > [ 195.017709] remove_vq_common+0x1f/0x3a0 [virtio_net] > [ 195.018178] virtnet_remove+0x5d/0x70 [virtio_net] > [ 195.018618] virtio_dev_remove+0x3d/0x90 > [ 195.018986] device_release_driver_internal+0x1aa/0x230 > [ 195.019466] bus_remove_device+0xd8/0x150 > [ 195.019841] device_del+0x18b/0x3f0 > [ 195.020167] ? kernfs_find_ns+0x35/0xd0 > [ 195.020526] device_unregister+0x13/0x60 > [ 195.020894] unregister_virtio_device+0x11/0x20 > [ 195.021311] device_release_driver_internal+0x1aa/0x230 > [ 195.021790] bus_remove_device+0xd8/0x150 > [ 195.022162] device_del+0x18b/0x3f0 > [ 195.022487] device_unregister+0x13/0x60 > [ 195.022852] ? vdpa_dev_remove+0x30/0x30 [vdpa] > [ 195.023270] vp_vdpa_dev_del+0x12/0x20 [vp_vdpa] > [ 195.023694] vdpa_match_remove+0x2b/0x40 [vdpa] > [ 195.024115] bus_for_each_dev+0x78/0xc0 > [ 195.024471] vdpa_mgmtdev_unregister+0x65/0x80 [vdpa] > [ 195.024937] vp_vdpa_remove+0x23/0x40 [vp_vdpa] > [ 195.025353] pci_device_remove+0x36/0xa0 > [ 195.025719] device_release_driver_internal+0x1aa/0x230 > [ 195.026201] pci_stop_bus_device+0x6c/0x90 > [ 195.026580] pci_stop_and_remove_bus_device+0xe/0x20 > [ 195.027039] disable_slot+0x49/0x90 > [ 195.027366] acpiphp_disable_and_eject_slot+0x15/0x90 > [ 195.027832] hotplug_event+0xea/0x210 > [ 195.028171] ? hotplug_event+0x210/0x210 > [ 195.028535] acpiphp_hotplug_notify+0x22/0x80 > [ 195.028942] ? hotplug_event+0x210/0x210 > [ 195.029303] acpi_device_hotplug+0x8a/0x1d0 > [ 195.029690] acpi_hotplug_work_fn+0x1a/0x30 > [ 195.030077] process_one_work+0x1e8/0x3c0 > [ 195.030451] worker_thread+0x50/0x3b0 > [ 195.030791] ? rescuer_thread+0x3a0/0x3a0 > [ 195.031165] kthread+0xd9/0x100 > [ 195.031459] ? kthread_complete_and_exit+0x20/0x20 > [ 195.031899] ret_from_fork+0x22/0x30 > [ 195.032233] </TASK> > > Fixes: ffbda8e9df10 ("vdpa/vp_vdpa : add vdpa tool support in vp_vdpa") > Tested-by: Lei Yang <leiyang@redhat.com> > Cc: stable@vger.kernel.org > Signed-off-by: Cindy Lu <lulu@redhat.com> Other than above, Acked-by: Jason Wang <jasowang@redhat.com> Thanks > --- > drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/vdpa/virtio_pci/vp_vdpa.c b/drivers/vdpa/virtio_pci/vp_vdpa.c > index 8fe267ca3e76..281287fae89f 100644 > --- a/drivers/vdpa/virtio_pci/vp_vdpa.c > +++ b/drivers/vdpa/virtio_pci/vp_vdpa.c > @@ -645,8 +645,8 @@ static void vp_vdpa_remove(struct pci_dev *pdev) > struct virtio_pci_modern_device *mdev = NULL; > > mdev = vp_vdpa_mgtdev->mdev; > - vp_modern_remove(mdev); > vdpa_mgmtdev_unregister(&vp_vdpa_mgtdev->mgtdev); > + vp_modern_remove(mdev); > kfree(vp_vdpa_mgtdev->mgtdev.id_table); > kfree(mdev); > kfree(vp_vdpa_mgtdev); > -- > 2.34.3 >
On Tue, 14 Feb 2023 at 14:24, Jason Wang <jasowang@redhat.com> wrote: > > On Tue, Feb 14, 2023 at 2:17 PM Cindy Lu <lulu@redhat.com> wrote: > > > > While unplugging the vp_vdpa device, the kernel will crash > > The root cause is the function vp_modern_get_status() called following the > > vp_modern_remove(). > > This needs some tweaking, maybe it's better to say > vdpa_mgmtdev_unregister() will access modern devices which will cause > a use after free. > > >So need to change the sequence in vp_vdpa_remove > > > > [ 195.016001] Call Trace: > > Let's paste the full log with the reason for the calltrace (e.g > general protection fault or whatever else). > sure, will post a new version Thanks Cindy > > [ 195.016233] <TASK> > > [ 195.016434] vp_modern_get_status+0x12/0x20 > > [ 195.016823] vp_vdpa_reset+0x1b/0x50 [vp_vdpa] > > [ 195.017238] virtio_vdpa_reset+0x3c/0x48 [virtio_vdpa] > > [ 195.017709] remove_vq_common+0x1f/0x3a0 [virtio_net] > > [ 195.018178] virtnet_remove+0x5d/0x70 [virtio_net] > > [ 195.018618] virtio_dev_remove+0x3d/0x90 > > [ 195.018986] device_release_driver_internal+0x1aa/0x230 > > [ 195.019466] bus_remove_device+0xd8/0x150 > > [ 195.019841] device_del+0x18b/0x3f0 > > [ 195.020167] ? kernfs_find_ns+0x35/0xd0 > > [ 195.020526] device_unregister+0x13/0x60 > > [ 195.020894] unregister_virtio_device+0x11/0x20 > > [ 195.021311] device_release_driver_internal+0x1aa/0x230 > > [ 195.021790] bus_remove_device+0xd8/0x150 > > [ 195.022162] device_del+0x18b/0x3f0 > > [ 195.022487] device_unregister+0x13/0x60 > > [ 195.022852] ? vdpa_dev_remove+0x30/0x30 [vdpa] > > [ 195.023270] vp_vdpa_dev_del+0x12/0x20 [vp_vdpa] > > [ 195.023694] vdpa_match_remove+0x2b/0x40 [vdpa] > > [ 195.024115] bus_for_each_dev+0x78/0xc0 > > [ 195.024471] vdpa_mgmtdev_unregister+0x65/0x80 [vdpa] > > [ 195.024937] vp_vdpa_remove+0x23/0x40 [vp_vdpa] > > [ 195.025353] pci_device_remove+0x36/0xa0 > > [ 195.025719] device_release_driver_internal+0x1aa/0x230 > > [ 195.026201] pci_stop_bus_device+0x6c/0x90 > > [ 195.026580] pci_stop_and_remove_bus_device+0xe/0x20 > > [ 195.027039] disable_slot+0x49/0x90 > > [ 195.027366] acpiphp_disable_and_eject_slot+0x15/0x90 > > [ 195.027832] hotplug_event+0xea/0x210 > > [ 195.028171] ? hotplug_event+0x210/0x210 > > [ 195.028535] acpiphp_hotplug_notify+0x22/0x80 > > [ 195.028942] ? hotplug_event+0x210/0x210 > > [ 195.029303] acpi_device_hotplug+0x8a/0x1d0 > > [ 195.029690] acpi_hotplug_work_fn+0x1a/0x30 > > [ 195.030077] process_one_work+0x1e8/0x3c0 > > [ 195.030451] worker_thread+0x50/0x3b0 > > [ 195.030791] ? rescuer_thread+0x3a0/0x3a0 > > [ 195.031165] kthread+0xd9/0x100 > > [ 195.031459] ? kthread_complete_and_exit+0x20/0x20 > > [ 195.031899] ret_from_fork+0x22/0x30 > > [ 195.032233] </TASK> > > > > Fixes: ffbda8e9df10 ("vdpa/vp_vdpa : add vdpa tool support in vp_vdpa") > > Tested-by: Lei Yang <leiyang@redhat.com> > > Cc: stable@vger.kernel.org > > Signed-off-by: Cindy Lu <lulu@redhat.com> > > Other than above, > > Acked-by: Jason Wang <jasowang@redhat.com> > > Thanks > > > --- > > drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/vdpa/virtio_pci/vp_vdpa.c b/drivers/vdpa/virtio_pci/vp_vdpa.c > > index 8fe267ca3e76..281287fae89f 100644 > > --- a/drivers/vdpa/virtio_pci/vp_vdpa.c > > +++ b/drivers/vdpa/virtio_pci/vp_vdpa.c > > @@ -645,8 +645,8 @@ static void vp_vdpa_remove(struct pci_dev *pdev) > > struct virtio_pci_modern_device *mdev = NULL; > > > > mdev = vp_vdpa_mgtdev->mdev; > > - vp_modern_remove(mdev); > > vdpa_mgmtdev_unregister(&vp_vdpa_mgtdev->mgtdev); > > + vp_modern_remove(mdev); > > kfree(vp_vdpa_mgtdev->mgtdev.id_table); > > kfree(mdev); > > kfree(vp_vdpa_mgtdev); > > -- > > 2.34.3 > > >
diff --git a/drivers/vdpa/virtio_pci/vp_vdpa.c b/drivers/vdpa/virtio_pci/vp_vdpa.c index 8fe267ca3e76..281287fae89f 100644 --- a/drivers/vdpa/virtio_pci/vp_vdpa.c +++ b/drivers/vdpa/virtio_pci/vp_vdpa.c @@ -645,8 +645,8 @@ static void vp_vdpa_remove(struct pci_dev *pdev) struct virtio_pci_modern_device *mdev = NULL; mdev = vp_vdpa_mgtdev->mdev; - vp_modern_remove(mdev); vdpa_mgmtdev_unregister(&vp_vdpa_mgtdev->mgtdev); + vp_modern_remove(mdev); kfree(vp_vdpa_mgtdev->mgtdev.id_table); kfree(mdev); kfree(vp_vdpa_mgtdev);
While unplugging the vp_vdpa device, the kernel will crash The root cause is the function vp_modern_get_status() called following the vp_modern_remove(). So need to change the sequence in vp_vdpa_remove [ 195.016001] Call Trace: [ 195.016233] <TASK> [ 195.016434] vp_modern_get_status+0x12/0x20 [ 195.016823] vp_vdpa_reset+0x1b/0x50 [vp_vdpa] [ 195.017238] virtio_vdpa_reset+0x3c/0x48 [virtio_vdpa] [ 195.017709] remove_vq_common+0x1f/0x3a0 [virtio_net] [ 195.018178] virtnet_remove+0x5d/0x70 [virtio_net] [ 195.018618] virtio_dev_remove+0x3d/0x90 [ 195.018986] device_release_driver_internal+0x1aa/0x230 [ 195.019466] bus_remove_device+0xd8/0x150 [ 195.019841] device_del+0x18b/0x3f0 [ 195.020167] ? kernfs_find_ns+0x35/0xd0 [ 195.020526] device_unregister+0x13/0x60 [ 195.020894] unregister_virtio_device+0x11/0x20 [ 195.021311] device_release_driver_internal+0x1aa/0x230 [ 195.021790] bus_remove_device+0xd8/0x150 [ 195.022162] device_del+0x18b/0x3f0 [ 195.022487] device_unregister+0x13/0x60 [ 195.022852] ? vdpa_dev_remove+0x30/0x30 [vdpa] [ 195.023270] vp_vdpa_dev_del+0x12/0x20 [vp_vdpa] [ 195.023694] vdpa_match_remove+0x2b/0x40 [vdpa] [ 195.024115] bus_for_each_dev+0x78/0xc0 [ 195.024471] vdpa_mgmtdev_unregister+0x65/0x80 [vdpa] [ 195.024937] vp_vdpa_remove+0x23/0x40 [vp_vdpa] [ 195.025353] pci_device_remove+0x36/0xa0 [ 195.025719] device_release_driver_internal+0x1aa/0x230 [ 195.026201] pci_stop_bus_device+0x6c/0x90 [ 195.026580] pci_stop_and_remove_bus_device+0xe/0x20 [ 195.027039] disable_slot+0x49/0x90 [ 195.027366] acpiphp_disable_and_eject_slot+0x15/0x90 [ 195.027832] hotplug_event+0xea/0x210 [ 195.028171] ? hotplug_event+0x210/0x210 [ 195.028535] acpiphp_hotplug_notify+0x22/0x80 [ 195.028942] ? hotplug_event+0x210/0x210 [ 195.029303] acpi_device_hotplug+0x8a/0x1d0 [ 195.029690] acpi_hotplug_work_fn+0x1a/0x30 [ 195.030077] process_one_work+0x1e8/0x3c0 [ 195.030451] worker_thread+0x50/0x3b0 [ 195.030791] ? rescuer_thread+0x3a0/0x3a0 [ 195.031165] kthread+0xd9/0x100 [ 195.031459] ? kthread_complete_and_exit+0x20/0x20 [ 195.031899] ret_from_fork+0x22/0x30 [ 195.032233] </TASK> Fixes: ffbda8e9df10 ("vdpa/vp_vdpa : add vdpa tool support in vp_vdpa") Tested-by: Lei Yang <leiyang@redhat.com> Cc: stable@vger.kernel.org Signed-off-by: Cindy Lu <lulu@redhat.com> --- drivers/vdpa/virtio_pci/vp_vdpa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)