| Message ID | 50F52677.80100@cn.fujitsu.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
Wanlong Gao <gaowanlong@cn.fujitsu.com> writes: > Add hot cpu notifier to reset the request virtqueue affinity > when doing cpu hotplug. You need to be careful to get_online_cpus() and put_online_cpus() here, so CPUs can't go up and down in the middle of operations. In particular, get_online_cpus()/put_online_cpus() around calls to virtscsi_set_affinity() (except within notifiers). > +static int virtscsi_cpu_callback(struct notifier_block *nfb, > + unsigned long action, void *hcpu) > +{ > + struct virtio_scsi *vscsi = container_of(nfb, struct virtio_scsi, nb); > + switch(action) { > + case CPU_ONLINE: > + case CPU_ONLINE_FROZEN: > + case CPU_DEAD: > + case CPU_DEAD_FROZEN: > + virtscsi_set_affinity(vscsi, true); > + break; > + default: > + break; > + } > + return NOTIFY_OK; > +} You probably want to remove affinities *before* the CPU goes down, and restore it after the CPU comes up. So, how about: switch (action & ~CPU_TASKS_FROZEN) { case CPU_ONLINE: case CPU_DOWN_FAILED: virtscsi_set_affinity(vscsi, true, -1); break; case CPU_DOWN_PREPARE: virtscsi_set_affinity(vscsi, true, (unsigned long)hcpu); break; } The extra argument to virtscsi_set_affinity() is to tell it to ignore a cpu which seems online (because it's going down). > static void virtscsi_init_vq(struct virtio_scsi_vq *virtscsi_vq, > struct virtqueue *vq) > { > @@ -888,6 +909,13 @@ static int __devinit virtscsi_probe(struct virtio_device *vdev) > if (err) > goto virtscsi_init_failed; > > + vscsi->nb.notifier_call = &virtscsi_cpu_callback; > + err = register_hotcpu_notifier(&vscsi->nb); > + if (err) { > + pr_err("virtio_scsi: registering cpu notifier failed\n"); > + goto scsi_add_host_failed; > + } > + You have to clean this up if scsi_add_host() fails, I think. Cheers, Rusty. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 01/16/2013 11:31 AM, Rusty Russell wrote: > Wanlong Gao <gaowanlong@cn.fujitsu.com> writes: >> Add hot cpu notifier to reset the request virtqueue affinity >> when doing cpu hotplug. > > You need to be careful to get_online_cpus() and put_online_cpus() here, > so CPUs can't go up and down in the middle of operations. > > In particular, get_online_cpus()/put_online_cpus() around calls to > virtscsi_set_affinity() (except within notifiers). Yes, I'll take care of this, thank you. > >> +static int virtscsi_cpu_callback(struct notifier_block *nfb, >> + unsigned long action, void *hcpu) >> +{ >> + struct virtio_scsi *vscsi = container_of(nfb, struct virtio_scsi, nb); >> + switch(action) { >> + case CPU_ONLINE: >> + case CPU_ONLINE_FROZEN: >> + case CPU_DEAD: >> + case CPU_DEAD_FROZEN: >> + virtscsi_set_affinity(vscsi, true); >> + break; >> + default: >> + break; >> + } >> + return NOTIFY_OK; >> +} > > You probably want to remove affinities *before* the CPU goes down, and > restore it after the CPU comes up. > > So, how about: > > switch (action & ~CPU_TASKS_FROZEN) { > case CPU_ONLINE: > case CPU_DOWN_FAILED: > virtscsi_set_affinity(vscsi, true, -1); > break; > case CPU_DOWN_PREPARE: > virtscsi_set_affinity(vscsi, true, (unsigned long)hcpu); > break; > } > > The extra argument to virtscsi_set_affinity() is to tell it to ignore a > cpu which seems online (because it's going down). OK, thank you very much for teaching this. > >> static void virtscsi_init_vq(struct virtio_scsi_vq *virtscsi_vq, >> struct virtqueue *vq) >> { >> @@ -888,6 +909,13 @@ static int __devinit virtscsi_probe(struct virtio_device *vdev) >> if (err) >> goto virtscsi_init_failed; >> >> + vscsi->nb.notifier_call = &virtscsi_cpu_callback; >> + err = register_hotcpu_notifier(&vscsi->nb); >> + if (err) { >> + pr_err("virtio_scsi: registering cpu notifier failed\n"); >> + goto scsi_add_host_failed; >> + } >> + > > You have to clean this up if scsi_add_host() fails, I think. Yeah, will do, thank you very much. Regards, Wanlong Gao > > Cheers, > Rusty. > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Il 16/01/2013 04:55, Wanlong Gao ha scritto: >>> >> Add hot cpu notifier to reset the request virtqueue affinity >>> >> when doing cpu hotplug. >> > >> > You need to be careful to get_online_cpus() and put_online_cpus() here, >> > so CPUs can't go up and down in the middle of operations. >> > >> > In particular, get_online_cpus()/put_online_cpus() around calls to >> > virtscsi_set_affinity() (except within notifiers). > Yes, I'll take care of this, thank you. > I squashed patch 1 (plus changes to get/put_online_cpus) in my multiqueue series, and applied this one as a separate patch. Paolo -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c index 16b0ef2..a3b5f12 100644 --- a/drivers/scsi/virtio_scsi.c +++ b/drivers/scsi/virtio_scsi.c @@ -20,6 +20,7 @@ #include <linux/virtio_ids.h> #include <linux/virtio_config.h> #include <linux/virtio_scsi.h> +#include <linux/cpu.h> #include <scsi/scsi_host.h> #include <scsi/scsi_device.h> #include <scsi/scsi_cmnd.h> @@ -109,6 +110,9 @@ struct virtio_scsi { /* Does the affinity hint is set for virtqueues? */ bool affinity_hint_set; + /* CPU hotplug notifier */ + struct notifier_block nb; + struct virtio_scsi_vq ctrl_vq; struct virtio_scsi_vq event_vq; struct virtio_scsi_vq req_vqs[]; @@ -738,6 +742,23 @@ static void virtscsi_set_affinity(struct virtio_scsi *vscsi, bool affinity) } } +static int virtscsi_cpu_callback(struct notifier_block *nfb, + unsigned long action, void *hcpu) +{ + struct virtio_scsi *vscsi = container_of(nfb, struct virtio_scsi, nb); + switch(action) { + case CPU_ONLINE: + case CPU_ONLINE_FROZEN: + case CPU_DEAD: + case CPU_DEAD_FROZEN: + virtscsi_set_affinity(vscsi, true); + break; + default: + break; + } + return NOTIFY_OK; +} + static void virtscsi_init_vq(struct virtio_scsi_vq *virtscsi_vq, struct virtqueue *vq) { @@ -888,6 +909,13 @@ static int __devinit virtscsi_probe(struct virtio_device *vdev) if (err) goto virtscsi_init_failed; + vscsi->nb.notifier_call = &virtscsi_cpu_callback; + err = register_hotcpu_notifier(&vscsi->nb); + if (err) { + pr_err("virtio_scsi: registering cpu notifier failed\n"); + goto scsi_add_host_failed; + } + cmd_per_lun = virtscsi_config_get(vdev, cmd_per_lun) ?: 1; shost->cmd_per_lun = min_t(u32, cmd_per_lun, shost->can_queue); shost->max_sectors = virtscsi_config_get(vdev, max_sectors) ?: 0xFFFF; @@ -925,6 +953,8 @@ static void __devexit virtscsi_remove(struct virtio_device *vdev) scsi_remove_host(shost); + unregister_hotcpu_notifier(&vscsi->nb); + virtscsi_remove_vqs(vdev); scsi_host_put(shost); }
Add hot cpu notifier to reset the request virtqueue affinity when doing cpu hotplug. Signed-off-by: Wanlong Gao <gaowanlong@cn.fujitsu.com> --- drivers/scsi/virtio_scsi.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)