| Message ID | 20240327082128.942818-1-wenjian1@xiaomi.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 730fffce4fd2eb7a0be2d0b6cd7e55e9194d76d5 |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next,v2] devlink: use kvzalloc() to allocate devlink instance resources | expand |
From: Jian Wen <wenjianhn@gmail.com> Date: Wed, 27 Mar 2024 16:21:28 +0800 > During live migration of a virtual machine, the SR-IOV VF need to be > re-registered. It may fail when the memory is badly fragmented. > > The related log is as follows. > > Mar 1 18:54:12 kernel: hv_netvsc 6045bdaa-c0d1-6045-bdaa-c0d16045bdaa eth0: VF slot 1 added > ... > Mar 1 18:54:13 kernel: kworker/0:0: page allocation failure: order:7, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 > Mar 1 18:54:13 kernel: CPU: 0 PID: 24006 Comm: kworker/0:0 Tainted: G E 5.4...x86_64 #1 > Mar 1 18:54:13 kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 > Mar 1 18:54:13 kernel: Workqueue: events work_for_cpu_fn > Mar 1 18:54:13 kernel: Call Trace: > Mar 1 18:54:13 kernel: dump_stack+0x8b/0xc8 > Mar 1 18:54:13 kernel: warn_alloc+0xff/0x170 > Mar 1 18:54:13 kernel: __alloc_pages_slowpath+0x92c/0xb2b > Mar 1 18:54:13 kernel: ? get_page_from_freelist+0x1d4/0x1140 > Mar 1 18:54:13 kernel: __alloc_pages_nodemask+0x2f9/0x320 > Mar 1 18:54:13 kernel: alloc_pages_current+0x6a/0xb0 > Mar 1 18:54:13 kernel: kmalloc_order+0x1e/0x70 > Mar 1 18:54:13 kernel: kmalloc_order_trace+0x26/0xb0 > Mar 1 18:54:13 kernel: ? __switch_to_asm+0x34/0x70 > Mar 1 18:54:13 kernel: __kmalloc+0x276/0x280 > Mar 1 18:54:13 kernel: ? _raw_spin_unlock_irqrestore+0x1e/0x40 > Mar 1 18:54:13 kernel: devlink_alloc+0x29/0x110 > Mar 1 18:54:13 kernel: mlx5_devlink_alloc+0x1a/0x20 [mlx5_core] > Mar 1 18:54:13 kernel: init_one+0x1d/0x650 [mlx5_core] > Mar 1 18:54:13 kernel: local_pci_probe+0x46/0x90 > Mar 1 18:54:13 kernel: work_for_cpu_fn+0x1a/0x30 > Mar 1 18:54:13 kernel: process_one_work+0x16d/0x390 > Mar 1 18:54:13 kernel: worker_thread+0x1d3/0x3f0 > Mar 1 18:54:13 kernel: kthread+0x105/0x140 > Mar 1 18:54:13 kernel: ? max_active_store+0x80/0x80 > Mar 1 18:54:13 kernel: ? kthread_bind+0x20/0x20 > Mar 1 18:54:13 kernel: ret_from_fork+0x3a/0x50 > > Changes since v1: > - Use struct_size(devlink, priv, priv_size) as suggested by Alexander Lobakin > > Signed-off-by: Jian Wen <wenjian1@xiaomi.com> Since it actually fixes a bug splat, you may want to send it with prefix "net" instead of "net-next" and add a "Fixes:" tag here blaming the first commit which added Devlink instance allocation. Let's see what others think. > --- > net/devlink/core.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/net/devlink/core.c b/net/devlink/core.c > index 7f0b093208d7..f49cd83f1955 100644 > --- a/net/devlink/core.c > +++ b/net/devlink/core.c > @@ -314,7 +314,7 @@ static void devlink_release(struct work_struct *work) > mutex_destroy(&devlink->lock); > lockdep_unregister_key(&devlink->lock_key); > put_device(devlink->dev); > - kfree(devlink); > + kvfree(devlink); > } > > void devlink_put(struct devlink *devlink) > @@ -420,7 +420,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, > if (!devlink_reload_actions_valid(ops)) > return NULL; > > - devlink = kzalloc(sizeof(*devlink) + priv_size, GFP_KERNEL); > + devlink = kvzalloc(struct_size(devlink, priv, priv_size), GFP_KERNEL); > if (!devlink) > return NULL; > > @@ -455,7 +455,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, > return devlink; > > err_xa_alloc: > - kfree(devlink); > + kvfree(devlink); > return NULL; > } > EXPORT_SYMBOL_GPL(devlink_alloc_ns); Thanks, Olek
On Wed, Mar 27, 2024 at 6:15 PM Alexander Lobakin <aleksander.lobakin@intel.com> wrote: > > From: Jian Wen <wenjianhn@gmail.com> > Date: Wed, 27 Mar 2024 16:21:28 +0800 > > > During live migration of a virtual machine, the SR-IOV VF need to be > > re-registered. It may fail when the memory is badly fragmented. > > > > The related log is as follows. > > > > Mar 1 18:54:12 kernel: hv_netvsc 6045bdaa-c0d1-6045-bdaa-c0d16045bdaa eth0: VF slot 1 added > > ... > > Mar 1 18:54:13 kernel: kworker/0:0: page allocation failure: order:7, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 > > Mar 1 18:54:13 kernel: CPU: 0 PID: 24006 Comm: kworker/0:0 Tainted: G E 5.4...x86_64 #1 > > Mar 1 18:54:13 kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 > > Mar 1 18:54:13 kernel: Workqueue: events work_for_cpu_fn > > Mar 1 18:54:13 kernel: Call Trace: > > Mar 1 18:54:13 kernel: dump_stack+0x8b/0xc8 > > Mar 1 18:54:13 kernel: warn_alloc+0xff/0x170 > > Mar 1 18:54:13 kernel: __alloc_pages_slowpath+0x92c/0xb2b > > Mar 1 18:54:13 kernel: ? get_page_from_freelist+0x1d4/0x1140 > > Mar 1 18:54:13 kernel: __alloc_pages_nodemask+0x2f9/0x320 > > Mar 1 18:54:13 kernel: alloc_pages_current+0x6a/0xb0 > > Mar 1 18:54:13 kernel: kmalloc_order+0x1e/0x70 > > Mar 1 18:54:13 kernel: kmalloc_order_trace+0x26/0xb0 > > Mar 1 18:54:13 kernel: ? __switch_to_asm+0x34/0x70 > > Mar 1 18:54:13 kernel: __kmalloc+0x276/0x280 > > Mar 1 18:54:13 kernel: ? _raw_spin_unlock_irqrestore+0x1e/0x40 > > Mar 1 18:54:13 kernel: devlink_alloc+0x29/0x110 > > Mar 1 18:54:13 kernel: mlx5_devlink_alloc+0x1a/0x20 [mlx5_core] > > Mar 1 18:54:13 kernel: init_one+0x1d/0x650 [mlx5_core] > > Mar 1 18:54:13 kernel: local_pci_probe+0x46/0x90 > > Mar 1 18:54:13 kernel: work_for_cpu_fn+0x1a/0x30 > > Mar 1 18:54:13 kernel: process_one_work+0x16d/0x390 > > Mar 1 18:54:13 kernel: worker_thread+0x1d3/0x3f0 > > Mar 1 18:54:13 kernel: kthread+0x105/0x140 > > Mar 1 18:54:13 kernel: ? max_active_store+0x80/0x80 > > Mar 1 18:54:13 kernel: ? kthread_bind+0x20/0x20 > > Mar 1 18:54:13 kernel: ret_from_fork+0x3a/0x50 > > > > Changes since v1: > > - Use struct_size(devlink, priv, priv_size) as suggested by Alexander Lobakin > > > > Signed-off-by: Jian Wen <wenjian1@xiaomi.com> > > Since it actually fixes a bug splat, you may want to send it with prefix > "net" instead of "net-next" and add a "Fixes:" tag here blaming the > first commit which added Devlink instance allocation. Let's see what > others think. Many commits that replace kzalloc() with kvzalloc() don't include the "Fixes:'' tag. Jiri, what do you think? > > > --- > > net/devlink/core.c | 6 +++--- > > 1 file changed, 3 insertions(+), 3 deletions(-) > > > > diff --git a/net/devlink/core.c b/net/devlink/core.c > > index 7f0b093208d7..f49cd83f1955 100644 > > --- a/net/devlink/core.c > > +++ b/net/devlink/core.c > > @@ -314,7 +314,7 @@ static void devlink_release(struct work_struct *work) > > mutex_destroy(&devlink->lock); > > lockdep_unregister_key(&devlink->lock_key); > > put_device(devlink->dev); > > - kfree(devlink); > > + kvfree(devlink); > > } > > > > void devlink_put(struct devlink *devlink) > > @@ -420,7 +420,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, > > if (!devlink_reload_actions_valid(ops)) > > return NULL; > > > > - devlink = kzalloc(sizeof(*devlink) + priv_size, GFP_KERNEL); > > + devlink = kvzalloc(struct_size(devlink, priv, priv_size), GFP_KERNEL); > > if (!devlink) > > return NULL; > > > > @@ -455,7 +455,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, > > return devlink; > > > > err_xa_alloc: > > - kfree(devlink); > > + kvfree(devlink); > > return NULL; > > } > > EXPORT_SYMBOL_GPL(devlink_alloc_ns); > > Thanks, > Olek
On Wed, 27 Mar 2024 11:15:08 +0100 Alexander Lobakin wrote: > > Changes since v1: > > - Use struct_size(devlink, priv, priv_size) as suggested by Alexander Lobakin The change log should go under --- and would be great to include a link to the previous version (from lore) > > Signed-off-by: Jian Wen <wenjian1@xiaomi.com> > > Since it actually fixes a bug splat, you may want to send it with prefix > "net" instead of "net-next" and add a "Fixes:" tag here blaming the > first commit which added Devlink instance allocation. Let's see what > others think. That's my initial reaction as well. We often treat memory pressure improvements as fixes. But thinking more we would need to check if any of the drivers puts a DMA buffer in its priv. Some FW mailbox, maybe? It's possible.
Hello: This patch was applied to netdev/net-next.git (main) by Jakub Kicinski <kuba@kernel.org>: On Wed, 27 Mar 2024 16:21:28 +0800 you wrote: > During live migration of a virtual machine, the SR-IOV VF need to be > re-registered. It may fail when the memory is badly fragmented. > > The related log is as follows. > > Mar 1 18:54:12 kernel: hv_netvsc 6045bdaa-c0d1-6045-bdaa-c0d16045bdaa eth0: VF slot 1 added > ... > Mar 1 18:54:13 kernel: kworker/0:0: page allocation failure: order:7, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 > Mar 1 18:54:13 kernel: CPU: 0 PID: 24006 Comm: kworker/0:0 Tainted: G E 5.4...x86_64 #1 > Mar 1 18:54:13 kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 > Mar 1 18:54:13 kernel: Workqueue: events work_for_cpu_fn > Mar 1 18:54:13 kernel: Call Trace: > Mar 1 18:54:13 kernel: dump_stack+0x8b/0xc8 > Mar 1 18:54:13 kernel: warn_alloc+0xff/0x170 > Mar 1 18:54:13 kernel: __alloc_pages_slowpath+0x92c/0xb2b > Mar 1 18:54:13 kernel: ? get_page_from_freelist+0x1d4/0x1140 > Mar 1 18:54:13 kernel: __alloc_pages_nodemask+0x2f9/0x320 > Mar 1 18:54:13 kernel: alloc_pages_current+0x6a/0xb0 > Mar 1 18:54:13 kernel: kmalloc_order+0x1e/0x70 > Mar 1 18:54:13 kernel: kmalloc_order_trace+0x26/0xb0 > Mar 1 18:54:13 kernel: ? __switch_to_asm+0x34/0x70 > Mar 1 18:54:13 kernel: __kmalloc+0x276/0x280 > Mar 1 18:54:13 kernel: ? _raw_spin_unlock_irqrestore+0x1e/0x40 > Mar 1 18:54:13 kernel: devlink_alloc+0x29/0x110 > Mar 1 18:54:13 kernel: mlx5_devlink_alloc+0x1a/0x20 [mlx5_core] > Mar 1 18:54:13 kernel: init_one+0x1d/0x650 [mlx5_core] > Mar 1 18:54:13 kernel: local_pci_probe+0x46/0x90 > Mar 1 18:54:13 kernel: work_for_cpu_fn+0x1a/0x30 > Mar 1 18:54:13 kernel: process_one_work+0x16d/0x390 > Mar 1 18:54:13 kernel: worker_thread+0x1d3/0x3f0 > Mar 1 18:54:13 kernel: kthread+0x105/0x140 > Mar 1 18:54:13 kernel: ? max_active_store+0x80/0x80 > Mar 1 18:54:13 kernel: ? kthread_bind+0x20/0x20 > Mar 1 18:54:13 kernel: ret_from_fork+0x3a/0x50 > > [...] Here is the summary with links: - [net-next,v2] devlink: use kvzalloc() to allocate devlink instance resources https://git.kernel.org/netdev/net-next/c/730fffce4fd2 You are awesome, thank you!
diff --git a/net/devlink/core.c b/net/devlink/core.c index 7f0b093208d7..f49cd83f1955 100644 --- a/net/devlink/core.c +++ b/net/devlink/core.c @@ -314,7 +314,7 @@ static void devlink_release(struct work_struct *work) mutex_destroy(&devlink->lock); lockdep_unregister_key(&devlink->lock_key); put_device(devlink->dev); - kfree(devlink); + kvfree(devlink); } void devlink_put(struct devlink *devlink) @@ -420,7 +420,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, if (!devlink_reload_actions_valid(ops)) return NULL; - devlink = kzalloc(sizeof(*devlink) + priv_size, GFP_KERNEL); + devlink = kvzalloc(struct_size(devlink, priv, priv_size), GFP_KERNEL); if (!devlink) return NULL; @@ -455,7 +455,7 @@ struct devlink *devlink_alloc_ns(const struct devlink_ops *ops, return devlink; err_xa_alloc: - kfree(devlink); + kvfree(devlink); return NULL; } EXPORT_SYMBOL_GPL(devlink_alloc_ns);
During live migration of a virtual machine, the SR-IOV VF need to be re-registered. It may fail when the memory is badly fragmented. The related log is as follows. Mar 1 18:54:12 kernel: hv_netvsc 6045bdaa-c0d1-6045-bdaa-c0d16045bdaa eth0: VF slot 1 added ... Mar 1 18:54:13 kernel: kworker/0:0: page allocation failure: order:7, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null),cpuset=/,mems_allowed=0 Mar 1 18:54:13 kernel: CPU: 0 PID: 24006 Comm: kworker/0:0 Tainted: G E 5.4...x86_64 #1 Mar 1 18:54:13 kernel: Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 Mar 1 18:54:13 kernel: Workqueue: events work_for_cpu_fn Mar 1 18:54:13 kernel: Call Trace: Mar 1 18:54:13 kernel: dump_stack+0x8b/0xc8 Mar 1 18:54:13 kernel: warn_alloc+0xff/0x170 Mar 1 18:54:13 kernel: __alloc_pages_slowpath+0x92c/0xb2b Mar 1 18:54:13 kernel: ? get_page_from_freelist+0x1d4/0x1140 Mar 1 18:54:13 kernel: __alloc_pages_nodemask+0x2f9/0x320 Mar 1 18:54:13 kernel: alloc_pages_current+0x6a/0xb0 Mar 1 18:54:13 kernel: kmalloc_order+0x1e/0x70 Mar 1 18:54:13 kernel: kmalloc_order_trace+0x26/0xb0 Mar 1 18:54:13 kernel: ? __switch_to_asm+0x34/0x70 Mar 1 18:54:13 kernel: __kmalloc+0x276/0x280 Mar 1 18:54:13 kernel: ? _raw_spin_unlock_irqrestore+0x1e/0x40 Mar 1 18:54:13 kernel: devlink_alloc+0x29/0x110 Mar 1 18:54:13 kernel: mlx5_devlink_alloc+0x1a/0x20 [mlx5_core] Mar 1 18:54:13 kernel: init_one+0x1d/0x650 [mlx5_core] Mar 1 18:54:13 kernel: local_pci_probe+0x46/0x90 Mar 1 18:54:13 kernel: work_for_cpu_fn+0x1a/0x30 Mar 1 18:54:13 kernel: process_one_work+0x16d/0x390 Mar 1 18:54:13 kernel: worker_thread+0x1d3/0x3f0 Mar 1 18:54:13 kernel: kthread+0x105/0x140 Mar 1 18:54:13 kernel: ? max_active_store+0x80/0x80 Mar 1 18:54:13 kernel: ? kthread_bind+0x20/0x20 Mar 1 18:54:13 kernel: ret_from_fork+0x3a/0x50 Changes since v1: - Use struct_size(devlink, priv, priv_size) as suggested by Alexander Lobakin Signed-off-by: Jian Wen <wenjian1@xiaomi.com> --- net/devlink/core.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)