| Message ID | 20200220150103.5183-1-jsavitz@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | mm/page_alloc: increase default min_free_kbytes bound | expand |
On Thu, 20 Feb 2020 10:01:03 -0500 Joel Savitz <jsavitz@redhat.com> wrote: > > Currently, the vm.min_free_kbytes sysctl value is capped at a hardcoded > 64M in init_per_zone_wmark_min (unless it is overridden by khugepaged > initialization). > > This value has not been modified since 2005, and enterprise-grade > systems now frequently have hundreds of GB of RAM and multiple 10, 40, > or even 100 GB NICs. We have seen page allocation failures on heavily > loaded systems related to NIC drivers. These issues were resolved by an > increase to vm.min_free_kbytes. > > This patch increases the hardcoded value by a factor of 4 as a temporary > solution. OK, better than nothing I guess. > Further work to make the calculation of vm.min_free_kbytes more > consistent throughout the kernel would be desirable. > > As an example of the inconsistency of the current method, this value is > recalculated by init_per_zone_wmark_min() in the case of memory hotplug > which will override the value set by set_recommended_min_free_kbytes() > called during khugepaged initialization even if khugepaged remains > enabled, however an on/off toggle of khugepaged will then recalculate > and set the value via set_recommended_min_free_kbytes(). > Yup, these hardcoded numbers are lame.
On 2/20/20 7:01 AM, Joel Savitz wrote: > > Further work to make the calculation of vm.min_free_kbytes more > consistent throughout the kernel would be desirable. > > As an example of the inconsistency of the current method, this value is > recalculated by init_per_zone_wmark_min() in the case of memory hotplug > which will override the value set by set_recommended_min_free_kbytes() > called during khugepaged initialization even if khugepaged remains > enabled, however an on/off toggle of khugepaged will then recalculate > and set the value via set_recommended_min_free_kbytes(). I took a shot at fixing some of those inconsistencies. https://lore.kernel.org/linux-mm/20200210190121.10670-1-mike.kravetz@oracle.com/
On 2/20/20 7:01 AM, Joel Savitz wrote: > > Currently, the vm.min_free_kbytes sysctl value is capped at a hardcoded > 64M in init_per_zone_wmark_min (unless it is overridden by khugepaged > initialization). > > This value has not been modified since 2005, and enterprise-grade > systems now frequently have hundreds of GB of RAM and multiple 10, 40, > or even 100 GB NICs. We have seen page allocation failures on heavily > loaded systems related to NIC drivers. These issues were resolved by an > increase to vm.min_free_kbytes. > > This patch increases the hardcoded value by a factor of 4 as a temporary > solution. > > Further work to make the calculation of vm.min_free_kbytes more > consistent throughout the kernel would be desirable. > > As an example of the inconsistency of the current method, this value is > recalculated by init_per_zone_wmark_min() in the case of memory hotplug > which will override the value set by set_recommended_min_free_kbytes() > called during khugepaged initialization even if khugepaged remains > enabled, however an on/off toggle of khugepaged will then recalculate > and set the value via set_recommended_min_free_kbytes(). > > Signed-off-by: Joel Savitz <jsavitz@redhat.com> > --- > mm/page_alloc.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > index 3c4eb750a199..32cbfb13e958 100644 > --- a/mm/page_alloc.c > +++ b/mm/page_alloc.c > @@ -7867,8 +7867,8 @@ int __meminit init_per_zone_wmark_min(void) > min_free_kbytes = new_min_free_kbytes; > if (min_free_kbytes < 128) > min_free_kbytes = 128; > - if (min_free_kbytes > 65536) > - min_free_kbytes = 65536; > + if (min_free_kbytes > 262144) > + min_free_kbytes = 262144; Would it be any better to at least use symbols, instead of numbers, in the routine? Like this: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3c4eb750a199..e705636bb644 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -149,6 +149,9 @@ DEFINE_STATIC_KEY_FALSE(init_on_free); #endif EXPORT_SYMBOL(init_on_free); +static const int MIN_FREE_KBYTES_LOWER_LIMIT = 128; +static const int MIN_FREE_KBYTES_UPPER_LIMIT = 262144; + static int __init early_init_on_alloc(char *buf) { int ret; @@ -7865,10 +7868,10 @@ int __meminit init_per_zone_wmark_min(void) if (new_min_free_kbytes > user_min_free_kbytes) { min_free_kbytes = new_min_free_kbytes; - if (min_free_kbytes < 128) - min_free_kbytes = 128; - if (min_free_kbytes > 65536) - min_free_kbytes = 65536; + if (min_free_kbytes < MIN_FREE_KBYTES_LOWER_LIMIT) + min_free_kbytes = MIN_FREE_KBYTES_LOWER_LIMIT; + if (min_free_kbytes > MIN_FREE_KBYTES_UPPER_LIMIT) + min_free_kbytes = MIN_FREE_KBYTES_UPPER_LIMIT; } else { pr_warn("min_free_kbytes is not updated to %d because user defined value %d is preferred\n", new_min_free_kbytes, user_min_free_kbytes); thanks,
On 2/21/20 2:53 AM, John Hubbard wrote: > On 2/20/20 7:01 AM, Joel Savitz wrote: >> >> Currently, the vm.min_free_kbytes sysctl value is capped at a hardcoded >> 64M in init_per_zone_wmark_min (unless it is overridden by khugepaged >> initialization). >> >> This value has not been modified since 2005, and enterprise-grade >> systems now frequently have hundreds of GB of RAM and multiple 10, 40, >> or even 100 GB NICs. We have seen page allocation failures on heavily >> loaded systems related to NIC drivers. These issues were resolved by an >> increase to vm.min_free_kbytes. >> >> This patch increases the hardcoded value by a factor of 4 as a temporary >> solution. >> >> Further work to make the calculation of vm.min_free_kbytes more >> consistent throughout the kernel would be desirable. >> >> As an example of the inconsistency of the current method, this value is >> recalculated by init_per_zone_wmark_min() in the case of memory hotplug >> which will override the value set by set_recommended_min_free_kbytes() >> called during khugepaged initialization even if khugepaged remains >> enabled, however an on/off toggle of khugepaged will then recalculate >> and set the value via set_recommended_min_free_kbytes(). >> >> Signed-off-by: Joel Savitz <jsavitz@redhat.com> >> --- >> mm/page_alloc.c | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/mm/page_alloc.c b/mm/page_alloc.c >> index 3c4eb750a199..32cbfb13e958 100644 >> --- a/mm/page_alloc.c >> +++ b/mm/page_alloc.c >> @@ -7867,8 +7867,8 @@ int __meminit init_per_zone_wmark_min(void) >> min_free_kbytes = new_min_free_kbytes; >> if (min_free_kbytes < 128) >> min_free_kbytes = 128; >> - if (min_free_kbytes > 65536) >> - min_free_kbytes = 65536; >> + if (min_free_kbytes > 262144) >> + min_free_kbytes = 262144; > > > Would it be any better to at least use symbols, instead of numbers, in the > routine? Like this: +1 Thanks
diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3c4eb750a199..32cbfb13e958 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7867,8 +7867,8 @@ int __meminit init_per_zone_wmark_min(void) min_free_kbytes = new_min_free_kbytes; if (min_free_kbytes < 128) min_free_kbytes = 128; - if (min_free_kbytes > 65536) - min_free_kbytes = 65536; + if (min_free_kbytes > 262144) + min_free_kbytes = 262144; } else { pr_warn("min_free_kbytes is not updated to %d because user defined value %d is preferred\n", new_min_free_kbytes, user_min_free_kbytes);
Currently, the vm.min_free_kbytes sysctl value is capped at a hardcoded 64M in init_per_zone_wmark_min (unless it is overridden by khugepaged initialization). This value has not been modified since 2005, and enterprise-grade systems now frequently have hundreds of GB of RAM and multiple 10, 40, or even 100 GB NICs. We have seen page allocation failures on heavily loaded systems related to NIC drivers. These issues were resolved by an increase to vm.min_free_kbytes. This patch increases the hardcoded value by a factor of 4 as a temporary solution. Further work to make the calculation of vm.min_free_kbytes more consistent throughout the kernel would be desirable. As an example of the inconsistency of the current method, this value is recalculated by init_per_zone_wmark_min() in the case of memory hotplug which will override the value set by set_recommended_min_free_kbytes() called during khugepaged initialization even if khugepaged remains enabled, however an on/off toggle of khugepaged will then recalculate and set the value via set_recommended_min_free_kbytes(). Signed-off-by: Joel Savitz <jsavitz@redhat.com> --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)