| Message ID | 20250211034856.629371-1-luizcap@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm: hugetlb: avoid fallback for specific node allocation of 1G pages | expand |
On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote: > When using the HugeTLB kernel command-line to allocate 1G pages from > a specific node, such as: > > default_hugepagesz=1G hugepages=1:1 > > If node 1 happens to not have enough memory for the requested number of > 1G pages, the allocation falls back to other nodes. A quick way to > reproduce this is by creating a KVM guest with a memory-less node and > trying to allocate 1 1G page from it. Instead of failing, the allocation > will fallback to other nodes. > > This defeats the purpose of node specific allocation. Also, specific > node allocation for 2M pages don't have this behavior: the allocation > will just fail for the pages it can't satisfy. > > This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() > for 1G boot-time allocation as this function falls back to other nodes > if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() > instead, which ensures that the allocation will only be satisfied from > the specified node. > > Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation") > > Signed-off-by: Luiz Capitulino <luizcap@redhat.com> Acked-by: Oscar Salvador <osalvador@suse.de> This was discussed yesterday in [1], ccing Frank for awareness. [1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/
On 11.02.25 04:48, Luiz Capitulino wrote: > When using the HugeTLB kernel command-line to allocate 1G pages from > a specific node, such as: > > default_hugepagesz=1G hugepages=1:1 > > If node 1 happens to not have enough memory for the requested number of > 1G pages, the allocation falls back to other nodes. A quick way to > reproduce this is by creating a KVM guest with a memory-less node and > trying to allocate 1 1G page from it. Instead of failing, the allocation > will fallback to other nodes. > > This defeats the purpose of node specific allocation. Also, specific > node allocation for 2M pages don't have this behavior: the allocation > will just fail for the pages it can't satisfy. > > This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() > for 1G boot-time allocation as this function falls back to other nodes > if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() > instead, which ensures that the allocation will only be satisfied from > the specified node. > > Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation") > > Signed-off-by: Luiz Capitulino <luizcap@redhat.com> > --- > mm/hugetlb.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 65068671e460..163190e89ea1 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) > > /* do node specific alloc */ > if (nid != NUMA_NO_NODE) { > - m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h), > + m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h), > 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid); > if (!m) > return 0; Yeah, documentation says "The node format specifies the number of huge pages to allocate on specific nodes." Likely the patch simply copied the memblock_alloc_try_nid_raw() call; memblock_alloc_exact_nid_raw() seems to be the right thing to do Acked-by: David Hildenbrand <david@redhat.com>
On 2025-02-11 04:06, Oscar Salvador wrote: > On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote: >> When using the HugeTLB kernel command-line to allocate 1G pages from >> a specific node, such as: >> >> default_hugepagesz=1G hugepages=1:1 >> >> If node 1 happens to not have enough memory for the requested number of >> 1G pages, the allocation falls back to other nodes. A quick way to >> reproduce this is by creating a KVM guest with a memory-less node and >> trying to allocate 1 1G page from it. Instead of failing, the allocation >> will fallback to other nodes. >> >> This defeats the purpose of node specific allocation. Also, specific >> node allocation for 2M pages don't have this behavior: the allocation >> will just fail for the pages it can't satisfy. >> >> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() >> for 1G boot-time allocation as this function falls back to other nodes >> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() >> instead, which ensures that the allocation will only be satisfied from >> the specified node. >> >> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation") >> >> Signed-off-by: Luiz Capitulino <luizcap@redhat.com> > > Acked-by: Oscar Salvador <osalvador@suse.de> > > This was discussed yesterday in [1], ccing Frank for awareness. > > [1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/ Interesting, thanks for the reference. I stumbled over this issue back in December when debugging a HugeTLB issue at Red Hat (David knows it ;) ) and had this patch pending for more than a week now...
On Tue, Feb 11, 2025 at 6:51 AM Luiz Capitulino <luizcap@redhat.com> wrote: > > On 2025-02-11 04:06, Oscar Salvador wrote: > > On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote: > >> When using the HugeTLB kernel command-line to allocate 1G pages from > >> a specific node, such as: > >> > >> default_hugepagesz=1G hugepages=1:1 > >> > >> If node 1 happens to not have enough memory for the requested number of > >> 1G pages, the allocation falls back to other nodes. A quick way to > >> reproduce this is by creating a KVM guest with a memory-less node and > >> trying to allocate 1 1G page from it. Instead of failing, the allocation > >> will fallback to other nodes. > >> > >> This defeats the purpose of node specific allocation. Also, specific > >> node allocation for 2M pages don't have this behavior: the allocation > >> will just fail for the pages it can't satisfy. > >> > >> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() > >> for 1G boot-time allocation as this function falls back to other nodes > >> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() > >> instead, which ensures that the allocation will only be satisfied from > >> the specified node. > >> > >> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation") > >> > >> Signed-off-by: Luiz Capitulino <luizcap@redhat.com> > > > > Acked-by: Oscar Salvador <osalvador@suse.de> > > > > This was discussed yesterday in [1], ccing Frank for awareness. > > > > [1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/ > > Interesting, thanks for the reference. > > I stumbled over this issue back in December when debugging a HugeTLB issue > at Red Hat (David knows it ;) ) and had this patch pending for more than a > week now... > Looks good, I'll drop the same change from my upcoming v4 series. This will create a contextual dependency, but that's ok, this one will go in first in any case. Reviewed-by: Frank van der Linden <fvdl@google.com>
diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 65068671e460..163190e89ea1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid) /* do node specific alloc */ if (nid != NUMA_NO_NODE) { - m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h), + m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h), 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid); if (!m) return 0;
When using the HugeTLB kernel command-line to allocate 1G pages from a specific node, such as: default_hugepagesz=1G hugepages=1:1 If node 1 happens to not have enough memory for the requested number of 1G pages, the allocation falls back to other nodes. A quick way to reproduce this is by creating a KVM guest with a memory-less node and trying to allocate 1 1G page from it. Instead of failing, the allocation will fallback to other nodes. This defeats the purpose of node specific allocation. Also, specific node allocation for 2M pages don't have this behavior: the allocation will just fail for the pages it can't satisfy. This issue happens because HugeTLB calls memblock_alloc_try_nid_raw() for 1G boot-time allocation as this function falls back to other nodes if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw() instead, which ensures that the allocation will only be satisfied from the specified node. Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation") Signed-off-by: Luiz Capitulino <luizcap@redhat.com> --- mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)