| Message ID | 20211122153233.9924-2-mhocko@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | extend vmalloc support for constrained allocations | expand |
On Mon, Nov 22, 2021 at 04:32:30PM +0100, Michal Hocko wrote: > From: Michal Hocko <mhocko@suse.com> > > vmalloc historically hasn't supported GFP_NO{FS,IO} requests because > page table allocations do not support externally provided gfp mask > and performed GFP_KERNEL like allocations. > > Since few years we have scope (memalloc_no{fs,io}_{save,restore}) APIs > to enforce NOFS and NOIO constrains implicitly to all allocators within > the scope. There was a hope that those scopes would be defined on a > higher level when the reclaim recursion boundary starts/stops (e.g. when > a lock required during the memory reclaim is required etc.). It seems > that not all NOFS/NOIO users have adopted this approach and instead > they have taken a workaround approach to wrap a single [k]vmalloc > allocation by a scope API. > > These workarounds do not serve the purpose of a better reclaim recursion > documentation and reduction of explicit GFP_NO{FS,IO} usege so let's > just provide them with the semantic they are asking for without a need > for workarounds. > > Add support for GFP_NOFS and GFP_NOIO to vmalloc directly. All internal > allocations already comply with the given gfp_mask. The only current > exception is vmap_pages_range which maps kernel page tables. Infer the > proper scope API based on the given gfp mask. > > Signed-off-by: Michal Hocko <mhocko@suse.com> > --- > mm/vmalloc.c | 22 ++++++++++++++++++++-- > 1 file changed, 20 insertions(+), 2 deletions(-) > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > index d2a00ad4e1dd..17ca7001de1f 100644 > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -2926,6 +2926,8 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, > unsigned long array_size; > unsigned int nr_small_pages = size >> PAGE_SHIFT; > unsigned int page_order; > + unsigned int flags; > + int ret; > > array_size = (unsigned long)nr_small_pages * sizeof(struct page *); > gfp_mask |= __GFP_NOWARN; > @@ -2967,8 +2969,24 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, > goto fail; > } > > - if (vmap_pages_range(addr, addr + size, prot, area->pages, > - page_shift) < 0) { > + /* > + * page tables allocations ignore external gfp mask, enforce it > + * by the scope API > + */ > + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) > + flags = memalloc_nofs_save(); > + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) > + flags = memalloc_noio_save(); > + > + ret = vmap_pages_range(addr, addr + size, prot, area->pages, > + page_shift); > + > + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) > + memalloc_nofs_restore(flags); > + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) > + memalloc_noio_restore(flags); > + > + if (ret < 0) { > warn_alloc(orig_gfp_mask, NULL, > "vmalloc error: size %lu, failed to map pages", > area->nr_pages * PAGE_SIZE); > -- > 2.30.2 > Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com> -- Vlad Rezki
On 11/22/21 16:32, Michal Hocko wrote: > From: Michal Hocko <mhocko@suse.com> > > vmalloc historically hasn't supported GFP_NO{FS,IO} requests because > page table allocations do not support externally provided gfp mask > and performed GFP_KERNEL like allocations. > > Since few years we have scope (memalloc_no{fs,io}_{save,restore}) APIs > to enforce NOFS and NOIO constrains implicitly to all allocators within > the scope. There was a hope that those scopes would be defined on a > higher level when the reclaim recursion boundary starts/stops (e.g. when > a lock required during the memory reclaim is required etc.). It seems > that not all NOFS/NOIO users have adopted this approach and instead > they have taken a workaround approach to wrap a single [k]vmalloc > allocation by a scope API. > > These workarounds do not serve the purpose of a better reclaim recursion > documentation and reduction of explicit GFP_NO{FS,IO} usege so let's > just provide them with the semantic they are asking for without a need > for workarounds. > > Add support for GFP_NOFS and GFP_NOIO to vmalloc directly. All internal > allocations already comply with the given gfp_mask. The only current > exception is vmap_pages_range which maps kernel page tables. Infer the > proper scope API based on the given gfp mask. > > Signed-off-by: Michal Hocko <mhocko@suse.com> Acked-by: Vlastimil Babka <vbabka@suse.cz> > --- > mm/vmalloc.c | 22 ++++++++++++++++++++-- > 1 file changed, 20 insertions(+), 2 deletions(-) > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > index d2a00ad4e1dd..17ca7001de1f 100644 > --- a/mm/vmalloc.c > +++ b/mm/vmalloc.c > @@ -2926,6 +2926,8 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, > unsigned long array_size; > unsigned int nr_small_pages = size >> PAGE_SHIFT; > unsigned int page_order; > + unsigned int flags; > + int ret; > > array_size = (unsigned long)nr_small_pages * sizeof(struct page *); > gfp_mask |= __GFP_NOWARN; > @@ -2967,8 +2969,24 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, > goto fail; > } > > - if (vmap_pages_range(addr, addr + size, prot, area->pages, > - page_shift) < 0) { > + /* > + * page tables allocations ignore external gfp mask, enforce it > + * by the scope API > + */ > + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) > + flags = memalloc_nofs_save(); > + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) > + flags = memalloc_noio_save(); > + > + ret = vmap_pages_range(addr, addr + size, prot, area->pages, > + page_shift); > + > + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) > + memalloc_nofs_restore(flags); > + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) > + memalloc_noio_restore(flags); > + > + if (ret < 0) { > warn_alloc(orig_gfp_mask, NULL, > "vmalloc error: size %lu, failed to map pages", > area->nr_pages * PAGE_SIZE); >
diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d2a00ad4e1dd..17ca7001de1f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2926,6 +2926,8 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, unsigned long array_size; unsigned int nr_small_pages = size >> PAGE_SHIFT; unsigned int page_order; + unsigned int flags; + int ret; array_size = (unsigned long)nr_small_pages * sizeof(struct page *); gfp_mask |= __GFP_NOWARN; @@ -2967,8 +2969,24 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, goto fail; } - if (vmap_pages_range(addr, addr + size, prot, area->pages, - page_shift) < 0) { + /* + * page tables allocations ignore external gfp mask, enforce it + * by the scope API + */ + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) + flags = memalloc_nofs_save(); + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) + flags = memalloc_noio_save(); + + ret = vmap_pages_range(addr, addr + size, prot, area->pages, + page_shift); + + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) + memalloc_nofs_restore(flags); + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) + memalloc_noio_restore(flags); + + if (ret < 0) { warn_alloc(orig_gfp_mask, NULL, "vmalloc error: size %lu, failed to map pages", area->nr_pages * PAGE_SIZE);