| Message ID | 1594789529-6206-3-git-send-email-iamjoonsoo.kim@lge.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/4] mm/page_alloc: fix non cma alloc context | expand |
On Wed 15-07-20 14:05:28, Joonsoo Kim wrote: > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > new_non_cma_page() in gup.c requires to allocate the new page that is not > on the CMA area. new_non_cma_page() implements it by using allocation > scope APIs. > > However, there is a work-around for hugetlb. Normal hugetlb page > allocation API for migration is alloc_huge_page_nodemask(). It consists > of two steps. First is dequeing from the pool. Second is, if there is no > available page on the queue, allocating by using the page allocator. > > new_non_cma_page() can't use this API since first step (deque) isn't > aware of scope API to exclude CMA area. So, new_non_cma_page() exports > hugetlb internal function for the second step, alloc_migrate_huge_page(), > to global scope and uses it directly. This is suboptimal since hugetlb > pages on the queue cannot be utilized. > > This patch tries to fix this situation by making the deque function on > hugetlb CMA aware. In the deque function, CMA memory is skipped if > PF_MEMALLOC_NOCMA flag is found. Now that this is in sync with the global case I do not have any objections. > Acked-by: Mike Kravetz <mike.kravetz@oracle.com> > Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Acked-by: Michal Hocko <mhocko@suse.com> Minor nit below [...] > @@ -1036,10 +1037,16 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) > static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) > { > struct page *page; > + bool nocma = !!(READ_ONCE(current->flags) & PF_MEMALLOC_NOCMA); READ_ONCE is not really needed because current->flags are always set on the current so no race is possible. > + > + list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { > + if (nocma && is_migrate_cma_page(page)) > + continue; > > - list_for_each_entry(page, &h->hugepage_freelists[nid], lru) > if (!PageHWPoison(page)) > break; > + } > + > /* > * if 'non-isolated free hugepage' not found on the list, > * the allocation fails. > @@ -1928,7 +1935,7 @@ static struct page *alloc_surplus_huge_page(struct hstate *h, gfp_t gfp_mask, > return page; > } > > -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, > +static struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, > int nid, nodemask_t *nmask) > { > struct page *page; > -- > 2.7.4
On 7/15/20 7:05 AM, js1304@gmail.com wrote: > From: Joonsoo Kim <iamjoonsoo.kim@lge.com> > > new_non_cma_page() in gup.c requires to allocate the new page that is not > on the CMA area. new_non_cma_page() implements it by using allocation > scope APIs. > > However, there is a work-around for hugetlb. Normal hugetlb page > allocation API for migration is alloc_huge_page_nodemask(). It consists > of two steps. First is dequeing from the pool. Second is, if there is no > available page on the queue, allocating by using the page allocator. > > new_non_cma_page() can't use this API since first step (deque) isn't > aware of scope API to exclude CMA area. So, new_non_cma_page() exports > hugetlb internal function for the second step, alloc_migrate_huge_page(), > to global scope and uses it directly. This is suboptimal since hugetlb > pages on the queue cannot be utilized. > > This patch tries to fix this situation by making the deque function on > hugetlb CMA aware. In the deque function, CMA memory is skipped if > PF_MEMALLOC_NOCMA flag is found. > > Acked-by: Mike Kravetz <mike.kravetz@oracle.com> > Signed-off-by: Joonsoo Kim <iamjoonsoo.kim@lge.com> Acked-by: Vlastimil Babka <vbabka@suse.cz>
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2660b04..fb2b5aa 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -509,8 +509,6 @@ struct page *alloc_huge_page_nodemask(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask); struct page *alloc_huge_page_vma(struct hstate *h, struct vm_area_struct *vma, unsigned long address); -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, - int nid, nodemask_t *nmask); int huge_add_to_page_cache(struct page *page, struct address_space *mapping, pgoff_t idx); diff --git a/mm/gup.c b/mm/gup.c index bbd36a1..4ba822a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1634,11 +1634,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private) struct hstate *h = page_hstate(page); gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); - /* - * We don't want to dequeue from the pool because pool pages will - * mostly be from the CMA region. - */ - return alloc_migrate_huge_page(h, gfp_mask, nid, NULL); + return alloc_huge_page_nodemask(h, nid, NULL, gfp_mask); } #endif if (PageTransHuge(page)) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3245aa0..514e29c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -29,6 +29,7 @@ #include <linux/numa.h> #include <linux/llist.h> #include <linux/cma.h> +#include <linux/sched/mm.h> #include <asm/page.h> #include <asm/tlb.h> @@ -1036,10 +1037,16 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) { struct page *page; + bool nocma = !!(READ_ONCE(current->flags) & PF_MEMALLOC_NOCMA); + + list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { + if (nocma && is_migrate_cma_page(page)) + continue; - list_for_each_entry(page, &h->hugepage_freelists[nid], lru) if (!PageHWPoison(page)) break; + } + /* * if 'non-isolated free hugepage' not found on the list, * the allocation fails. @@ -1928,7 +1935,7 @@ static struct page *alloc_surplus_huge_page(struct hstate *h, gfp_t gfp_mask, return page; } -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, +static struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask) { struct page *page;