From patchwork Mon Jul 20 04:56:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joonsoo Kim X-Patchwork-Id: 11672851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 52C0F13B4 for ; Mon, 20 Jul 2020 04:57:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F7BE20758 for ; Mon, 20 Jul 2020 04:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iYXv9eft" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F7BE20758 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4FCC96B0005; Mon, 20 Jul 2020 00:57:38 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4AE4D8D0001; Mon, 20 Jul 2020 00:57:38 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 39D166B0007; Mon, 20 Jul 2020 00:57:38 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 25A376B0005 for ; Mon, 20 Jul 2020 00:57:38 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D82B918473E55 for ; Mon, 20 Jul 2020 04:57:37 +0000 (UTC) X-FDA: 77057246154.18.crib78_0c1860726f22 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 8ADFD100CE359 for ; Mon, 20 Jul 2020 04:56:47 +0000 (UTC) X-Spam-Summary: 1,0,0,cfe6700321dc333e,d41d8cd98f00b204,js1304@gmail.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1345:1359:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2693:2898:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4250:4321:4385:4605:5007:6261:6653:6742:7576:7903:8957:9010:9413:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:12986:14181:14394:14721:14819:21080:21212:21444:21451:21627:21666:21740:21990:30003:30054:30064:30070,0,RBL:209.85.214.193:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100;04yfaxw6qzo93mgsiqm8n1feubu38ocd5xd1cz7dnp51g6apgjmsh87rim5o6kp.ogpqnsgu1gnnnbmyogb6ufij5m9npcbn5kpfyi9rt4qrhtechqzgbk64e59iw7k.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: crib78_0c1860726f22 X-Filterd-Recvd-Size: 6965 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Mon, 20 Jul 2020 04:56:47 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id b9so8120689plx.6 for ; Sun, 19 Jul 2020 21:56:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WxQvsZ6QmcUkdn7bmiYYmJhxUakC2I6RnR8rzuhXmYI=; b=iYXv9eftR84ZHA/raJZbczowGT4NKjrV8526WC8U95iR7EuvEcIqalEjIbB59y3oPu OL6AvUcKU6L2I3zjzLUwCs2cAgbVakZ1QPgvgNIZOTdluu3dtek4uaCnu0oa3xzURgs/ QG7wjeVXnfxVo8JhZvFjcAFtbSfOVWVCP6WUpAacx1/q1PJCwkbVowxQTpi61dV9SOpm NPX0wCUSOtDwA7tWaflIHY3p607XyjKY+pqHlyPYUdczUX8QKOXAdFn+5ib1roKDJexC n9iFbleMRgpfi8GCy8b5Z2hdJJj1TZZ1NFx1ksaxeffoCcxYEnVAWtyJJlDE4/LxxiNT tfrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WxQvsZ6QmcUkdn7bmiYYmJhxUakC2I6RnR8rzuhXmYI=; b=FANCjrS435nIH9R+Rbl2cA4vm9XqcFJPm3YucYq7CZoFcBbg+ugCjfGEsNskjcDvQX cJBbSf6hv9dHStj+ODOWEprJVkCjEw9YxINs3/RygOL/L7ueMBoLd0WBkYu2YDLkDWD9 S12Mu2xhpIlZGVtzOl2WFY3nOkJ9UqGzP1ONlZcizKJ7QO1z6QKbxY9AapwqckDnUoQt DrF64v2ix67qz/H0TQTaB1BfnoFnB4dlpLohNKF1I5EtINp9/x5Hoa4iswkz/ETxlq62 ZJUok1ZfEFcpQsOvTfHwvd2uqiZXZcjOy1K/MCNWqmVsDx7suPdVSnOa0Nz/njRrihCv BBgw== X-Gm-Message-State: AOAM532Xod5IQ+pEcR/lXwSz4wHnKdeNg+k0cYESVTtK4ZxzrNKwq1Pf 6CnraR2I8D9M5wWEKZ0uCaM= X-Google-Smtp-Source: ABdhPJw0rKezzDSAU6tqpn6flMPuV+16J+ebbVLnAeiSiY9g1d3Fxj4/34wxckGgtOme7S/i7vBDOw== X-Received: by 2002:a17:90a:32cb:: with SMTP id l69mr19545057pjb.205.1595221006220; Sun, 19 Jul 2020 21:56:46 -0700 (PDT) Received: from localhost.localdomain ([114.206.198.176]) by smtp.gmail.com with ESMTPSA id w2sm10177532pjt.19.2020.07.19.21.56.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 19 Jul 2020 21:56:45 -0700 (PDT) From: js1304@gmail.com X-Google-Original-From: iamjoonsoo.kim@lge.com To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@lge.com, Vlastimil Babka , Christoph Hellwig , Roman Gushchin , Mike Kravetz , Naoya Horiguchi , Michal Hocko , "Aneesh Kumar K . V" , Joonsoo Kim Subject: [PATCH v2 3/4] mm/hugetlb: make hugetlb migration callback CMA aware Date: Mon, 20 Jul 2020 13:56:17 +0900 Message-Id: <1595220978-9890-3-git-send-email-iamjoonsoo.kim@lge.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1595220978-9890-1-git-send-email-iamjoonsoo.kim@lge.com> References: <1595220978-9890-1-git-send-email-iamjoonsoo.kim@lge.com> X-Rspamd-Queue-Id: 8ADFD100CE359 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joonsoo Kim new_non_cma_page() in gup.c requires to allocate the new page that is not on the CMA area. new_non_cma_page() implements it by using allocation scope APIs. However, there is a work-around for hugetlb. Normal hugetlb page allocation API for migration is alloc_huge_page_nodemask(). It consists of two steps. First is dequeing from the pool. Second is, if there is no available page on the queue, allocating by using the page allocator. new_non_cma_page() can't use this API since first step (deque) isn't aware of scope API to exclude CMA area. So, new_non_cma_page() exports hugetlb internal function for the second step, alloc_migrate_huge_page(), to global scope and uses it directly. This is suboptimal since hugetlb pages on the queue cannot be utilized. This patch tries to fix this situation by making the deque function on hugetlb CMA aware. In the deque function, CMA memory is skipped if PF_MEMALLOC_NOCMA flag is found. Acked-by: Mike Kravetz Acked-by: Vlastimil Babka Acked-by: Michal Hocko Signed-off-by: Joonsoo Kim --- include/linux/hugetlb.h | 2 -- mm/gup.c | 6 +----- mm/hugetlb.c | 11 +++++++++-- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2660b04..fb2b5aa 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -509,8 +509,6 @@ struct page *alloc_huge_page_nodemask(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask); struct page *alloc_huge_page_vma(struct hstate *h, struct vm_area_struct *vma, unsigned long address); -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, - int nid, nodemask_t *nmask); int huge_add_to_page_cache(struct page *page, struct address_space *mapping, pgoff_t idx); diff --git a/mm/gup.c b/mm/gup.c index bbd36a1..4ba822a 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1634,11 +1634,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private) struct hstate *h = page_hstate(page); gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); - /* - * We don't want to dequeue from the pool because pool pages will - * mostly be from the CMA region. - */ - return alloc_migrate_huge_page(h, gfp_mask, nid, NULL); + return alloc_huge_page_nodemask(h, nid, NULL, gfp_mask); } #endif if (PageTransHuge(page)) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3245aa0..d9eb923 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -1036,10 +1037,16 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) { struct page *page; + bool nocma = !!(current->flags & PF_MEMALLOC_NOCMA); + + list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { + if (nocma && is_migrate_cma_page(page)) + continue; - list_for_each_entry(page, &h->hugepage_freelists[nid], lru) if (!PageHWPoison(page)) break; + } + /* * if 'non-isolated free hugepage' not found on the list, * the allocation fails. @@ -1928,7 +1935,7 @@ static struct page *alloc_surplus_huge_page(struct hstate *h, gfp_t gfp_mask, return page; } -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, +static struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask) { struct page *page;