From patchwork Fri Jul 31 07:35:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joonsoo Kim X-Patchwork-Id: 11694255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE6FA1392 for ; Fri, 31 Jul 2020 07:35:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B604206FA for ; Fri, 31 Jul 2020 07:35:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rBH5vfGv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B604206FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61E1C8D0014; Fri, 31 Jul 2020 03:35:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5A8018D000B; Fri, 31 Jul 2020 03:35:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4476D8D0014; Fri, 31 Jul 2020 03:35:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id 2C2488D000B for ; Fri, 31 Jul 2020 03:35:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E6F9C181AEF09 for ; Fri, 31 Jul 2020 07:35:34 +0000 (UTC) X-FDA: 77097560988.30.baby56_141125c26f82 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id BEB1D180B3AA7 for ; Fri, 31 Jul 2020 07:35:34 +0000 (UTC) X-Spam-Summary: 1,0,0,9963295eaab7663b,d41d8cd98f00b204,js1304@gmail.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1345:1359:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2693:2898:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:4117:4250:4321:4385:4605:5007:6261:6653:6742:7576:7903:8957:9010:9413:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12679:12895:14181:14394:14721:14819:21080:21212:21444:21451:21627:21666:21740:21990:30003:30054:30064:30070,0,RBL:209.85.216.67:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100;04yg45fw18rep1yajhwbr3g6c667tocd5xd1cz7dnp51g6apgjmsh87rim5o6kp.ogpqnsgu1gnnnbmyogb6ufij5m9npcbn5kpfyi9rt4qr79xy9chg5sxpwpodydk.y-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: baby56_141125c26f82 X-Filterd-Recvd-Size: 6966 Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Fri, 31 Jul 2020 07:35:34 +0000 (UTC) Received: by mail-pj1-f67.google.com with SMTP id mt12so6596545pjb.4 for ; Fri, 31 Jul 2020 00:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XG1OzhoIoStzjs92yyazweiEB/4D+J+cafe51bT7rtQ=; b=rBH5vfGv+lomki5x/FN7kDdxQuPJ/LbAKwn/7qcZm7yfqD9YWeERFo3wZYH916c2TI 6dO1NSPBQrCnpqCBPSPrZiIyiIqtiDRFPZh1kFUM+oZ1/Vz6TWvZFVYVwBNIgo/NGz2S W6z/Ty9lCinBEdohxajE9OPtOW3Mcw0HezJHQZN5W9Z+O54looju/8JRn5WgJlWFsP0v r5/UZXqvX7/iPStaYtBjRGRV3j3f5i7OCOvwGTwAzbP7g5UDY92BARxWUdiSA5L/s0Ra pIM7EJQUCMECpUmsxBDlbjg230gfWo9XH7jHV0ayyXvgKqoFjYg4JZmMYN2B1EsWdiKe UwsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XG1OzhoIoStzjs92yyazweiEB/4D+J+cafe51bT7rtQ=; b=fwwXAqZ47VhryWH9T3XW7N8G6scdOGchS2Mi5odxygwRokvbamll7IKOpjbDxp0PfN Us2+NdCQEqg8Hd0aIE7K7gbKWqAKpc1+sOQuyz4hsOTEJnL4ru/dGWcoTXe+s1XkdjQe 8mA05unRqVANK0SgLuc3OZ7zTS9TwNfLnZEnsJYvfYKnmSDYFG2YtpVx5SQpcMAtcyS6 tTa7sV8AiXeeNX57/4ZSubzVjnnJ4bZLOWAiQFgwPJcG4HDKxO7DQeuzUPWCxx1D6M7h Fu1PZCGQ8Tsvlb2dEiQgcpTAdCLmHjf9hB2k79oAUzsAsa/twSeZgHneBB8sui4YbN4j 114g== X-Gm-Message-State: AOAM531YHQmiBkU+x8tXiaFhzfIZ0PIYjYUdm+8FFK7Iq3saQwtNfiNY ubAYALFHBN7amCkgUBr+oGQ= X-Google-Smtp-Source: ABdhPJwnVQv/CqRvsbNm14kCYy6anShj+SJtR2oELJWKTlAR9pnZ5I27qpNb0kZ3Hj8ZMwKVTRJuTw== X-Received: by 2002:a17:90a:1589:: with SMTP id m9mr3025242pja.122.1596180933494; Fri, 31 Jul 2020 00:35:33 -0700 (PDT) Received: from localhost.localdomain ([114.206.198.176]) by smtp.gmail.com with ESMTPSA id 13sm8355522pfp.3.2020.07.31.00.35.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jul 2020 00:35:32 -0700 (PDT) From: js1304@gmail.com X-Google-Original-From: iamjoonsoo.kim@lge.com To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@lge.com, Vlastimil Babka , Christoph Hellwig , Roman Gushchin , Mike Kravetz , Naoya Horiguchi , Michal Hocko , "Aneesh Kumar K . V" , Joonsoo Kim Subject: [PATCH v3 2/3] mm/hugetlb: make hugetlb migration callback CMA aware Date: Fri, 31 Jul 2020 16:35:05 +0900 Message-Id: <1596180906-8442-2-git-send-email-iamjoonsoo.kim@lge.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1596180906-8442-1-git-send-email-iamjoonsoo.kim@lge.com> References: <1596180906-8442-1-git-send-email-iamjoonsoo.kim@lge.com> X-Rspamd-Queue-Id: BEB1D180B3AA7 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joonsoo Kim new_non_cma_page() in gup.c requires to allocate the new page that is not on the CMA area. new_non_cma_page() implements it by using allocation scope APIs. However, there is a work-around for hugetlb. Normal hugetlb page allocation API for migration is alloc_huge_page_nodemask(). It consists of two steps. First is dequeing from the pool. Second is, if there is no available page on the queue, allocating by using the page allocator. new_non_cma_page() can't use this API since first step (deque) isn't aware of scope API to exclude CMA area. So, new_non_cma_page() exports hugetlb internal function for the second step, alloc_migrate_huge_page(), to global scope and uses it directly. This is suboptimal since hugetlb pages on the queue cannot be utilized. This patch tries to fix this situation by making the deque function on hugetlb CMA aware. In the deque function, CMA memory is skipped if PF_MEMALLOC_NOCMA flag is found. Acked-by: Mike Kravetz Acked-by: Vlastimil Babka Acked-by: Michal Hocko Signed-off-by: Joonsoo Kim --- include/linux/hugetlb.h | 2 -- mm/gup.c | 6 +----- mm/hugetlb.c | 11 +++++++++-- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2660b04..fb2b5aa 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -509,8 +509,6 @@ struct page *alloc_huge_page_nodemask(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask); struct page *alloc_huge_page_vma(struct hstate *h, struct vm_area_struct *vma, unsigned long address); -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, - int nid, nodemask_t *nmask); int huge_add_to_page_cache(struct page *page, struct address_space *mapping, pgoff_t idx); diff --git a/mm/gup.c b/mm/gup.c index 3990ddc..7b63d72 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1616,11 +1616,7 @@ static struct page *new_non_cma_page(struct page *page, unsigned long private) struct hstate *h = page_hstate(page); gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); - /* - * We don't want to dequeue from the pool because pool pages will - * mostly be from the CMA region. - */ - return alloc_migrate_huge_page(h, gfp_mask, nid, NULL); + return alloc_huge_page_nodemask(h, nid, NULL, gfp_mask); } #endif if (PageTransHuge(page)) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4645f14..d1706b7 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include @@ -1041,10 +1042,16 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) { struct page *page; + bool nocma = !!(current->flags & PF_MEMALLOC_NOCMA); + + list_for_each_entry(page, &h->hugepage_freelists[nid], lru) { + if (nocma && is_migrate_cma_page(page)) + continue; - list_for_each_entry(page, &h->hugepage_freelists[nid], lru) if (!PageHWPoison(page)) break; + } + /* * if 'non-isolated free hugepage' not found on the list, * the allocation fails. @@ -1973,7 +1980,7 @@ static struct page *alloc_surplus_huge_page(struct hstate *h, gfp_t gfp_mask, return page; } -struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, +static struct page *alloc_migrate_huge_page(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask) { struct page *page;