From patchwork Mon Feb 8 10:38:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 12074799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40217C433E0 for ; Mon, 8 Feb 2021 10:38:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B437C64E54 for ; Mon, 8 Feb 2021 10:38:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B437C64E54 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2CDB76B006E; Mon, 8 Feb 2021 05:38:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 27C2E6B0070; Mon, 8 Feb 2021 05:38:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 121036B0071; Mon, 8 Feb 2021 05:38:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0210.hostedemail.com [216.40.44.210]) by kanga.kvack.org (Postfix) with ESMTP id C40746B006E for ; Mon, 8 Feb 2021 05:38:17 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 89A803629 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-FDA: 77794751034.27.kick11_5f00c2e275fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 6B3F93D669 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-HE-Tag: kick11_5f00c2e275fe X-Filterd-Recvd-Size: 3175 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Mon, 8 Feb 2021 10:38:16 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id A85BBB039; Mon, 8 Feb 2021 10:38:15 +0000 (UTC) From: Oscar Salvador To: Mike Kravetz Cc: David Hildenbrand , Muchun Song , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [RFC PATCH 1/2] mm,page_alloc: Make alloc_contig_range handle in-use hugetlb pages Date: Mon, 8 Feb 2021 11:38:11 +0100 Message-Id: <20210208103812.32056-2-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20210208103812.32056-1-osalvador@suse.de> References: <20210208103812.32056-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: alloc_contig_range is not prepared to handle hugetlb pages and will fail if it ever sees one, but since they can be migrated as any other page (LRU and Movable), it makes sense to also handle them. For now, do it only when coming from alloc_contig_range. Signed-off-by: Oscar Salvador --- mm/compaction.c | 17 +++++++++++++++++ mm/vmscan.c | 5 +++-- 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index e5acb9714436..89cd2e60da29 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -940,6 +940,22 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, goto isolate_fail; } + /* + * Handle hugetlb pages only when coming from alloc_contig + */ + if (PageHuge(page) && cc->alloc_contig) { + if (page_count(page)) { + /* + * Hugetlb page in-use. Isolate and migrate. + */ + if (isolate_huge_page(page, &cc->migratepages)) { + low_pfn += compound_nr(page) - 1; + goto isolate_success_no_list; + } + } + goto isolate_fail; + } + /* * Check may be lockless but that's ok as we recheck later. * It's possible to migrate LRU and non-lru movable pages. @@ -1041,6 +1057,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, isolate_success: list_add(&page->lru, &cc->migratepages); +isolate_success_no_list: cc->nr_migratepages += compound_nr(page); nr_isolated += compound_nr(page); diff --git a/mm/vmscan.c b/mm/vmscan.c index b1b574ad199d..0803adca4469 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1506,8 +1506,9 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, LIST_HEAD(clean_pages); list_for_each_entry_safe(page, next, page_list, lru) { - if (page_is_file_lru(page) && !PageDirty(page) && - !__PageMovable(page) && !PageUnevictable(page)) { + if (!PageHuge(page) && page_is_file_lru(page) && + !PageDirty(page) && !__PageMovable(page) && + !PageUnevictable(page)) { ClearPageActive(page); list_move(&page->lru, &clean_pages); } From patchwork Mon Feb 8 10:38:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 12074801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBE30C433E6 for ; Mon, 8 Feb 2021 10:38:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 60B3064E59 for ; Mon, 8 Feb 2021 10:38:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60B3064E59 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8EBA86B0070; Mon, 8 Feb 2021 05:38:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 822286B0071; Mon, 8 Feb 2021 05:38:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ECA56B0072; Mon, 8 Feb 2021 05:38:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0186.hostedemail.com [216.40.44.186]) by kanga.kvack.org (Postfix) with ESMTP id 2FB3D6B0071 for ; Mon, 8 Feb 2021 05:38:18 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F191B180AD804 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-FDA: 77794751034.17.hands04_0307c0e275fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id D02F6180D0181 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-HE-Tag: hands04_0307c0e275fe X-Filterd-Recvd-Size: 4283 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 2109AAEB3; Mon, 8 Feb 2021 10:38:16 +0000 (UTC) From: Oscar Salvador To: Mike Kravetz Cc: David Hildenbrand , Muchun Song , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [RFC PATCH 2/2] mm,page_alloc: Make alloc_contig_range handle free hugetlb pages Date: Mon, 8 Feb 2021 11:38:12 +0100 Message-Id: <20210208103812.32056-3-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20210208103812.32056-1-osalvador@suse.de> References: <20210208103812.32056-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Free hugetlb pages are trickier to handle as to in order to guarantee no userspace appplication disruption, we need to replace the current free hugepage with a new one. In order to do that, a new function called alloc_and_dissolve_huge_page in introduced. This function will first try to get a new fresh hugetlb page, and if it succeeds, it will dissolve the old one. Signed-off-by: Oscar Salvador --- include/linux/hugetlb.h | 6 ++++++ mm/compaction.c | 11 +++++++++++ mm/hugetlb.c | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ebca2ef02212..f81afcb86e89 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -505,6 +505,7 @@ struct huge_bootmem_page { struct hstate *hstate; }; +bool alloc_and_dissolve_huge_page(struct page *page); struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve); struct page *alloc_huge_page_nodemask(struct hstate *h, int preferred_nid, @@ -773,6 +774,11 @@ static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; +static inline bool alloc_and_dissolve_huge_page(struct page *page) +{ + return false; +} + static inline struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve) diff --git a/mm/compaction.c b/mm/compaction.c index 89cd2e60da29..7969ddc10856 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -952,6 +952,17 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, low_pfn += compound_nr(page) - 1; goto isolate_success_no_list; } + } else { + /* + * Free hugetlb page. Allocate a new one and + * dissolve this is if succeed. + */ + if (alloc_and_dissolve_huge_page(page)) { + unsigned long order = buddy_order_unsafe(page); + + low_pfn += (1UL << order) - 1; + continue; + } } goto isolate_fail; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 18f6ee317900..79ffbb64c4ee 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2253,6 +2253,41 @@ static void restore_reserve_on_error(struct hstate *h, } } +bool alloc_and_dissolve_huge_page(struct page *page) +{ + NODEMASK_ALLOC(nodemask_t, nodes_allowed, GFP_KERNEL); + struct page *head; + struct hstate *h; + bool ret = false; + int nid; + + if (!nodes_allowed) + return ret; + + spin_lock(&hugetlb_lock); + head = compound_head(page); + h = page_hstate(head); + nid = page_to_nid(head); + spin_unlock(&hugetlb_lock); + + init_nodemask_of_node(nodes_allowed, nid); + + /* + * Before dissolving the page, we need to allocate a new one, + * so the pool remains stable. + */ + if (alloc_pool_huge_page(h, nodes_allowed, NULL)) { + /* + * Ok, we have a free hugetlb-page to replace this + * one. Dissolve the old page. + */ + if (!dissolve_free_huge_page(page)) + ret = true; + } + + return ret; +} + struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve) {