From patchwork Mon Feb 8 10:38:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 12074801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBE30C433E6 for ; Mon, 8 Feb 2021 10:38:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 60B3064E59 for ; Mon, 8 Feb 2021 10:38:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60B3064E59 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8EBA86B0070; Mon, 8 Feb 2021 05:38:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 822286B0071; Mon, 8 Feb 2021 05:38:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ECA56B0072; Mon, 8 Feb 2021 05:38:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0186.hostedemail.com [216.40.44.186]) by kanga.kvack.org (Postfix) with ESMTP id 2FB3D6B0071 for ; Mon, 8 Feb 2021 05:38:18 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F191B180AD804 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-FDA: 77794751034.17.hands04_0307c0e275fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id D02F6180D0181 for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-HE-Tag: hands04_0307c0e275fe X-Filterd-Recvd-Size: 4283 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Mon, 8 Feb 2021 10:38:17 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 2109AAEB3; Mon, 8 Feb 2021 10:38:16 +0000 (UTC) From: Oscar Salvador To: Mike Kravetz Cc: David Hildenbrand , Muchun Song , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Oscar Salvador Subject: [RFC PATCH 2/2] mm,page_alloc: Make alloc_contig_range handle free hugetlb pages Date: Mon, 8 Feb 2021 11:38:12 +0100 Message-Id: <20210208103812.32056-3-osalvador@suse.de> X-Mailer: git-send-email 2.13.7 In-Reply-To: <20210208103812.32056-1-osalvador@suse.de> References: <20210208103812.32056-1-osalvador@suse.de> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Free hugetlb pages are trickier to handle as to in order to guarantee no userspace appplication disruption, we need to replace the current free hugepage with a new one. In order to do that, a new function called alloc_and_dissolve_huge_page in introduced. This function will first try to get a new fresh hugetlb page, and if it succeeds, it will dissolve the old one. Signed-off-by: Oscar Salvador --- include/linux/hugetlb.h | 6 ++++++ mm/compaction.c | 11 +++++++++++ mm/hugetlb.c | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ebca2ef02212..f81afcb86e89 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -505,6 +505,7 @@ struct huge_bootmem_page { struct hstate *hstate; }; +bool alloc_and_dissolve_huge_page(struct page *page); struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve); struct page *alloc_huge_page_nodemask(struct hstate *h, int preferred_nid, @@ -773,6 +774,11 @@ static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; +static inline bool alloc_and_dissolve_huge_page(struct page *page) +{ + return false; +} + static inline struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve) diff --git a/mm/compaction.c b/mm/compaction.c index 89cd2e60da29..7969ddc10856 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -952,6 +952,17 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, low_pfn += compound_nr(page) - 1; goto isolate_success_no_list; } + } else { + /* + * Free hugetlb page. Allocate a new one and + * dissolve this is if succeed. + */ + if (alloc_and_dissolve_huge_page(page)) { + unsigned long order = buddy_order_unsafe(page); + + low_pfn += (1UL << order) - 1; + continue; + } } goto isolate_fail; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 18f6ee317900..79ffbb64c4ee 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2253,6 +2253,41 @@ static void restore_reserve_on_error(struct hstate *h, } } +bool alloc_and_dissolve_huge_page(struct page *page) +{ + NODEMASK_ALLOC(nodemask_t, nodes_allowed, GFP_KERNEL); + struct page *head; + struct hstate *h; + bool ret = false; + int nid; + + if (!nodes_allowed) + return ret; + + spin_lock(&hugetlb_lock); + head = compound_head(page); + h = page_hstate(head); + nid = page_to_nid(head); + spin_unlock(&hugetlb_lock); + + init_nodemask_of_node(nodes_allowed, nid); + + /* + * Before dissolving the page, we need to allocate a new one, + * so the pool remains stable. + */ + if (alloc_pool_huge_page(h, nodes_allowed, NULL)) { + /* + * Ok, we have a free hugetlb-page to replace this + * one. Dissolve the old page. + */ + if (!dissolve_free_huge_page(page)) + ret = true; + } + + return ret; +} + struct page *alloc_huge_page(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve) {