From patchwork Fri Aug 14 17:31:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Minchan Kim X-Patchwork-Id: 11715051 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3F46739 for ; Fri, 14 Aug 2020 17:31:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A06E820768 for ; Fri, 14 Aug 2020 17:31:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZDk4pXeR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A06E820768 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A5526B0095; Fri, 14 Aug 2020 13:31:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 309848D0002; Fri, 14 Aug 2020 13:31:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1593A6B009A; Fri, 14 Aug 2020 13:31:45 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id D96676B0095 for ; Fri, 14 Aug 2020 13:31:44 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8FD43127C for ; Fri, 14 Aug 2020 17:31:44 +0000 (UTC) X-FDA: 77149866528.07.jar62_14181ec26ffe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id E74881803F9C0 for ; Fri, 14 Aug 2020 17:31:41 +0000 (UTC) X-Spam-Summary: 1,0,0,53daccf7a0fc110c,d41d8cd98f00b204,minchan.kim@gmail.com,,RULES_HIT:41:355:379:541:560:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1543:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2731:3138:3139:3140:3141:3142:3354:3870:3871:3874:3876:4118:4321:4605:5007:6120:6261:6653:7901:8957:9010:10004:11026:11473:11658:11914:12043:12296:12297:12438:12517:12519:12555:12895:12986:13894:14096:14181:14721:21080:21324:21444:21451:21627:21990:30054,0,RBL:209.85.210.194:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100;04yg6chbtkhactfbgw758mn8brhsgycxacu9u9xu4qs1f7cmpfp66wb4p5ryci8.zcy3pss8kafm3a3z4su5xj5hrtbpnwcos8i8tmggxp3sfa8ujfzeddqj8haixzw.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: jar62_14181ec26ffe X-Filterd-Recvd-Size: 7005 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 14 Aug 2020 17:31:40 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id d188so4879250pfd.2 for ; Fri, 14 Aug 2020 10:31:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WJpxQfWuLdEkvuYVJgijB6hFX/x7FtsK/XxVIYu8htw=; b=ZDk4pXeR6FAMjoS0yPpCqgUCJ4U+gI08oxZBQFFoXdDp/GqP48qMKGfORSzg7H1/KJ PDScmt3LlDHAARpuGj5CkzmrYHuLEw0LwTOSIpHACzmTJtc3bSaEVMCXYiTNFmG3N05C tyH+Lyzy5mZ/D09wWk2u1CekQhaNW5zJqqJ6AjdwEQUyVEA5BaabuSU2uzzCq2RcmGPt mmMAm0FNWBteNbN5KcVS6BzAqRWmUuLWjCCKaEtf0HnB2zi+Vk46q3eeC+w44/fY5fr3 1fUARfRhexKKNFB44fKL+N2a877G3GEP0wb4lELss1GEe2wVmATgWdyQcBLPwRWQoSch E3Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=WJpxQfWuLdEkvuYVJgijB6hFX/x7FtsK/XxVIYu8htw=; b=Yp1QoSEdC4WsfoZvZC7q7XFf3F82OWRzBMmsPsaUShWVs5cd+n8Og3V+eJC1eh5R8t yzdeoJ/QUC7IQomwNNPkvJjJInn6PbhFG1cqnZla9fLDSX3y7bWIgiZK7s9KSieD+4Us ReBVYepOFY5OrWr/MqQPUj5EQqkNjbeuMGFZSvALLKIpyQpJfC8ojf4/9XPUc+xe6guc gHuM0VRCpujy/zTz/fc2JbaOzaaEs1/pQzO2BUo5C4NbU/q215UigDPrzTZeQJkXT77Y r1Nyd02bGO8M+Y6LYjqE3+4YfxifcmYyVfNCSTXDohBxJcPaeGI7AGrMbV4Qf+TwrSOM 0fyQ== X-Gm-Message-State: AOAM533bmGzjeJ9gwLDMjafAe8hDgoibQTcTVLMRnkJmluawNVl+ycZD zgD2ifKnuwiYWwlSd6vd4aw= X-Google-Smtp-Source: ABdhPJxS173hg461uEBdXz771hL5VcL7CVZSin+58iHnkkss5F1xbgR6RTLPK1Jd/Sllu+HaMOYZ0g== X-Received: by 2002:a63:2543:: with SMTP id l64mr2514599pgl.164.1597426299852; Fri, 14 Aug 2020 10:31:39 -0700 (PDT) Received: from bbox-1.mtv.corp.google.com ([2620:15c:211:1:7220:84ff:fe09:5e58]) by smtp.gmail.com with ESMTPSA id n22sm8522973pjq.25.2020.08.14.10.31.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Aug 2020 10:31:38 -0700 (PDT) From: Minchan Kim To: Andrew Morton Cc: linux-mm , Joonsoo Kim , Vlastimil Babka , John Dias , Suren Baghdasaryan , pullip.cho@samsung.com, Minchan Kim Subject: [RFC 1/7] mm: page_owner: split page by order Date: Fri, 14 Aug 2020 10:31:25 -0700 Message-Id: <20200814173131.2803002-2-minchan@kernel.org> X-Mailer: git-send-email 2.28.0.220.ged08abb693-goog In-Reply-To: <20200814173131.2803002-1-minchan@kernel.org> References: <20200814173131.2803002-1-minchan@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: E74881803F9C0 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: split_page_owner has assumed that a high-order page allocation is always split into order-0 allocations. This patch enables splitting a high-order allocation into any smaller-order allocations. Signed-off-by: Minchan Kim --- include/linux/page_owner.h | 10 ++++++---- mm/huge_memory.c | 2 +- mm/page_alloc.c | 2 +- mm/page_owner.c | 7 +++++-- 4 files changed, 13 insertions(+), 8 deletions(-) diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h index 8679ccd722e8..60231997edb7 100644 --- a/include/linux/page_owner.h +++ b/include/linux/page_owner.h @@ -11,7 +11,8 @@ extern struct page_ext_operations page_owner_ops; extern void __reset_page_owner(struct page *page, unsigned int order); extern void __set_page_owner(struct page *page, unsigned int order, gfp_t gfp_mask); -extern void __split_page_owner(struct page *page, unsigned int order); +extern void __split_page_owner(struct page *page, unsigned int order, + unsigned int new_order); extern void __copy_page_owner(struct page *oldpage, struct page *newpage); extern void __set_page_owner_migrate_reason(struct page *page, int reason); extern void __dump_page_owner(struct page *page); @@ -31,10 +32,11 @@ static inline void set_page_owner(struct page *page, __set_page_owner(page, order, gfp_mask); } -static inline void split_page_owner(struct page *page, unsigned int order) +static inline void split_page_owner(struct page *page, unsigned int order, + unsigned int new_order) { if (static_branch_unlikely(&page_owner_inited)) - __split_page_owner(page, order); + __split_page_owner(page, order, new_order); } static inline void copy_page_owner(struct page *oldpage, struct page *newpage) { @@ -60,7 +62,7 @@ static inline void set_page_owner(struct page *page, { } static inline void split_page_owner(struct page *page, - unsigned int order) + unsigned int order, unsigned int new_order) { } static inline void copy_page_owner(struct page *oldpage, struct page *newpage) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 07007a8b68fe..2858a342ce87 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2420,7 +2420,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, ClearPageCompound(head); - split_page_owner(head, HPAGE_PMD_ORDER); + split_page_owner(head, HPAGE_PMD_ORDER, 0); /* See comment in __split_huge_page_tail() */ if (PageAnon(head)) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cf0b25161fea..8ce30cc50577 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3205,7 +3205,7 @@ void split_page(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); - split_page_owner(page, order); + split_page_owner(page, order, 0); } EXPORT_SYMBOL_GPL(split_page); diff --git a/mm/page_owner.c b/mm/page_owner.c index 360461509423..c7a07b53eb92 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -204,7 +204,8 @@ void __set_page_owner_migrate_reason(struct page *page, int reason) page_owner->last_migrate_reason = reason; } -void __split_page_owner(struct page *page, unsigned int order) +void __split_page_owner(struct page *page, unsigned int order, + unsigned int new_order) { int i; struct page_ext *page_ext = lookup_page_ext(page); @@ -213,9 +214,11 @@ void __split_page_owner(struct page *page, unsigned int order) if (unlikely(!page_ext)) return; + VM_BUG_ON_PAGE(order < new_order, page); + for (i = 0; i < (1 << order); i++) { page_owner = get_page_owner(page_ext); - page_owner->order = 0; + page_owner->order = new_order; page_ext = page_ext_next(page_ext); } }