From patchwork Sun Apr 24 03:27:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?5p2O5Z+56ZSL?= X-Patchwork-Id: 12824746 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14993C433EF for ; Sun, 24 Apr 2022 03:28:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 88FA26B0074; Sat, 23 Apr 2022 23:28:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 817136B0075; Sat, 23 Apr 2022 23:28:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66A526B0078; Sat, 23 Apr 2022 23:28:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 513C96B0074 for ; Sat, 23 Apr 2022 23:28:10 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2E6FF620C2 for ; Sun, 24 Apr 2022 03:28:10 +0000 (UTC) X-FDA: 79390339140.27.D5310A3 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2067.outbound.protection.outlook.com [40.107.117.67]) by imf14.hostedemail.com (Postfix) with ESMTP id AF4E6100038 for ; Sun, 24 Apr 2022 03:28:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WN78kriZbZpsSVFXC0gyiDlBbvwHe2ucP67NJzmffIrHLQ15PhHqIbtRarNbeWgoVKgMvwts7A/T6E6CumlgmHp8Q/eI1vR++yolprkhYunBNjyzX85yxrZrC7V+Cdpr1A304G0vykbMyB6KxkUh9gMu/6ErGUU3sZ1hbUBDOx66BGNnVs9E+PsMA9VMzOyGQmukEXTh6z1ZLwF0g2LRKF++Kbul6xcQ0VSoBKYg3Zit1Slpb7CqNDoHXBH+Uz4AL1fqAdHnqb7cHqWRyLe7K66gkxSzSxR40kXUM74J/owRSfhKx0ZHL9x3C19pTTszBdHu+HJ/fYRb/c+h048HFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pHxqsDccDxmcC1wRN5mxcS73GIzXKkyHnLYJUW+G9W8=; b=FNNvrjuGXX9cY3+zcC5jZrYdYCILp3dc4y6U4pj5iglYsDCARi1FntDC24I54veWZuY05ax+vEX5kH+iULeuU9IM47Oydlx6n8EPD8hDJnQvDaf2TYwZCdkbjXWj7kfPXZLT+ctWGkXxYBCaymx9Ion8IBFpW7vUCjfc+NVT5HXW6OX7FhqWcjx0D/s5WYpikCr3CVr1cz+eCCqduyEk2lntg6StW0SKawZIPu8/eK73E7kkbBvO9VPCgvDvIiJrIFJ2mXIWVhliZY9c41NauOgBpF5kmypugvZLw3y9lk+v1oHXmmZwGOm5uqlHwoVUBE52zwrKOk8272UpjFqC1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oppo.com; dmarc=pass action=none header.from=oppo.com; dkim=pass header.d=oppo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oppo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pHxqsDccDxmcC1wRN5mxcS73GIzXKkyHnLYJUW+G9W8=; b=uFlMjp8jyURyA2t/+N34+s6/07ic6+KLjIVFFIoCvMrcoXpAWYVazq2cXW8xdhXbsT0SL1Vq15yVwdUGu51m3BR5MvrVyS2TeYZhS+dy1c6n10Iqv7Trz8JcvYwrXd3uANpYiBdo9nRxwk8RqIA4gG4SqcBPESdkoNt++rw3Evk= Received: from TY2PR02MB4431.apcprd02.prod.outlook.com (2603:1096:404:8003::13) by SEYPR02MB5797.apcprd02.prod.outlook.com (2603:1096:101:5d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.20; Sun, 24 Apr 2022 03:28:02 +0000 Received: from TY2PR02MB4431.apcprd02.prod.outlook.com ([fe80::8179:176f:a62e:c998]) by TY2PR02MB4431.apcprd02.prod.outlook.com ([fe80::8179:176f:a62e:c998%5]) with mapi id 15.20.5186.020; Sun, 24 Apr 2022 03:28:02 +0000 From: lipeifeng@oppo.com To: akpm@linux-foundation.org Cc: peifeng55@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, 21cnbao@gmail.com, zhangshiming@oppo.com, lipeifeng Subject: [PATCH] mm/page_alloc: give priority to free cma-pages from pcplist to buddy Date: Sun, 24 Apr 2022 11:27:33 +0800 Message-Id: <20220424032734.1542-1-lipeifeng@oppo.com> X-Mailer: git-send-email 2.31.1.windows.1 X-ClientProxiedBy: TYXPR01CA0052.jpnprd01.prod.outlook.com (2603:1096:403:a::22) To TY2PR02MB4431.apcprd02.prod.outlook.com (2603:1096:404:8003::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 60966777-a3ca-4092-8d8e-08da25a26fc5 X-MS-TrafficTypeDiagnostic: SEYPR02MB5797:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6LfatoxbP5fG4IEpxt3/TEY1sliCmoFh0lllBJLRREayf1Z/NajPnN1iu30At8GbpZ7IUCxmXF7KQXqxE2SwIhQLdh91Z0jEoJZl3iLyaRfexmFlLmJ3wanVkzFiVjAYoblefqq1oGTfNQ8N+Kthc3HoBwTOsOmbu+jdJKOzuXy7q+0hXrKd1AjCYgHozTQEiTGVF/V9wk6DiHA2paupxM30rkVz04MD48clOfh4M3Ve9aKT+avp8jyIBNTGt0b2NszgjY/ktBATp76A/zcytCAo0rFnzFefpTwkjzTsI4YL7OFUAElbldInkluQGe0Sp1GBMCc0ZTvGMwA0t4EBjyCMJvr7ZcuCRponFOKKc5KtffVe77ODx+dt36HrlqhvnmZcVmj4bL6wURbp2ogd2mdPZSIyI65KZCFCim7fir0uf1D5rIl1RziLhBbmU/jhE7Lf5S8FCWZI65lXDC6ygagsjxZJSO1i7EozWk+VR5edfRlGTk6AuWRhY/j6EIdyRIYZb6+6ZHUd5ZgIIF8X3xxClUjqdStUZurecCd7tEebUVylTglYZqZe3VRpWSTKR+prmwRj22v1ZzGfOVm3akfgypfTcfv2bmPDb369geb0RPhPf8DaLP0XRDXLhzmd/tGPr/4E/J36fkVEnQUS2ZiAlz+3eAk35+tpdfkxQwM//0H72DZaHLEJVdWbwv6RcvKLr4X78ZssFsG7/4y9ZA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TY2PR02MB4431.apcprd02.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(6666004)(83380400001)(2906002)(508600001)(6486002)(5660300002)(8936002)(6506007)(52116002)(36756003)(1076003)(107886003)(2616005)(316002)(38100700002)(38350700002)(186003)(6916009)(26005)(8676002)(66946007)(6512007)(66556008)(66476007)(9686003)(86362001)(4326008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ecgpZcX7PCkO/mYrmRl0aQ/XGvEwWdNymSNsqLKBYSA3/2BQZLF4CX2i7QGWdW+BlW4Kn2n9Tt82oid/6BCEMS7Ds4J15PqYNKOY1F1pPOQPlrmkoeODC1UgpRNL2JHise2leDnuTnf/tkisOtpdzE6UmVko8niXUKXhALlUlm4/uRCTJ5CkGXbBwzm3wkIVcUogT23lh0cpxmxx0ZXPo670VQtNcFQkBUF6ARQihCrd1QywJbcLAEF8n9XT6cFPzTmCyUCORFqxw25AGjhMJsTbk70fXNFMPoQZDYW5Xj/mOa19Qec1UY7ek4SAEAfx4klg2GYSGyIKtSQACPOafgsn2RxamZhdv9U5PztZA/oHHeWT5VWepoTm6XlTJEsKK2pYgcewuZOY+bNBdLkXeXBCwHE0nh7ufusWqWFcz2sJsqbTab+fvzWDjJ7DVFQvSOnNH3D2cfj2hvsh9goYgSuQGpEjDoxsxQnEkDaqHYlSAb1hAPs30aq1QHfNJZ1aLUWlqXgHiuImTrExquoaOHQRIgL+LJ1pLpZAdaVn4HpqA9b2rWd84n6V2KBE9OnseGDMgillA7mXxLRe6gLXr0PsmUfNG91sdJmYRpM4SgI2y+xuhrDCggllAnUdvA95Cv2/L0D7FiaJInyG61t37EHn1VnycEUNCjgfUUbogxRz9lIP1lcdqa1XKvr2HBtnJidVAAIwB+ECkC2bkuoiMEieq35rT2Idh4qW4rtrnf6/GMjk2OvRgbX83AhGoIEm9MoLQ4Jm6bJUOHyzZh4SljtZa2mU6+lNWaH0wn6qWqc+zaIJSg7A+hRjzrTCUkTEcXTbGxhV/57uoTFXSDIE++Bply+5Vf3r16FlnMPsjMWKNcN6jtnivpWyRTYomMzRmPGt7EaX8rCisnX6XE9lWAQHdXojs9tZPF7BzJnR84l1VyhO1mGWBh/WRi9BTJxEOWP1iYCPCO6xXUqfHBVMDEpP0duoYnqzyafmkKGlNRAyDAPcbE5MgHk0QLCYytFoYKlFSLh5wp8I1/aRg0Mg9ZOcOPHgqcRHvr/ZMhUu060MOeW61EdxjtZcYShzXcIOihZoaNvrh/nYlg25iymaL1p+lSq7q5M0L26VJvD3PhuPvK4HIfhjxsJFvOTaMK2q7GKr4Yf0VyFCS3o4iIW6QTfkc48IKbOCddyncms/8/0JDqPrNjv0s9M+NN3GAWjbHplspzvlhURIM9yIL3cNOd6yBJ63MpvRgwcppbLp+t0UnigLDO/H6jj9P1hDZXgDOYE8MWR/dejJDfGjobEuDw3c4oh5JM3tJxHo0ZSHP+wgYmm44bQZUzyOhdHU3ke6jStvGROoTufbyaoKGTLwuMBIJ4hn8UHN0ECCZO53KlYJVGKjbMreOvYB4ePAr27AYsgTKVTNW20vXU3549SUgPks4fEIHM/J13rE/LyT2OnV30znbA/j8CGanbG/m2uwK1MFqM26zauC4BPn/dp0yfmbl5UnrJHibvZVsREVPR5GsDm57wGtSoMK8caiax9sBMUj4gGMTWDK81s8I1lJ/yLK4TOEuulcVZvLn1/ah6Gzft+oHm5UAsHsI6CSxa63LMCVu/d3DPgm6Hw5PCKGqkuKoKSZbzIxJKVPB6dY/I0l/WhjC/s9IpvUHMFqWb2mPphCXRFowWPo8dLVxmjSOeN2Z5wGkEXPOMHdMBHaibifGpCvPdZ0HrDyVeOMmpnMgsN8VTDATHb+zl9eIYA2qAJtoC4JlfxUyobpNX6LLzc= X-OriginatorOrg: oppo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60966777-a3ca-4092-8d8e-08da25a26fc5 X-MS-Exchange-CrossTenant-AuthSource: TY2PR02MB4431.apcprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2022 03:28:01.9150 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f1905eb1-c353-41c5-9516-62b4a54b5ee6 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pCfj696N6ejORTUIsb9JPKt+TwZCCyI58810RUHrIssfltlRbNjq6uaZ3G/XF3RHauEise6fyefa5EvKZWvJSA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR02MB5797 X-Rspamd-Queue-Id: AF4E6100038 X-Stat-Signature: a697e48re6y7mrr17obe4swebrih5b5q Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=oppo.com header.s=selector1 header.b=uFlMjp8j; dmarc=pass (policy=quarantine) header.from=oppo.com; spf=pass (imf14.hostedemail.com: domain of lipeifeng@oppo.com designates 40.107.117.67 as permitted sender) smtp.mailfrom=lipeifeng@oppo.com X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1650770887-392799 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: lipeifeng Cma-pages will be fallback to movable pages in many scenarios.when cma pages are freed to pcplist, we give priority to free it from pcplist to buddy in order to avoids cma-pages to be used as movable-pages soon if there is enough free-movable-pages, which saves more cma-pages in buddy to decrease pages migration when cma_alloc. Signed-off-by: lipeifeng --- mm/page_alloc.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3589feb..69369ed 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3372,7 +3372,7 @@ static int nr_pcp_high(struct per_cpu_pages *pcp, struct zone *zone) } static void free_unref_page_commit(struct page *page, unsigned long pfn, - int migratetype, unsigned int order) + int migratetype, unsigned int order, bool fast_free) { struct zone *zone = page_zone(page); struct per_cpu_pages *pcp; @@ -3382,7 +3382,10 @@ static void free_unref_page_commit(struct page *page, unsigned long pfn, __count_vm_event(PGFREE); pcp = this_cpu_ptr(zone->per_cpu_pageset); pindex = order_to_pindex(migratetype, order); - list_add(&page->lru, &pcp->lists[pindex]); + if (fast_free) + list_add_tail(&page->lru, &pcp->lists[pindex]); + else + list_add(&page->lru, &pcp->lists[pindex]); pcp->count += 1 << order; high = nr_pcp_high(pcp, zone); if (pcp->count >= high) { @@ -3400,6 +3403,7 @@ void free_unref_page(struct page *page, unsigned int order) unsigned long flags; unsigned long pfn = page_to_pfn(page); int migratetype; + bool fast_free = false; if (!free_unref_page_prepare(page, pfn, order)) return; @@ -3419,9 +3423,15 @@ void free_unref_page(struct page *page, unsigned int order) } migratetype = MIGRATE_MOVABLE; } + /* + * Give priority to free cma-pages to buddy in order to + * decrease pages migration when cma_alloc. + */ + if (migratetype == MIGRATE_CMA) + fast_free = true; local_lock_irqsave(&pagesets.lock, flags); - free_unref_page_commit(page, pfn, migratetype, order); + free_unref_page_commit(page, pfn, migratetype, order, fast_free); local_unlock_irqrestore(&pagesets.lock, flags); } @@ -3459,6 +3469,8 @@ void free_unref_page_list(struct list_head *list) local_lock_irqsave(&pagesets.lock, flags); list_for_each_entry_safe(page, next, list, lru) { + bool fast_free = false; + pfn = page_private(page); set_page_private(page, 0); @@ -3467,11 +3479,19 @@ void free_unref_page_list(struct list_head *list) * to the MIGRATE_MOVABLE pcp list. */ migratetype = get_pcppage_migratetype(page); + + /* + * Give priority to free cma-pages to buddy in order to + * decrease pages migration when cma_alloc. + */ + if (migratetype == MIGRATE_CMA) + fast_free = true; + if (unlikely(migratetype >= MIGRATE_PCPTYPES)) migratetype = MIGRATE_MOVABLE; trace_mm_page_free_batched(page); - free_unref_page_commit(page, pfn, migratetype, 0); + free_unref_page_commit(page, pfn, migratetype, 0, fast_free); /* * Guard against excessive IRQ disabled times when we get