From patchwork Thu Mar 25 23:09:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12165633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB1F7C433C1 for ; Fri, 26 Mar 2021 02:32:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 679C461A25 for ; Fri, 26 Mar 2021 02:32:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 679C461A25 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EE71A6B0036; Thu, 25 Mar 2021 22:32:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBDB76B006E; Thu, 25 Mar 2021 22:32:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC19E6B0070; Thu, 25 Mar 2021 22:32:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id AE2306B0036 for ; Thu, 25 Mar 2021 22:32:28 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 728C28249980 for ; Fri, 26 Mar 2021 02:32:28 +0000 (UTC) X-FDA: 77960451576.34.193E0CD Received: from userp2120.oracle.com (userp2120.oracle.com [156.151.31.85]) by imf18.hostedemail.com (Postfix) with ESMTP id 609092000243 for ; Fri, 26 Mar 2021 02:32:27 +0000 (UTC) Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12PNA6go136814; Thu, 25 Mar 2021 23:10:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=lPo7Ti03CLdH4oj2DM8QS4tfFwfDCyyGc1h07TMfb2U=; b=SCJu1NnZxzGgLQ7S7QXfreRj7QqgRg2lsJnIyOXFE/AUTpKi5EdiyGtOGs7ZFLgsbigw 0PC/bH2LCQPzPkRc4epDpPN+QDTPj/i3cTjIb/dltxRmwdte0s6KSVmToBdFRRjt1CRX HA4dDXNwWxK5A9+OClMW52QQyx0X+lBGG0IXwmWcq9NEzmtTXjaqgoPfndFRCB/IZhtz QQOg6IFMnHjZCJTjlLnWR0eEZHOcGsjYRB54qFYZnxhvqtSGXKBuUyEaCLCepK0O3Mcz v3iIgaOebqxsMOy/qlYsg2f0QJV338vIlTfUyGP5URY2ymr/UED42eUsiCf3RsCZ55J9 nw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 37h13e8dxm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 25 Mar 2021 23:10:24 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 12PN1Ko6002948; Thu, 25 Mar 2021 23:10:24 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2177.outbound.protection.outlook.com [104.47.56.177]) by aserp3030.oracle.com with ESMTP id 37h140nhh8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 25 Mar 2021 23:10:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FlNOMrkhrGa8uNUt9EJC6kYU9JVSQi8xV9ga4v7MxEjy4qTAmsuUg+tBHx5wMizNXLd43tP9l2OjGl7R4qsI8lsObt4hnPzH03KYpprvKD1tbpmwPXMtFk++685K6xio/RCqTMqCewhSSc4KQuTmnp07WI9Vc9AFhHgt+w9IHR6noj4H2ioG1X7/LIePA/npZMY3X8MwdQEVHZsKVmgvhP3Ig9BYMRpCzGeQXnBPrmupwvBczZ2dbijqXgCIdFWKNHvxolmn8P1c+X1t5XgXqO4KkWA7mbExmrggBCnNGzSOOzs02hIvgk8iS+OxdSaEe720dx/Ak3llTAAzlz7KGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lPo7Ti03CLdH4oj2DM8QS4tfFwfDCyyGc1h07TMfb2U=; b=oOB69t4RKHex0+j9nt9T+TZlayMIb7cN/AJXiHA/YuVTNo2j+Dh4LNQELGO3cnkcB2Smw4lNYMTzZ7VASEQ2t1uWJmbd47j5KMvbmvgS6xNLWfIPFlhUMmdYueS+NSFaHlP6MuF0zY/aBspyKclWM1ALm1Yz98bbxxpntzd29TClIW0VGAQzPL5M+rJZ2C0CBHgcP1I3hsBajWb8QqgE4p8OEVLhY5BPO7LMh8owyKtTOlMGNUklVvhKQopc+ELzT0cIWONUdZ7SKesEW6rRtQTGvB+171rTmL80ad90QeUMe4Evuy544hRJmRGcP8MopfkaTDIO1uyzcYtRfEbHRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lPo7Ti03CLdH4oj2DM8QS4tfFwfDCyyGc1h07TMfb2U=; b=bYxTxW3KR0QJo2ptTtesjdAVIsZQb3SuLXpWlx9aK3F2yk+8wvvjkToU1tz0pgc+Wi1YmabefNTF9i7d0VIrSw2boZ8e4cAUIGPiSOk0kcJmy5hi4mU/prlHGZfQ0NzgD3v3svXQtHVoXs/nJGhtdzsFBUpzn1fHE438kALAgHc= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3077.namprd10.prod.outlook.com (2603:10b6:a03:8c::12) by BYAPR10MB2853.namprd10.prod.outlook.com (2603:10b6:a03:92::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.25; Thu, 25 Mar 2021 23:10:20 +0000 Received: from BYAPR10MB3077.namprd10.prod.outlook.com ([fe80::507f:a466:3318:65db]) by BYAPR10MB3077.namprd10.prod.outlook.com ([fe80::507f:a466:3318:65db%7]) with mapi id 15.20.3955.027; Thu, 25 Mar 2021 23:10:20 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Ira Weiny , linux-nvdimm@lists.01.org, Matthew Wilcox , Jason Gunthorpe , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Joao Martins Subject: [PATCH v1 09/11] mm/page_alloc: reuse tail struct pages for compound pagemaps Date: Thu, 25 Mar 2021 23:09:36 +0000 Message-Id: <20210325230938.30752-10-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210325230938.30752-1-joao.m.martins@oracle.com> References: <20210325230938.30752-1-joao.m.martins@oracle.com> X-Originating-IP: [94.61.1.144] X-ClientProxiedBy: LO2P265CA0082.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8::22) To BYAPR10MB3077.namprd10.prod.outlook.com (2603:10b6:a03:8c::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from paddy.uk.oracle.com (94.61.1.144) by LO2P265CA0082.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:8::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3955.24 via Frontend Transport; Thu, 25 Mar 2021 23:10:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ad2cc3f9-8144-4bc0-5b37-08d8efe32952 X-MS-TrafficTypeDiagnostic: BYAPR10MB2853: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: C+le9RwC/ssugtmFYnZxMmv9+VJj3BKbPX2Oxefp2BU8ZZbjgd9m0lxPaaNzw3rkUKyzp0mnYYPI6uKUXBTK2RkZgs9QDdZUsKTZUkSr5A7MNJmI7sTf61ZT8h2Osq9CWyW3SbE0pHiwMq7Nxr0yM3lsWKYS2lt+d6kt01VSeJqgojENHdELCyZBlmCwp6/gWjkrNesKJI8U71ofc0N+HArGp44Po2huE/GQjVK94MTJ4IkMuKoD4bys9SDCAd1c/WxOYdy01tK/ZYvtnbYK+5koiH8daIKuTW6pUm4MyhyltAnwIs+TAl7Uejm/eebO8S3itcHbSHOb7gr/4tUfPpSOQfNka7PlX4HpEu5dPY8kiPVJDaKtwPT3D7YJAdV+vXLb+Pe89lVuamN+0sv/iwudFo6iBG7yljvMxWz31n45iSLy0oVRhUXVVl19K04JcpqKhKEKzYeRN/vHJI7fydCbU4PK5j5g2FlO3Jcbgw50G57fRQfkfjjVvaCY+FyWZo2o0lDiI3doQot2VaovTaNkRM9fAIWRcWjuN241tZwP1aPLKct+WkAHyj95BaBEcaQSmepuPlhthAVWHTWebj+uC6P6Kf+tP2oxbq6jh3NTWHwegfQetRPLg8BSo3o7UxWtB9WYcgi2pZMmwQZZIG+Yj99sPE5jnVC3LNzO9R0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3077.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(346002)(39860400002)(376002)(396003)(366004)(136003)(38100700001)(4326008)(8676002)(1076003)(2616005)(8936002)(956004)(6666004)(6916009)(107886003)(66476007)(478600001)(103116003)(86362001)(186003)(83380400001)(7696005)(5660300002)(52116002)(316002)(66556008)(6486002)(36756003)(16526019)(2906002)(26005)(54906003)(66946007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 15hNaVsw1F0i21kGlUWylfbw3kiFhsYcxUlbqo6x1MxYxMidB1BkXXVbHWZ3mmNQTV1QMKuxmA1GpxZcqTF46kJ3MUuaeQhI6SWER/DxTLSD+MJbJcJtlaTy2Rb8YlwtIAGgVb3zi5Ev4uvP9ptD1+U1eXiouQkuu+IYv+QIqNntH4on3orO/uf+Q0bTXHdjnDy0tM8IaX7iLeFe7EsfFDyHceaeugUBw/SJIIcedppD7bkwhNMVNTHhybsvVM6pW9sr2be6syngH7Rv9ZunNDbWrLGOjPmzKJIl7TONRfGXag6qeYibkEEiGSrrq3KFyWv5AAE0hJ5TRnr9R4VFEVC5Q/U2YXkfjjOyu1uPGpGVZCDvjrOsWfNxUNY+XA+28fdzW0a+lCfDYVInxkmllwoJaIamAhF6w5obHs2NyLVEH76gPRO/J6IAcFDvYubMvZqVd3YuIo62/kGYs0lQppyV2OgVyuKu95b3zDSJ5qMzRtlpUflYqCJP6bkYfA7zSZLLNCOtasNH/qLD0Lp8jpIW4qyOiNJMjdxiW11WIkatkhUwV9ooQHrNjH9WgAVEaT8dgvWOikuIxp8XjPg7mXBL1Cm2UQV6hNrFB2Msc9f8GwRFUKwTKS0meJ19gP+5NuKa6ExbssZoA5N+zb+3DC90eKwzWzIz0F7sFDPIyd0OshZ80h4LUDhrK7lWENnSva6LeLZik+WdNu0FvKTWdsxEokrU6K7MqrJd4F1fJF+ZjYpAFov9fIIwhleF3ETJvmUpOmDWzSbSrtwGi6tJFaxuw3384O52XvpbehAV7rzsHo3JPoI9OruPuqS23E7il22dmn133ft2/DcKR2JDGOfvao4TQ65//nIIknkj0MHBFb72G7YHT+iDwXlH81nVN3j/RDXUKzC5HV83rsZKF8cywLcB1FaiS+hZATERvOGaQd7b0aNJXLPLZlRfyrZljkyiAKuNLpVQr/IDgp8xdYuRABF+DFxaKHNJICmyJNfUioTFC98blfphdYxpJNt5L5lYFzvGbYEDvmXySvMDjnHhNmOdfWHX2+ybwlybXMkcAPtFU26lg/f9RlQxGcVv1O9gXypYpL0B07iE+bbHsb3DkmloFtei5BAqFb/btqC2mYfWYG639Gd4humfSIPtGOw04DUbhfpTp3YghZITCX2+q9FzyX101HceOHpk/mNf9PGFPsLWNWSNCeyXWsXEnpHLEGnhPtVo6hayeoW2DLbDcee3HQdI3NFLueGB/txpX8nGKqXd/ZFhfB5WaGVWlpBkv+5RacnFYDUjdXIDQeda8vmv4xwaCIAVeXYu+Elabcxc8kJdRB8zf6Cy74fC X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ad2cc3f9-8144-4bc0-5b37-08d8efe32952 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3077.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2021 23:10:20.4681 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5bZPo4M1iS9RArrgZTQ4scNML5a0J6knRCS+/uOZ9gNge7AH/s+p8QYPJjr5HY5qCSByHfy43kQzC7adwUxIVpJugzjYql483LLDw1vo7WA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2853 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9934 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 mlxscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103250000 definitions=main-2103250170 X-Proofpoint-ORIG-GUID: 9SFP2AaPkjf1RiUFbRtNNwvkEne0BrnQ X-Proofpoint-GUID: 9SFP2AaPkjf1RiUFbRtNNwvkEne0BrnQ X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9934 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 lowpriorityscore=0 bulkscore=0 malwarescore=0 priorityscore=1501 suspectscore=0 impostorscore=0 phishscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103250000 definitions=main-2103250171 X-Stat-Signature: e8gei9ju7j97jnriktn7xyumgjq475g9 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 609092000243 Received-SPF: none (oracle.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=userp2120.oracle.com; client-ip=156.151.31.85 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616725947-114629 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a pgmap @align is set, all pages are mapped at a given huge page alignment and thus uses compound pages to describe them as opposed to a struct per 4K. With @align > PAGE_SIZE and when struct pages are stored in ram (!altmap) most tail pages are reused. Consequently, the amount of unique struct pages is a lot smaller that the total amount of struct pages being mapped. When struct pages are initialize in memmap_init_zone_device, make sure that only unique struct pages are initialized i.e. the first 2 4K pages per @align which means 128 struct pages, instead of 32768 for 2M @align or 262144 for a 1G @align. Keep old behaviour with altmap given that it doesn't support reusal of tail vmemmap areas. Signed-off-by: Joao Martins --- mm/page_alloc.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3a77f9e43f3a..948dfad6754b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6277,6 +6277,8 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } } +#define MEMMAP_NR_PAGES (2 * (PAGE_SIZE/sizeof(struct page))) + void __ref memmap_init_zone_device(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, @@ -6287,6 +6289,7 @@ void __ref memmap_init_zone_device(struct zone *zone, struct vmem_altmap *altmap = pgmap_altmap(pgmap); unsigned int pfn_align = pgmap_pfn_align(pgmap); unsigned int order_align = order_base_2(pfn_align); + unsigned long ntails = min_t(unsigned long, pfn_align, MEMMAP_NR_PAGES); unsigned long zone_idx = zone_idx(zone); unsigned long start = jiffies; int nid = pgdat->node_id; @@ -6302,6 +6305,7 @@ void __ref memmap_init_zone_device(struct zone *zone, if (altmap) { start_pfn = altmap->base_pfn + vmem_altmap_offset(altmap); nr_pages = end_pfn - start_pfn; + ntails = pfn_align; } for (pfn = start_pfn; pfn < end_pfn; pfn += pfn_align) { @@ -6315,7 +6319,7 @@ void __ref memmap_init_zone_device(struct zone *zone, __SetPageHead(page); - for (i = 1; i < pfn_align; i++) { + for (i = 1; i < ntails; i++) { __init_zone_device_page(page + i, pfn + i, zone_idx, nid, pgmap); prep_compound_tail(page, i);