From patchwork Wed Jul 14 19:35:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12377787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90EB7C1B08C for ; Wed, 14 Jul 2021 19:36:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 48189613CB for ; Wed, 14 Jul 2021 19:36:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 48189613CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1D32B6B0095; Wed, 14 Jul 2021 15:36:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 185546B0096; Wed, 14 Jul 2021 15:36:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF1926B0098; Wed, 14 Jul 2021 15:36:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id C64B36B0095 for ; Wed, 14 Jul 2021 15:36:31 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D1D712A002 for ; Wed, 14 Jul 2021 19:36:29 +0000 (UTC) X-FDA: 78362200098.23.6BA453C Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf11.hostedemail.com (Postfix) with ESMTP id 50A0DF0000A3 for ; Wed, 14 Jul 2021 19:36:29 +0000 (UTC) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16EJVMMW031363; Wed, 14 Jul 2021 19:36:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=DgFa36vKxyDbO5Oq+1ov6YbXaB8wMCO9g9s3XnYQq5E=; b=C6KNc5jOO+paSUiDwOEPLvPr9wElDqMzZg5VF1a0d8Yz7TwZapaOL2M7OaMebXT0aVLA 1ZrSNPY+LOHenLm2pqFfj4jVW958BRsKhmKpesN+PQrieAbkR51P9rSkdEE/kpKaARqd 7DlEl3s1kbyA+JhDEXDkvkAokj5P4VC1ANUoXT8mQO1Fnvm275NC07cI0hl5dkLOL9o/ To5zIvWOPPXLmUrTaKDy4xxLlMvSETkBFXuUFVAxZwf/shRHnqJtpxvnOIA+GM6vGzht M9uTQ1fEMT0c4UtAXCWle/loxc7Gg7I1WfKE1U2TaX7wi1j/aloqxp1CRpO1PqA8DTT2 QQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2020-01-29; bh=DgFa36vKxyDbO5Oq+1ov6YbXaB8wMCO9g9s3XnYQq5E=; b=Bx9OnzGP9tq2kdP/DNtAeRVQhEUEaII5ovcKu6i2k/g0xMzqInXgRV8VQgSaHoSkaQkn kgQxcRc3i/SalyqeHaegzn2rWFqvz6bJUeadFTMeopvLJoVJJ+CzoOc8CtLK4qHO2bYo ZNlSGcaqyFGy4KMI/4THPUrbeE3CHsuYCT97f/lQvQO0RHJLLw7GJ5ImkWCFkR+lOjw6 A3nHQbHF8/Brn+L1cO88C3EKJmu2wgu6bTkMNY3z7GLKGOT2PmCKZ2lmG2fUaHOodv0b 53eZCZ0tbX3GpjOqgiqfZ3pc062lsJqbam0YsWhCuUoKmT1dVIC9awN9XyLP2Ma/JlUT 0w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 39rqkb5knd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jul 2021 19:36:25 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 16EJUXgJ021507; Wed, 14 Jul 2021 19:36:23 GMT Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam07lp2047.outbound.protection.outlook.com [104.47.51.47]) by userp3030.oracle.com with ESMTP id 39q0p98max-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Jul 2021 19:36:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FGjcXfZ1rwCjNTNfi7hw1qwfatSBYGogAHnJ9wyiVacuRa3ybyIRDq9y653sURhsIBNJ88ebSZwzp1wZrngTfxWBMaE8GIaEyzUfPdCr889ltUBzKQNpL6rVWMrZVS7O0/y/qjCWo5dDZSXv6z5CHocgNpZjzMjUD+/B3FYB6nV2B7ODUIiBCUrUKU2JA90ULIIVybRNRQL9zaUg7KCa9KP5CHVhyM05oJJRbw8UlGVHTE7fyTih26P6bH4v4y7J4oMij7I91EfihyIr7rrHEjiYcq1zL63a0UNazs+bu1m9Q/wFy8GS9fXJess/xYRIe+ZcSi8dJvEncj03YCkGtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DgFa36vKxyDbO5Oq+1ov6YbXaB8wMCO9g9s3XnYQq5E=; b=cAonrxfh2hbNGU/Ei5E3qDowR+Ws1LznVog6qugEhbPxzJ7+G3AdovYxPbnrov0DATg/S/uwHmTLino5b3YJezY/9OnhsOqLzlAvzdecPpoEzhumwJMzVDhfXplJObhWX2Nwd6CEHHIfxlUygnTze3ovZvv+a+snmNC6O4fVsEPyJKNL5N4tLYWpTFJRXPGAok21ImGm28tLAWH/tvPkg7h4yr5Gkwm18TdYdFA/v0u81TEh63a/whtCR4W1YAv8qcoz0a1UOSQHvUyx4ZnWSHsHm47XBL7d+TOxiN1WM44Y91fP25NbEDwO7qUjKStOgqzFk19J6N7X9Or3JzJlng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DgFa36vKxyDbO5Oq+1ov6YbXaB8wMCO9g9s3XnYQq5E=; b=NGcJAWTIePrnribpIPZQz/bUPjoCqwvHUSkjxAROYfhLb/cp3EbiDpmZ1sAtgnA/LjsqbHY7GzTeHRUWqQbF9nBWytxIdzSWnnqZu5tI39wAYUZvqM6i/RDw0FAKV9PD2QfVYGfupyrGZoGVGCz6Kfw3I3CLPlzmAmTW8PTYEqo= Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by BLAPR10MB5204.namprd10.prod.outlook.com (2603:10b6:208:328::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.22; Wed, 14 Jul 2021 19:36:21 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::5833:5ab2:944c:7360]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::5833:5ab2:944c:7360%9]) with mapi id 15.20.4331.022; Wed, 14 Jul 2021 19:36:21 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v3 09/14] mm/page_alloc: reuse tail struct pages for compound pagemaps Date: Wed, 14 Jul 2021 20:35:37 +0100 Message-Id: <20210714193542.21857-10-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210714193542.21857-1-joao.m.martins@oracle.com> References: <20210714193542.21857-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM4PR0101CA0048.eurprd01.prod.exchangelabs.com (2603:10a6:200:41::16) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from paddy.uk.oracle.com (94.61.1.144) by AM4PR0101CA0048.eurprd01.prod.exchangelabs.com (2603:10a6:200:41::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4331.21 via Frontend Transport; Wed, 14 Jul 2021 19:36:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2dc6b64f-a22f-4e25-1c62-08d946fea87a X-MS-TrafficTypeDiagnostic: BLAPR10MB5204: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z6ExAPWA9gSVMUJZYbq3jwbgIPnTpH4doipXlA1f5AdpVvj+dFRdvGft7SrA4qkgapC6+9iGlbZQV7vNR1aaCxRLGItkG21t+TWdeEZ8npvyUYucqiQW1FUO4+nKr/nGaKZaBwvUZQimH1roqF+fDm7/yNRT1Wf0rIIlvn4VoHC3KXYC7TVp5wRlhu+JlNAOSTs6N8My3uiuzuwGgD8wjlgWxGkqcD1JjW6+cuJg8BMtysukfXraLznuINwznCDvacDkRx87+FDx3mnuH4ZKnqwdJthhw459NjkW7iwyI48fwJ+pk92/SVZOK7Mjv+WnE/aMRTNr5TZsFDGfmOZbeJrpYYLwarQYS9eYKi+BFVzTGWvAq3XwrojQsdVQJL/kWyi8Y5BtLAYvZ1G884RCi5BbKfbZtLTxcljY0Pe/TuncHvqPFuupxrOPlEHmFOCYQ9wAeydKrh8qv1GdhQDnHO7Vj1LiuK0MdGIZnTsHBjhZWqJhtYS28RAFtWRFFCzfKlG3PYaxA3QwVO2z2hrSJ25/5HEKNSMwoewnT6FUMOpITt9Od2Bve6g23u8HOJ4eCxxiwsO37B9aZbaTIfDUthdEPMP5AYKGyWkKj3E8Xg4KiSdhpJb7h1GOwlyOAIZYtTz0uagi9PiXl2Nq4LlnpcUroaMjZmSn5/7eGUxhdbs9thpWSnALNrqD0Gp02IcLBhI8rIvgYiMnZCeoitSziA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(396003)(366004)(136003)(376002)(8676002)(4326008)(86362001)(6916009)(7416002)(2906002)(956004)(107886003)(2616005)(6486002)(1076003)(8936002)(66476007)(5660300002)(36756003)(103116003)(83380400001)(38350700002)(38100700002)(54906003)(6666004)(66556008)(316002)(66946007)(478600001)(186003)(52116002)(7696005)(26005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xCdSzq4kC4+LesmXrsbwWRIHsPdvuCJ9fP65dTlOSBSKKNemMDLcLJp/YhTRxWJme3KTK7eavRZ+P/hFigUDkXbr5v7IEur6FqmcfFI0hQJfnqb6hTvDmRbETDi68SqE3l7KJCrQQyCJPyAsBcik7aAfDHLgmzF0xaM3wGrkHCBEmQtsqBfBgV38lex+u9es1nILvybEgErMNGgUgcyBWCD9G9qooHz4VMrOW2HusKEhq8OKNYJ+++NpbCD41s/x+exE3CLnLAxVDA828V7AB5tfoaNXsc9MNliuycerQEl6/zT7O4pIuFGFtb/IJINLOSKozu6IHeL1IyvFsYM8GnPgJoNI9G5/5Ki1gwzfliihgRHF7hmd7e+/IDEBgrR9z53NplPlgmc2A5RtVl1YoRCafSQnojdD3sUPM+t7144xZNeaxYHRFxvnC+hhP6khJwxIkeCTWlLMZqembSON56UxOlqHI9DhMddWrA/OHgxJpf2i0AGgrbFOs53R2WrZM3N+SpqibZ1PQGEbX/gJBhunNJz7DDH9XBOdj16E8bW8x4plMJsNaoLNYxSQVpwpASym5ld6B8CPzKihzFOjy2uje0TpFH3b3Pb2FwFdOdcVGbrGKNXp5WRc62kW6mem8gCmw5LEEaOGv+QBS9IRsIDSTUhrY3TBS+ZQRmW+xGQYa/bhzM5nFsQgrTMyzxdZ3P3sTlsbRX3i+A5DiqUudH0vLbuSLXQuJNqZuozBF3u5PIySRuhsfvxl5j7NEPBa59veoeaDyRVFf4ANYFfIUOWNpNjBTvaxt/u56gSprd2sooQEBWtfmgKJriRhWgB2vqtre+oPA0QiZk3Rv9nAPLpihWgQbTTcto1sRWlUhNbdIkV1KSgN/eSvqhoOIu9SLp1+SGxCa26DOMG3uXHZ00ehH6l88zJzSEHzJxDvy3Gb6FGv1e0+wE0aALJCaFcjSoybwOPuAsvuFvU4GpTqlYT01Ju+FoTXPD3H5iq7tn5ZzyMqEdo7xiTNsutw7/LRAFd1U//WJahT3QFcXhh8I0Y7tWZtf7LKfP+/inte/ILqvSMXwrRIcAzaPmIm2+RxNVUk1FJOgtoWq4IQgxsCGAZww9XUyNT6o3tTVUFXV987dVuBrQU6iMjYJeFRTy/b1MztqK6vZpHuswO6biru5E2rrMTqJLdiqNNnEOdigrrT4Riwx87ppySNmTVqeMmOti/BUQfAHXfAOALmL8rhUjaXJWUABg6gWqU2Z+mw9mH4FLtlxaROdgw/GCpcGlTjedlLDn/5Hf9qYlH+MFw3w53AessotxYc2RPaDV21PQ4iz2ILhdCCbko19VmiG/lr X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2dc6b64f-a22f-4e25-1c62-08d946fea87a X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jul 2021 19:36:21.4178 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Sin5DG/cO6/Bfdlm/uP6T5eRg+O8IYknnIgCIM2kjAED9UAjq0r24LsHKeb0RTU5QezqHrllPRhNGkk1Nr9B5in5GLltp/bDBQQ1m/lxypw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5204 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10045 signatures=668682 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 adultscore=0 phishscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2107140115 X-Proofpoint-ORIG-GUID: PNMdpS0IzVgF6E6OfWgr2zg7oxkWjhQK X-Proofpoint-GUID: PNMdpS0IzVgF6E6OfWgr2zg7oxkWjhQK X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 50A0DF0000A3 X-Stat-Signature: nk85kwwnmncz9b3w9iw45etgikxfdj58 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=C6KNc5jO; dkim=pass header.d=oracle.com header.s=corp-2020-01-29 header.b=Bx9OnzGP; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=NGcJAWTI; dmarc=pass (policy=none) header.from=oracle.com; spf=none (imf11.hostedemail.com: domain of joao.m.martins@oracle.com has no SPF policy when checking 205.220.165.32) smtp.mailfrom=joao.m.martins@oracle.com X-HE-Tag: 1626291389-647196 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently memmap_init_zone_device() ends up initializing 32768 pages when it only needs to initialize 128 given tail page reuse. That number is worse with 1GB compound page geometries, 262144 instead of 128. Update memmap_init_zone_device() to skip redundant initialization, detailed below. When a pgmap @geometry is set, all pages are mapped at a given huge page alignment and use compound pages to describe them as opposed to a struct per 4K. With @geometry > PAGE_SIZE and when struct pages are stored in ram (!altmap) most tail pages are reused. Consequently, the amount of unique struct pages is a lot smaller that the total amount of struct pages being mapped. The altmap path is left alone since it does not support memory savings based on compound pagemap geometries. Signed-off-by: Joao Martins --- mm/page_alloc.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 188cb5f8c308..96975edac0a8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6600,11 +6600,23 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, static void __ref memmap_init_compound(struct page *page, unsigned long pfn, unsigned long zone_idx, int nid, struct dev_pagemap *pgmap, + struct vmem_altmap *altmap, unsigned long nr_pages) { unsigned int order_align = order_base_2(nr_pages); unsigned long i; + /* + * With compound page geometry and when struct pages are stored in ram + * (!altmap) most tail pages are reused. Consequently, the amount of + * unique struct pages to initialize is a lot smaller that the total + * amount of struct pages being mapped. + * See vmemmap_populate_compound_pages(). + */ + if (!altmap) + nr_pages = min_t(unsigned long, nr_pages, + 2 * (PAGE_SIZE/sizeof(struct page))); + __SetPageHead(page); for (i = 1; i < nr_pages; i++) { @@ -6657,7 +6669,7 @@ void __ref memmap_init_zone_device(struct zone *zone, continue; memmap_init_compound(page, pfn, zone_idx, nid, pgmap, - pfns_per_compound); + altmap, pfns_per_compound); } pr_info("%s initialised %lu pages in %ums\n", __func__,