From patchwork Fri Nov 12 15:08:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616941 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 149802C80 for ; Fri, 12 Nov 2021 15:09:27 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACEQENk008613; Fri, 12 Nov 2021 15:09:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=blBh2EDfpufy4WWuhG5UL/QpYFWDSu5K0S3/ZSg8MR4=; b=SsEdWSdcmqD0wMH4GyoyB1+uRnR7ELyuqirQZlY2q++ICfRWFNJyh1qFc/MCTl5bnhBx 31FYaXgmeYM0ABovHwcoysuFg6VbJmg5I8Zh0yHa4btPSBbWa+IWSexLTz+eIdJDLHcI 07SeCLSBxbYiptGV4ByEGLF2EON0APoBlnumDryjMFP6aE8J6ctK1keW/JtJ3vjNwMhN 0LEfVWQ2AX4HHvZ/P51NolV+YgQ5HKoIn8dBgIMJqc82w5daX768tXbNEzc8tW7aKDY2 siG380DHRtMA/EQwQUPfCqm7+6gGOLLD5wSm99BGRi7z/9xr81Cb0/Z8lqbXPSqTSm01 kQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9kn51ya6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:04 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF5w0B196349; Fri, 12 Nov 2021 15:09:03 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2046.outbound.protection.outlook.com [104.47.57.46]) by userp3030.oracle.com with ESMTP id 3c842f8x77-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZYyLEf7Nmj2IEK62lY3uSCY3IM5eDZBq9QUblxKH+ltIrqGfUi6TaKZ0QX6vTnuK4wIYX43kTa4OQbWtfT0Xs1rLdq7vyiFBt8nP0+ccw+0qGNIFEmb4zeOp2EOHks1V6ra+UPEhW4kaOS0UnbWUDAYL/qIoons7vakAQYDTFBJMZzLd7z4VIpd0LR2Czw1Co2yEbKO2RQHAWe09JJ8NW8CJMQ8hgfSGgT3AUShJrXQAYTvdd/hnu4UQ7SKRZ9cQPjzfIBfrl0diyETC7jOhQK9kmF3innd+4seEMMXfyFNVYZT6YE471qMQGDpEYomSi/ZoFcD7T7QJcLT4b2aeoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=blBh2EDfpufy4WWuhG5UL/QpYFWDSu5K0S3/ZSg8MR4=; b=BQeWXtpEBYV73M73+GCodwssjWsQjUDSASc/pLNjmSCoUBwXeJww7wWhd7FXRm0QmiRB4FEuk0JJA1PuLJh3LAmW0NWaHHWWe0OWX+XyjSVkijuIT+RctX2IR7Pj0nHOaA9YSW50OWf2ojyv8SBigixxkzkvI+BVMQsNpwOsvaM+jWkU4WHOgRC2LOgkMJPsGx9oWSmLt+TtWa9wwYwtATdh3hGOnh1LUEoDFX62arV50S+VKRWfpnkGfVtjKHQCTxiA+ScJhsb84fHyIS20kPkUQaq/d4mFXvn/J0ubkf8nWeKqGaB4QcvoaGVoI+D3XGjxbEhVee5oBbR6TZemvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=blBh2EDfpufy4WWuhG5UL/QpYFWDSu5K0S3/ZSg8MR4=; b=Ra0EYswM63d+EuiS4i0VXFTn2iTLxnWJA1SNdvhGqIxpKqXd4Ex8f9xU4fUA20uTInhdf9r9a0tOpVcRLX+/c/KNs4dYSzpKCdNnIkJ3BwfMU4EyndFy8JpHwhR2Gp2TSd8LuR2SqWUqLLDCjiVx7m8NiHbYndhgYkNNXbj47vI= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:02 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:01 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 1/8] memory-failure: fetch compound_head after pgmap_pfn_valid() Date: Fri, 12 Nov 2021 16:08:17 +0100 Message-Id: <20211112150824.11028-2-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:08:57 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 85a4c399-ebdf-4a2c-7665-08d9a5ee5bf2 X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AKDjxXOFdQWpKPnxX7nL0n98chJq9NDCfWOnwqz25a5Ld+v4PJTcmcHLpbKoRT5uXTdcduokI4xBnoxnwvXZYMzwN+BjCCVJoO+pt5EaPNqrsr9qTNxMmRod1kaKpBcMsLBs6+bC3nwNFrFi8NwSL/bvuUpg5hWTbKl0ivaQa9+bZvYvAWQbmqZZtZvnXjcOdDNLwF+bQ8QnjGibeFWVTeDUJbWYY2LWEvokANO2ItjChUOQfh9RbB5UOb3e/cO5eSsL926zA8YORvyVWZFpWzQ3ek4xWPpsZF1SLwAWl3V/2gwqV7/I6kt5VbqDgSBSD9zgTGlD9ojP2tuWya06/nwdSEx3b4AjLHK9Jrmu6EwXWtR84ER3AA/THTRf9wJ3zYiAgKnGHJ0Y4Tm4oWbreGu8IO2rZxMbNOqt4Ju6QyuW6aSuDWp9Y5N8I8ocqJkJ9ju40ZdIn8st7KinDRH5UrwrnPsA5/PdUIvBchzSKDhhspdJeebV7Y97ApyaJd1z9RtPdx+VtYxiZstsN7N3egxtNJ1dk978mPrMncOLjTsLhVRFKq11+o2j/verzQX4NCaSQooZtCqEpSQWuTWgwuZA6oTinp6QL6gZZAJEKK3QRu6b/AVjASmB7Ei5h7P6njuMVV+b/cPSgXU9UPQNAkdfx7CfQfgiLPQGOvuzmy+Is0nrs2QkAJZvgCe/V+cwNGTk/TRWvAU1JwITXNB8Zw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nAqCBXgUnzBourGkMEcdi7XeGGyf8hp2EbM1MYcFJbDpfas7k0BREXv1MkxZ7K3GqJzQEN10s6ifnWiRLh5e9JuQTltIOM6mEPPyA4p8NgzSVtin1RyfKwiXQ83VnYJxK9Nod1G/tuRe2XGKOPKdF/Lbdhq1UaPTINfGWzuR4kvSKT+0Ctpgfw/casgQWGSpq4kU7eD4urgtgdIATYTQ1siNv9mrIyVyvfOIqPz4lU1q4/RH4z2tPOiXM3yxYDloP+nnIREfH2h99aRetY2t537etPOgw1ryZGOptQmbC+0d1OaZcrZ0yFCNyYYsA+mw1KUbZkRb5/zebNlft/HhPuor04he577gXbpjvHfubp+8YyTphtNZzuWPOpmXPtauNtikOz3QftZO0CX0HWEHp2xbWZRMIoqWDr5Od4BnN2IGU0KZRNxFoeQFQYb8cYycTE45Q4yrOSeUAsV+e4Zq7rm6qFVKUBy8DMbQrb7CKISnPljPvAzaTKDHxZDaELLN0Ge3IwhPQZsev/e0zf6o2Q7HWwVhEAHuG3kff48kDYcHbN+niy33WvcVZrNc0fvCJpGP+Aq07nMfTRf1T3lvZJoYHukzCVdz95fBBaBp2T5mbKsyCLl17A/MaXur1oXgt8YtoFpazd9z6ISQPju0DjTr0RalQqwKoheY6LaOcFJHONSAaEpsykpDsrWp4NWBSEPIDcKsf0sqlqC7EiKuDHxweiG6DHAFSVgqFWwJomDt9dhy4MMJL7/JSQG3ebqY57n6zXEMp2EqMUq68gpCZRR3YnZSt8VPMqcrzCxE1vy6uWXdZyiE7uPYWFnnYFRYniI9MN5F3qpD/Bp4jDXk231faqVP3UuGNOjKMHNQNNhSX12A0FiL0VDdjDQlrhkYFug6WTQS7bSpWMvCc3NsPguVP/s7TfmbUicjaEBP+LVFh/2NjhaVAsSzl7/BTkYTKAUVGRwL/GuH51RxT6MMm71ZqcZZTLWrY9UopaJZWu1arv3pDEnV3d4tCxbrzxOFrESTCw/FDhXCTGNKARstExMWDYoR8KGTZww93hrq+immBPGpM/9subbklxoi7IQr6zRbDagTI8NamdfEcuhvnVjjX8xfDTVD4lkdLKMT6GK/3e7HdDlQd+pmyRzt+muc6dLBK33fx5uZImnz8fTs5EMhoVZOlf1+MnAiUf72X1xO7gCx9anuAeIwjOxKqQrLBU8PkUToi86Gsf4oP0w7p0GC8GxLPknCUUIq5NpwOI5iZ/L9Os9xf9iY+BwcCLVuA+gXLexU8ZIaUl3Z4eFtjRZWURK7cB+VIIQ58o/rVJyBpk7t/TRVgo/sEdkGW+nTNUlJ8/+i0pHckkow4JLsfn9tcbTdFcJqLQ+W3NJLAev1bN9XbP0p9Vx3rHgUvINt6go4Js5lPMN1SN5p+9D/sz00NURY5fCGBvKDEAoUVgvGnFWRpf6kxWXFwi9QNrk5bM+V/ogQ8FxEGs1kud1paa1Rm1Z2Qj8YahZmFcqYFiBUCew78lMyp3eEokCOaoQUGyRYa/TIBbK3gRRqMfaDiilLKqJF0GcfGsr1ramJ79aHwJ5t5vx1vZ8YT7NnbFDPq7LdfQUVZXYyhV+nox5SjUPNNmUx1xnWhCfRW9w7zvJMyThl8rn/A2/UuLDRG+MknQtj+aUVyX2852n5drONRQ== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 85a4c399-ebdf-4a2c-7665-08d9a5ee5bf2 X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:01.8358 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: s9fvt1VNJ5i8N/4erZFF0ae2/JyYd0/8e758iD6sn4si9QxOS0hFnfPJdDLwzUV/0NbuLdUFVdGMKBXO1NNCwz2asUwLbIztsrd2FlmU8wM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: b53EmZGhpguVN8bzemXtEM5n4C3GIk_i X-Proofpoint-GUID: b53EmZGhpguVN8bzemXtEM5n4C3GIk_i memory_failure_dev_pagemap() at the moment assumes base pages (e.g. dax_lock_page()). For devmap with compound pages fetch the compound_head in case a tail page memory failure is being handled. Currently this is a nop, but in the advent of compound pages in dev_pagemap it allows memory_failure_dev_pagemap() to keep working. Reported-by: Jane Chu Signed-off-by: Joao Martins Reviewed-by: Naoya Horiguchi Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm/memory-failure.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index f64ebb6226cb..0c4b08da1a02 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -1565,6 +1565,12 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags, goto out; } + /* + * Pages instantiated by device-dax (not filesystem-dax) + * may be compound pages. + */ + page = compound_head(page); + /* * Prevent the inode from being freed while we are interrogating * the address_space, typically this would be handled by From patchwork Fri Nov 12 15:08:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616939 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C0152C80 for ; Fri, 12 Nov 2021 15:09:24 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACER7J6008739; Fri, 12 Nov 2021 15:09:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=GNQT0rdlCOrwq/ZrZF16oWrUySXVl5z5ltnGWNuSce8=; b=0zdRTGXFE5TrdKW+VyNYvDUEwdRGUpFHm46eNkd3Yxd7bLBRd5uar1SmtpuU78IIjnqc S+fVueOKWs83/0pf5FESZHuo/BDeoTrhQf72cgXGlvjp7s0qa0IRpWCXOcKlGlTXhB3r H/DgJE5AdIAgC1+4P2c3UCM9YuymYHELZcfJzydgjCE2Sj+yMmV5u76WSfJzghaQD1cx 6l8rGkd0hWEKz3yKLXnZU5TBGsyijzFsQLfa77Q8jYrn2NhUg95ZNIjFJmkgOoB/1htK WA+WmlARkILTMUclR7ocO/AR4DmwXbOcqQlMfBjd+u7vjYjzidrGpn5KB6R8DZb+Tdjr Jg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9kn51yad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:08 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF6F6f094734; Fri, 12 Nov 2021 15:09:07 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2048.outbound.protection.outlook.com [104.47.57.48]) by aserp3030.oracle.com with ESMTP id 3c5frjhw08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ROCLEhUH8TKIVg1698a+9I/4fHjva8Zgp7nYhTL2AHk2NY91d1V0geBE4cs27cQqDJQZ79+3+z08GdBorwp7U6AaYREmHjWjVHi4DYVoBxkqyHuCmrEvpdAhss75YEzPNBTR21N1kYpsPUq/XVrcYp7J+y2yjbFImAEw00VRePHNCfh/+jaeYtpiCxxEefVu50n7N9iUONmXElb2buFFuLGo2S6y8h+WW1FFUgmWQV89b1KyPgHm0utl7zmns2NJnrhP252PXRFyLjEqACjfbA9SvIJcb8+JBEXfktbMtAQ3foUacTnqjA71xxX/JFUIrlUiGtj6/Km4JPVGGReqtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GNQT0rdlCOrwq/ZrZF16oWrUySXVl5z5ltnGWNuSce8=; b=msU1CfdVqCBPHivMBBgdirlQBl9nUHCwKUUyZDtemi1MHOhOsmOnHsQRq+mvCoYSRIr863BfwIG8VN/MdVBVFlbOO9H/E8MxEIfIIV8VzvIDof+vsTWx68MJuN5ECEdp/JTHc2USkWv2Ef0M1MECrTRWBVeiivoZzmzpx44WbfC72xVY+RIFFnHSp7lABHGApgcymEXxyNzId2DrnMF7cBzvpqqFJDbufdNolKgyExEYob9N50/F1hUFyRMweHX6cwdaiBjcFocKMuvD5vGiW+a3vkTTsVZI64ry0qwyi2KOCBBo5DGL/04GKQmEl7KNyXBuQ3DhhnCggO23U7k9xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GNQT0rdlCOrwq/ZrZF16oWrUySXVl5z5ltnGWNuSce8=; b=kCGFXtUtwuBDBm4ru6tqljR/5tOO07VU1VjHACcOeFFsRkjAL2lEdi3dvSfsUELTsZ7yUbe2S9gJ8fKwb+qjsBXLvFXPry6nP4o+HpYdmegZakLy8eEitfp7qFFynxTmrvEKoVrgsxCDSMQkb1KJ8y0osi7BvcDhZcz0oOMffjE= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:05 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:05 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 2/8] mm/page_alloc: split prep_compound_page into head and tail subparts Date: Fri, 12 Nov 2021 16:08:18 +0100 Message-Id: <20211112150824.11028-3-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:02 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 38c139f9-0d7c-4f36-fb5a-08d9a5ee5dfd X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1284; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RdQ60aP6orIelUOzyoIsNqcpQLdBiG/oBJg5U/NXFWUSdPKuRgFrlFRP+YoP6gQuAnN6zUmx64NfDs8eDGKdgvSJOntut8UP93hLE+gzNqgvCflhTJRepjwJe6PD7HLUxxUw+crCqxwkNEBL1RAcfTY00mL6fTWEOsONE6YOW4pCVqyqv3x6UWQl4Dr4Shd920bSvPaEUk8yJPkSSFQxkSJkaenxWrunc5JYi/XHnNezA5UURphUSOZfPCplKB3CMnonWAlIUuow1E05rd3fw+KP8i5DjeSSEbxpFlT2nIm9EzJY5x3XcOSedqA/8smMKnNYKLOYRlbiJffszqWJJHd2BI9egBdtY7v4dVa5mdzhItcmHueCj24GXhlVD+AEJYIV3XdMPk4gXCfNA4ZLZuo3PHEiq5C+uYiCIARapJXN3MzyfirdGn68zpbmuV9uVb0YcS9W824VPCrMEAhG2W+62IGLkzWvBk+SmwHYwQDbRHGMsdBOdhjkSRA2V2JR53sHlR3uqpckjLO8W9++ueVbE1j2aPyXRGJY/8Q3PhYlCUZiFktFB+do9Zw1CairJMTcjVf2SgJEnydFJn+HF40OZbonDm6qzrW2lTL2VxxMoNrY2pvYGBevoQA/9PQp+CBReqCy2xKAm4KBr0cTkwQga+Vg0MOV/tlqw1Gh03c6nQrD1QFZFLtctpikCefriJyOBswKiaNKR8ORBlkgiw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: cLrKldoK1A0VG31EAZEgFDwrbjZ17MQWrMR7IBpjtkbuOpnGU2pMniEOKIg7HNDv2UOpRRmtOd7m3kpbWZ2Ny8jCqwKBFPZHypgqJVzw9+RvgpiCioUiJLkJHuntco5pWrNVs6GkUoci41P7PMnZCM/Wwrr02UZxvSqWXe/LymmFTxW6ISr9nhIdeg+KMpyah3xEMVt7eZuQdxRNiN+tUQ76/jImw0v/UvkOvY3o4D3KgnmuCqKM9ClH1VYJQH1gR1P5UR/PlxpavSOX/U2ZvETt3grtbXDgl6nfa5xMsR3y6waLYW1pkxMyJybZmF2G3m8Ya++OnpWQ0Cf/QoPxsbFwtVpx2Whv5aWKstHs/+t6S/kSrbAxizsbuCD4tXvrP5Y8IedyDF+QAtawBRXs7gihJbto8QefaEaniWsdLH1rcnsD16Kkl6t4hj/fosSSzxr1i3vorTXUeIEuBftlaAB6OoHEIIfeFD0iQcWHH0sBVTILofkQTIiDMOQPPt6EkZ0Bglz014gmLER+dZcpNhY7I4Be6OR9+iZC+M/fA95JgF5DazhXufzDCYWKXbZtbT7j8HXuSHxV0b+WvQnfLQdYrZSuvu6PId6TbFgCp/zb3kVbLu2Zvhd9YmQrzdG34sDCUWJx8q7qexFWgxtOLINQZJ+GaAsH0QKExdU+C47viUHfy9E9aj8XhyUwWx7j/XbdknlNhM6EZyKoGkVkRF4tFpgW+rkRXxZ4KZJ4KVFiwqbjZYuqZ1ZWo6r8NyB9lRnIICtqPNVPmL1vKnCSN31qiA2NIPFSA6JJYCSJlPgsajY4T7eNCO4IH4ifGusqSxo0Ld5iIOiOzUpYJ+Pp03u3+oSiWAFA0LKgHKgO63+r4y4ZLJjqA2ZVfSkrZq5pD25cywnPDxYprd3+luw1dJPQstGdy4dTy9e9lqmsV8p/KhVYDzG5z7f/AFeOY4suude/PKPobX8QcOvPbPVzcR2mYlSjMEuXVUfbVgeTSgcnOa1ZOcE+cq+OWqNY0gw0FzaUTrV7ox5s1esTBxC3tQwpq2i7VOIZhuu+yiso14wY9wztbZwql4cuPCK+ujGMkaOTCGIwb/GrFVXf//nuN+XhhsY0FpK1tJoPnk8eL3hnOHtA7VJuynPHaKNFSsVklb0LEdAkBEfKCuOJWvVWvYTgs13SkDf5kUjmjcjovNEDgCToAqYWEa42sKYzlinakUwgd536pFYQ4jAYdcJqp09wQ0jN4y5IZDBtK4meiNwinPrjCtdbZsqaJ+RUZNh+a4IXyf3ETM/7mE2kmmvS8+ZKZUVk+y+u1ebMxWeUqXRH8ozix80gx9UJPLCS+6TbLFJl3Y0yty04CeJFizAcpsDgf9y7mVP9aE5U8MrSPJ+ekrD8qS5rgBO8tcJngwQ80CVO66m7CDzqyQjPGDb0bHpiJkWJI5ubET1cMxTQPD3NDXnCRBgvWERw/Es08QdiLMgVu5WX1cgesc2f+Dd+3Sn7dwmU2E46ehkoylIIJQ2rFTK1l+Qn+cjNJlW5f7LOvpu7vVE/6plpiCzzROAQGruNmbG/ldRDR8c/qgCF+S62ZZleJn/38dk4TFWJRl8xjdqhWPljBbjiW3TjhiT2TCGVK7eFnaBDIqBgP+8NqJuRi15HESzNdXgiXvpfkiX9ITwfEUzJxGtxyxGUXmLPiQ== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 38c139f9-0d7c-4f36-fb5a-08d9a5ee5dfd X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:04.9830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S7NXfJlglHME6v2DWjFtOx7e1hqSNK/zynFhRO2S3YO554HQLUhoI+qwnm24NaRBMkOtn7X8ga/oVl7d7QhFdQCkaHVmCrtQ71Ojxsb8FB0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: GJtNzXO6OKfVOKS6ky7CuWHczRdw-WsU X-Proofpoint-GUID: GJtNzXO6OKfVOKS6ky7CuWHczRdw-WsU Split the utility function prep_compound_page() into head and tail counterparts, and use them accordingly. This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Acked-by: Mike Kravetz Reviewed-by: Dan Williams Reviewed-by: Muchun Song --- mm/page_alloc.c | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c5952749ad40..20b9db0cf97c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -726,23 +726,33 @@ void free_compound_page(struct page *page) free_the_page(page, compound_order(page)); } +static void prep_compound_head(struct page *page, unsigned int order) +{ + set_compound_page_dtor(page, COMPOUND_PAGE_DTOR); + set_compound_order(page, order); + atomic_set(compound_mapcount_ptr(page), -1); + if (hpage_pincount_available(page)) + atomic_set(compound_pincount_ptr(page), 0); +} + +static void prep_compound_tail(struct page *head, int tail_idx) +{ + struct page *p = head + tail_idx; + + p->mapping = TAIL_MAPPING; + set_compound_head(p, head); +} + void prep_compound_page(struct page *page, unsigned int order) { int i; int nr_pages = 1 << order; __SetPageHead(page); - for (i = 1; i < nr_pages; i++) { - struct page *p = page + i; - p->mapping = TAIL_MAPPING; - set_compound_head(p, page); - } + for (i = 1; i < nr_pages; i++) + prep_compound_tail(page, i); - set_compound_page_dtor(page, COMPOUND_PAGE_DTOR); - set_compound_order(page, order); - atomic_set(compound_mapcount_ptr(page), -1); - if (hpage_pincount_available(page)) - atomic_set(compound_pincount_ptr(page), 0); + prep_compound_head(page, order); } #ifdef CONFIG_DEBUG_PAGEALLOC From patchwork Fri Nov 12 15:08:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616937 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB95C68 for ; Fri, 12 Nov 2021 15:09:23 +0000 (UTC) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACEQENp008613; Fri, 12 Nov 2021 15:09:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=XT+ISnWwYhe5gFe4TDLrao6T9ENv/t2qqtZgPA+hHcg=; b=Hs/NHMbJ7DmjyunD7VAfaQ7Q3gYfMiqstvZjWODT5dpTGdfQuUq8bH7v+CPp1r7c026T Uw89Hj/12pp8QB79mOv4f4tCA29gmQJ1vFgwQTVjxYj4iZSXlgzEWX/VlPzTxB8qgs0B V1iaP2vSaDwDtmtL5RaSwg+Xnr/Pg2f9p2BQWxA1zzOSzR3W6rEWblDAxeXwcKAQB4oy wp73HTbA9HATtRlM666rHrYa41FAmk0A8j7OUJmcmFdrQxatLwnsgDBpqyenCrUB7+TK 6qunpa28KJ9VWL/7apZCoDdKNnx2KOTFEpo8k2uqqWajP/0HQnlF7pFgJLRHwKqV2iyr pQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9kn51yag-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:10 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF6E7r094536; Fri, 12 Nov 2021 15:09:10 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2042.outbound.protection.outlook.com [104.47.57.42]) by aserp3030.oracle.com with ESMTP id 3c5frjhw3m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RRrDcBtMunGWoZSDcLJ1SlD0rbnxJph1jxD7mqOYy0aZGPHVU4YA89Ji/dk+jRX0S7fdYNmpCWahiLy2RrenE2VqjwLghyB6xJ9lNQfTvxieKCFSHq69uSz4EbNOH7qvb/7uBk4VxITZUPoy5213wmp+G1GQGfd7fJXSU21dHD0nJQYK6wpW62YWpUbuWYIjLOUN0YioLnyZbapInoeLF7Sq6y8U6BVbzmRizlwF0NAkL6gpL765TybqdjvP/AX/HcrKVBQjlZUUnhTqiDNb0Er2TrnG0xP8uYK8fnrzmhOZus/NYBf1XezuUNr14MeBav2GETnRDkv4HZh5cRp+5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XT+ISnWwYhe5gFe4TDLrao6T9ENv/t2qqtZgPA+hHcg=; b=hh7uFJmVj1I6OU78722rfN+SeUpXjXlT4WPC5l8unrOEpwd0Rzc9lX/Bp7vZK1q7QWVGvamEkadey7TxqBQI7XO5EtVs950xS4Zncat06+/kJ03GYyjZ6W8n5JFLgP0DA0EZeQcbNoDiMEmC+rGKOxp17QcNIznrWZtPV/1+vgKPObEyzDtINc2lSX4QqMnQeEPuFkVXkJIMmzTHETxG1pUZEcRcoi+qxP6SeGXaSjHpdGJhXLWfTySkNOe2jlNGZaGIA6/J1uxGn3x5ZT/X0l78bsIOLw6Gv34pJKJ04dYEXN8KsE1ohdR3qUBpg8s9GQmh+qp//JNl9ziyEnIkfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XT+ISnWwYhe5gFe4TDLrao6T9ENv/t2qqtZgPA+hHcg=; b=nZG7gp2ARF1rRw+6GIXkYBLibP3+n2FaM12X3gwhU21FmrXsihKl0EDm52/7TddIR61+z0X6UKUV19xYyufsUeYmzUy9b234AHZ6bJBepk6YoM06ECYFTcaIGsuKqK4mNFQZauGSoTSk8UzluCjTYfGb4NHVNN2x8RWtCYRDikc= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:08 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:08 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 3/8] mm/page_alloc: refactor memmap_init_zone_device() page init Date: Fri, 12 Nov 2021 16:08:19 +0100 Message-Id: <20211112150824.11028-4-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: edee9e87-7b8a-4e85-b6e2-08d9a5ee5ffe X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EtDS2SGrHt1GhetLPp8kv+JiXtj+PCMl+c5a4T8+U/6DscA7bjd9CO5z1/jUi7Sr1kPWdivq9AXFuCVUOwfrZrivfvTvoKjIE0ycvcjyMD9QKNB0gB5vesSy+vKDITlPbT5jXrDMkOabEvEbHw5Gl0dBcWjiTK2IAjCQREupbOXzmmYZRgVbRrI3MaOGHAjnvGIY0JR5QBCM6lBLp5b5YacErm25Cp/JlMXqhVtU+f2GRDsPEXwGps62TvJZziukekskoCVvHI8jxJarcC1QWkcgSYXGwgNbwt4rRjSVNPkb+G4xC6IpyAGhVMQllKBTXQ9btZ+J7wzVaMY8XJKk8MDgbagd6Qwzv8DmwYUs45eTXo9Gr/rzLoTYSazSdRrQnq1qwnQkd0tbzbjPM7W6Mr4+E5Z9oI7xbi7gRzov6DEbDvJ7cpEBMvmylyoez8y28PXK9ECoxo0WOVRkRs70P3VIW8pAD2YWc7wkLHgdrfzzi3CrJE684NPHNxejOEJ5kVpo/aDdEy2Xu17zuBjB2NOrx9rvy2Sb/1oM7rc00pSawc+pOzM0OC/uz+W8v6Z2QFZkbN3jORybEII4enQydO2MLYBjS4A4vOooI7f3BGcf8KZcx8qFi4DGMJHMJ3AMfc9TOYY2JUBskIXKKLYrWQyIqc5ZhkIUhowPId9aZkJ3RLY0HiJp6x3sdKm2Xjax0j1pz8D9ORT8TLrNftzEvg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: f/C01gTGxdpALRE9sDY449/FNX5nu4MM5RAL2cvK15L3ZzeBvRQbezJj/LiTE+1unfLv9alOdH/37WkctMLLbjDg03DSrLm/a+hX4LjV92zxTspNNDRm50pzhAcE8QML8OrrcpwnF9bFRWF1IeVfpygErRVZB7+iOP7ORZytNDbZT6vUjaUHliRDCSVlv7q7xuwbBokUMeeatG9+aWmY535fU2uwk5/cqLekjDAOvrUw7xDtwp8Oblz2L8G7XLztsM/fRZBIKpfddHvQi9wGM9jKJAprOpegl40m4mXR4NSLVucylOCWGh18rg/G5Y1qGPXGCDZgSTYQwkIoKtlzPFrOtgmUvCqy7aryhWZE96zgqcNjlRE+IIzMVjjZWAnOryFIcgPUjrvPoRIjialXDyCjTNPlIiJ+1JwHoLxvpfASl+XxwuA+sfRu4WWx7i7o4SZu1A4kt/sr1XYFaBvrN1MDoA7s/BipW26jp+0F5R30jFpW4475CMTGyWyAa6NctT4g4zl3/52oj1Y2pfmGGBFzsjjvZZ6W0V78wNChqhNfnrol/7oWm77tFDG/LuPDGHxpb98/jK8WNf/hYN5X2+YjBS2m1UhDoNssU1Eej6PsjPA4wEUAcdVo0fyDE8z9T7bPL9BiRdJJxyDUSqaaczK2ClKGsoU7DvQYFPn3QHK26nsk8SdjLMaWgHVBguMn3u9bQnpy26VZF5mRCtKoKvemuv2GxdUWjyMgZDZze702Jvip0Ewm0h85dRYvVM9SoLy4D+SYqO6rpwwT3d94CEysB7ECgJlyFcEYmTBG1FpEUcMos2cvcRkmfezUtXpjntNV4t8Eid5k+uKdYNuji8Kf0fyVWZcBq6dmR+GFOMBk4y7JShtlUAHHoher4VEdsHX04soujSU8nbwuMa8kKPOcWIPTSJ0c5eQOSWFkBXwQu6F9HBBjaRqlbB7XIHVe+eqGOIueK/sDJXj1QKK6IBubl6ik4FtgQ6iLXe/2/nbOrK4JeWbavUnqV2Ws/U43wGbeSOogTp7WqK8CQFtCD8nifylzLwvnwgIWnMq5UiKIIEREvGKU9ynpMOBbmZ+4tksgdgPtwHYyFrpbnrSzCLXiHIojxs47s+AjHuK3NFcjD46psHXbyPVpaV+LjatRpHSg98XxXiRLKgAeocxupwN/tXiaWNl+l5AVHBAKAcYfcno026ozTbR5mSre4qf/SpOxMn7CObdA12IGOxGhtr/jugTRGGLcVfB4ellWNfMja9imu1A9QgDO4N6bqpTFb95WHJSYzK0jey0xZVvJIOURmHfeqoq4sphR5YIoV6JUFCX4o/N7gShPP2tvDs7i8SIjPCDyglThe63yyJToVzfpfY2pKyJo/NpkIRA8aaByzHtFy1T5k43igQdpGAwAttsUpO7Sc0P7/H4OdlfEJ7bwQpz8qhzFqLcjUPxHnHgFFQIg+UlaVCPJ7g8GBPi1Vfaw18DP43JJu1x8Fu4fEP2QjaSriYGh6MkVWkdbgBPIqkqJ+2Q9i66Ob9n5XSpppaYKPOiEGs8o2uUbAOHEtNX/4CxfFqvqU0f0T7+W/SVABzUM4k8De+q9rzurB3xmo7gFmTCqnB57UCQ1/X5bmm17R/Ojhz/qsDZNJ8FvhGj1gTcslu3ygkQ5678Y9KNAMbUIfllxvIxyxFM9PWi9ew== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: edee9e87-7b8a-4e85-b6e2-08d9a5ee5ffe X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:08.3174 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lqglSBoPOCXsTnIb1xkFJxITXijHlx0gYfHTiacHBRB7hNVe5UySBodCKe/YMxkCFNNhskZYx4VYQnHocK8J4CMtOT4YqityiUK+SnShXLY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 spamscore=0 phishscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: 2w9kIrxEcBcE6R3mx1NROGfNht0pAuTm X-Proofpoint-GUID: 2w9kIrxEcBcE6R3mx1NROGfNht0pAuTm Move struct page init to an helper function __init_zone_device_page(). This is in preparation for sharing the storage for compound page metadata. Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- mm/page_alloc.c | 74 +++++++++++++++++++++++++++---------------------- 1 file changed, 41 insertions(+), 33 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 20b9db0cf97c..23045a2a1339 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6572,6 +6572,46 @@ void __meminit memmap_init_range(unsigned long size, int nid, unsigned long zone } #ifdef CONFIG_ZONE_DEVICE +static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, + unsigned long zone_idx, int nid, + struct dev_pagemap *pgmap) +{ + + __init_single_page(page, pfn, zone_idx, nid); + + /* + * Mark page reserved as it will need to wait for onlining + * phase for it to be fully associated with a zone. + * + * We can use the non-atomic __set_bit operation for setting + * the flag as we are still initializing the pages. + */ + __SetPageReserved(page); + + /* + * ZONE_DEVICE pages union ->lru with a ->pgmap back pointer + * and zone_device_data. It is a bug if a ZONE_DEVICE page is + * ever freed or placed on a driver-private list. + */ + page->pgmap = pgmap; + page->zone_device_data = NULL; + + /* + * Mark the block movable so that blocks are reserved for + * movable at startup. This will force kernel allocations + * to reserve their blocks rather than leaking throughout + * the address space during boot when many long-lived + * kernel allocations are made. + * + * Please note that MEMINIT_HOTPLUG path doesn't clear memmap + * because this is done early in section_activate() + */ + if (IS_ALIGNED(pfn, pageblock_nr_pages)) { + set_pageblock_migratetype(page, MIGRATE_MOVABLE); + cond_resched(); + } +} + void __ref memmap_init_zone_device(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, @@ -6600,39 +6640,7 @@ void __ref memmap_init_zone_device(struct zone *zone, for (pfn = start_pfn; pfn < end_pfn; pfn++) { struct page *page = pfn_to_page(pfn); - __init_single_page(page, pfn, zone_idx, nid); - - /* - * Mark page reserved as it will need to wait for onlining - * phase for it to be fully associated with a zone. - * - * We can use the non-atomic __set_bit operation for setting - * the flag as we are still initializing the pages. - */ - __SetPageReserved(page); - - /* - * ZONE_DEVICE pages union ->lru with a ->pgmap back pointer - * and zone_device_data. It is a bug if a ZONE_DEVICE page is - * ever freed or placed on a driver-private list. - */ - page->pgmap = pgmap; - page->zone_device_data = NULL; - - /* - * Mark the block movable so that blocks are reserved for - * movable at startup. This will force kernel allocations - * to reserve their blocks rather than leaking throughout - * the address space during boot when many long-lived - * kernel allocations are made. - * - * Please note that MEMINIT_HOTPLUG path doesn't clear memmap - * because this is done early in section_activate() - */ - if (IS_ALIGNED(pfn, pageblock_nr_pages)) { - set_pageblock_migratetype(page, MIGRATE_MOVABLE); - cond_resched(); - } + __init_zone_device_page(page, pfn, zone_idx, nid, pgmap); } pr_info("%s initialised %lu pages in %ums\n", __func__, From patchwork Fri Nov 12 15:08:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616943 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72F112C80 for ; Fri, 12 Nov 2021 15:09:28 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACESmWC007211; Fri, 12 Nov 2021 15:09:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=BI6yNaHxxMrzht4ixewrCrLdJzu85PxHJWx8g0dNNIA=; b=yG76r1pyDAlbLLT5VOBbQYeA0Wg+ImhAMfyGLosQvfczN3G/w7BVE52ROQ03w4GBi+0W //rWbjj7vOxfJRoOnU2mJPY8UaMyeKIsUUn4FAWHfROJyvindzo5PkcrUxi3EKAsT0X0 h5NprzQKTNSAEo8ffm5ubVdbnB8ZlG4iovUHY2fctVqd6CXS1j428HDVFBJakvkn7kvh 6L0dOZd/GrdWQSwH5mwxlaJzfMNJCR9lZRxB8GoNie2YzrQJItvY2hpot6V3RXske7qy mo1QkhKGDwDdqbUOiXkdoIttmRACCRFXUmOhkHUfV6eSBQbBH3SV3wqs+J5Vf8YUcA6b VQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9gvs2fcp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:15 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF5xBM196462; Fri, 12 Nov 2021 15:09:14 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2041.outbound.protection.outlook.com [104.47.57.41]) by userp3030.oracle.com with ESMTP id 3c842f8xgr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TF2mZRVJeM/5yQNw0yThxA1Mbo3DyO6S9/xrn4L77Lf3OXmEisdQZoFT5nS1VCXvWnSvfb8rWzzSe1ydiJbvXvWblFMj5QwY1HfdkX9saGEKVZGQyHROqY7fzdgNvZ6u+Qjm00uR75WigVvKiixNoYSWKAaQhrjChpiO2yPIh8WLAZ9n6O3rj7IqUx8g4CTYrCCEcWtMBYqgaaXQhtgdeGuv7tpCrgyz1UFd71CSndxM0moJ/yfgVwt1QgKSebGBlMflYW80FEKsc0JivwDPqiQv8Qqi2mGR5LFFfffO/xGlYq0HtSpDnpTM7b/ziThMlrP6WVQ4CiUpyZBFfAFeqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BI6yNaHxxMrzht4ixewrCrLdJzu85PxHJWx8g0dNNIA=; b=Ek3vb2dFQJFUNpavI2h5XBDqWJPiwszio9KCS3Q5W0zThEj4bAMCnzE3MAyuIvVJG4NeuecgKSBKc4mwG7Jr46KomNjAtxQsXw+ahIQr9D5wH2M+kPPAcgBI7srHA1nkI/o831xJO3ERoq0w3eF4usP/DcA5icLkwYbldt5uvG3jp6s8TbVVnszQ7s4VhpFMxfHJ7aDM1JC63k0M6qlS2xcNCmhKZtih6Sw3fU7FrclP2ik0uyaCiXvWUjIaqzik3DWmLUfnrFPMbot0An/qs9Ua6jOuvn+fd9cJ8dSez5LJHihWbWr4K1eb7khaD1Bz2+aiyX2YIkr+APLRDmtrkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BI6yNaHxxMrzht4ixewrCrLdJzu85PxHJWx8g0dNNIA=; b=Y94sOvoMggCi/8cQjKUi2GvBrQmscI2jjz7Ums3axUzukde0w5Gu4ofRDE8sxZ1Y3fJ6ngbnvPFDNX7iiOD7U4BDxHozEoa3KzukQAWWU1JsKpeZw+O9JH67r63/dTsK+EOgCVcFOQnEp2MTzE8VHYSDiZbdyD08MpIBfDQOOKw= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:12 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:12 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 4/8] mm/memremap: add ZONE_DEVICE support for compound pages Date: Fri, 12 Nov 2021 16:08:20 +0100 Message-Id: <20211112150824.11028-5-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ff84066d-01e4-4a77-8af5-08d9a5ee622e X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:459; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MMySkFnFbRiiL3kgXJVrpHfRZz189gmjNKPjLixi1cWGWD5rzdODmerqSG0DFv6bnv2n+bfu5KnAe95JCvP6pkALQQWxXQkWbfhWA4MgD117YJ0DuM4K5oH8P4sS+/5mRSKZkPoS6THLJwPVXs7eawS2UP9Ex7ftpUXcL4NVkBPdCP691bBEpstgqMzw1y9LZxzAgxIjME+CahC24YQZbdgGyGdHy5DUyvOvrXvGzComHN3cKbhD4n5NMTZKG41Y2aJylhC+BPLopEihi1u5cLxO5KsYlQELGUUcHOsX3XrXEDwOb7PCG9ANyjFRtpVpxhaQsk4omcaNSbMEs8aFip46aXR4yCdKIOVZ9zbD6C00KN7BIk6a3FeI7JoV856qsziE5Bim8JkuFD4qJYJ3G1rVzRuyBvrfXXKibDfm2xXkZDqcvWoZZT6tAxnTL21j6GbtCYY6VulzJk/eE080gK4+6XpKrVYgs8luEntlptYMxoJqNVCkm41g3BiiS1crs0i5wbW8g1agLhK/2R0BVzNDqyDU8kqXDRXFiIxD16OikxLdUbHWB+Tm8iWu7YwL3eYSUYduP9Cfwm2fF6YQ4j9LCLvNyq7n5q094R1w0G0D4cRoq6D+73v9D70dPiseW6l6eCbJtkEyboznZtodR+eCKSwe7GZ1S4kyAYWkm9AvFkp2BwlSMMA/aI+uS9Z+5yRRF6tcZ0Wr9OwnydkWWA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /4oUOK/DIL8tMBSoAnVIEGmcQ/sSkFqNPbLbjJUrvKFMJ4rjQ4z+Nk3q5D9VleHJX0s7x/Fh9FvYHyxUrns0s87JBneP+u8usdOssGfsaX4PWJHzPIBntqrKzIowz11EmXDus3D8LigCK42Wm2HFPB+PD9SXT0gV2iqNdM+lsAdtnShA7eKDI1EsI08Prs/96Xwy49tkiwFkBnHPmFN+qZ4bxgNaETGoFRktcOzih/0ih9QC4KN7adG3Aib5qQBpch7nMa3p94pJuMtKINtJyemd+lOFNyeBLBe8U/1Nt9Nzen2fOLU6KFrKZo9Wv5IR8VyUG+GVR8ktUkk8prfKps69EeBFsW/o52EGjVwx9chbmItXdBQyJzLR9mWYSPQKDFvoezzqS41GEuVRe9nRE2jvODcujIOCJGN/Bx+l9Il/lnFCVUc/OkVrMMRGpz/F7v2n3MBvk4Bh8+A6hYTV+YuaRZimFE1EVp5oeugZUs05vjiU/8w/jc7iU8SX236S9zyigpY0QmN3BM9WFvRVc7p0FMvj8GEdHnoR20DCUZqNAvcamPQekkYjk7cSm7ik5TwppIm23KKlLmzZZ4NIouRxOFtDXDSKl+VQG8GcZyRCK+/Jgun/0vI2+JKNCbQhf/BKL+AeYot4z0AiSVG12oHXnrLwwzGOXs59d077vtuTuyoiGuBqghxdDrtReFjtLIFiHJlyQ7173nydaTC5ha3YxYGvHcsu6c1KAQ3BsiDnGfhIac2eR9X8G2DlGkuLMNqp+qltTfooPZ/S29KUynS7wT+uZs/x8yqJZl79NxA3bCMXMHSeXCp9LsDoRiC7Bn+fweSqwuWJzfjc8VdPWl+qLdKHAUH6p7GYBc9C/fEDsOGA/zMEzHFPVeDBXdzU4J+puRtK7LOP/pgNfOQlV63/SgjbroTL+xDYdQApXtQBBzd8FLAYwxhDsTuGWtBomWSTpZy6I1rJNW61iQxvpgkbycmiA081bGhfBNevQVzVFaEIsotY2WzNH5ctdVnJp1+K/4gevABG0WbCO3Xgjac/ITuz98Pgsl56wAdV0O0PfjmV+Cyua8rDE268FbCa5jW/RsHQIwqjsv3ACDrLCMmqV8lByr1cFTeLgv1Dhxv7A48A4oO0WMNwMGxidJNGJogg/97mAUNRAWfGVGJu7U1nxfxCO9wl59+N4Ky9djI6ZhupkE0n1OlYxscYWFeNod7B2bybLzn4fMz1qqC2skE7FGAOAWdM3xvCiGgdie0XRUZa4rsk6cD2KW/R5rds4o6rzy0Ok6SenFyWoUyteIRhC9+lRdHvjJhFUvDxLUIHr82fy8+RscCPJqQ8e7PxxjE/cd9okCBJ4wWyqd41b3nbPGPlQvFlZNXU+Y8MTaQKeDVzpzJMsIMflhF7OEkAQKcH6VvM9bGCNwbIvfzz+item1xOquoCAacUr8rKIbBWVYpva3uvSOcxN15R09jFyyr9K4FyZfKxigZj1Xo5LLsvkQ1dhGns9ZwNBNIYEPpylJsFoz2yoLxsilAkJxhSatZC35OMRRf6xRBgip+2ivD6TdDR4BM4ukztJT9c9IQHJqX1vg2NYbAobsPSlidBhAZSvYCtP6Fn1XJUD5bUoHa0xqDdTXH8/yp5FWh5KvgTcNDKVXYt7o+qH+aGU8HssDbrxdQB5xnXVFscbgO7gw== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ff84066d-01e4-4a77-8af5-08d9a5ee622e X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:11.9833 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VZVEjsN6yn3JO+2qO+KUl87tpgFD2bLSckBriUkfCe65gung9UDIXPLfaZCiKN2x3uhCNYTJ9QDqapxQSVvboss98HtzS1JFwQYUas/QSO4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-GUID: RpdZ555mqTT4iYSxr_C90VYnkrZ-6N6a X-Proofpoint-ORIG-GUID: RpdZ555mqTT4iYSxr_C90VYnkrZ-6N6a Add a new @vmemmap_shift property for struct dev_pagemap which specifies that a devmap is composed of a set of compound pages of order @vmemmap_shift, instead of base pages. When a compound page devmap is requested, all but the first page are initialised as tail pages instead of order-0 pages. For certain ZONE_DEVICE users like device-dax which have a fixed page size, this creates an opportunity to optimize GUP and GUP-fast walkers, treating it the same way as THP or hugetlb pages. Additionally, commit 7118fc2906e2 ("hugetlb: address ref count racing in prep_compound_gigantic_page") removed set_page_count() because the setting of page ref count to zero was redundant. devmap pages don't come from page allocator though and only head page refcount is used for compound pages, hence initialize tail page count to zero. Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- include/linux/memremap.h | 11 +++++++++++ mm/memremap.c | 12 ++++++------ mm/page_alloc.c | 38 +++++++++++++++++++++++++++++++++++++- 3 files changed, 54 insertions(+), 7 deletions(-) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 119f130ef8f1..aaf85bda093b 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -99,6 +99,11 @@ struct dev_pagemap_ops { * @done: completion for @internal_ref * @type: memory type: see MEMORY_* in memory_hotplug.h * @flags: PGMAP_* flags to specify defailed behavior + * @vmemmap_shift: structural definition of how the vmemmap page metadata + * is populated, specifically the metadata page order. + * A zero value (default) uses base pages as the vmemmap metadata + * representation. A bigger value will set up compound struct pages + * of the requested order value. * @ops: method table * @owner: an opaque pointer identifying the entity that manages this * instance. Used by various helpers to make sure that no @@ -114,6 +119,7 @@ struct dev_pagemap { struct completion done; enum memory_type type; unsigned int flags; + unsigned long vmemmap_shift; const struct dev_pagemap_ops *ops; void *owner; int nr_range; @@ -130,6 +136,11 @@ static inline struct vmem_altmap *pgmap_altmap(struct dev_pagemap *pgmap) return NULL; } +static inline unsigned long pgmap_vmemmap_nr(struct dev_pagemap *pgmap) +{ + return 1 << pgmap->vmemmap_shift; +} + #ifdef CONFIG_ZONE_DEVICE bool pfn_zone_device_reserved(unsigned long pfn); void *memremap_pages(struct dev_pagemap *pgmap, int nid); diff --git a/mm/memremap.c b/mm/memremap.c index 84de22c14567..3afa246eb1ab 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -102,11 +102,11 @@ static unsigned long pfn_end(struct dev_pagemap *pgmap, int range_id) return (range->start + range_len(range)) >> PAGE_SHIFT; } -static unsigned long pfn_next(unsigned long pfn) +static unsigned long pfn_next(struct dev_pagemap *pgmap, unsigned long pfn) { - if (pfn % 1024 == 0) + if (pfn % (1024 << pgmap->vmemmap_shift)) cond_resched(); - return pfn + 1; + return pfn + pgmap_vmemmap_nr(pgmap); } /* @@ -130,7 +130,7 @@ bool pfn_zone_device_reserved(unsigned long pfn) } #define for_each_device_pfn(pfn, map, i) \ - for (pfn = pfn_first(map, i); pfn < pfn_end(map, i); pfn = pfn_next(pfn)) + for (pfn = pfn_first(map, i); pfn < pfn_end(map, i); pfn = pfn_next(map, pfn)) static void dev_pagemap_kill(struct dev_pagemap *pgmap) { @@ -315,8 +315,8 @@ static int pagemap_range(struct dev_pagemap *pgmap, struct mhp_params *params, memmap_init_zone_device(&NODE_DATA(nid)->node_zones[ZONE_DEVICE], PHYS_PFN(range->start), PHYS_PFN(range_len(range)), pgmap); - percpu_ref_get_many(pgmap->ref, pfn_end(pgmap, range_id) - - pfn_first(pgmap, range_id)); + percpu_ref_get_many(pgmap->ref, (pfn_end(pgmap, range_id) + - pfn_first(pgmap, range_id)) >> pgmap->vmemmap_shift); return 0; err_add_memory: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 23045a2a1339..d59023a676ed 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6612,6 +6612,35 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } } +static void __ref memmap_init_compound(struct page *head, + unsigned long head_pfn, + unsigned long zone_idx, int nid, + struct dev_pagemap *pgmap, + unsigned long nr_pages) +{ + unsigned long pfn, end_pfn = head_pfn + nr_pages; + unsigned int order = pgmap->vmemmap_shift; + + __SetPageHead(head); + for (pfn = head_pfn + 1; pfn < end_pfn; pfn++) { + struct page *page = pfn_to_page(pfn); + + __init_zone_device_page(page, pfn, zone_idx, nid, pgmap); + prep_compound_tail(head, pfn - head_pfn); + set_page_count(page, 0); + + /* + * The first tail page stores compound_mapcount_ptr() and + * compound_order() and the second tail page stores + * compound_pincount_ptr(). Call prep_compound_head() after + * the first and second tail pages have been initialized to + * not have the data overwritten. + */ + if (pfn == head_pfn + 2) + prep_compound_head(head, order); + } +} + void __ref memmap_init_zone_device(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, @@ -6620,6 +6649,7 @@ void __ref memmap_init_zone_device(struct zone *zone, unsigned long pfn, end_pfn = start_pfn + nr_pages; struct pglist_data *pgdat = zone->zone_pgdat; struct vmem_altmap *altmap = pgmap_altmap(pgmap); + unsigned int pfns_per_compound = pgmap_vmemmap_nr(pgmap); unsigned long zone_idx = zone_idx(zone); unsigned long start = jiffies; int nid = pgdat->node_id; @@ -6637,10 +6667,16 @@ void __ref memmap_init_zone_device(struct zone *zone, nr_pages = end_pfn - start_pfn; } - for (pfn = start_pfn; pfn < end_pfn; pfn++) { + for (pfn = start_pfn; pfn < end_pfn; pfn += pfns_per_compound) { struct page *page = pfn_to_page(pfn); __init_zone_device_page(page, pfn, zone_idx, nid, pgmap); + + if (pfns_per_compound == 1) + continue; + + memmap_init_compound(page, pfn, zone_idx, nid, pgmap, + pfns_per_compound); } pr_info("%s initialised %lu pages in %ums\n", __func__, From patchwork Fri Nov 12 15:08:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616949 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1F892C93 for ; Fri, 12 Nov 2021 15:09:28 +0000 (UTC) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACEeV4L005374; Fri, 12 Nov 2021 15:09:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=6K3dV9kT1TePMQHNEat9vbBQo8pBmM+K6iChvFRe6Gk=; b=X4rsohgvpwKbYk45fF9eYWadE/tfrlGhd8HDj1/mZI5K2pLPu5N7VBgg8a9O2mnPWs0j EbaneeEmylB+vTWBjmTLlTd9F4X5vKkVW3pktmwAUk1fGmolbe6tV5k4Pn26FtA9fgzU jLzUFqDiMYV0B7imRfBYm/ke226+ILLr8d1568cHDsiRKAwmF7MXUvJefnELH2dCzjsr ulyGFhc1chvTewN4FOaNLAE2di5PjlxFi+PEAC74TufMH89kynK4Jz0OuMy0DuPIyMWp 0BKWoOz4Q9+nHdVDS57JIHan1cpUetS0pPE9a4EIwwC0p91cWHuv7FT8acyHc6s5kCo6 4w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9ruc8k06-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:20 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF5wiD196345; Fri, 12 Nov 2021 15:09:19 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2043.outbound.protection.outlook.com [104.47.57.43]) by userp3030.oracle.com with ESMTP id 3c842f8xn2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:19 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZWw6Shegvz6GfqqCyc7hvigeiWrB5yzvSmVCyYDpflpGAXos8LgwVHmY8B81kxD6yPuHSDFuL8qqHcgMQ9+FTS7TSPjbG3cpU/8lpJ6tPuoYtqosJ30ieW/F+M/pCS9vCOaL5cbkDroZPpp8txenp+gx5NPBYL4l8E5acE5h/AR8kunnBUzNoF6yaagX2NkqFtP16akIS/HuP1LK5vkzBUWwI9p1XzNolHkfV3hAyGnErG3qqbX5ahvNSewGWM3XVR/IspJBaPOYtosiMVotwmObNivbtkN0E+t1GtYIEBYwzKz+EwQrepbm+HQyCoHHvOR/5EG06tw6Mhwi7ac5ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6K3dV9kT1TePMQHNEat9vbBQo8pBmM+K6iChvFRe6Gk=; b=gIdcO+nYtWE7FVzuGybWdlL9sncrrc46NsSGdonlcMrxsMGv5QPUrZGKVzYqQKnlzMk6xStZhHi4kKz4qLaoZPLgmRhLe92n6R9hLAankoyNtLZ+gCq96J8jcP5XF/RH/hzWIzQchyWHWyj+ZdCnTi5bRzHE+lmZhIffZ+MK2JdjiXLyD0j2pXabBND6MHgXct0gMZYNBR9VjrgDzHZ3FRDh+muCvIOe8QT6pV9Eq6zeRkeNP5eISzqxnX91TGrW8CL/jQUmF03wwIBeNtq0K30B9uuCNrSSZ9LDg20jc/3Kva2Yzzl1uCH/ojvqJBIyHJNWb0XeAWRdsa7Y/7MvZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6K3dV9kT1TePMQHNEat9vbBQo8pBmM+K6iChvFRe6Gk=; b=BXp3qkkztlpCo8aPnz5HoM0ppec/QnNkDkwypW7UIgWQRmbH/anF5H7C4NBHqmu2+WO7MCzCoLUG/N96/DRrHsbrcBU+laZK8jjF5ru0xdxKFwWu1dMB3hAz1oz7yW1H2ZOV5rEqqi13yRfix334YmX1h7yw1Qd6db7ASo6OjIQ= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:17 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:17 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 5/8] device-dax: use ALIGN() for determining pgoff Date: Fri, 12 Nov 2021 16:08:21 +0100 Message-Id: <20211112150824.11028-6-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:12 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 404dfb93-7827-401e-080c-08d9a5ee651d X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8NuI8sMaR8nnbyJF/ag8vSkrznw+YjLw9DZxlHPYEY+Vi4PNfpj7Q5PSt0SvAAueU69ymX59184wGJWHlgIR6bW0NSrn3ZVXAFYnYkmcMIgSfDo+Kxzy5IGEobCbo2k5Vn+RTDE0OFeKMjEX5oMNnUEhIgqV2a/tLy/irVV0UUP2qhxA8gKGAvwfJWlJCyPukqcrokSqlUDBAffxFUCP5uRJhDdVhek07IZGjoIcCNiUm4/an/L2Bgc+ihTb5SAFDIh+D0JR8CYBpcbv8FQVfAWly/NpOiPL4EsF0ZwZgknr8pqGOIyZcFdf+beShclF3Bm8Ofb8QWI8xIiyppK6z4kAWqTRl4aU13jJU21hmSlsAA0IkcQ05+HQ/fHp6t33eci3P4+FOd3aZbT8OWyci+VMNr3s/OWzItI+fkvx3rZEQMjTLdrLn6qEjlThGnyZ1J+YWH3yj57Qwelz/d8kmxO2vbCnnCN0luu2SXVfW9nmjctpXlakly/E4mdkx/MJPZzk8mR+4Tv7fYUKNI5peM58VoL1TdwQJcy88TDMSy20bbVkgbztU6ZlWBYiyqaZ9Gh/nG/YsOZn8xO6QuesLR3SV2XRH9GTawQVmtbncDx8GhxXtKHNrbwg9xzV9hzdFBasSAMR5IbwtaGZHSaXcv6tR9Z3Ctrh9NF7kxVUJzNeW7CfEb5pK/L0EaDxPcqwi0+pBqDkeuIRkxtqhuCdiQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(4744005)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: NxIHogO7ILVXRFcUssSlXbfJu1J2FnrKCo13Z2AtH6anAgeLXBplQ8PJUqE4vqdBOVeqLsYrSM8iQwHkIwTLVFVaghE1/IiPOU5euDQOUEmIUgGskEU9TOZHd/BZ2s37tq5/sR33fMNW+xbU9CDzgn1Th6+uJOKeGS+flGwc8goHSCsCiXSppON5OG78nbRpewmtHVwEf9ICFZBpMzjAQPMLU8YLyLq/YQfoQwT6pXDlfZgLPUkMfbJhTMIePyK7tET9C6NnoV7dHfpA+Rx+svS8xdrF0kTdLB9czqImFNlyrbfI9ZLEsdSPX4ROwUsWTs5tO3BwsAXW/gSc2qp54YlA30SI7HadbNJzu+0fklL0P90P3lF4uiu5SkNo5obAQF2vVe3Fp5crY03Sc58BNP5qNC5wiJfADBPazvOlbXLHeGgdHA5VJWHmFnFRF91tP9b/jjAfraxDyiJCHAr/XHGPy16Rdc2/BvMQqYYwADNCxi0n/CpQvko6SyMLrDiT1cdCE6FcZ9Rg4EaiUM+V+dqHUxo6FFblevMoIZVlWorIPUVGF7AAX0a8uWS5wJz5Mc+vqNBnYn/6/ReQtJD6jRqK4JG0NLZM3OA9ucdcqLkGc4dDRIDYZwudPAjSerV63sUDm1jFN9noBnbAHPe4EJ2/q8B3lVQ9aNYDE7wMeAwu+tmkYEw4az11k6RKB0nkC00aUu+cjvVGPqu5ISBAsWroIOexn+FXgRI6GyiX6oMHeKlLc5GVVKa1wD+gXC6ZClH/eKXEG2HP6CeEG5mXgrNAWJE5dxghYdajmpRl7muAH3qzKPR00x/QdzN4m87Kc5ow5N1dj3U8IrgP6pk7lXAWuePnX+Ev/PGvCkFdlJYZOmcoMZk4T1qgzNeErQKdOD8F0T6+IOGYra29jFrOUr6OJNdrnRNAJ44BMoMikzd3xXi+PlKBNU1Q8BZL4c7CS5n7v8KwvxtvS2pTjef5Kpokk0a+TjhjPpAZBS+pfnud6gU92cHENpimE1DYLaL0/d4avVP51UGrDf/ZUSA74hfTcnfS63imhDK1sNwY+aR+5CpPyNhrqWm0mcfKlOPxWvqV5QWI1MxH6dZ/J/dCxSKePZgv2OFl5RS9FNHnQSlMNufkMBby5PTvlr1R/ucsTINIjK0S0EYX5LeJ2J2kHYvLb/we/P9UteSJRd+ZAelk+P/5DZ/ZPDXvRZKEZeF97OMIg0tjA0viDKkHqtIgD7I5uL05VcpiIisrpGMaX3NrYEwNTgtYFpPH3c80VnOAsmmcTTkoB1X4w/3sCi5u1dijpD3GsVRRkeZEnkzs0Onrpet2UyUsHdthegSvTSL81b0ECMicN4BmznSvBqvSd1VONi2gvea5le5wk9jUIavBn3h4Ao56pR4AbCZ8F4tjqoBACk5f1Gc8Y4lAdCTqq6conQRCwiZJi8aul3l8dXjsdfTy+8p5TxVYPWjp5Ckdp+abq9oZnSEinYzSt6FJMLPnRhi1P21Z5ZxLha835/Rn/82Ha4BZmObS4hX343PAz0Rlnx4FJi2tvnblst8JOGrUHsbi/NMh59y4Q1O+RxaPWb2Yc1CBIrqnHHky5J5OV7AYGeTeFYsdTQvoG8yMe3iEEbpPfgp0vnYqxG1MyTX3vGIeLxUk463ci4QvYhHWHYqsoaOcbBvSVo5pT/aBIA== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 404dfb93-7827-401e-080c-08d9a5ee651d X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:17.0928 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GylyENWzD5pB74llvh5f1fYOo/Ru5Qx1HzhRuwmHj2nZHHApAdHm9cKHHZrJPEy09k1dzsXhrG3rWzB0mU/9qEji0DqC/JBX7hrXK5M5HvQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: RyaQEB82cnhYmcSIJz4N1WJtjqEKqYVt X-Proofpoint-GUID: RyaQEB82cnhYmcSIJz4N1WJtjqEKqYVt Rather than calculating @pgoff manually, switch to ALIGN() instead. Suggested-by: Dan Williams Signed-off-by: Joao Martins Reviewed-by: Dan Williams --- drivers/dax/device.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index dd8222a42808..0b82159b3564 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -234,8 +234,8 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault *vmf, * mapped. No need to consider the zero page, or racing * conflicting mappings. */ - pgoff = linear_page_index(vmf->vma, vmf->address - & ~(fault_size - 1)); + pgoff = linear_page_index(vmf->vma, + ALIGN(vmf->address, fault_size)); for (i = 0; i < fault_size / PAGE_SIZE; i++) { struct page *page; From patchwork Fri Nov 12 15:08:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616947 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED7C12C82 for ; Fri, 12 Nov 2021 15:09:28 +0000 (UTC) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACESmWJ007211; Fri, 12 Nov 2021 15:09:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=A0HEozXxStZ0eIcVZFkPdFIHA26rV61+onQuEltLdQg=; b=rDaZuA/8K1pVJ4Q9DUcFC2CmnC99CdD4l1dU64nhlIvDGJ37G+n+dscIxcq8ntuOMxoq Om71mhorEgvhii74PkeQAM2K16S72ZJdeYMQsVcwIEGujhZi+PuQfK8qP+wnDJgMUzNp NLB+UQJf4ZNy/2+GrVlbICFMxEuCG5evlVZlCdvL1SDd5o6XBSsQO1fb87HUqq75d+t2 eU1vP8eqMNMLAuRY0EbyzXmVWzba/PnoNA81rUuQHFHlQHUpwgsrTjzAjKaUozffyo7N S5RFONLKBCKK9j5ulUpXF10GATSP/KGPj8wV1lPyF1TeHEx1Jy01zo6+Wm9CXhjnw74f TA== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9gvs2fd7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:24 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF6rPI055870; Fri, 12 Nov 2021 15:09:22 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2041.outbound.protection.outlook.com [104.47.57.41]) by userp3020.oracle.com with ESMTP id 3c63fxpt2s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lJA+k6rFEqCXZQfNYWrXzMsWYxG2sZ0NTfENnzMjUpIkrCQM90W29OmIjJFe3VbFJvixZJoimyhPAEJ4LRC1Ctkj/6CDmle1jQrvimWWDgeRhR/gP5+iTvSGyDg0UsKFXs59mqjoWe3t5Nwb2kjYKdFVnI76BFOCI8T8JH54F60RwbBfLRbHQ7Ubf38bLm/XZSKb0MMmiV6pCggSChAMrN030vUwAMIzzqpo6gzlKb8PQXU8+aC9lxrJwMT390TYCi3IXxEf/7qNONn2XDML1WDOFEVnUVBuBu+NpR7UmxGp7DNA1AEUn7WwtNvbTHV8jo1RUl9ytRIclzaUAwNDpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=A0HEozXxStZ0eIcVZFkPdFIHA26rV61+onQuEltLdQg=; b=gohK6DyplEw7Zpz6M7XZjOfbBbXo3dUjrjlFdywvohAB7BMaj1YwUeIk82/r94v3HFk2hZWyn7MdKtCW/lYsXQNtLGS2drCiRjcEnxCYbJE8mbgubCaqOpkIbWuOKt+xsqQESTFRJLMZSrgghws/KufPgeeQc5gzwBq3tz0EpOxgjp7qacQgIkaqU3xjKgmXUxjQmJ/rrSXaCVjC6OhdQShapJH2oQD3wLtz0YWk1ynt/9mNF1Tx1IUYPOKrVBDLtzl3AAmAzKnTil1I8Oyy679B/x9WAESctIIW0HWt0adLpui1Xt4cBjmPON76vFyTLxdG0RwrjnUAArbTz3FGvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A0HEozXxStZ0eIcVZFkPdFIHA26rV61+onQuEltLdQg=; b=fbjWatErfbSdXOEEBrb/uaMqwtikvHfiq5KygefpGx5EJerWuU0h82D/HR27XxKFyhix6loZfROBn+wJbj1CwuXA3ojKXfTNJMv9gFwSvEkETjCXS0D47bY119DdyHMo1N/LpJnmViir0sSIJ95Ya6bULTa/LmHxmHU3Pf8arXM= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:20 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:20 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 6/8] device-dax: use struct_size() Date: Fri, 12 Nov 2021 16:08:22 +0100 Message-Id: <20211112150824.11028-7-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 31d0da88-349d-42d8-aec5-08d9a5ee674e X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: agiRxy1mjafRIlqAE7R6jb6oucIL1k5NVy83I4bDQM/4/4Fh78h1/kb8tmyDWUUkkz9hqiBkhqR1bf0LW2JU+ophOuZUB1urBlpuYPW5yh/CbD6VprmFrTO4V5NgWEK6QbI+TNHa0hcl8vSxQz2DRNPbO6eYopB8fnLT3mI8J5ekuhTZRsmrOOgEW1g4Xajus4TTDeCggNra6EgT43uUWKKOyRmMHzlPeTvtKMAxvAqFu03obrHHMwza+lV1t1xup0DGLEhQByvdFXLbwKY7T6CAGThX1AtmyRFDlM1zyPoyw/VnElVP1ZHO2dnMCBhcEXxoMm3zva+UannZomjKd+EFllZJ+XwETSsMwuv50IAMKyuN3wEbcju1dGLF08qUxt2RPqXJDMP/U/1kOy6TjXovnVGOc91cUCa1rJCKKhsZmL23f6LGZeIfnang5ilhSEufBa4KYVEc/1Bbvmy/vMtTUIJVMgPyCS95qd5v4Amk/Gvy82C/6geEPjcEFN9nVY2hyczYuvF7vc0Dg+ooEOTZM+2oT4VRDBDx83PZID4FoLm/zcww7RtPeaKNRJ6bktH2J5ZxaesF48AxuN+Po55q9FpQFi2Jb7f612sWmLl+1vJKFkttQ1Z3D7eZa9ZnEn/KUJbVpyn+gqfcncwtEGceBBW/JGjJZ1fr/oiNg5cldSJBNzPgWG67hkkov372D9kbxCmVhWOWJbJ6oJXnWw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(6666004)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(4744005)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HHbqmKjRPaGAKalNSCO9xaf2ue8PoXDNplou9J75dojlskW1VHQ/lm2ubKQptlo5DSei+r3mKmfbNntihyw1tNzHPrHmvYWk7/ybwJhvxn3jAiax4qEE33wc8uN2XZj3UauFX0osiPZR5uazVloElosrlXPwqvTqhwVnC0LBChi7TELQ7zjnt+4HfjTTMbE/74yZpet6Q+jt6CMPMOkqfOcwmCW1SZwUJRJwAP+MZTzsllDaXDUgbSjK0IDYl46uqt+0887rpuj9IK3iJsxOZFJmM8XqdY5TL4xy9L5Cg0f7qUFE37eu/g4pMv1tyGtxMSU7HfbyUBNGwTZ7mrj+JhYV39WgqoWivxfI0r9CPJbnCbKadeVx2tIUHYXgyvYL7Wk3jU3TQemSNa+PCqpiJBjfTdc52z9Hy7HYNurRYi8LUO3D2bRwC0oFboqwjUmgn37U1QvfLce24mNwaxOWbWVNxQOuyw/W4m4rMBRHCor6je3bmXgRH+89cuvnu8quiWffSTo9qUtmJltGBbh9MGd9kT3aD7JHj45LDLGpYU/zfHuFOtdGefBfWwClkP30hDL0JEok2Qq/gbPzUwi+oAiXu7mHy+qRbZt+hpJmmpDu/d7A0KgnQUanObixzaC8Uz7yGZG4mC91Yy3iZkln3N5HKP7/S2+QObyiGT1WHtInOmrXCAsgtQw4RBtsW4+4D9TPgvFm+ljhe5qAJPwSZ0vcrqfdK4fOeW8IcDYYHsPB+5lSDqRLpXkwwvn3EopVOAKSnSvEI+F74FL3lTzJh3SvJ2bBgziqhzKQ7wt92dK7nCcHTHjJsY2G0Ok6+/QqIdPOoPtZVPp/OfPfFGu9rU7mNSPPNyQsrS7aVXVAFh8+Bhmwh71/Qj+tv+Xa6S8/7/YYHPv1k4zZvxxAe5g3mi9NzVa0iOmO4NDJyAui4F6ZOEi0O9gNtt9CAcHwYFthBSdg26cmWSHO+WtzOWEXAvRaOxPlMbhBNBXldOhyMyMB6fWQsw+LdkgTi++nCuTfBSVbt9PDfttfEcYSPFD47ywX0D176/TGH0D4wDaNXtzg/NPX3nQ41cY5jwf8ccibuFWRMK1Qq2pyOCMgq70y9C0RQyuuTYiDp6pHPSV9s1/rOe8+en2KMx+p6s2N4vx9TE/ORLhAVvcjWks/DmbCLwV1REb6xxk0eiM2MWF86N5DpKvkvylH5i33KebpdPKKyoR6wxKkvTdf43M8Fqtrujtm4IRdDFGPunmrtKWBBGLsjRCzlGWJX/YQTukY1neCN0XAXcNK9ILAGo3i69g/glBbUdQPpJKjxd9Lp3Mu2S/GbfskcBkUVmmgCCybXZB2TWgiOcPSAR8AVkir9lAp64lpuEo9ptSkoaEHKbc3Z7Qqeb/G6kl31g2y2J7op4VSzhYEG1Ep5E5bN1Bl2LBoF2f+8zTQooqUGdIQMBNR3eufXMPre052UXZtitLY2TvYbrDqHQ4UKx6CgIUbaC4Zt8HLJH3BPx959NaWi1msRtMI6PXyZcXerJQRFvoX2WJPa3GmuKYkZULzFLGQgPno8J2YAseEzkdknmts+CwBJOfx45zwN7EX8uh9JebTEMquS23BxElwlKM3zF8LZpNaplbYjx4H0oIHHYBmmp5XBUVwZJkPzGOKnr7j6hnrUJffTyi8BWuIXSLBwr/YYwoPlw== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 31d0da88-349d-42d8-aec5-08d9a5ee674e X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:20.7856 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3FsxOSjWHaO4nlt8gv7Pi+AZ7YYrQzeAfRQJ8ubEpDu02Z4Sv9v0t2xjKoN31k34LAO3GRtKqMEBxFZ5ebRIbXWurGrgRP4mucJ2KiG4c8s= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 spamscore=0 mlxscore=0 bulkscore=0 phishscore=0 suspectscore=0 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-GUID: a3weCfMS9cwz878PgOtF95JnMNhkMlWQ X-Proofpoint-ORIG-GUID: a3weCfMS9cwz878PgOtF95JnMNhkMlWQ Use the struct_size() helper for the size of a struct with variable array member at the end, rather than manually calculating it. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 0b82159b3564..d6796a3115a6 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -404,8 +404,9 @@ int dev_dax_probe(struct dev_dax *dev_dax) return -EINVAL; if (!pgmap) { - pgmap = devm_kzalloc(dev, sizeof(*pgmap) + sizeof(struct range) - * (dev_dax->nr_range - 1), GFP_KERNEL); + pgmap = devm_kzalloc( + dev, struct_size(pgmap, ranges, dev_dax->nr_range - 1), + GFP_KERNEL); if (!pgmap) return -ENOMEM; pgmap->nr_range = dev_dax->nr_range; From patchwork Fri Nov 12 15:08:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616951 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF2D968 for ; Fri, 12 Nov 2021 15:09:40 +0000 (UTC) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACDqgVY018116; Fri, 12 Nov 2021 15:09:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=8HFEZphhYZDWiuCy0vyL75HwDr9OvafkX9Jq8eoYs6k=; b=Ftl6KoJiLuNl1RgRNdsfxeKLzT2gGhexPlxw9nvPhWs5dRDK1kL8Xbyd/cVRp0A7OI1U ZZ6lMbn7YzzqJAlFaE/smYkIPcuD2Z88/85pGxbIDLXSlABcNghpCBTcoJt6zVg53Tbv k90tSXq5LADi3isB0FE5KtxcjN7dX/Ubia4074rlKtjIAeVw4qOuQoD5XSOEbq9LMr0z mKgjt4nSn9N9Copap3Qgnj+49o+mw9gsJyOujLudldv9gXKuSvZkap/p72DtTUeMMaLT wg/bEptSCQDbt3txYJzr9VMzpVg/oFrkEofQKordAYMPfy7J21CAgXIVvrFKahhh359V 1Q== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9qx3rujv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:28 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF5wAi196340; Fri, 12 Nov 2021 15:09:26 GMT Received: from nam02-sn1-obe.outbound.protection.outlook.com (mail-sn1anam02lp2046.outbound.protection.outlook.com [104.47.57.46]) by userp3030.oracle.com with ESMTP id 3c842f8xvb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:26 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Okr4Qaob2azCNsH8mg5Pn9ofSSDwXvfBkI+p+Le3cSEdnnYkRDOJsvl9i2V79bA75S+6A5krAf9LfEFlJwgqSMW74XlwWPGO8/m7TfRspfpZtgtyBZ684xQI+wtQ+sFzBnVzCq67TM8ve/DSdJ0J9vv6yTwnjiXzK1udu0eVE6GYNFlioJDzQpzpppFYJJ0ps9zqtABab3SUQI/l3HkcpWssBgLuPdWx3OJNwOR9W5/O3YJnR3gaboHOs8v8eIwZRVArXnriZR17bFL4Zo9Wm7P9U6e3SYQ8KfpnOimsVIEAnwtlTgAkDcbxaxQJJrU3YQZMva8viBlFg6X8ZdpwsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8HFEZphhYZDWiuCy0vyL75HwDr9OvafkX9Jq8eoYs6k=; b=fZyMbu4huXc+mTG6FCBSc6ow+xGFMvqovUjce4tePFQN0iuH3pNjgpbYe4UaPd5bUEQ7LMHFv2rZiDHwfM8ctaV5NVGgxeG0l3auZwng1zLhKtNkANaIGyUwOTNXclDx1qmllydDjHyGtMcxEfjlA90tPIhC5pGiKi44c/aCMVEE0JAhiV+baBLjfasiefkWDjeWZPkteBXD1F1L6a/0UBNjefhZO2Z2UvPPRGNQLwoAwQAVswE913CUmW0ew616H3ZMVTWVyis8TuOk4YxdrIS8rsF0SWpUqz6C0/LIbBCVkrMxrnUw6dQuadQQl5c15vK9aFfvL4ye5ywzLBGYQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8HFEZphhYZDWiuCy0vyL75HwDr9OvafkX9Jq8eoYs6k=; b=p0t4wFmUQ0nSadHvxIklIPchLjhXrbRhUhyJWXucEIAgCpEBOrJb5cMYvl7jz+dTXEuRjrqMPcUrgVmR/m1C3qHjgEfvIroA5HuQdD4jso9Y8UhWpXGgmsTwL5R6QGtA7hH+HBShNle5nM83JEPSTppx9iR5i8sHVeMzZxKtXtE= Authentication-Results: kvack.org; dkim=none (message not signed) header.d=none;kvack.org; dmarc=none action=none header.from=oracle.com; Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by MN2PR10MB4223.namprd10.prod.outlook.com (2603:10b6:208:1dd::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4669.17; Fri, 12 Nov 2021 15:09:24 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:24 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 7/8] device-dax: ensure dev_dax->pgmap is valid for dynamic devices Date: Fri, 12 Nov 2021 16:08:23 +0100 Message-Id: <20211112150824.11028-8-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bd626ec2-f4a6-4509-34b4-08d9a5ee69cd X-MS-TrafficTypeDiagnostic: MN2PR10MB4223: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4941; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b3CEA/6Ad3CV9bemuMsmJB30Q5RQ4VVKtaCp6o63DX/amAgrh4KSugH4O4ck+g3BlibgAtKemFSkA1N3H2U5k7LnpckljSv3pr5JR2Ug8wWzCSjJHkCSb0Vct11N7LJ4WVVFw9txLlhxqlfpVekmTPpfvQVHqo3qe4HJwB8V3X0Sa4B4zbYAla6qJP6q9mXfJ9j7EMRaUjtUmZb8fMOStwYSzaRvJESIbwd4mQiOJArCW4CUpFj36LDZBEHRLcGihUj6Rkqx2eKrqyfxDMUELNA9ooroxTjSLYmoJG80lsgoxzLJHqk1Da2kvjafU6qmx0BNiE5KU54f+amQZTVir8CV0E7qCxL5fGKgPLYz5yuT0UkGV0HnMROblnrBsq7bvqOQ61m36dysxp22lxQ4Lt8ZDjEnaIPA4asZWpNGQIXEw1FOKa7bNWaIc5zlZMliDH1rC3e80W8yBCwmmEkduPP91z8aNhr1PTojaeTnG4Bo1O4qjEOEOHGg6oqi4h9N1/SbFnW8dHx+QscIPyeDB4S0TYAX1ovXKCtGW4tJX/H+hFyPrtNXV95dcUWo0l3StYRDnjSfw0H4Uje8RzdS2ZJPDLSf7XcgXaG/O9zOYHxj7XYJdp+NofTYfba62B2hpaD8KZNlPNBmJXhVfVdU0JlXsJ5kRBl420bKjiW9aWlCDAPbqh1cZOSWp6xkKB9wsivedy4C8nzyWXwEHJDV+g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7696005)(5660300002)(2616005)(107886003)(2906002)(38350700002)(38100700002)(66946007)(6916009)(6486002)(66476007)(54906003)(1076003)(66556008)(52116002)(4326008)(8936002)(86362001)(36756003)(103116003)(186003)(316002)(26005)(956004)(83380400001)(508600001)(8676002)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: vyFi6z35zhCoy8Urz3+fMj6JkiPWXYxaitPyCSHl+k2RJwHVonfLx1J+wFGO9jTFFePWjJCfmSDXhZYD78dKmv3mR8188+vO+ShvPXTabrTVMMseblN/47s9VnbTVbkwnBrTFH9u+h8KCveJaF9cTcqpMP16AQQkfbE0A/vQTtEA1fSqQGTX1jhnrG4tM4kyRJns9YjgAAR3jN3SParVWtpaqPYkA0iu1Xv44Z0tHHOCijx3Tjgzvwt85vP+L8Wqd7RiL7p77IzY/ndneyJdfBCtqoIwk1YiyZw2zH/gQB+ytGwdQZq4920KQTKCGRFCB4XvfzT30O5Q/Gx8s4zb45YvxnXA4saFvlMtS+H85BZLS+DlvbDVeQudBbJsaE6ppLdr25QP9n56XFVHwDdRTLT32NnbAcDwhqflMOdQYLt5lKFOMJdOmHQhil4iFTFowCETE7PMyxIsC/mmFHwTRhBv9fnqlKRHAQBITiIGdUbTYSbmAdytJMEX0gjASFph/8R+Mx5QaRC86BgmFKLD4CEMSzajYt7o4rqpZ+vcA4LRxcDf78MfdVKd1Kvy678j8oWViIl5tApkGNrs10E2dbepcVbSC8JmMxpYMaHW+s3nnaqdNS3IoWJXULebGbAVvxLIksNmpkSzdM973ytXhNNjtJWfElklERpAsDswEDrXDzneKLNCe2qF+X3H7qygy47XvU3W273TANXfIX25Ih8yOssWbpvphnAMUWZkJoFVoX/KTdKggBs0HV4jXhDIP4gDshci2OEckd3rSuzqYRrACHB7/x2+59jL2fViPHQSBfilUhykWYZ3PlwLX/ouyfXb9bue7vdskRlhblrPXA1RwceCslPk6/gcZufmCAHEwpkw7DSF5X2ztDRY8ad9NVZVrBbSIHA4ZU2cJ+Ws6GkzRaiOgJQvfJZNgzG+29zDsw1QVy7ojNxvmVlnH009NHYRTFP6km4bovTGAHb4qCU+SNN4LuNIe8oXESn6T4qGx87Ade00xA9yJ5vcO0EofXpAB+wd84bZJdyFxdx8NlJQXNiDjNS1UDVtIkLj5KLozwPDzXfMxAUW5Y4Qw7Dtx7jPtE0SOp7+ZSxSGfuWMl4ZwVLkWv1ax0jv94L3050UD06ZL1Qz5YbzqZ3/LoKQFmavajD7TsPzzPd1ykUcQCwsGuiy4GC6Y26p2iQ424DAb2kWR7W8JRg4VQrSqw9XVdpxT9dcgYVMgJBEGXmfoaup0bFlv0PMW9ZjfB/bJ0HgjYXjvpPDf3I2USbb9DuNQJua5BEZa2VA6urXDFRcLGUxlHtZvA6s3unJw033cjAIwNxX3lJL1+X/1/OsScFdMngTJ5Tj2hvqZxXJoInKFIp1c14Q+l9Cp1WXLQCCLtETNbnwiJFEmdgcox+aMqZFkhGTmfz61FfiLEGLFZWMF6pSETa8O/bATMdS/CFC+t/pJU6m4XtpfrlPFc+GoZV9ReylROMrJxRaBfqkXwbjfykp+wikvkxwGWN0yVu3aDR8Ypr+ZUD59Sy8QScYwo3Yj8oe489Xyy7G69jv+LyBWzoIC9Er13IbCvS0aUkajtNCYLuK40GrSkH9/6pUhnogJxMRT9oZ+p2/KeylVsn/ZmEuMKwvKy2DBjTknV8Bu7Dvf+dKbBNA6PS0VLcZQcuA+pxxTOhe8ZH6UD9QcZ0OXA== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: bd626ec2-f4a6-4509-34b4-08d9a5ee69cd X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:24.7801 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nbDGrtPkjDU/q7aehirImxWWqOODiw0ZkjMZyMKh+pj55k0CdiJnJ3jtdUfMK2BAZEy7vS6xojvXGohrTRUCUP4rXEtdU/ruTPr8TRE0NRs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4223 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 spamscore=0 suspectscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: qbW8LDSLTyJgXfixe6qoLCmncEGGYIi6 X-Proofpoint-GUID: qbW8LDSLTyJgXfixe6qoLCmncEGGYIi6 Right now, only static dax regions have a valid @pgmap pointer in its struct dev_dax. Dynamic dax case however, do not. In preparation for device-dax compound devmap support, make sure that dev_dax pgmap field is set after it has been allocated and initialized. dynamic dax device have the @pgmap is allocated at probe() and it's managed by devm (contrast to static dax region which a pgmap is provided and dax core kfrees it). So in addition to ensure a valid @pgmap, clear the pgmap when the dynamic dax device is released to avoid the same pgmap ranges to be re-requested across multiple region device reconfigs. Add a static_dev_dax() and use that helper in dev_dax_probe() to ensure the initialization differences between dynamic and static regions are more explicit. While at it, consolidate the ranges initialization when we allocate the @pgmap for the dynamic dax region case. Suggested-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/bus.c | 14 ++++++++++++++ drivers/dax/bus.h | 1 + drivers/dax/device.c | 26 +++++++++++++++++++------- 3 files changed, 34 insertions(+), 7 deletions(-) diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c index 6cc4da4c713d..19dd83d3f3ea 100644 --- a/drivers/dax/bus.c +++ b/drivers/dax/bus.c @@ -134,6 +134,12 @@ static bool is_static(struct dax_region *dax_region) return (dax_region->res.flags & IORESOURCE_DAX_STATIC) != 0; } +bool static_dev_dax(struct dev_dax *dev_dax) +{ + return is_static(dev_dax->region); +} +EXPORT_SYMBOL_GPL(static_dev_dax); + static u64 dev_dax_size(struct dev_dax *dev_dax) { u64 size = 0; @@ -363,6 +369,14 @@ void kill_dev_dax(struct dev_dax *dev_dax) kill_dax(dax_dev); unmap_mapping_range(inode->i_mapping, 0, 0, 1); + + /* + * Dynamic dax region have the pgmap allocated via dev_kzalloc() + * and thus freed by devm. Clear the pgmap to not have stale pgmap + * ranges on probe() from previous reconfigurations of region devices. + */ + if (!static_dev_dax(dev_dax)) + dev_dax->pgmap = NULL; } EXPORT_SYMBOL_GPL(kill_dev_dax); diff --git a/drivers/dax/bus.h b/drivers/dax/bus.h index 1e946ad7780a..4acdfee7dd59 100644 --- a/drivers/dax/bus.h +++ b/drivers/dax/bus.h @@ -48,6 +48,7 @@ int __dax_driver_register(struct dax_device_driver *dax_drv, __dax_driver_register(driver, THIS_MODULE, KBUILD_MODNAME) void dax_driver_unregister(struct dax_device_driver *dax_drv); void kill_dev_dax(struct dev_dax *dev_dax); +bool static_dev_dax(struct dev_dax *dev_dax); #if IS_ENABLED(CONFIG_DEV_DAX_PMEM_COMPAT) int dev_dax_probe(struct dev_dax *dev_dax); diff --git a/drivers/dax/device.c b/drivers/dax/device.c index d6796a3115a6..a65c67ab5ee0 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -398,18 +398,33 @@ int dev_dax_probe(struct dev_dax *dev_dax) void *addr; int rc, i; - pgmap = dev_dax->pgmap; - if (dev_WARN_ONCE(dev, pgmap && dev_dax->nr_range > 1, - "static pgmap / multi-range device conflict\n")) + if (static_dev_dax(dev_dax) && dev_dax->nr_range > 1) { + dev_warn(dev, "static pgmap / multi-range device conflict\n"); return -EINVAL; + } + + if (static_dev_dax(dev_dax)) { + pgmap = dev_dax->pgmap; + } else { + if (dev_dax->pgmap) { + dev_warn(dev, + "dynamic-dax with pre-populated page map\n"); + return -EINVAL; + } - if (!pgmap) { pgmap = devm_kzalloc( dev, struct_size(pgmap, ranges, dev_dax->nr_range - 1), GFP_KERNEL); if (!pgmap) return -ENOMEM; + pgmap->nr_range = dev_dax->nr_range; + dev_dax->pgmap = pgmap; + + for (i = 0; i < dev_dax->nr_range; i++) { + struct range *range = &dev_dax->ranges[i].range; + pgmap->ranges[i] = *range; + } } for (i = 0; i < dev_dax->nr_range; i++) { @@ -421,9 +436,6 @@ int dev_dax_probe(struct dev_dax *dev_dax) i, range->start, range->end); return -EBUSY; } - /* don't update the range for static pgmap */ - if (!dev_dax->pgmap) - pgmap->ranges[i] = *range; } pgmap->type = MEMORY_DEVICE_GENERIC; From patchwork Fri Nov 12 15:08:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 12616953 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D91C62C83 for ; Fri, 12 Nov 2021 15:09:41 +0000 (UTC) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1ACEgY3v005379; Fri, 12 Nov 2021 15:09:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=corp-2021-07-09; bh=Xce7fuO8ugtcHQyaqzDt+3x29zeBRhwbNDjHZ/Fbtqs=; b=Qxb6X0SgM6qF90FZBoZ61PxOhnmRWO0y2Mec8e1wolAtzL2CAYIHYp358TmhENSdWbKh vv24RkOz20YoRjO0a3i/irOvHLvHla4yds/wp8VqF59abuCeljsg+4jp3jczKrAXdkCq QyvXWEeTKD+gvopmLw7PLCZp29tDvdSrNGWUoUyxNOn4B1mejcgYOxKo/l2eceokJvEF uaajMbp5JkJ1XQ9SUBaQiugMsn3IOGcIC8pQazNt59SL62BLRPb1AkLDkHiPFRMD6r2j tCGUsZLIO/+rtQbWALofRYQm9qcLdpSquTBZvC6dGc7TisYionBPyOUs6KOI8HUVYazJ qA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by mx0b-00069f02.pphosted.com with ESMTP id 3c9ruc8k16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:32 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 1ACF6S9H104501; Fri, 12 Nov 2021 15:09:30 GMT Received: from nam12-bn8-obe.outbound.protection.outlook.com (mail-bn8nam12lp2176.outbound.protection.outlook.com [104.47.55.176]) by aserp3020.oracle.com with ESMTP id 3c5hh87m5m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Nov 2021 15:09:30 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JZT4QuYtV7+q6oVRacYy2cESAKpW3KFlmWUkTg29GYxPwdIj+D/BiwpjTSGxBjs2mzvR16Qb7GT19dKjNGrVAMYMWjKa1m2+z6KYyfET93xCNf50hNkbpY/OcVejs/wtF6e0CJq20rh9aaxtGMk0AelPty5Y+qbwcL9CfjVKiSjsaZJXDwlB1UPs7SmF90CtnTT4Y/au8Ww7S31e/Px+TyX7XoDpkhaEQEdi86Q6vfvus0FUPUjMtE65PQvjXQJiIVHQlOWhqt8vABXTPawUvAJdRIOq2RlNpgDK7j16I8q90ehTh8vH8YLhnUoeuqFmkS51r+BRMeOEQBbzvoZPzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Xce7fuO8ugtcHQyaqzDt+3x29zeBRhwbNDjHZ/Fbtqs=; b=N6CqULY8ZDO0uVaQNSxWjDCfrNCTw78f5U+Aed8HTiaGuIZuMpOyaOxcnZS4mEJZDBdE7XNRyeSvNpbNQplpB6ULDVDqnXJIgWy2tk9R+PVQWUcQSyYPYs8P8QAJc1HZc3VoZC8azZXwXeMJlo3Cf1QUy4ac4Tt0ulut6LQ84QcKSyKAfI/zCVYaDKGJwdfY3AiNN7JC5pqjkbhdYgkrq8B001ofQK4y/jvlbxBjGGEeVedZ8XUvs4U73TAw7fapj0TW7RDzax0b0MPUmRiYX1Or/w/MYnWh9Zgk6uImiUr0aNknWpVDyYzgYGKDdyg3kSoVVwaf1HkXm5J0SJq6mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Xce7fuO8ugtcHQyaqzDt+3x29zeBRhwbNDjHZ/Fbtqs=; b=A/XWoa3L9WmuiUo+mREP86AbrMN7wnvgDAeVi5mObpr//tI3zEAZnbxWVGPBfxtUSQy8oQ3AJhPuZr1kVxI/6UBfVBFvR46do/dB74ny8cfNUXK1Y5kpykdZuohquw/ZUVHV1mfWb7fclwQflxWSKTDZwpM6E7gyy97b023eayU= Received: from BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) by BL0PR10MB2916.namprd10.prod.outlook.com (2603:10b6:208:77::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4690.26; Fri, 12 Nov 2021 15:09:28 +0000 Received: from BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6]) by BLAPR10MB4835.namprd10.prod.outlook.com ([fe80::d809:9016:4511:2bc6%8]) with mapi id 15.20.4669.016; Fri, 12 Nov 2021 15:09:28 +0000 From: Joao Martins To: linux-mm@kvack.org Cc: Dan Williams , Vishal Verma , Dave Jiang , Naoya Horiguchi , Matthew Wilcox , Jason Gunthorpe , John Hubbard , Jane Chu , Muchun Song , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, linux-doc@vger.kernel.org, Joao Martins Subject: [PATCH v5 8/8] device-dax: compound devmap support Date: Fri, 12 Nov 2021 16:08:24 +0100 Message-Id: <20211112150824.11028-9-joao.m.martins@oracle.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20211112150824.11028-1-joao.m.martins@oracle.com> References: <20211112150824.11028-1-joao.m.martins@oracle.com> X-ClientProxiedBy: AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) To BLAPR10MB4835.namprd10.prod.outlook.com (2603:10b6:208:331::11) Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received: from paddy.uk.oracle.com (138.3.204.9) by AM3PR07CA0134.eurprd07.prod.outlook.com (2603:10a6:207:8::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.4690.12 via Frontend Transport; Fri, 12 Nov 2021 15:09:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 159c9a54-3c35-4223-c21e-08d9a5ee6bb7 X-MS-TrafficTypeDiagnostic: BL0PR10MB2916: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:758; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EXiymA5Pb3qM74lSKasX/BcsP9zZbkvPuXg3SdTTP9Ol0vyZP0ZwLz0JdtxQbRFuzYjvGwxl20BpdwbLgPY4PpZq1n4hZfltdH2nDHQ46hJv9DuZx8z7JlEqX0OSnBSjGo1c0BI0jyWuzVJ/5Lm5qR/g9HbS5DsmI8UB4QuyPdsIy4AoQniP0eHGuwNL8SsMZpQUGLsNhSWL5bfPO3WiiscBRMDpVaxyIPvQvDYhERpf25vPOYTGtc9dwi1zLWBdY1z9Z+ZDSPKYI0/GL+JzZwnOyJTz7fUfkK4tw46T9G+GPFh6L0T5dOJ9qlj2wdoRzy6QhLAi4WiLeAkjy8LBoEhmp9D49uCG+bS/2irm1RzDz0uHW/Hz4d85nJzxEyWT6v+TUkA5sVM7hXxvwuw1erubLp48d1AfmL7ZnWTc9fgDr1KsWSUFkEhukQcL/dgy1e/S9peTJFg1NXnZ2uv4WTcvlug4eQFNd8pkCUW1lqmfafyqqRRJd4cjzXbT67Tq8F5cPWO9m+QdTY0dk0xctKM9kcBm84RvK0GKEd4ANt+sF0HQPKmBMje+pgBMxRfxIaOUMOT6iUS7rk+aTwRQqQgprYMPoRzRhyHp4uVUTsOf/NP9AxgQ3nm/rYjn8DyDJ1wKxjwzIRK39o7BlquGN0i2lkyhJewWfCIwT59INMLQxprNiKp12+JUtwCpSLbvEMm4FQGhTlYT0ER1Mk8KgSQTyAGBowF8tlDKyn4GL0OIpr49MvEmvDlhYLQSEe1c1OLIl16451tRCxNAqCBCL9o44RUrDOKC2TywFbxkOWY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB4835.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(956004)(103116003)(316002)(2616005)(7416002)(54906003)(2906002)(38350700002)(7696005)(1076003)(38100700002)(8936002)(83380400001)(52116002)(66476007)(66556008)(66946007)(5660300002)(6916009)(26005)(8676002)(36756003)(107886003)(6486002)(86362001)(508600001)(186003)(966005)(4326008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: H9oJGCl47LzbNReuERULlUcAl2OG3WiCGnOJd7rjQVqwOrCLbmEoBzO5DlCQ5+/Z65FLO6Y13jZ5yWJdW5zg7Jpb8MDF5EuYfMpJBtvCeW3+fZp6FpFGbziAPsgnqxxtITSVhk446IHkTNMHALYeuXHFxvyC2drJVUVcGyhnrgTQPRPgoWytRnptCCONd7qiyxMy6ROH6C1ql/a4eltl321Hmn1FNC8K+FNTD44ILxRxY/5R0/V6lqpB/6V5AxjV2o5I4MzyJlJJMkcpAclv4hUudr51ISY/QSDWxweT+GwrzGSWk3PbpY9IfOmcss5rEybbayoADhwllGcFLdKVcUtZZmJGtMIYzqrAQ2hqILyrYoZVkncD9kOjDV+X1qxMIByfkCFlYNYb1IaqvSNuN1Bbq+StPdU5xLNAtKa96T54Li4PGKcwJwvmKeTKKaS3O5ZjtcOGfKpN71VLgViHxv2f462A74skOAU6UrOMg8cKOlMu4A/iZueYLnH6bHWMWDAJNalVRExFHKOPjag9IPSpGQ0cTckxUrbddUdwDExVaoaoSB3/WZ2C+FVpbu1JG7LReCGQb68f02MFfWXC3yEgu55e2vemOsA8dJMEsv+2nO3iAeCPgKmXTU8aGS3R9g+7Lu845/j8qRpQeY2iFdaOjb3HhxGEgGZRn7dzKdinweTy+rCkUSdEPkN8eqF6nG68bFXru9qmwkmh9pBohjf9E76o+pH5MBuTp/ZJjV+FAs9PtgSbS/uy3FLqA5vkR8QFCHLToUGMoicbcQcaLvDSB94x+wJGEdReLRo1hpR9IKwFgFe0s9LqqqGHz0h/MvseuSrAR7Ls27i+I72oPf5esCspuHg0K3w1AgH8ebLAP+g6k/0jvz+wbNOjZkHiO0RhtHa37dP5z5MvJfUn63JLU+3ysx74WMJoLZy7pQqELfBbX27RBuIimgGSqgEMDbjEU4wic6Goyl4JnHfwNlmwijt82wT3AFaW2yvj9yvgjNZIysuBYRhfnTfOJaD1BFHIxB68OQ12jYv5m+Nd06rcxXRmPZ4yzwJFPDUSlRa5hWatiTzcCZlJ1S/F0wl8Tr9PtcLLeYt7s7pJpaN9nl8K4vNan1FmngkjYzp8BRMTdsldZuAhk9v0dJeu5XE68iKHqk9Q5yR6YkBydkmJIp3t2E6ugHj+RYynEtB6guJ7rboAgfcmHk6Ta6S1qejsqdW8ba335/7h8FfOWxGK1A8Gl6G3gglHRYdZHIPraTeRC8jsk7aP3YkdK3pTezx3y+FPHRrWzGR0KPvvf1BZG1cj/ANO5KI6uVqLWugItqK8VqNHCCqTd6Aj9FhYNQQpmdndu7OsHlsO1E+7hNxjHGrieLlrZDKGAnqCUMLWayPQklw6nGZiiqFnG2LL8VGa60+/O56iQgj2TQJKgiLIiRk6PjP6LhOxbZehppCC15hfB6unA7DFmj/wwlIEjx6suGkNRL7o3tHBG5yskVltaUou68oZIQjmgVuLUzLjxsp7wY+OZphcByi/kBUsdJAypKsaFVLGi0UBmFKTwyuCqKHgge6Nyv9YCxbqHxSKgjUlLSPsunqMwusvoOJJd9ZlHrZMf/MkFh3y0i/2UEoQhshh1rGLb6kX80R1ASS3ag3e1FCIVYX49P20GhCmLsZft4wYwF94mBWnQFQ7J6O6Jg== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 159c9a54-3c35-4223-c21e-08d9a5ee6bb7 X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB4835.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2021 15:09:28.0527 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fOsAEodz6w2qaJKBouH4BzLrFtEeO0Fh3PnX2Euv3h+1ra3KThNpM4w2b5qfJIGb7dbJNA+hlmkHC8ZZLhIEUTSJ71oj9DqfBLJ2UDMjiPk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB2916 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10165 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111120087 X-Proofpoint-ORIG-GUID: 3KR7T6_ER0mVygb6tZ3CFV1lZuH7XSTn X-Proofpoint-GUID: 3KR7T6_ER0mVygb6tZ3CFV1lZuH7XSTn Use the newly added compound devmap facility which maps the assigned dax ranges as compound pages at a page size of @align. dax devices are created with a fixed @align (huge page size) which is enforced through as well at mmap() of the device. Faults, consequently happen too at the specified @align specified at the creation, and those don't change throughout dax device lifetime. MCEs unmap a whole dax huge page, as well as splits occurring at the configured page size. Performance measured by gup_test improves considerably for unpin_user_pages() and altmap with NVDIMMs: $ gup_test -f /dev/dax1.0 -m 16384 -r 10 -S -a -n 512 -w (pin_user_pages_fast 2M pages) put:~71 ms -> put:~22 ms [altmap] (pin_user_pages_fast 2M pages) get:~524ms put:~525 ms -> get: ~127ms put:~71ms $ gup_test -f /dev/dax1.0 -m 129022 -r 10 -S -a -n 512 -w (pin_user_pages_fast 2M pages) put:~513 ms -> put:~188 ms [altmap with -m 127004] (pin_user_pages_fast 2M pages) get:~4.1 secs put:~4.12 secs -> get:~1sec put:~563ms .. as well as unpin_user_page_range_dirty_lock() being just as effective as THP/hugetlb[0] pages. [0] https://lore.kernel.org/linux-mm/20210212130843.13865-5-joao.m.martins@oracle.com/ Signed-off-by: Joao Martins Reviewed-by: Dan Williams Signed-off-by: Joao Martins --- drivers/dax/device.c | 57 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 13 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index a65c67ab5ee0..0c2ac97d397d 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -192,6 +192,42 @@ static vm_fault_t __dev_dax_pud_fault(struct dev_dax *dev_dax, } #endif /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ +static void set_page_mapping(struct vm_fault *vmf, pfn_t pfn, + unsigned long fault_size, + struct address_space *f_mapping) +{ + unsigned long i; + pgoff_t pgoff; + + pgoff = linear_page_index(vmf->vma, ALIGN(vmf->address, fault_size)); + + for (i = 0; i < fault_size / PAGE_SIZE; i++) { + struct page *page; + + page = pfn_to_page(pfn_t_to_pfn(pfn) + i); + if (page->mapping) + continue; + page->mapping = f_mapping; + page->index = pgoff + i; + } +} + +static void set_compound_mapping(struct vm_fault *vmf, pfn_t pfn, + unsigned long fault_size, + struct address_space *f_mapping) +{ + struct page *head; + + head = pfn_to_page(pfn_t_to_pfn(pfn)); + head = compound_head(head); + if (head->mapping) + return; + + head->mapping = f_mapping; + head->index = linear_page_index(vmf->vma, + ALIGN(vmf->address, fault_size)); +} + static vm_fault_t dev_dax_huge_fault(struct vm_fault *vmf, enum page_entry_size pe_size) { @@ -225,8 +261,7 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault *vmf, } if (rc == VM_FAULT_NOPAGE) { - unsigned long i; - pgoff_t pgoff; + struct dev_pagemap *pgmap = dev_dax->pgmap; /* * In the device-dax case the only possibility for a @@ -234,17 +269,10 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault *vmf, * mapped. No need to consider the zero page, or racing * conflicting mappings. */ - pgoff = linear_page_index(vmf->vma, - ALIGN(vmf->address, fault_size)); - for (i = 0; i < fault_size / PAGE_SIZE; i++) { - struct page *page; - - page = pfn_to_page(pfn_t_to_pfn(pfn) + i); - if (page->mapping) - continue; - page->mapping = filp->f_mapping; - page->index = pgoff + i; - } + if (pgmap->vmemmap_shift) + set_compound_mapping(vmf, pfn, fault_size, filp->f_mapping); + else + set_page_mapping(vmf, pfn, fault_size, filp->f_mapping); } dax_read_unlock(id); @@ -439,6 +467,9 @@ int dev_dax_probe(struct dev_dax *dev_dax) } pgmap->type = MEMORY_DEVICE_GENERIC; + if (dev_dax->align > PAGE_SIZE) + pgmap->vmemmap_shift = + order_base_2(dev_dax->align >> PAGE_SHIFT); addr = devm_memremap_pages(dev, pgmap); if (IS_ERR(addr)) return PTR_ERR(addr);