From patchwork Tue May 24 20:50:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 12860532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA3E4C433F5 for ; Tue, 24 May 2022 20:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D59D8D0005; Tue, 24 May 2022 16:50:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 884CA8D0001; Tue, 24 May 2022 16:50:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 723AD8D0005; Tue, 24 May 2022 16:50:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 650DB8D0001 for ; Tue, 24 May 2022 16:50:38 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2DDBC34480 for ; Tue, 24 May 2022 20:50:38 +0000 (UTC) X-FDA: 79501830156.01.B969906 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf15.hostedemail.com (Postfix) with ESMTP id 49081A0002 for ; Tue, 24 May 2022 20:50:18 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24OJT7E4018614; Tue, 24 May 2022 20:50:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=ISxhDVlPxp/UugQFvW0vUmVQQcd+qDNUR3rDl2UEFz4=; b=pFBNUpuTP/WHda2X8XztA7bpSjQCAutBI5yNowZXs9S/AP2Gq64lJSCJlqDTqZR9giGd W941Y+HCoC06dSyWLbxqamEaKksJSKDmD/8+BTKUa7ok/CQwFzzMOk13dddBB/BJon3h WvgfPMq2Ss7Cri30MiwqIcToQB8hhNR3a0iIBRCw+k3sngUbSffzHb5TycpLDVlPQtsG wwA6PeVSArHZ6N7YBXweC0JPcvDDwmuWxVf/OjihqJ0pCWYEYhfulcD8bDNHg5JIxBry KGAGXNo72NAGlDvEdrEXL7w+JkMD0TR/zqt2rhuLN9PKGBbvHQORo5baS0JNCccXzi43 6g== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3g93tc0f7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 24 May 2022 20:50:36 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 24OKoJQS016791; Tue, 24 May 2022 20:50:36 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2109.outbound.protection.outlook.com [104.47.55.109]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3g93x549d5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 24 May 2022 20:50:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X5rvJJ4L/mKyTCyIJITGcoV6NyhFvhfk3eWCe9yQ4rrgPHrjva8PC/z7yQrgIE3aIDkQqZk03jDH5L49NmfaY570rxSO4FAcM8RJ0lqMvHWlZFA7eMDii3seqkswbtyCB/RpnbFVQMsUOfa/3oWb4vdfAkXv0FCBVD3WY00FtF+zwQz4xIiyJURIZSqtwKEAd9FGECzXb1lmtblmFieMx7A5CMbG6wrK3F1dI7ZowyQgo9yL/NatRsqCYsZO7rtn/TDS4F770NVw+pLU4nA6S7pVxGMCgarOd4kvIGatKzzZRsGwCfP5iPolYIOS9wCl4VUDEhen+DidA4z5CxD0Wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ISxhDVlPxp/UugQFvW0vUmVQQcd+qDNUR3rDl2UEFz4=; b=YcdsxyoQ0XrYbWqaUcn77Wiy2er+mb76/5FJax2aDsdGfppcSQU+xlGc8NbArnyNxS23gca3ACnWS3/z1SIa0xtKo/+vvJPBYili4+FqQaPSk5aBz5vdCSrBu1nmQXk3QlB4Elh5gyXzqhs+ktfj98dEGLRByWH54GDmnJwvTIncjfMUTxj2OCglygHdsmYeW0DNPpuA2l+XDQWmZ+TEYZ4zKMXIw5XmChKN2COtyZyS5zcmb5PBdz4L/TyNiy7PP8x1P67BZpkznHOBGidQUFm/i9F9oUuNi+Rls3oAVWGu1Uq1wndaCKv+6vjzooHXXrR99eW4XNtgYXnyzazplg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ISxhDVlPxp/UugQFvW0vUmVQQcd+qDNUR3rDl2UEFz4=; b=GCp/e3/g5CUhECLJ2ojnPhFurb1IGOnVdNi1XYIZ2qOPHg98Xxt9KDLyoWhn+otlPeOtTpcfJO7oY8Bs9mTR418a9tNiHbg4XoeERd+iwB064WiyeTgp5lrb/lqBrufGTZH/7qSueFSc02GgEmpPD/Qd7MuG98Bw5qDxvHb8x70= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by CO6PR10MB5618.namprd10.prod.outlook.com (2603:10b6:303:149::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.16; Tue, 24 May 2022 20:50:34 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::2125:9bb7:bfeb:81f9]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::2125:9bb7:bfeb:81f9%9]) with mapi id 15.20.5273.023; Tue, 24 May 2022 20:50:34 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Mike Kravetz , stable@vger.kernel.org Subject: [PATCH] hugetlb: fix huge_pmd_unshare address update Date: Tue, 24 May 2022 13:50:03 -0700 Message-Id: <20220524205003.126184-1-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.35.3 X-ClientProxiedBy: MW4P223CA0005.NAMP223.PROD.OUTLOOK.COM (2603:10b6:303:80::10) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5be55647-7911-4a82-d387-08da3dc70c06 X-MS-TrafficTypeDiagnostic: CO6PR10MB5618:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qJExZkxCp/t2Ms2dXtGdOdoatbMRD8IaQ+ZbOZFJp/MFFW1RpabH2mqVtrv7fEG7dijlToX1cjH5g+uGxPpbpZJOxYCLDPKQAbaq//WobgBu2ccWsbIGubYyMP1QeqjK1UPdt/ml0gwVWu0efvZokaD1VKSs/5eTQ1EYXM4TOl3Sv9yHeYRPkQ86UjZVC3EWLlpwILXKuvo7WnisSvuhi3iyVf4jH78/jtiNolxzjXrwQbsg064WVO7YBq8YymzAliBsmBYNsH4uXae2RI3EnEOUJAovETr/l5TQ1WQaJP3yCHAbBAYa83iAKZhMhBMzjd53ZR3eljvln01RA4nP3XDhR1C/5aLZRjjwvpVxlqizvOVXX+4npj8u082I966BdwpmMlyrMbvoWs0kX1OpAzO+KNJ4w2xZ/576/LKLzIR2vATSPhuOLHCKDOsih0sfWm9WLd6AtpRiiD1zN3STCDFHOaycnDCqGIyWWdzMcDsFINv4dEQw8lfhxV17CRVVkc0GflhnX/l0+Q4VImdcFUZxLwHu8l6YZ97cxSutbOzpLPlmmqyK6XVXgff563gfxEZ72JnLkZaBP9Wrgaz/hdhvqLhHuLd81Xr1LpjgP/YdVJBl90BH+ODMg6Iyuj/MGMnzuKtg9aVIrXYqINDymGRosdv6LDJzsFfcgD4skV04Lgk9Uhep8WkTPWZ4lbzMjc7dQ3gtf3GWsgNPK5WPtwCwEKsGC0/O17tfp9X+9qQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(6486002)(6666004)(44832011)(316002)(5660300002)(54906003)(8936002)(508600001)(2906002)(83380400001)(6506007)(66556008)(66476007)(38350700002)(6512007)(26005)(36756003)(66946007)(4326008)(2616005)(15650500001)(52116002)(1076003)(8676002)(186003)(38100700002)(14583001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 24NM6F7GljyWyM1yVJC4Y9bAynT9YuyQ08tW1ei16ZWckePZsJeo4JVoW+rQHQ3IFyx1tq5ZNkIgzo/6WMnwmCMuRWYK1pevXl1u2b38aPdKqmxDebx/GLi3nnJNyFIq3m7vo1iCel+r+1EDhDJF7YZHMI2k166olwq3emaKJGrvaGLSV9yVSYS9/XdpQJb+ZHxZPXjm2weMYzrdChHioAUGepIu6LLQqZxzpsp0xyu4deLjBCtUB07VHfqeBg/xBV6li+wN3Z/rl1eAMSx6rZIIroNE3dwiUrHtN+FsyRcbvMTdntgSYQ30erFoNMfoH8fNrqTmlxKSrYf1CTGRhXDSCkWR8T0nxLV0OP+FQMRVeuudB3j02JoJtNmdoSME6UGwpxQbK3ugcld9/wuYDn//OfG36g3uTIpm5VvQfmrll47F5aVzv1j44rpSPQ0tw5t1Ujd154/7LlSjrjFEnhzN42bjOn2ClcFSWYn6b9bWnGYmbecIr0fAyNnsmI5KnK2lPvh7Zoo5cwwi94Zu1KSYaMgNeBsoa5YAy95UeP4TWD/0enMNmKgDGQ/6GPgB9LzhDEyxNvq1No5BvMlM7PzRAxOSNyg+dg6p5NKRu1v+UI9h+TByjujxfh1TVb0IzqsDeHGnLLSRj7bEf6LzbTO6F2RekkZajwyOWhwijSDPARrQpJA6gh/cpjGca74ww/mT6mF8e8D3kuPbt5dyBvnEuyalyMkIMimQf+zd5vBj14kNWL6ah9pq8MNWMB/uIiVuX3qtRIGKLev9KiuZIbBWN6LkhSpwN1M1vf+yjBvCS3jfZaZRTvyEmQZ5K6bewo24PRrfh4L8Znn9pJ2lpf0dMTC1P7iWfLA4xijeaaQ+8OkHwNiRlvASy7jeAfjXcdYNKmZT2RLdqNNDUJJ/cFeNJI2Zr/dKQo7LYSD7C0ruyl1FYQd0xIojvJ8rnNx+6ZsHfMdO4sep/YMVe7oR+vTpQ76Ty8Y45H8RC44FzYMMx3szYJ0W02iowmMIvp7r4EWCWUMZWx9z4czIbadvClEeG5r+edd070ClQIciVhV5FVnOndPMU6OlOQF54IXdkTCnAj4CEOEV8ep9TJzRVN/tJQPerAGh7XuM3jePhkYXGhDjsNZuUU4eE37hjVGTa+ECMNgUhgNgtzDaDMphsoit9K6+jxskA9iAm3sA35in835b9UeHqwyaIiTNKr6/1xZxKp7HDsw5MEiDsrU/CgTYQ3AzJeEVL9uvY/IW2Xuokwjh4N2tM1tWCSDM7NJtijiQ6HFSVF/azyeW0gwk1toahAwD4PdiK0GhEt+kPnkn69sNt1hRdPIj8esOv3zAFH5pwy2E9slOnj8vPePLt84q0zdPThj2p3NdFGYY/Ay+4G+C9bGvWTb77Uxx4ttIBF+DFVGFHisNhI8791p5Z7SyUWkOdI6FLErYPDFEmW8bPeTXmFRoJ721j3wNYGpzjzJ3dFUJ2FEQ5WhY4f8JlSfOxuI8UbWa75SHt5ViR5glhM1J/sq6Y3WchMXNzZsyqtETocI2ngVGGRjNUcJyRbXq8npgraagTXpfGP9OFZt9GNa919Bvbflr9vnsq8/I2+aofsmMwgBcukLLoBC9uKdF69uMMzvHfucdGOiwefE12oE9C7hfF7J7BUz/vJxvBGxn8QSas8Gg4mFq+uzpQy7X9xkHv1AkUNh599tlN5myoTDDpfgEwDKL4nOkmIy4D5gG2RwS6YpSMfJLbFweRN8xHqul1PMGxcGf+OXqc40= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5be55647-7911-4a82-d387-08da3dc70c06 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 May 2022 20:50:33.9827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: w3qtrAidsrr2V8OoHPtJti6yoPjDVchHRilpkyyZuzALsZ0V0UGBPask6K021/oGVXoSKDEt0nkdTDNcNUuxVw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR10MB5618 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.874 definitions=2022-05-24_08:2022-05-23,2022-05-24 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=820 adultscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000 definitions=main-2205240102 X-Proofpoint-ORIG-GUID: 5l9193N3t5EELc0hV_JtXyCrDR8iqoBQ X-Proofpoint-GUID: 5l9193N3t5EELc0hV_JtXyCrDR8iqoBQ Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b=pFBNUpuT; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b="GCp/e3/g"; spf=none (imf15.hostedemail.com: domain of mike.kravetz@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com X-Rspam-User: X-Stat-Signature: xy6irow3yqwjkpdi4fo7enqjxz94wwwj X-Rspamd-Queue-Id: 49081A0002 X-Rspamd-Server: rspam01 X-HE-Tag: 1653425418-870727 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The routine huge_pmd_unshare is passed a pointer to an address associated with an area which may be unshared. If unshare is successful this address is updated to 'optimize' callers iterating over huge page addresses. For the optimization to work correctly, address should be updated to the last huge page in the unmapped/unshared area. However, in the common case where the passed address is PUD_SIZE aligned, the address is incorrectly updated to the address of the preceding huge page. That wastes CPU cycles as the unmapped/unshared range is scanned twice. Cc: Fixes: 39dde65c9940 ("shared page table for hugetlb page") Signed-off-by: Mike Kravetz Acked-by: Muchun Song --- mm/hugetlb.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 01f0e2e5ab48..7c468ac1d069 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6755,7 +6755,14 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, pud_clear(pud); put_page(virt_to_page(ptep)); mm_dec_nr_pmds(mm); - *addr = ALIGN(*addr, HPAGE_SIZE * PTRS_PER_PTE) - HPAGE_SIZE; + /* + * This update of passed address optimizes loops sequentially + * processing addresses in increments of huge page size (PMD_SIZE + * in this case). By clearing the pud, a PUD_SIZE area is unmapped. + * Update address to the 'last page' in the cleared area so that + * calling loop can move to first page past this area. + */ + *addr |= PUD_SIZE - PMD_SIZE; return 1; }