From patchwork Wed May 4 01:14:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liam R. Howlett" X-Patchwork-Id: 12840176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DD9EC433F5 for ; Thu, 5 May 2022 20:17:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 242516B0071; Thu, 5 May 2022 16:17:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F1346B0073; Thu, 5 May 2022 16:17:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01BA46B0074; Thu, 5 May 2022 16:17:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E639D6B0071 for ; Thu, 5 May 2022 16:17:20 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BC1DE2E54A for ; Thu, 5 May 2022 20:17:20 +0000 (UTC) X-FDA: 79432799040.18.3DFF39D Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf01.hostedemail.com (Postfix) with ESMTP id 3313340096 for ; Thu, 5 May 2022 20:17:11 +0000 (UTC) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 243KnJSF013484; Wed, 4 May 2022 01:14:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2021-07-09; bh=7Yn9V/1cTpY9CxoJnkNHIjhQ3Gvdb8BYbJIaBvjK20U=; b=MX/Mt+aiksR8+9b7Hip7Jwjz2b18U8gugyioNPt4kEkIfrPaUbDKInsmDR1lw9XSr2EM L+SYZOH88QxmThlXdDe+BQ5Gr/YAtrnPW+v9SG1D+PISpy51ttMcK2xg+GrsO1hIGisr FiG++u8XkMWGssz4hKxa4ALucA73WZ+PcZQGeULiM3g4mUQtRwmja/58nomJUWxFPPL4 Jck1/TOkHypn6pZqddnwxypT6svpq81f97OoIYEWt78KsxdYpz2vnKyFOfqIjcEuO9Lq FiAVNqj8vY5Hvvbl5YLiKvRqmWJq5ibYJZ+v+U/477C633/jUmNJ5IUk1RHCKoQ8taO/ hg== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3frvqsf6fd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 04 May 2022 01:14:40 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.16.1.2/8.16.1.2) with SMTP id 2441BXQv013207; Wed, 4 May 2022 01:14:40 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2049.outbound.protection.outlook.com [104.47.66.49]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com with ESMTP id 3fsvbmv41r-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 04 May 2022 01:14:40 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EdhKywSsE1S9J0G3chjP9X6iylUBIIGO2TFBNWIyuN/cQyGTrA3SYn+Qub1xFKOxxx40Czff6IEqfj3BdpuAysoQOmEMJm+FVehLUhMViKfYvX2i+MlQeAp8WcIBEtOwr/NCeprtJO3CKEYPgxO+2T0zGfGuaNd4MBFDuPfjzvLRLYdJDpKyaRJ2gU6UNwHAxrinmcgc4sjma+nyMuc8zi+bj5wuuSVt0JMt2NQ3xXE+4OGRHoNm0bizfIV1EurKIifxbM7P64+bHVXqPo02oWDECA0CwOK/ZSkabwScQGCW1C3QvZtUz6e5WXxtmUkWXgH+Q3aTBmEjKMdIqgQZgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7Yn9V/1cTpY9CxoJnkNHIjhQ3Gvdb8BYbJIaBvjK20U=; b=aYuabL9SlIpafQxDp3y8st1ApaMR1/P9ySAoKWhjSKFn+D3D9u3fzPXPL23BedkKk48XV7XBy8nMg39kP7ACYK13O28/YLiO8EXFpN9Me9A7eDLP6mhUdTi5NpUmaj6IOF31UE1tpDo5NLt3P1twfLNzNUFxtotGeQoGyZNU57QmAMbbkXqI0gZMedFkBWVI8AT8E2twwkZYFdQFjvZqKnkPnykb9y8r1U8sBGOTEAuSHjRoab+tLrCugCenJ1Vxm945HoDv8P+G+zBh9hgevgrwi5pzftFHuyPfkxHzqSmAUoaWoUx09eYifmeCusqJ06AJ2oGcuGwKZa5QHzn2OQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7Yn9V/1cTpY9CxoJnkNHIjhQ3Gvdb8BYbJIaBvjK20U=; b=DGAnHZlSvTVuZMrK1j/GXIo8bqWK9pVBx6hmchplR5uZqBISqEjCY+1NL3A1kIyLbkfhqnESALv0h9sBhnWETOr9YD61IMk/KEpFN3B8Vj+IT4vBkWlQD8iSGlqqIK0GAlNNihgF+KEEwcvXQWjhnAgTarjVKDcjgyUbfCE8lJU= Received: from SN6PR10MB3022.namprd10.prod.outlook.com (2603:10b6:805:d8::25) by SN6PR10MB2688.namprd10.prod.outlook.com (2603:10b6:805:4e::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Wed, 4 May 2022 01:14:37 +0000 Received: from SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::318c:d02:2280:c2c]) by SN6PR10MB3022.namprd10.prod.outlook.com ([fe80::318c:d02:2280:c2c%7]) with mapi id 15.20.5206.024; Wed, 4 May 2022 01:14:37 +0000 From: Liam Howlett To: "maple-tree@lists.infradead.org" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Andrew Morton Subject: [PATCH v9 52/69] mm/khugepaged: stop using vma linked list Thread-Topic: [PATCH v9 52/69] mm/khugepaged: stop using vma linked list Thread-Index: AQHYX1Q9bf9C/gs9X0Wx+jxWdBRhTg== Date: Wed, 4 May 2022 01:14:02 +0000 Message-ID: <20220504011345.662299-37-Liam.Howlett@oracle.com> References: <20220504002554.654642-1-Liam.Howlett@oracle.com> <20220504011345.662299-1-Liam.Howlett@oracle.com> In-Reply-To: <20220504011345.662299-1-Liam.Howlett@oracle.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.35.1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7c0442e6-7800-412c-631c-08da2d6b7532 x-ms-traffictypediagnostic: SN6PR10MB2688:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YJGZ4YI5jZi35va+SHehdzGKCWYnPLoqx81JcYJuaxFq6USxs9QzY4XeoOH+Wa1Rscv5v50xsBlX9qURl1ryOpskGnM+rFbutM9Fwn4uCtRrXhsnn2ELias4tS9mY73lzbbaXUpVvCsthAZkpzl0AuyplMDUadRSD1PbdvDcPM9NYp/AvaWo88+St0hTKO4KR0ipQB0iwZP7SdM1o5xFdTMdXC3AcdEWLOU4geAcezJhQA4e26aYhuZs+FnDLL7PY6IEk7Y1FnDzrbrF+dzgu33NLBh1kGJv4OAHF1VV+YDXJ0AblUe5ughlPqrHJjUl0PHU3QXXNZbfAJkNEe1T784deSEFFnyyW9Vm39tTvmjESRuoi2dIwvkTDieop8csM7mZ090dpHat4wOt3H+zqgFkpGQBBi9yKueYCBEE3iVSZSQUKJO1J3AKvSqoHhaRAAAOFLS3VosH2Cry7owZyjoiw8aEfVrv4uWpsh6L38t3OTh9uD1XbYl8gTKbEYUL/GNEawF6/k1LvTYarGFSvnBa8AA7Wl4lpe3Cy75FjjvCco8S1e2acYBtcV2TX3xL9JqcbgAW/WPm7xrw1HNW0n7sBCoi+BvUpwcGHmQcWuIqo8P5YP3E3gJ4ezBxxtfHJTjPyGY6Cl2e2qMiXqLC7TR+OEfDOitxzqip2RF+SE8Px+BYmu72wIcoVF39UOkXll2XRuO/CYfoIfLt99VbuQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR10MB3022.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(186003)(122000001)(1076003)(2616005)(66476007)(8676002)(66446008)(64756008)(38070700005)(36756003)(316002)(38100700002)(66556008)(66946007)(76116006)(91956017)(110136005)(26005)(6512007)(71200400001)(8936002)(6666004)(6506007)(2906002)(86362001)(5660300002)(6486002)(508600001)(83380400001)(44832011);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?yJe9CcPCywcl5ycNLyCj8gT?= =?iso-8859-1?q?6YjiaUBvPXoJyczCAOrArHOyWI20HLxmhiYHCqaKDewYBciF4HxpB2kRA137?= =?iso-8859-1?q?ZgcN+PX7iTjg2TctOLjWyS85n7H3FrLE4XbLKAIZ304gajb6H1VyXf6/uL64?= =?iso-8859-1?q?qF/N5PIGm4eON+Bc3RCAkJwfYfYMLnYiHSDY1mrpdASnbBAXIVTE/cqjtj7i?= =?iso-8859-1?q?r1Rfcyr1uwZD/tX9HDcql3Y7zXBiqXchMJ4h4SaeSSM7/c5YPnqGut6L5YjX?= =?iso-8859-1?q?HCDOnkxuBlNzyl185LRrhijdKxTHEqCr701BPJpxLFpjTKUWvZG3lOgZGQGX?= =?iso-8859-1?q?SzhB4OCYES5bCc/z9z60PJ+otIya8kYBH6GPBWd+XzVWnURkh0ca6CxNgUe3?= =?iso-8859-1?q?fIE4rhSIlUPyGR5xiCNCazo20wILpPfIvWPe68h43ZMxK274wH3xICRxz9XW?= =?iso-8859-1?q?5i1t7/dlzi2WazPQt1ApY4oWbB9ECigKd6xEvkylhmgCxvOq2OmuHyAhJ5HV?= =?iso-8859-1?q?RuNupf6xdhOiZFG7zIs7/A27rB9u2qUgSycVT240DLKRKhmnZMzZ1tTDf7tT?= =?iso-8859-1?q?MbbqBTb5s6QMdkB5tzHoAypXwcS6g0Wk9hcmFLr14bjA02dFL+jNGYh+KAfs?= =?iso-8859-1?q?ctDuvpoTA08XyPfu4evLTv9APCX7xXE9xoSTw0nTtWfPKb8Lppc48Y1TKOie?= =?iso-8859-1?q?9CoY3pv1kcDMCcBIFbdDX5rQnVGnWzTrUnz33rJb7VL6XXiEF0adtOukj0zP?= =?iso-8859-1?q?Er2d6UAl8VdpvmBQQsP9Y8gKNV5KOQHFc8i4K4XkLhaFWPNV/C8pyDh5t5Jw?= =?iso-8859-1?q?fReqT0tjtFqOzTmcb7sOtY+zNUPQ+DOjgowQrm/Zwx/UUP2JvlyQz00H/+qZ?= =?iso-8859-1?q?zhe7dJuTq82BJSnPoIrlWYwKI6ucOz2jdQp2GZnznNhk3nq7eL4t7FUQ9l07?= =?iso-8859-1?q?zKLuQQSLtsKqqa+MplFzbxr0UgW5G/6p2aAiyHVNO7AkQud8yUMmC4Eo2SoM?= =?iso-8859-1?q?9VTsMVGT/yFKj11cAZ+YusYXFsjBvJr0EXcokoRJpQ2iq5ZxOKzeAkj1wvfc?= =?iso-8859-1?q?NuR6AeE7R5zLjGRCq23jTEJ/CuiQgP9lCMBf+vSMUeodMYf+whEyEso3Qx8a?= =?iso-8859-1?q?/fs0YTC48Kvs+uf1IU2V8fb1VlvnK6RoYvvsdupv9jkVaUBIMAmrrpn9yzd0?= =?iso-8859-1?q?J7ltLzd3QavprcOICBjKs4QfQw1hHTudZFbB2ooPO7izXJXWlAtszrobm9GY?= =?iso-8859-1?q?jKYKlvb2rXU7NNKcpxCbBWsU9Vx4v/9nNdZphgD7jFfbf6MUhPUJ7o7ujd0z?= =?iso-8859-1?q?lxbNmPw1cuX+P3z087bAF0xQpGBoUXevFqMRHSVyHZNap/kejyu1S8KHIcEo?= =?iso-8859-1?q?XBI0CeRGvQuEPiLXX6vDMzMHmh/lc7LxvN24bW+iX64jQc/6QqbMACwVoRm7?= =?iso-8859-1?q?sHrKsc+NTbeinUAs5CZPNxm69yOQfjEL4KTK4C7zP1FcZmbR++9LNmQa0Cet?= =?iso-8859-1?q?hx2lcuPb3wRCLoBJ3HYKyjciH1gLLEgrjHBn+ctJ8PpJfGgKLzYyKnV8YkQc?= =?iso-8859-1?q?A2rGrVYJeARd1JRHoLla0Za2fpi974qLBd51zbwmMQtO6rAaWd3Oq4jwllp7?= =?iso-8859-1?q?Ic41ydbYBmyGbA2uhes9hJLgHaLysoJZzgnyjPX2E4DI/E5DalZTKjCHy2hP?= =?iso-8859-1?q?NoyXxjl9evopmsNhkCPs1tFrsbXsrYWlC12tBCIzCIzBKYA3SkatwvqI9iQt?= =?iso-8859-1?q?0yATcCALgtmfjc4DYNFijVxbZcnmOV646bXh0f9le8xrtQqnxkb7Dhp1IBqg?= =?iso-8859-1?q?6DW/zpPY=3D?= MIME-Version: 1.0 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7c0442e6-7800-412c-631c-08da2d6b7532 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 May 2022 01:14:02.5013 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: vPJBG2lz+lG9QbCHqciMYk6WKuTGokjlAzKclcIyb3rqPNzWApY5zeTDqpFyjZXKHvfOPXqYVXzKJb+A3Mm0Dw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2688 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-05-03_10:2022-05-02,2022-05-03 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 spamscore=0 malwarescore=0 adultscore=0 suspectscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205040006 X-Proofpoint-GUID: 09lkM9QnYoqPZ40tU-qhDICuYAKoCUCJ X-Proofpoint-ORIG-GUID: 09lkM9QnYoqPZ40tU-qhDICuYAKoCUCJ X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 3313340096 X-Stat-Signature: tydyog8q8a4jbenamys5kkethheigmez Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2021-07-09 header.b="MX/Mt+ai"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=DGAnHZlS; spf=none (imf01.hostedemail.com: domain of liam.howlett@oracle.com has no SPF policy when checking 205.220.177.32) smtp.mailfrom=liam.howlett@oracle.com; dmarc=pass (policy=none) header.from=oracle.com X-HE-Tag: 1651781831-129950 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Use vma iterator & find_vma() instead of vma linked list. Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Liam R. Howlett --- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 11 ++++++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c468fee595ff..c72827d9cf04 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2221,11 +2221,11 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, split_huge_pmd_if_needed(vma, end); /* - * If we're also updating the vma->vm_next->vm_start, + * If we're also updating the next vma vm_start, * check if we need to split it. */ if (adjust_next > 0) { - struct vm_area_struct *next = vma->vm_next; + struct vm_area_struct *next = find_vma(vma->vm_mm, vma->vm_end); unsigned long nstart = next->vm_start; nstart += adjust_next; split_huge_pmd_if_needed(next, nstart); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 03fda93ade3e..208fc0e19eb1 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -2089,10 +2089,12 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, __releases(&khugepaged_mm_lock) __acquires(&khugepaged_mm_lock) { + struct vma_iterator vmi; struct mm_slot *mm_slot; struct mm_struct *mm; struct vm_area_struct *vma; int progress = 0; + unsigned long address; VM_BUG_ON(!pages); lockdep_assert_held(&khugepaged_mm_lock); @@ -2116,11 +2118,14 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, vma = NULL; if (unlikely(!mmap_read_trylock(mm))) goto breakouterloop_mmap_lock; - if (likely(!khugepaged_test_exit(mm))) - vma = find_vma(mm, khugepaged_scan.address); progress++; - for (; vma; vma = vma->vm_next) { + if (unlikely(khugepaged_test_exit(mm))) + goto breakouterloop; + + address = khugepaged_scan.address; + vma_iter_init(&vmi, mm, address); + for_each_vma(vmi, vma) { unsigned long hstart, hend; cond_resched();